Commit graph

91 commits

Author SHA1 Message Date
WANG MINGMING
ca724865b9
fix(stream): parName is null (#30592) 2025-04-19 10:26:32 +08:00
liuyao
fac36002c1
ci(stream): add ci and fix issue (#30270)
* ci(stream): add ci and fix issue

* ci: adj test

* ci: add test and fix issue

* ci: adj test

* ci(stream): add test for fill operator and fix issue

* fix: fix issue

* fix: fix issue

* fix: adjust defalut rec value
2025-03-20 10:34:31 +08:00
Haojun Liao
dfe32225de chore: merge main. 2025-03-17 14:51:11 +08:00
Haojun Liao
ab92886820
fix(stream): reduce the consensus checkpoint id trans. (#30105)
* fix(stream): reduce the consensus checkpoint id trans.

* refactor(stream): add some logs.

* refactor(stream): set the max checkpoint exec time 30min.

* refactor(stream): add checkpoint-consensus trans conflict check.

* refactor(stream): remove unused local variables.

* fix(stream): fix syntax error.

* fix(stream): 1. fix free memory error 2. continue if put result into dst hashmap failed.

* fix issue

* fix issue

* fix(mnd): follower mnode not processes the timer event.

* fix(stream): print correct error msg.

* fix(stream): add some logs.

* fix(stream): add some logs.

* fix(stream): add some logs.

* fix(stream): add some logs.

* fix(stream): add some logs.

* fix(stream): add some logs.

* fix(stream): add some logs.

* fix(stream): add some logs.

* fix(stream): truncate long subtable name

* fix(stream): add buffer len.

* refactor(stream): update some logs.

* fix issue

* refactor(stream): update some logs.

* refactor(stream): update some logs.

* fix(stream): check return value.

* fix(stream): fix syntax error.

* fix(stream): check return value.

* fix(stream): update the timer check in mnode.

* fix(stream): add restart stage tracking.

* fix(stream): track the start task stage for meta.

* fix(stream): fix error in log.

* refactor(stream): adjust log info.

* fix mem issue

* fix(stream): check the number of required tasks for consensus checkpointId.

* fix(stream): lock the whole start procedure.

* fix(stream): add lock during start all tasks.

* fix(stream): update logs.

* fix(stream): update logs.

* fix(stream): update logs.

* fix(stream): fix dead-lock.

* fix(stream): fix syntax error.

* fix(stream): not drop the scan-history task.

* fix(stream): fix syntax error.

* fix(stream): wait for executor stop before restarting.

* fix(stream): add some logs.

* fix(stream): add some logs.

* fix(stream): add some logs.

* fix(stream): disable some logs.

* fix(stream): reset the start info if no task left.

---------

Co-authored-by: 54liuyao <54liuyao@163.com>
Co-authored-by: Jinqing Kuang <kuangjinqingcn@gmail.com>
2025-03-17 10:20:17 +08:00
liuyao
2de6736160
feat(stream): add new trigger continuous_window_close (#30125)
* opt stream build twa result

* opt force window close memory

* feat(stream):optimize new interval and scan operator

* adj log

* opt code

* opt code

* fill history

* fix issue for fill history

* add ci

* feat(stream): add new stream nonblock interval operator

* adjust code

* use new scan operator

* use new scan operator

* add log

* fix issue

* recover stream scan next function

* fix issue

* fix issue

* fix issue

* ignore disorder data

* fix issue

* fix issue for interval sliding

* fix issue

* fix ci issue

* fix ci issue

* fix ci issue

* add semi && final nonblock interval operator

* fix issue

* fix issue

* fix issue

* fix issue

* fix issue

* fix issue

* fix issue

* fix issue

* refactor(stream): track the msgId for each upstream tasks.

* fix(stream): fix race condition.

* fix(stream): update the task last msgId when putting into input queue succ.

* fix issue

* fix issue

* put recalculate data to rocksdb

* fix issue

* fix issue

* enh(query)[TD-33071]: add support for saving and restoring tsdbReader scan progress

- Implement functionality to save scan progress during tsdbReader operations
- Enable resuming scans from the last saved position

* fix issue

* fix issue

* fix issue

* fix issue

* fix issue

* add rec interval check

* enh(stream):add recalculate tasks.

* enh(stream): support the re-calculate the tasks.

* fix issue && do refactor

* do refactor

* fix issue

* fix issue

* update backend opt

* add new interface

* add new session operator

* support blob

* add new session operator

* fix issue

* add rec state for rec task

* fix invalid read

* add new session window

* enh(stream): update the stream tasks backend.

* new session operator

* add pull data

* fix(stream): fix error in expand stream backend.

* fix issue

* fix issue

* fix issue

* merge code

* fix issue

* fix(stream): check for null ptr.

* fix(stream): add more check.

* fix issue

* fix issue

* fix issue

* add debug code

* fix issue

* fix issue

* fix issue

* set rec end flag

* fix(stream): opt re-calculate stream tasks.

* fix issue

* fix issue

* add new operator

* enh(stream): dispatch recalculate block to agg tasks.

* fix issue

* fix issue

* fix(stream): adjust filter.

* fix issue

* refactor

* refactor(stream): adjust the recalculate end block.

* fix issue

* fix(stream): set correct create trigger block.

* fix issue

* fix(stream): fix error in build recalculate end block.

* fix(stream): check null ptr.

* add stream client && fix issue

* fix mem leak

* fix(stream): free msg.

* add stream client

* fix(stream): fix error.

* add stream client && fix issue

* add stream client

* refactor(stream): set the recalculate task complete.

* add wend and group_id for session window dest table

* feat(stream): refactor and set the recalcul agg task complete.

* add cfg for adapter

* fix issue

* add state && event operator

* feat(stream): support fill-history task.

* add new fill operator

* fix(stream): set correct backend when updating fill-history task to recalculate task.

* add new fill operator

* fix(stream): s2025-03-06 11:10:31.272
et ops always open in scan tsdb

* fix(stream):set the correct taskType for sink task.

* new fill operator

* adj stream fill operator

* fix issue

* fix issue

* fix issue

* fix issue

* fix issue

* fix issue

* fix issue

* fix issue

* adj test

* fix issue

* fix(stream): fix issue

* fix(steam): fix issue

* fix(steam): fix issue

* fix(steam): fix issue

* fix(steam): fix issue

* fix(stream): fix issue

* fix(stream): fix issue

* fix(stream): fix issue

* fix: ut com error

* fix(stream): fix mem leak and adjust operator type check rule

* fix(stream): fix mem leak and adjust test case

* refactor code

* fix(stream): free items.

* fix(stream): free fix memory leak.

* fix(stream): fix syntax error.

* fix: ignore unexpect block

* fix: adjust op type

---------

Co-authored-by: Haojun Liao <hjliao@taosdata.com>
Co-authored-by: Jinqing Kuang <kuangjinqingcn@gmail.com>
Co-authored-by: yihaoDeng <luomoxyz@126.com>
2025-03-14 20:14:01 +08:00
wangjiaming
f50fcb85b8
feat(decimal): support decimal data type (#30060)
* decimal: create table

* decimal: add test case decimal.py

* decimal: add decimal.c

* support input decimal

* decimal test

* refactor svalue

* fix test cases

* add decimal unit test

* add decimal test cmake

* support insert and query decimal type

* define wide integer, support decimal128

* support decimal128 divide

* set decimal type expr res types

* scalar decimal

* convert to decimal

* fix decimal64/128 from str and to str

* fix decimal from str and decimal to str

* decimal simple conversion

* unit test for decimal

* decimal conversion and unit tests

* decimal + - * /

* decimal scalar ops and comparision

* start to refactor GET_TYPED_DATA

* support decimal max func, cast func

* refactor GET_TYPED_DATA interface

* decimal scalar comparision

* start to implement sum for decimal

* support sum and avg for decimal type

* decimal tests

* add decimal test

* decimal add test cases

* decimal use int256/int128

* decimal testing

* fix decimal table meta and add tests for decimal col streams

* fix create stream and create tsma

* test insert decimal values

* decimal from str

* test decimal input

* test parse decimal from string

* add taos_fetch_field_e api

* decimal insert tests

* test decimal operators

* decimal operator test

* feat:support decimal in raw block

* decimal operator tests

* decimal test

* feat:support decimal in raw block

* feat:support decimal in raw block

* feat:add schemaExt to SMqDataRsp

* feat:remove add schemaExt to SMqDataRsp

* feat:remove add schemaExt to SMqDataRsp

* feat:remove add schemaExt to SMqDataRsp

* decimal test operators

* decimal operator test

* test decimal operators

* test decimal compare operators

* decimal unary operator test

* decimal col with decimal col oper test

* test decimal col filtering

* fix decimal float operator test

* decimal test where filtering

* fix decimal filtering

* fix decimal order by

* fix decimal op test

* test decimal agg funcs

* test decimal functions

* remove assert

* fix ci build for ret check

* fix decimal windows build

* fix ci ret check

* skip decimal ret check

* skip decimal ret check

* fix decimal tests

* fix decimal ci test

* decimal test

* fix(tmq): heap user after free

* fix(tmq): double free

* fix(tmq): double free

* fix decimal tests

* fix(decimal): decimal test ci build

* fix(decimal): windows build

* fix(decimal): decimal test build

* fix(decimal): fix decimal build and tests

* fix(decimal): fix decimal tests

* fix(decimal): fix taos_fetch_fields_e api

* fix(decimal): fix decimal taos_fetch_fields_e api

* fix(decimal): rebase 3.0

* fix(decimal): fix decimal functions

* fix(decimal): fix decimal test case memory leak

* fix(decimal): fix decimal tests

* fix(decimal): fix decimal test case

* fix(decimal): fix decimal tests

* feat(decimal): fix unit tests

* feat(decimal): fix deicmal unit test

---------

Co-authored-by: wangmm0220 <wangmm0220@gmail.com>
Co-authored-by: yihaoDeng <yhdeng@taosdata.com>
2025-03-14 18:08:07 +08:00
WANG MINGMING
4a3eda51e0
Merge branch '3.0' into feat/TD-32642 2024-12-10 14:11:47 +08:00
wangmm0220
0165684131 feat:[TD-32642] merge from 3.0 2024-12-05 23:47:46 +08:00
54liuyao
2a8263efea opt force window close memory 2024-12-03 09:11:46 +08:00
wangmm0220
ea9c5d3ce6 feat:[TD-32642] add timezone for connection support 2024-11-28 18:59:54 +08:00
54liuyao
9d668150d5 fix stream issue 2024-11-27 16:24:46 +08:00
54liuyao
e63b8a0ccb fix issue 2024-11-27 10:55:52 +08:00
54liuyao
0669910da5 fix issue 2024-11-27 08:58:39 +08:00
54liuyao
22023575ba opt stream force fill op 2024-11-26 19:46:36 +08:00
54liuyao
f37ae57a87 opt stream row buff 2024-11-26 15:23:57 +08:00
54liuyao
527c3d59a9 fix issue 2024-11-06 11:44:02 +08:00
54liuyao
48d5f0e7cd fix issue 2024-11-05 18:14:35 +08:00
54liuyao
e9b9ebeb6e fic comp issue 2024-10-31 18:52:24 +08:00
54liuyao
5b9f434532 fix issue for value 2024-10-31 17:47:54 +08:00
54liuyao
5158bf6b96 fix issue 2024-10-31 15:50:56 +08:00
54liuyao
cea454b002 Merge branch '3.0' of https://github.com/taosdata/TDengine into fix/TD-30837 2024-10-24 19:24:55 +08:00
wangjiaming0909
d1d6689684 add not fill exprs for fill operator 2024-10-23 15:16:16 +08:00
54liuyao
a62b5a3efd fix issue 2024-10-17 14:55:07 +08:00
54liuyao
39379af1a2 add twa ci 2024-10-16 17:34:03 +08:00
54liuyao
b24c9c5d20 fix ci check 2024-10-16 13:39:32 +08:00
54liuyao
3e49f40c74 stream twa 2024-10-16 12:16:59 +08:00
54liuyao
58d8b9f84e fix ci issue 2024-10-09 11:32:20 +08:00
54liuyao
c9308e7482 Merge branch '3.0' of https://github.com/taosdata/TDengine into fix/TD-30837 2024-09-30 17:12:25 +08:00
54liuyao
2a6e0ed386 calculate stream fill row buff size 2024-09-30 11:33:44 +08:00
54liuyao
d44529e62f Merge branch '3.0' of https://github.com/taosdata/TDengine into fix/TD-30837 2024-09-25 15:12:15 +08:00
Jinqing Kuang
83f1a590b0 fix(query)[TD-32258]. Fix error handling in operator interface functions
- Contain errors within individual operators, preventing error
  propagation to upper-level operators
- Use longjmp to jump directly to the outermost error handler
  for unresolvable issues, avoiding unnecessary error code
  returns through multiple layers
- Simplify error-handling logic for better maintainability
2024-09-24 18:15:14 +08:00
lyh250-666
116844208d enh:modify error code passing 2024-09-20 13:23:44 +08:00
54liuyao
442bf96b87 add check 2024-09-18 17:24:00 +08:00
54liuyao
88d01d8980 Merge branch '3.0' of https://github.com/taosdata/TDengine into fix/TD-30837 2024-09-09 11:21:23 +08:00
Haojun Liao
0236db51f6 fix(query): fix memory leak and do some internal refactor. 2024-09-06 15:44:25 +08:00
54liuyao
43390b1c36 Merge branch '3.0' of https://github.com/taosdata/TDengine into fix/TD-30837 2024-09-04 09:20:11 +08:00
lyh250-666
9c8f630759 enh:modify error code passing in libs/executor/ 2024-09-03 11:01:02 +08:00
54liuyao
9d8e8892b4 Merge branch '3.0' of https://github.com/taosdata/TDengine into fix/TD-30837 2024-08-30 16:04:09 +08:00
xsren
d43df78537 fix: free operator 2024-08-29 19:43:59 +08:00
Haojun Liao
09e31670ce
Merge pull request #27495 from taosdata/fix/syntax
fix(query): fix memory leak.
2024-08-28 11:58:03 +08:00
Haojun Liao
c118df0ec4 fix(query): fix memory leak. 2024-08-27 17:04:44 +08:00
54liuyao
42a31d4ca1 fix(query):fix mem leak for fill 2024-08-26 15:32:50 +08:00
54liuyao
2d4ff65509 adj state cache 2024-08-22 15:59:42 +08:00
54liuyao
3583237942 Merge branch '3.0' of https://github.com/taosdata/TDengine into fix/TD-30837 2024-08-22 14:47:16 +08:00
54liuyao
99bdcb3f58 enh(stream):remove assert of stream operator 2024-08-20 15:02:09 +08:00
54liuyao
4fe68034cf fix issue 2024-08-16 17:46:14 +08:00
54liuyao
a82c220645 fix issue 2024-08-15 17:38:29 +08:00
54liuyao
cf796a45c1 fix issue 2024-08-15 17:03:32 +08:00
54liuyao
6ab21a0d17 fix issue 2024-08-15 13:20:42 +08:00
54liuyao
057f09ffbf fix issue && add ci 2024-08-14 19:20:21 +08:00