Commit graph

37 commits

Author SHA1 Message Date
liuyao
2de6736160
feat(stream): add new trigger continuous_window_close (#30125)
* opt stream build twa result

* opt force window close memory

* feat(stream):optimize new interval and scan operator

* adj log

* opt code

* opt code

* fill history

* fix issue for fill history

* add ci

* feat(stream): add new stream nonblock interval operator

* adjust code

* use new scan operator

* use new scan operator

* add log

* fix issue

* recover stream scan next function

* fix issue

* fix issue

* fix issue

* ignore disorder data

* fix issue

* fix issue for interval sliding

* fix issue

* fix ci issue

* fix ci issue

* fix ci issue

* add semi && final nonblock interval operator

* fix issue

* fix issue

* fix issue

* fix issue

* fix issue

* fix issue

* fix issue

* fix issue

* refactor(stream): track the msgId for each upstream tasks.

* fix(stream): fix race condition.

* fix(stream): update the task last msgId when putting into input queue succ.

* fix issue

* fix issue

* put recalculate data to rocksdb

* fix issue

* fix issue

* enh(query)[TD-33071]: add support for saving and restoring tsdbReader scan progress

- Implement functionality to save scan progress during tsdbReader operations
- Enable resuming scans from the last saved position

* fix issue

* fix issue

* fix issue

* fix issue

* fix issue

* add rec interval check

* enh(stream):add recalculate tasks.

* enh(stream): support the re-calculate the tasks.

* fix issue && do refactor

* do refactor

* fix issue

* fix issue

* update backend opt

* add new interface

* add new session operator

* support blob

* add new session operator

* fix issue

* add rec state for rec task

* fix invalid read

* add new session window

* enh(stream): update the stream tasks backend.

* new session operator

* add pull data

* fix(stream): fix error in expand stream backend.

* fix issue

* fix issue

* fix issue

* merge code

* fix issue

* fix(stream): check for null ptr.

* fix(stream): add more check.

* fix issue

* fix issue

* fix issue

* add debug code

* fix issue

* fix issue

* fix issue

* set rec end flag

* fix(stream): opt re-calculate stream tasks.

* fix issue

* fix issue

* add new operator

* enh(stream): dispatch recalculate block to agg tasks.

* fix issue

* fix issue

* fix(stream): adjust filter.

* fix issue

* refactor

* refactor(stream): adjust the recalculate end block.

* fix issue

* fix(stream): set correct create trigger block.

* fix issue

* fix(stream): fix error in build recalculate end block.

* fix(stream): check null ptr.

* add stream client && fix issue

* fix mem leak

* fix(stream): free msg.

* add stream client

* fix(stream): fix error.

* add stream client && fix issue

* add stream client

* refactor(stream): set the recalculate task complete.

* add wend and group_id for session window dest table

* feat(stream): refactor and set the recalcul agg task complete.

* add cfg for adapter

* fix issue

* add state && event operator

* feat(stream): support fill-history task.

* add new fill operator

* fix(stream): set correct backend when updating fill-history task to recalculate task.

* add new fill operator

* fix(stream): s2025-03-06 11:10:31.272
et ops always open in scan tsdb

* fix(stream):set the correct taskType for sink task.

* new fill operator

* adj stream fill operator

* fix issue

* fix issue

* fix issue

* fix issue

* fix issue

* fix issue

* fix issue

* fix issue

* adj test

* fix issue

* fix(stream): fix issue

* fix(steam): fix issue

* fix(steam): fix issue

* fix(steam): fix issue

* fix(steam): fix issue

* fix(stream): fix issue

* fix(stream): fix issue

* fix(stream): fix issue

* fix: ut com error

* fix(stream): fix mem leak and adjust operator type check rule

* fix(stream): fix mem leak and adjust test case

* refactor code

* fix(stream): free items.

* fix(stream): free fix memory leak.

* fix(stream): fix syntax error.

* fix: ignore unexpect block

* fix: adjust op type

---------

Co-authored-by: Haojun Liao <hjliao@taosdata.com>
Co-authored-by: Jinqing Kuang <kuangjinqingcn@gmail.com>
Co-authored-by: yihaoDeng <luomoxyz@126.com>
2025-03-14 20:14:01 +08:00
Jinqing Kuang
eb5d463490 feat(stream)[TS-5469]. add support for window event notifications in stream processing
- Introduce new syntax to specify notification type and destination address
- Collect relevant event information during window computations
- Implement websocket-based notification delivery to the specified address
2025-01-21 17:39:46 +08:00
dapan1121
a8561fba4f Merge remote-tracking branch 'origin/3.0' into feat/TD-30268 2024-11-11 11:01:22 +08:00
dapan1121
c5c85efeb5 enh: adjust mode and retire policy 2024-11-06 19:26:14 +08:00
lyh250-666
5f6802e0a1 enh:modify addede rror code passing 2024-09-24 16:29:28 +08:00
lyh250-666
116844208d enh:modify error code passing 2024-09-20 13:23:44 +08:00
Haojun Liao
1e57724533
Merge pull request #27688 from taosdata/fix/3_liaohj
fix(stream): handle continuous retrieve during checkpoint procedure.
2024-09-06 09:11:08 +08:00
Haojun Liao
277528996e fix(stream): handle continuous retrieve during checkpoint procedure. 2024-09-05 16:04:36 +08:00
Pan Wei
aeb6fa9b25
Merge pull request #27613 from taosdata/fix/3.0/TD-31682-TD-31793
fix querytask.c memory leak when random malloc error
2024-09-03 18:30:22 +08:00
lyh250-666
9c8f630759 enh:modify error code passing in libs/executor/ 2024-09-03 11:01:02 +08:00
wangjiaming0909
38e8d9f2fa fix querytask.c memory leak when random malloc error
remove (void)
2024-09-03 09:13:40 +08:00
Yihao Deng
7b09a9f8b1 change log qid 2024-08-27 08:25:28 +00:00
Yihao Deng
e1998d151b update qid log 2024-08-24 10:19:25 +00:00
wangjiaming0909
7c6be8c357 fix exec task memory leaks 2024-08-13 11:23:00 +08:00
Haojun Liao
59d190508d fix(query):check return code 2024-08-06 16:46:40 +08:00
Haojun Liao
5524d42c2d fix: check return value and fix bugs. 2024-08-06 10:19:22 +08:00
Haojun Liao
3772f25840 fix(stream): check return value. 2024-08-05 18:02:32 +08:00
Haojun Liao
aacf1c3e9a fix(stream): check return value. 2024-07-24 17:08:08 +08:00
Haojun Liao
eefee54c6a fix(stream): check return value. 2024-07-23 17:31:41 +08:00
wangjiaming0909
11a98ffab2 QueryAutoQWorker 2024-07-01 11:08:39 +08:00
wangmm0220
912cb09839 Merge branch 'feat/TS-4243-3.0' of https://github.com/taosdata/TDengine into feat/TS-4243-tmq 2024-03-22 18:22:13 +08:00
wangmm0220
f1853ced26 feat:[TS-4243] add logic for primark key in tmq 2024-03-22 17:48:59 +08:00
Haojun Liao
93c808326b fix(meta): fix repeatly reader lock for tsdb meta. 2024-03-16 22:03:46 +08:00
dapan1121
143c657571 Merge remote-tracking branch 'origin/3.0' into feat/TD-24700 2023-08-16 14:50:31 +08:00
wangjiaming0909
5c276fa547 fix: make kill query work for sysscanoperator 2023-08-14 15:21:58 +08:00
dapan1121
c9656d3f77 fix: fix memory issues 2023-08-11 17:39:41 +08:00
dapan1121
fb63294289 fix: hash join res issue 2023-07-25 09:03:52 +08:00
Haojun Liao
c3be7b14be fix: set correct function ptr. 2023-05-25 17:51:03 +08:00
Haojun Liao
1f7f326bed refactor: do some internal refactor and set the api function ptr. 2023-05-24 13:22:05 +08:00
Haojun Liao
66477a28ca refactor: do some internal refactor. 2023-05-23 19:10:50 +08:00
Haojun Liao
e67b532d8a refactor: refactor the module dependency. 2023-05-23 18:29:23 +08:00
Haojun Liao
09a04052ec refactor: do some internal refactor. 2023-05-04 16:15:14 +08:00
Haojun Liao
05d416f3b9 fix(stream): fix invalid access when handling error, not start stream if tsDisablestream is set. 2023-04-28 23:53:53 +08:00
Haojun Liao
da0d9c78ef refactor: do some internal refactor. 2023-04-28 11:42:34 +08:00
Haojun Liao
6482ef9ad1 refactor: do some internal refactor. 2023-04-28 09:19:53 +08:00
Haojun Liao
8205168428 fix(query): fix memory leak. 2023-04-28 09:16:11 +08:00
Haojun Liao
7adc21028a refactor: do some internal refactor. 2023-04-28 00:23:38 +08:00