Elgato_dark/TDengine
1
0
Fork 0
mirror of https://github.com/taosdata/TDengine synced 2026-05-24 10:09:01 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 4
78642 commits 554 branches 514 tags 1.7 GiB
Commit graph

131 commits

Author SHA1 Message Date
dapan1121
0a01e7f4f1 fix: add create stream log 2025-05-20 08:15:23 +08:00
dapan1121
e4cd39affc fix: add stream object debug log 2025-05-19 17:46:59 +08:00
dapan1121
8ed0f7214e enh: add create snode processing 2025-05-13 14:37:07 +08:00
dapan1121
73d13075b1 enh: add snode failover 2025-05-12 15:19:22 +08:00
dapan1121
ca2241f6ad fix: add snode task redeploy 2025-05-10 17:16:39 +08:00
dapan1121
2dd6d3eb28 enh: add drop stream processing 2025-04-30 18:39:59 +08:00
dapan1121
330cf9982d fix: compile issues 2025-04-24 09:11:54 +08:00
dapan1121
ae0e1ded7d enh: add hb processing 2025-04-22 17:46:17 +08:00
dapan1121
21e5aae05c enh: add stream hb processing 2025-04-19 18:58:51 +08:00
dapan1121
45d0af0a63 enh: add create stream 2025-04-10 19:41:37 +08:00
dapan1121
675790a63c fix: remove codes 2025-04-10 09:09:30 +08:00
Jinqing Kuang
c82398e175 fix(stream): fix stream processing for virtual tables 
- Fixed status check in virtual table stream processing
- Fixed multi-way merge logic for super virtual tables
- Fixed generation of target subtable names during merge
 2025-03-20 01:15:54 +08:00
Haojun Liao
dfe32225de chore: merge main. 2025-03-17 14:51:11 +08:00
Haojun Liao
ab92886820
fix(stream): reduce the consensus checkpoint id trans. (#30105) 
* fix(stream): reduce the consensus checkpoint id trans.

* refactor(stream): add some logs.

* refactor(stream): set the max checkpoint exec time 30min.

* refactor(stream): add checkpoint-consensus trans conflict check.

* refactor(stream): remove unused local variables.

* fix(stream): fix syntax error.

* fix(stream): 1. fix free memory error 2. continue if put result into dst hashmap failed.

* fix issue

* fix issue

* fix(mnd): follower mnode not processes the timer event.

* fix(stream): print correct error msg.

* fix(stream): add some logs.

* fix(stream): add some logs.

* fix(stream): add some logs.

* fix(stream): add some logs.

* fix(stream): add some logs.

* fix(stream): add some logs.

* fix(stream): add some logs.

* fix(stream): add some logs.

* fix(stream): truncate long subtable name

* fix(stream): add buffer len.

* refactor(stream): update some logs.

* fix issue

* refactor(stream): update some logs.

* refactor(stream): update some logs.

* fix(stream): check return value.

* fix(stream): fix syntax error.

* fix(stream): check return value.

* fix(stream): update the timer check in mnode.

* fix(stream): add restart stage tracking.

* fix(stream): track the start task stage for meta.

* fix(stream): fix error in log.

* refactor(stream): adjust log info.

* fix mem issue

* fix(stream): check the number of required tasks for consensus checkpointId.

* fix(stream): lock the whole start procedure.

* fix(stream): add lock during start all tasks.

* fix(stream): update logs.

* fix(stream): update logs.

* fix(stream): update logs.

* fix(stream): fix dead-lock.

* fix(stream): fix syntax error.

* fix(stream): not drop the scan-history task.

* fix(stream): fix syntax error.

* fix(stream): wait for executor stop before restarting.

* fix(stream): add some logs.

* fix(stream): add some logs.

* fix(stream): add some logs.

* fix(stream): disable some logs.

* fix(stream): reset the start info if no task left.

---------

Co-authored-by: 54liuyao <54liuyao@163.com>
Co-authored-by: Jinqing Kuang <kuangjinqingcn@gmail.com>
 2025-03-17 10:20:17 +08:00
liuyao
2de6736160
feat(stream): add new trigger continuous_window_close (#30125) 
* opt stream build twa result

* opt force window close memory

* feat(stream):optimize new interval and scan operator

* adj log

* opt code

* opt code

* fill history

* fix issue for fill history

* add ci

* feat(stream): add new stream nonblock interval operator

* adjust code

* use new scan operator

* use new scan operator

* add log

* fix issue

* recover stream scan next function

* fix issue

* fix issue

* fix issue

* ignore disorder data

* fix issue

* fix issue for interval sliding

* fix issue

* fix ci issue

* fix ci issue

* fix ci issue

* add semi && final nonblock interval operator

* fix issue

* fix issue

* fix issue

* fix issue

* fix issue

* fix issue

* fix issue

* fix issue

* refactor(stream): track the msgId for each upstream tasks.

* fix(stream): fix race condition.

* fix(stream): update the task last msgId when putting into input queue succ.

* fix issue

* fix issue

* put recalculate data to rocksdb

* fix issue

* fix issue

* enh(query)[TD-33071]: add support for saving and restoring tsdbReader scan progress

- Implement functionality to save scan progress during tsdbReader operations
- Enable resuming scans from the last saved position

* fix issue

* fix issue

* fix issue

* fix issue

* fix issue

* add rec interval check

* enh(stream):add recalculate tasks.

* enh(stream): support the re-calculate the tasks.

* fix issue && do refactor

* do refactor

* fix issue

* fix issue

* update backend opt

* add new interface

* add new session operator

* support blob

* add new session operator

* fix issue

* add rec state for rec task

* fix invalid read

* add new session window

* enh(stream): update the stream tasks backend.

* new session operator

* add pull data

* fix(stream): fix error in expand stream backend.

* fix issue

* fix issue

* fix issue

* merge code

* fix issue

* fix(stream): check for null ptr.

* fix(stream): add more check.

* fix issue

* fix issue

* fix issue

* add debug code

* fix issue

* fix issue

* fix issue

* set rec end flag

* fix(stream): opt re-calculate stream tasks.

* fix issue

* fix issue

* add new operator

* enh(stream): dispatch recalculate block to agg tasks.

* fix issue

* fix issue

* fix(stream): adjust filter.

* fix issue

* refactor

* refactor(stream): adjust the recalculate end block.

* fix issue

* fix(stream): set correct create trigger block.

* fix issue

* fix(stream): fix error in build recalculate end block.

* fix(stream): check null ptr.

* add stream client && fix issue

* fix mem leak

* fix(stream): free msg.

* add stream client

* fix(stream): fix error.

* add stream client && fix issue

* add stream client

* refactor(stream): set the recalculate task complete.

* add wend and group_id for session window dest table

* feat(stream): refactor and set the recalcul agg task complete.

* add cfg for adapter

* fix issue

* add state && event operator

* feat(stream): support fill-history task.

* add new fill operator

* fix(stream): set correct backend when updating fill-history task to recalculate task.

* add new fill operator

* fix(stream): s2025-03-06 11:10:31.272
et ops always open in scan tsdb

* fix(stream):set the correct taskType for sink task.

* new fill operator

* adj stream fill operator

* fix issue

* fix issue

* fix issue

* fix issue

* fix issue

* fix issue

* fix issue

* fix issue

* adj test

* fix issue

* fix(stream): fix issue

* fix(steam): fix issue

* fix(steam): fix issue

* fix(steam): fix issue

* fix(steam): fix issue

* fix(stream): fix issue

* fix(stream): fix issue

* fix(stream): fix issue

* fix: ut com error

* fix(stream): fix mem leak and adjust operator type check rule

* fix(stream): fix mem leak and adjust test case

* refactor code

* fix(stream): free items.

* fix(stream): free fix memory leak.

* fix(stream): fix syntax error.

* fix: ignore unexpect block

* fix: adjust op type

---------

Co-authored-by: Haojun Liao <hjliao@taosdata.com>
Co-authored-by: Jinqing Kuang <kuangjinqingcn@gmail.com>
Co-authored-by: yihaoDeng <luomoxyz@126.com>
 2025-03-14 20:14:01 +08:00
WANG MINGMING
3d053e2c9b
feat(stream): optimize client logic of creating stream if with history (#30059) 
* feat:[TS-5617]use last_row cache mode if creating stream in fill_history

* feat(stream): optimize client logic of creating stream if with history

* feat(stream): optimize client logic of creating stream if with history

* feat(stream): optimize client logic of creating stream if with history

* feat(stream): optimize client logic of creating stream if with history

* feat(stream): optimize client logic of creating stream if with history

* feat(stream): optimize client logic of creating stream if with history

* feat(stream): optimize client logic of creating stream if with history

* feat(stream): optimize client logic of creating stream if with history

* feat(stream): optimize client logic of creating stream if with history

* feat(stream): optimize client logic of creating stream if with history

* feat(stream): optimize client logic of creating stream if with history

* fix: heap use after free

* feat: add log

* fix: ci case error

* fix: compile error in windows

* fix: ci case error

* fix: heap user after free

* fix: memory leak

* fix: ci case error

* fix: ci case error

* fix: ci case error

---------

Co-authored-by: yihaoDeng <yhdeng@taosdata.com>
 2025-03-14 13:55:52 +08:00
Shengliang Guan
f16ae7abe1
Merge pull request #29809 from taosdata/merge/mainto3.0 
merge: from main to 3.0 branch
 2025-02-20 10:27:20 +08:00
Jinqing Kuang
fc2d2672b9 feat(stream)[TS-5469]: add more stream event notifications 
- Add event notifications for all window types
- Implement window invalidation notifications
- Add statistics for event notification delivery
 2025-02-16 11:07:16 +08:00
Haojun Liao
267f7d3b08 refactor(stream): limit the maximum number of consensus checkpoint trans. 2025-02-11 10:31:11 +08:00
Haojun Liao
0ea46585f4 fix(stream): update checkpoint-info after check the failed checkpointId, and update the consensus-checkpoint id in mnode. 2025-01-21 16:29:00 +08:00
Haojun Liao
df6ec3afc2 fix(stream): check null ptr. 2025-01-21 11:21:53 +08:00
Haojun Liao
08092aeb0c fix(stream): not clear task list if check failed, and add more checks. 2025-01-21 10:27:39 +08:00
Haojun Liao
7900c725d5 fix(stream): fix syntax error. 2025-01-21 09:15:27 +08:00
Haojun Liao
efd33aa4d7 fix(stream): check the checkpoint-report transId and checkpointId, and identify the expired checkpoint-report info. 2025-01-21 01:25:53 +08:00
54liuyao
e6ac454303 Merge branch 'main' of https://github.com/taosdata/TDengine 2024-12-17 18:02:36 +08:00
54liuyao
63a6428e34 fix(stream):adjust status of ins_streams 2024-12-17 17:59:53 +08:00
Haojun Liao
d2146a2efb other: merge main 2024-12-17 09:23:13 +08:00
Shengliang Guan
4c1499ebeb
Merge pull request #29122 from taosdata/enh/3.0/TD-31709 
Replace unsafe memory functions with safe versions in wal&sync&mnode.
 2024-12-16 21:37:33 +08:00
Haojun Liao
779c1bd8ad refactor(stream): inject the reject dispatch error in stream to test the effect of fix. 2024-12-16 13:53:13 +08:00
Shengliang Guan
3311e71cd9
Merge pull request #29152 from taosdata/main 
merge: from main to 3.0 branch
 2024-12-16 11:27:02 +08:00
xiao-77
7b921b81d2 Fix ci. 2024-12-13 17:48:51 +08:00
xiao-77
c66a834d1e Replace unsafe memory functions with safe versions in wal&sync. 2024-12-13 15:19:29 +08:00
Haojun Liao
d230082a08 refactor(stream): check mnode when issue the nodeEp update trans. 2024-12-10 22:52:20 +08:00
Shengliang Guan
84ea0f1ddd Merge branch 'main' into merge/mainto2.0 2024-12-03 15:50:18 +08:00
Haojun Liao
67d6458254 fix: check return value. 2024-11-30 00:34:13 +08:00
Haojun Liao
9a9a1828c1 refactor: display the time window for force_window_close. 2024-11-29 13:47:20 +08:00
Haojun Liao
187997d5f0 fix(stream): record the failed checkpointId, after receive reset task info. 2024-11-14 17:40:37 +08:00
54liuyao
158eda13f9 Merge branch '3.0' of https://github.com/taosdata/TDengine into fix/TD-30837 2024-10-31 17:49:58 +08:00
Haojun Liao
39495ec935 refactor: do some internal refactor. 2024-10-09 10:07:08 +08:00
Haojun Liao
6170bb5c1c enh(stream): check the existence for snode. 2024-10-09 09:39:32 +08:00
54liuyao
d44529e62f Merge branch '3.0' of https://github.com/taosdata/TDengine into fix/TD-30837 2024-09-25 15:12:15 +08:00
54liuyao
e53fafc08f Merge branch '3.0' of https://github.com/taosdata/TDengine into fix/TD-30837 2024-09-20 15:31:03 +08:00
Haojun Liao
0a7022693c refactor: do some internal refactor. 2024-09-19 18:13:46 +08:00
Haojun Liao
720d1c2486 fix(stream): fix memory leak. 2024-09-18 16:43:18 +08:00
54liuyao
c4566f98b8 Merge branch '3.0' of https://github.com/taosdata/TDengine into fix/TD-30837 2024-09-18 16:26:37 +08:00
Haojun Liao
0335799e1c refactor: remove void. 2024-09-12 21:48:54 +08:00
54liuyao
7c32cc7796 Merge branch '3.0' of https://github.com/taosdata/TDengine into fix/TD-30837 2024-09-12 09:08:40 +08:00
Haojun Liao
568b18d4ed fix(stream): update the acceptcode for task epset update. 2024-09-10 14:34:53 +08:00
Haojun Liao
81b0bdadc1 fix(stream): update the acceptcode for task epset update. 2024-09-10 14:31:24 +08:00
54liuyao
88d01d8980 Merge branch '3.0' of https://github.com/taosdata/TDengine into fix/TD-30837 2024-09-09 11:21:23 +08:00