Pan Wei
bcc25e56bc
feat: new stream ( #31678 )
...
* fix: windows compile issue
* test: add vtable cases (#31829 )
* fix: windows compile issues
* test:add test cases
* fix: windows compile issue
* case: em-4 stream case submit
* test: stream4_sub1 found bug2
* test: submit test_scene_meters_bug2.py
* add stream parameters example
* feat: [TS-6100] Do not translate const value as column.
* Feat/ts 6100 3.0 zlv (#31747 )
* modify asan exampel
* modify asan exampel
* add example
* add example
* modify case example
---------
Co-authored-by: zelv01 <1101510017@qq.com>
* feat(stream): fix memory leak
* modify sliding example
* test: update test case.
* feat(stream): fix conflicts
* fix: add offset case 10a 10s 10m 10h 10d
* feat(stream): fix conflicts
* chore(stream): rename case name #TS-6100
* add case
* modify example
* fix: windows compile issues
* fix: data null check
* feat: [TS-6100] Forbid where when using %%trows (#31827 )
* feat: [TS-6100] Forbid where when using %%trows
* test: update cases
* feat: [TS-6100] Fix leaks.
---------
Co-authored-by: Simon Guan <guanshengliang@qq.com>
* test: reproduce bugs
* test: update test case.
* test: update test case.
* feat: [TS-6100] Fix leaks.
* test: add cases
* Feat/ts 6100 3.0.pw10 (#31841 )
* enh: add operator reset func
* fix: merge join reset issue
* fix: memory issues
* fix: add debug assert
* fix: memory issues
* fix: memory leak
* fix: memory issues
* fix taos log miss
* fix: case issue
* fix: case issue
* fix: case issues
* fix: drop dnode issue
* fix: memory issues
* fix: memory issues
* fix: memory leak issues
* fix: recalculate time range issue
* fix: add debug log
* fix: memory issues
* fix: enable case asan
* Update streamlist_for_ci.task
* fix: case asan issue
* fix: stream name issue
* fix: external window compile issues
* fix: deploy memory issue
* fix: ahandle issue
* fix: ahandle issue
* fix: ahandle issue
* fix: virtual table reader list issue
* fix: log info
* fix: msg error
* fix: virtual table addr list issue
* fix: memory issues
* fix: memory leak issue
* fix: memory issues
* fix: memory free issues
* fix: memory issues
* fix: snode deploy issue
* fix: mnode reader issue
* fix: memory issues
* fix: add debug test
* enh: add ignore nodata trigger
* fix: memory leaks
* fix: configuration issue
* fix: memory issue
* fix: external window issue
* fix: external window issues
* fix: external window placeholder issue
* fix: placeholder function init issues
* fix: memory leak issue
* fix: add debug log
* fix: compile issues
* fix: double free issue
* fix: runner addr update issue
* fix: msg rsp issue
* fix: external window reset issue
* fix: configuration issue
* fix: deploy msg issue
* fix: compile issue
---------
Co-authored-by: huohong <sallyhuo@taosdata.com>
* test: reproduce bugs
* fix: add sliding interval combine case
* test: add cases
* test: add recalc test.
* test: reproduce bugs
* case : add vt ts is null check
* modify case
* bug: submit test_idmp_meters_bug3.py
* test: add test for recalc.
* test: add cases
* fix: error code check
* test: add cases
* fix(stream): scan wal with schema in that version
* add case
* test: add cases
* test: update test case.
* fix: windows compile issues
* add case
* test: add cases (#31845 )
* modify case
* fix: reset interpPrev
* test: add test_idmp_meters bug4 and bug3
* add case
* fix(stream): opti wal interface
* fix: remove test_idmp_meters_bug5.py
* test: add cases
* fix(stream): fix ts data fetch for virtual tables
* cancel asan case
* test: update test case.
* test: update test case.
* add case
* test: add cases
* test: add cases
* test: add case test_idmp_meters_bug5.py
* test: update test case.
* fix(stream): tmq error
* test: add cases
* feat: [TS-6100] Restore deleted code in mndSma.c since they are still in use.
* fix(stream): optimize val scan logic
* test: add test_recalc_expired_time.py to ci.
* test: update test case.
* test: update test case.
* feat: [TS-6100] Fix fill range check
* fix(stream): optimize val scan logic
* add case
* test: modify for partition by %%1
* test: add fun case stream4_sub7
* fix(stream): optimize val scan logic
* add case
* feat: [TS-6100] Rename OPTIONS to STREAM_OPTIONS.
* test: add test for recalc.
* test: use stream_options.
* fix: some cases error.
* test: remove recalc from ci.
* fix: ci case issues (#31880 )
* enh: add operator reset func
* fix: merge join reset issue
* fix: memory issues
* fix: add debug assert
* fix: memory issues
* fix: memory leak
* fix: memory issues
* fix taos log miss
* fix: case issue
* fix: case issue
* fix: case issues
* fix: drop dnode issue
* fix: memory issues
* fix: memory issues
* fix: memory leak issues
* fix: recalculate time range issue
* fix: add debug log
* fix: memory issues
* fix: enable case asan
* Update streamlist_for_ci.task
* fix: case asan issue
* fix: stream name issue
* fix: external window compile issues
* fix: deploy memory issue
* fix: ahandle issue
* fix: ahandle issue
* fix: ahandle issue
* fix: virtual table reader list issue
* fix: log info
* fix: msg error
* fix: virtual table addr list issue
* fix: memory issues
* fix: memory leak issue
* fix: memory issues
* fix: memory free issues
* fix: memory issues
* fix: snode deploy issue
* fix: mnode reader issue
* fix: memory issues
* fix: add debug test
* enh: add ignore nodata trigger
* fix: memory leaks
* fix: configuration issue
* fix: memory issue
* fix: external window issue
* fix: external window issues
* fix: external window placeholder issue
* fix: placeholder function init issues
* fix: memory leak issue
* fix: add debug log
* fix: compile issues
* fix: double free issue
* fix: runner addr update issue
* fix: msg rsp issue
* fix: external window reset issue
* fix: configuration issue
* fix: deploy msg issue
* fix: compile issue
* fix: external window idx issue
* fix: ci issues
---------
Co-authored-by: huohong <sallyhuo@taosdata.com>
* fix(stream): fix compilation error
* fix(stream): optimize val scan logic
* test:add test cases
* test: modify case
* fix: external agg error
* test(stream): tobacco scene testing #TD-36514
* test: add stream cases (#31885 )
* fix: windows compile issue
* fix: calc timerange
* fix: windows compile issue
* modify case
* fix(stream): compile error
* test: remove one debug test case file
* test: modify
* test: add test cases
* test: reproduce bugs
* test: reproduce bugs
* feat: [TS-6100] Placeholder function should only appera in SELECT and… (#31868 )
* feat: [TS-6100] Placeholder function should only appera in SELECT and WHERE and FROM.
* test: update case
---------
Co-authored-by: Simon Guan <guanshengliang@qq.com>
* add example
* add example
* modify case example
* modify case
* test:alter sql
* test: add stream5 case
* fix(stream): get schema error with version
* test: add delete recalc test py.
* test: remove bug cases
* test: stream5 case test passed
* test: add state cases (#31893 )
* fix(stream): compile error
* test: modify case
* test: add cases
* test: add test.
* test: update test case.
* chore(test): fix case err
* test: update test case.
* fix: align data get
* fix(stream): fix row index of datablock written into data cache
* fix: put align data
* test: update test case.
* test: add test cases for virtual table
* chore(test): fix case err #TD-36514
* add case
* test: add test for water mark.
* test: add meters bug6 for stream5
* test: add cases (#31903 )
* test: add test for recalc.
* feat: [TS-6100] %%trows can only be used when event type is window close.
* test: add precision of database for ms/us/ns
* modify case
* add case
* add case
* test: add test to ci.
* modify case
* fix: ci case issues (#31904 )
* enh: add operator reset func
* fix: merge join reset issue
* fix: memory issues
* fix: add debug assert
* fix: memory issues
* fix: memory leak
* fix: memory issues
* fix taos log miss
* fix: case issue
* fix: case issue
* fix: case issues
* fix: drop dnode issue
* fix: memory issues
* fix: memory issues
* fix: memory leak issues
* fix: recalculate time range issue
* fix: add debug log
* fix: memory issues
* fix: enable case asan
* Update streamlist_for_ci.task
* fix: case asan issue
* fix: stream name issue
* fix: external window compile issues
* fix: deploy memory issue
* fix: ahandle issue
* fix: ahandle issue
* fix: ahandle issue
* fix: virtual table reader list issue
* fix: log info
* fix: msg error
* fix: virtual table addr list issue
* fix: memory issues
* fix: memory leak issue
* fix: memory issues
* fix: memory free issues
* fix: memory issues
* fix: snode deploy issue
* fix: mnode reader issue
* fix: memory issues
* fix: add debug test
* enh: add ignore nodata trigger
* fix: memory leaks
* fix: configuration issue
* fix: memory issue
* fix: external window issue
* fix: external window issues
* fix: external window placeholder issue
* fix: placeholder function init issues
* fix: memory leak issue
* fix: add debug log
* fix: compile issues
* fix: double free issue
* fix: runner addr update issue
* fix: msg rsp issue
* fix: external window reset issue
* fix: configuration issue
* fix: deploy msg issue
* fix: compile issue
* fix: external window idx issue
* fix: ci issues
* fix: ci case issues
* fix: drop dnode issue
---------
Co-authored-by: huohong <sallyhuo@taosdata.com>
* fix(stream): ci error
* test: update test case.
* feat: [TS-6100] Disable some failed UT.
* feat: [TS-6100] Fix virtual table
* test: add bug 5.
* test: add test delete recalc to ci.
* test: add bug 6.
* test(stream): tobacco scene #TD-36514
* fix: reqCids,reqCols memory leak in SSTriggerRealtimeContext
Co-authored-by: Tony Zhang <tonyzhang@taosdata.com>
* test: add case stream6
* fix(stream): implement some pending features in trigger task
* modify case
* modify case
* fix: case issues
* modify case
* test: add recalc for warter mark.
* fix(stream): fix count window trigger of virtual tables
* fix(stream): memory leak
* test: fix run err.
* test: add stream6 bug7
* fix: adjust format
* test(stream): tobacco scene testing #TD-36514
* test: change bug7 with update window1 and 2
* test: add test bug 7.
* case: restore write 3 window
* fix: windows compile issue
* fix: notify
* test: add cases
* modify case
* test: update test case.
* test(stream): toobacco scene testing #TD-36514
---------
Co-authored-by: Simon Guan <slguan@taosdata.com>
Co-authored-by: plum-lihui <huili@taosdata.com>
Co-authored-by: Alex Duan <417921451@qq.com>
Co-authored-by: zelv01 <1101510017@qq.com>
Co-authored-by: Jing Sima <simondominic9997@outlook.com>
Co-authored-by: xiangyang guo <66111494+happyguoxy@users.noreply.github.com>
Co-authored-by: wangmm0220 <wangmm0220@gmail.com>
Co-authored-by: Haojun Liao <hjliao@taosdata.com>
Co-authored-by: zyyang90 <zyyang@taosdata.com>
Co-authored-by: Alex Duan <51781608+DuanKuanJun@users.noreply.github.com>
Co-authored-by: facetosea <285808407@qq.com>
Co-authored-by: Simon Guan <guanshengliang@qq.com>
Co-authored-by: huohong <sallyhuo@taosdata.com>
Co-authored-by: Li Hui <52318143+plum-lihui@users.noreply.github.com>
Co-authored-by: Jinqing Kuang <kuangjinqingcn@gmail.com>
Co-authored-by: xiao-77 <berylbao@taosdata.com>
Co-authored-by: Zhixiao Bao <62235797+xiao-77@users.noreply.github.com>
Co-authored-by: happyguoxy <happy_guoxy@163.com>
Co-authored-by: Tony Zhang <34825804+Tony2h@users.noreply.github.com>
Co-authored-by: Tony Zhang <tonyzhang@taosdata.com>
2025-07-16 14:42:16 +08:00
Haojun Liao
dfe32225de
chore: merge main.
2025-03-17 14:51:11 +08:00
Haojun Liao
ab92886820
fix(stream): reduce the consensus checkpoint id trans. ( #30105 )
...
* fix(stream): reduce the consensus checkpoint id trans.
* refactor(stream): add some logs.
* refactor(stream): set the max checkpoint exec time 30min.
* refactor(stream): add checkpoint-consensus trans conflict check.
* refactor(stream): remove unused local variables.
* fix(stream): fix syntax error.
* fix(stream): 1. fix free memory error 2. continue if put result into dst hashmap failed.
* fix issue
* fix issue
* fix(mnd): follower mnode not processes the timer event.
* fix(stream): print correct error msg.
* fix(stream): add some logs.
* fix(stream): add some logs.
* fix(stream): add some logs.
* fix(stream): add some logs.
* fix(stream): add some logs.
* fix(stream): add some logs.
* fix(stream): add some logs.
* fix(stream): add some logs.
* fix(stream): truncate long subtable name
* fix(stream): add buffer len.
* refactor(stream): update some logs.
* fix issue
* refactor(stream): update some logs.
* refactor(stream): update some logs.
* fix(stream): check return value.
* fix(stream): fix syntax error.
* fix(stream): check return value.
* fix(stream): update the timer check in mnode.
* fix(stream): add restart stage tracking.
* fix(stream): track the start task stage for meta.
* fix(stream): fix error in log.
* refactor(stream): adjust log info.
* fix mem issue
* fix(stream): check the number of required tasks for consensus checkpointId.
* fix(stream): lock the whole start procedure.
* fix(stream): add lock during start all tasks.
* fix(stream): update logs.
* fix(stream): update logs.
* fix(stream): update logs.
* fix(stream): fix dead-lock.
* fix(stream): fix syntax error.
* fix(stream): not drop the scan-history task.
* fix(stream): fix syntax error.
* fix(stream): wait for executor stop before restarting.
* fix(stream): add some logs.
* fix(stream): add some logs.
* fix(stream): add some logs.
* fix(stream): disable some logs.
* fix(stream): reset the start info if no task left.
---------
Co-authored-by: 54liuyao <54liuyao@163.com>
Co-authored-by: Jinqing Kuang <kuangjinqingcn@gmail.com>
2025-03-17 10:20:17 +08:00
liuyao
2de6736160
feat(stream): add new trigger continuous_window_close ( #30125 )
...
* opt stream build twa result
* opt force window close memory
* feat(stream):optimize new interval and scan operator
* adj log
* opt code
* opt code
* fill history
* fix issue for fill history
* add ci
* feat(stream): add new stream nonblock interval operator
* adjust code
* use new scan operator
* use new scan operator
* add log
* fix issue
* recover stream scan next function
* fix issue
* fix issue
* fix issue
* ignore disorder data
* fix issue
* fix issue for interval sliding
* fix issue
* fix ci issue
* fix ci issue
* fix ci issue
* add semi && final nonblock interval operator
* fix issue
* fix issue
* fix issue
* fix issue
* fix issue
* fix issue
* fix issue
* fix issue
* refactor(stream): track the msgId for each upstream tasks.
* fix(stream): fix race condition.
* fix(stream): update the task last msgId when putting into input queue succ.
* fix issue
* fix issue
* put recalculate data to rocksdb
* fix issue
* fix issue
* enh(query)[TD-33071]: add support for saving and restoring tsdbReader scan progress
- Implement functionality to save scan progress during tsdbReader operations
- Enable resuming scans from the last saved position
* fix issue
* fix issue
* fix issue
* fix issue
* fix issue
* add rec interval check
* enh(stream):add recalculate tasks.
* enh(stream): support the re-calculate the tasks.
* fix issue && do refactor
* do refactor
* fix issue
* fix issue
* update backend opt
* add new interface
* add new session operator
* support blob
* add new session operator
* fix issue
* add rec state for rec task
* fix invalid read
* add new session window
* enh(stream): update the stream tasks backend.
* new session operator
* add pull data
* fix(stream): fix error in expand stream backend.
* fix issue
* fix issue
* fix issue
* merge code
* fix issue
* fix(stream): check for null ptr.
* fix(stream): add more check.
* fix issue
* fix issue
* fix issue
* add debug code
* fix issue
* fix issue
* fix issue
* set rec end flag
* fix(stream): opt re-calculate stream tasks.
* fix issue
* fix issue
* add new operator
* enh(stream): dispatch recalculate block to agg tasks.
* fix issue
* fix issue
* fix(stream): adjust filter.
* fix issue
* refactor
* refactor(stream): adjust the recalculate end block.
* fix issue
* fix(stream): set correct create trigger block.
* fix issue
* fix(stream): fix error in build recalculate end block.
* fix(stream): check null ptr.
* add stream client && fix issue
* fix mem leak
* fix(stream): free msg.
* add stream client
* fix(stream): fix error.
* add stream client && fix issue
* add stream client
* refactor(stream): set the recalculate task complete.
* add wend and group_id for session window dest table
* feat(stream): refactor and set the recalcul agg task complete.
* add cfg for adapter
* fix issue
* add state && event operator
* feat(stream): support fill-history task.
* add new fill operator
* fix(stream): set correct backend when updating fill-history task to recalculate task.
* add new fill operator
* fix(stream): s2025-03-06 11:10:31.272
et ops always open in scan tsdb
* fix(stream):set the correct taskType for sink task.
* new fill operator
* adj stream fill operator
* fix issue
* fix issue
* fix issue
* fix issue
* fix issue
* fix issue
* fix issue
* fix issue
* adj test
* fix issue
* fix(stream): fix issue
* fix(steam): fix issue
* fix(steam): fix issue
* fix(steam): fix issue
* fix(steam): fix issue
* fix(stream): fix issue
* fix(stream): fix issue
* fix(stream): fix issue
* fix: ut com error
* fix(stream): fix mem leak and adjust operator type check rule
* fix(stream): fix mem leak and adjust test case
* refactor code
* fix(stream): free items.
* fix(stream): free fix memory leak.
* fix(stream): fix syntax error.
* fix: ignore unexpect block
* fix: adjust op type
---------
Co-authored-by: Haojun Liao <hjliao@taosdata.com>
Co-authored-by: Jinqing Kuang <kuangjinqingcn@gmail.com>
Co-authored-by: yihaoDeng <luomoxyz@126.com>
2025-03-14 20:14:01 +08:00
Jinqing Kuang
fc2d2672b9
feat(stream)[TS-5469]: add more stream event notifications
...
- Add event notifications for all window types
- Implement window invalidation notifications
- Add statistics for event notification delivery
2025-02-16 11:07:16 +08:00
Jinqing Kuang
eb5d463490
feat(stream)[TS-5469]. add support for window event notifications in stream processing
...
- Introduce new syntax to specify notification type and destination address
- Collect relevant event information during window computations
- Implement websocket-based notification delivery to the specified address
2025-01-21 17:39:46 +08:00
54liuyao
f3b38edb98
add stream event notify
2025-01-21 11:18:49 +08:00
54liuyao
22023575ba
opt stream force fill op
2024-11-26 19:46:36 +08:00
54liuyao
f37ae57a87
opt stream row buff
2024-11-26 15:23:57 +08:00
54liuyao
48d5f0e7cd
fix issue
2024-11-05 18:14:35 +08:00
54liuyao
5158bf6b96
fix issue
2024-10-31 15:50:56 +08:00
54liuyao
3e49f40c74
stream twa
2024-10-16 12:16:59 +08:00
54liuyao
669c3aa4db
fix windows ci issue
2024-10-09 19:48:24 +08:00
54liuyao
69b8505a8d
fix ci issue
2024-10-09 17:49:24 +08:00
54liuyao
8709a12ce2
fix ci issue
2024-10-09 17:00:09 +08:00
54liuyao
e6f11c180c
fix ci issue
2024-10-09 16:03:09 +08:00
54liuyao
d13819bbf4
Merge branch '3.0' of https://github.com/taosdata/TDengine into fix/TD-30837
2024-09-30 09:05:46 +08:00
54liuyao
7517228dde
fix(stream):fix issue for stream count window state buff
2024-09-29 14:16:16 +08:00
54liuyao
b8cd739383
add ci and fix issue
2024-09-18 11:08:47 +08:00
54liuyao
4fe68034cf
fix issue
2024-08-16 17:46:14 +08:00
54liuyao
d59eb19c37
add ci
2024-08-16 10:54:52 +08:00
54liuyao
057f09ffbf
fix issue && add ci
2024-08-14 19:20:21 +08:00
54liuyao
5a2b89c357
fix issue
2024-08-13 14:05:16 +08:00
54liuyao
3a14454fb5
add ci
2024-08-08 19:30:18 +08:00
54liuyao
b26750c200
add ci
2024-08-02 13:22:14 +08:00
54liuyao
fe9d61b243
add node to json
2024-07-29 14:14:34 +08:00
54liuyao
b5f432bd50
Merge branch '3.0' of https://github.com/taosdata/TDengine into fix/TD-30837
2024-07-26 16:44:18 +08:00
54liuyao
3cdb957f22
adj function of stream operator result
2024-07-15 15:32:11 +08:00
54liuyao
1c9011e820
stream time slice
2024-07-12 11:03:45 +08:00
54liuyao
7b50663240
optimize stream operator checkpoint
2024-05-16 16:24:58 +08:00
