liuyao
2de6736160
feat(stream): add new trigger continuous_window_close ( #30125 )
...
* opt stream build twa result
* opt force window close memory
* feat(stream):optimize new interval and scan operator
* adj log
* opt code
* opt code
* fill history
* fix issue for fill history
* add ci
* feat(stream): add new stream nonblock interval operator
* adjust code
* use new scan operator
* use new scan operator
* add log
* fix issue
* recover stream scan next function
* fix issue
* fix issue
* fix issue
* ignore disorder data
* fix issue
* fix issue for interval sliding
* fix issue
* fix ci issue
* fix ci issue
* fix ci issue
* add semi && final nonblock interval operator
* fix issue
* fix issue
* fix issue
* fix issue
* fix issue
* fix issue
* fix issue
* fix issue
* refactor(stream): track the msgId for each upstream tasks.
* fix(stream): fix race condition.
* fix(stream): update the task last msgId when putting into input queue succ.
* fix issue
* fix issue
* put recalculate data to rocksdb
* fix issue
* fix issue
* enh(query)[TD-33071]: add support for saving and restoring tsdbReader scan progress
- Implement functionality to save scan progress during tsdbReader operations
- Enable resuming scans from the last saved position
* fix issue
* fix issue
* fix issue
* fix issue
* fix issue
* add rec interval check
* enh(stream):add recalculate tasks.
* enh(stream): support the re-calculate the tasks.
* fix issue && do refactor
* do refactor
* fix issue
* fix issue
* update backend opt
* add new interface
* add new session operator
* support blob
* add new session operator
* fix issue
* add rec state for rec task
* fix invalid read
* add new session window
* enh(stream): update the stream tasks backend.
* new session operator
* add pull data
* fix(stream): fix error in expand stream backend.
* fix issue
* fix issue
* fix issue
* merge code
* fix issue
* fix(stream): check for null ptr.
* fix(stream): add more check.
* fix issue
* fix issue
* fix issue
* add debug code
* fix issue
* fix issue
* fix issue
* set rec end flag
* fix(stream): opt re-calculate stream tasks.
* fix issue
* fix issue
* add new operator
* enh(stream): dispatch recalculate block to agg tasks.
* fix issue
* fix issue
* fix(stream): adjust filter.
* fix issue
* refactor
* refactor(stream): adjust the recalculate end block.
* fix issue
* fix(stream): set correct create trigger block.
* fix issue
* fix(stream): fix error in build recalculate end block.
* fix(stream): check null ptr.
* add stream client && fix issue
* fix mem leak
* fix(stream): free msg.
* add stream client
* fix(stream): fix error.
* add stream client && fix issue
* add stream client
* refactor(stream): set the recalculate task complete.
* add wend and group_id for session window dest table
* feat(stream): refactor and set the recalcul agg task complete.
* add cfg for adapter
* fix issue
* add state && event operator
* feat(stream): support fill-history task.
* add new fill operator
* fix(stream): set correct backend when updating fill-history task to recalculate task.
* add new fill operator
* fix(stream): s2025-03-06 11:10:31.272
et ops always open in scan tsdb
* fix(stream):set the correct taskType for sink task.
* new fill operator
* adj stream fill operator
* fix issue
* fix issue
* fix issue
* fix issue
* fix issue
* fix issue
* fix issue
* fix issue
* adj test
* fix issue
* fix(stream): fix issue
* fix(steam): fix issue
* fix(steam): fix issue
* fix(steam): fix issue
* fix(steam): fix issue
* fix(stream): fix issue
* fix(stream): fix issue
* fix(stream): fix issue
* fix: ut com error
* fix(stream): fix mem leak and adjust operator type check rule
* fix(stream): fix mem leak and adjust test case
* refactor code
* fix(stream): free items.
* fix(stream): free fix memory leak.
* fix(stream): fix syntax error.
* fix: ignore unexpect block
* fix: adjust op type
---------
Co-authored-by: Haojun Liao <hjliao@taosdata.com>
Co-authored-by: Jinqing Kuang <kuangjinqingcn@gmail.com>
Co-authored-by: yihaoDeng <luomoxyz@126.com>
2025-03-14 20:14:01 +08:00
Simon Guan
6826edf5e0
Merge branch 'main' into merge/mainto3.0
2025-03-03 14:01:43 +08:00
Haojun Liao
bf1f162aa0
fix(stream): drop task in synch model in write thread.
2025-02-27 00:15:36 +08:00
Haojun Liao
c14ba08d7c
refactor(stream): drop task in async ways
2025-02-27 00:12:11 +08:00
Shengliang Guan
3a739beaa5
Merge branch 'main' into merge/mainto3.0
2025-02-22 20:07:40 +08:00
Haojun Liao
ecc667c22b
Revert "refactor(stream): drop task in async ways"
2025-02-21 01:10:18 +08:00
Shengliang Guan
2bee60a41a
Merge branch 'main' into merge/mainto3.0
2025-02-20 18:12:02 +08:00
Haojun Liao
c4cd6dd62d
fix(stream): drop task in synch model in write thread.
2025-02-17 10:06:09 +08:00
Jinqing Kuang
fc2d2672b9
feat(stream)[TS-5469]: add more stream event notifications
...
- Add event notifications for all window types
- Implement window invalidation notifications
- Add statistics for event notification delivery
2025-02-16 11:07:16 +08:00
Haojun Liao
fd96b096ea
refactor(stream): drop task in async ways
2025-02-13 23:41:06 +08:00
Jinqing Kuang
eb5d463490
feat(stream)[TS-5469]. add support for window event notifications in stream processing
...
- Introduce new syntax to specify notification type and destination address
- Collect relevant event information during window computations
- Implement websocket-based notification delivery to the specified address
2025-01-21 17:39:46 +08:00
dapan1121
d77147d296
Merge remote-tracking branch 'origin/3.0' into feat/TD-30268
2024-12-06 17:43:19 +08:00
Jinqing Kuang
c2256e9443
feat(query)[TS-5058]: support AUTO OFFSET in INTERVAL clause
...
Add the AUTO keyword, which allows automatic determination of the
INTERVAL OFFSET based on the WHERE condition. It simplifies usage by
allowing users to rely on the system to infer the correct offset
without manual specification.
2024-12-04 10:06:47 +08:00
dapan1121
85fc4dfc2b
fix: data sink memory issues
2024-11-15 10:16:15 +08:00
dapan1121
a8561fba4f
Merge remote-tracking branch 'origin/3.0' into feat/TD-30268
2024-11-11 11:01:22 +08:00
dapan1121
371094cc13
Merge remote-tracking branch 'origin/3.0' into feat/TD-30268
2024-11-08 10:13:20 +08:00
Jinqing Kuang
43a337ef74
enh(query)[TD-32732]: add client Id generation and usage
...
To address the issue of non-unique identifiers for query tasks, a unique
client ID (cid) is now generated during client initialization. This cid
is included in every task message sent to the server. The server will
use a composite key of <qid, cid, tid, eid> to uniquely identify each
query task.
2024-11-07 08:47:31 +08:00
54liuyao
b6d16eccdb
Merge branch '3.0' of https://github.com/taosdata/TDengine into fix/TD-30837
2024-10-30 14:54:42 +08:00
Shengliang Guan
ea9ed2e538
Merge pull request #28300 from taosdata/enh/TS-5441-3.0
...
enh:[TS-5441] cost too long in tmq write meta data by cache meta and vg info
2024-10-30 13:43:41 +08:00
54liuyao
d065d71821
Merge branch '3.0' of https://github.com/taosdata/TDengine into fix/TD-30837
2024-10-30 10:09:48 +08:00
54liuyao
6b2d338a20
enh(executor):avoid use dangerous functions
2024-10-25 13:47:09 +08:00
dapan1121
4f3484b823
Merge remote-tracking branch 'origin/3.0' into feat/TD-30268
2024-10-24 16:49:38 +08:00
54liuyao
e9550a6b2c
feat(stream):save force window close scan range
2024-10-18 15:40:37 +08:00
wangmm0220
bc05289192
enh:[TS-5441] cost too long in tmq write meta data by cache meta and vg info
2024-10-18 09:28:16 +08:00
wangmm0220
42b7520d34
enh:[TS-5441] cost too long in tmq write meta data by cache meta and vg info
2024-10-10 18:27:42 +08:00
wangmm0220
e6bf8dcfde
enh:[TS-5441] cost too long in tmq write meta data by cache meta and vg info
2024-10-09 18:10:43 +08:00
Haojun Liao
c5eca0486e
Merge remote-tracking branch 'origin/fix/TD-30837' into fix/TD-30837
...
# Conflicts:
# source/dnode/vnode/src/tq/tq.c
# source/libs/stream/inc/streamInt.h
# source/libs/stream/src/streamSched.c
2024-09-19 16:39:54 +08:00
Haojun Liao
e1d8e98b11
feat(stream): support the force_window_close trigger.
2024-09-19 16:23:51 +08:00
Haojun Liao
1a90e9612d
refactor: check return value.
2024-09-13 23:04:41 +08:00
Haojun Liao
b3e5997336
fix(stream): reset error code.
2024-09-08 14:54:08 +08:00
dapan1121
1831deb2f0
Merge remote-tracking branch 'origin/3.0' into feat/TD-30268
2024-08-02 16:12:39 +08:00
dapan1121
dfc3fc4987
Merge remote-tracking branch 'origin/3.0' into feat/TD-30268
2024-07-29 11:16:36 +08:00
Haojun Liao
79ad7585ef
fix(stream): check return value.
2024-07-29 10:29:40 +08:00
wangmm0220
3f2278ebcf
fix:[TD-31017]process return value in vnode for tmq
2024-07-24 17:56:29 +08:00
dapan1121
adfd7b4504
Merge remote-tracking branch 'origin/3.0' into feat/TD-30268
2024-07-22 19:24:56 +08:00
dapan1121
4c2c9a71d5
fix: table info issue
2024-07-15 10:05:57 +08:00
dapan1121
55ba9054ea
enh: add stat info
2024-07-02 19:25:33 +08:00
wangjiaming0909
11a98ffab2
QueryAutoQWorker
2024-07-01 11:08:39 +08:00
Haojun Liao
895ee30cf3
Merge branch '3.0' into enh/opt_prj
2024-06-09 23:51:38 +08:00
Haojun Liao
6d7e1eb576
fix(query): check for the server ep and client ep to set the compress flag.
2024-06-04 15:07:02 +08:00
54liuyao
441a150d66
optimize tmq snapshot meta
2024-05-23 17:35:54 +08:00
Haojun Liao
e62dd2ff5c
refactor: do some internal refactor.
2024-02-19 10:31:25 +08:00
wangmm0220
a656d75ca7
opti:consume data excluded from some source
2024-02-05 17:07:50 +08:00
kailixu
b9f389a49f
Merge branch '3.0' into enh/TD-23769-3.0x
2023-11-08 08:36:35 +08:00
kailixu
6b738884a4
chore: rsma checkpoint
2023-11-07 19:59:05 +08:00
Haojun Liao
d440da0d8c
enh(stream): execute scan history by using time slice 1s.
2023-11-05 01:26:53 +08:00
Haojun Liao
53b2158c54
refactor: do some internal refactor.
2023-11-03 13:51:17 +08:00
yihaoDeng
6fe649c458
Merge branch 'enh/3.0' into enh/triggerCheckPoint2
2023-08-21 09:12:40 +08:00
dapan1121
6ce28e8886
Merge remote-tracking branch 'origin/3.0' into feat/TD-24700
2023-08-14 10:49:56 +08:00
dapan1121
c9656d3f77
fix: fix memory issues
2023-08-11 17:39:41 +08:00
