Commit graph

273 commits

Author SHA1 Message Date
wangmm0220
e6dfbf6295 fix(stream): add interface of get vtable info 2025-05-28 18:34:19 +08:00
wangmm0220
3aa8c175f5 feat(stream): reset agg operator & reset tsdb reader in fetch request 2025-05-26 14:52:02 +08:00
wangmm0220
a716fabbbc feat(stream): set tag value in trigger cols 2025-05-22 15:45:57 +08:00
wangmm0220
2d717c3a4f feat(stream): set tag value in trigger cols 2025-05-22 15:38:32 +08:00
wangjiaming0909
0ffe9e55f3 stream runner force output impl 2025-05-20 03:13:22 +00:00
factosea
2860309b6c fix: crate subtable tags 2025-05-17 14:13:08 +08:00
wangjiaming0909
0c402fbb38 new-stream: runner impl external window and force output 2025-05-16 08:29:34 +00:00
wangjiaming0909
5300a66f5e new-stream: runner impl 2025-05-16 01:41:31 +00:00
wangmm0220
0297c7be2c feat(stream): add ts slotid/blockId in external window 2025-05-15 14:22:46 +08:00
wangmm0220
9ffe95755b feat(stream): add interface of getting cols value of groupid 2025-05-15 10:03:57 +08:00
wangjiaming0909
ef7d67efe0 new-stream: runner notification 2025-05-14 09:55:07 +00:00
facetosea
0a7ac81aeb fix: table schema version start from 1 2025-05-14 17:41:51 +08:00
wangmm0220
5e7aed840e feat(stream): add filter if with external window 2025-05-13 10:53:39 +08:00
dapan1121
009e8aa41e Merge remote-tracking branch 'origin/3.0' into feat/TS-6100-3.0 2025-05-12 16:24:22 +08:00
wangmm0220
df42d12b06 feat(stream): return calc cols and trigger cols from stream reader 2025-05-10 21:22:11 +08:00
wangmm0220
f1155c54ad feat(stream): return calc cols and trigger cols from stream reader 2025-05-10 18:34:01 +08:00
Jing Sima
84016904ff
fix: [TS-6445] Fix virtual tables not detecting schema changes. (#30999)
Ensure virtual tables invalidate cached schema metadata and reload it when a schema change (e.g., ALTER VTABLE) is detected. This prevents stale schema references and query errors after structural updates.
2025-05-10 10:34:54 +08:00
facetosea
a11fc0d7cd fix: data insert, build STSchema 2025-05-09 17:05:56 +08:00
factosea
1fafaae137 fix: save table uid 2025-05-09 10:03:35 +08:00
factosea
e4dd28df92 feat inserter param check 2025-05-08 23:00:37 +08:00
wangmm0220
332d21630c feat(stream): reset task if fetchMsg.reset is true for calc 2025-05-08 19:53:40 +08:00
wangmm0220
9dfcb4d63b feat(stream): add logic of fetch rsp for calc 2025-05-08 17:39:12 +08:00
wangmm0220
bc1c0f272d feat(stream): add logic of fetch rsp for calc 2025-05-08 17:27:09 +08:00
factosea
cd9a532064 feat: data inserter 2025-05-08 15:07:04 +08:00
wangjiaming0909
9fd5f6364c new-stream: runner impl 2025-05-08 14:58:41 +08:00
factosea
f3027e938e feat: stream data insert 2025-05-08 09:48:53 +08:00
wangjiaming0909
1ccc6c69a8 new-stream: runner impl 2025-04-29 15:31:17 +08:00
wangmm0220
c632dcf0c1 feat(stream): add logic of stream reader 2025-04-28 19:21:47 +08:00
wangmm0220
a78b09577a feat(stream): add thread worker for stream reader 2025-04-28 14:37:05 +08:00
wangmm0220
5ce115cc23 feat(stream): fix compile error 2025-04-24 18:02:12 +08:00
dapan1121
330cf9982d fix: compile issues 2025-04-24 09:11:54 +08:00
WANG MINGMING
c85b195421
Merge branch 'feat/TS-6100-marks' into feat/TS-6100-3.0 2025-04-19 16:18:09 +07:00
wangmm0220
80f368ba51 feat(stream): add stream reader logic 2025-04-19 17:10:43 +08:00
wangmm0220
919628568d feat(stream): add stream reader logic 2025-04-17 19:50:37 +08:00
wangjiaming0909
163dd33a0a new-stream: impl stream runner 2025-04-16 18:16:55 +08:00
Jinqing Kuang
c82398e175 fix(stream): fix stream processing for virtual tables
- Fixed status check in virtual table stream processing
- Fixed multi-way merge logic for super virtual tables
- Fixed generation of target subtable names during merge
2025-03-20 01:15:54 +08:00
Jing Sima
ccaa0b91ae
feat: [TS-4897] Add more test and fix bugs. (#30208)
* feat: [TS-4897] Fix state window wrong res when all state col is NULL.

* fix: [TD-34074] Forbid decimal type in virtual table.
2025-03-19 17:15:17 +08:00
Jing Sima
410324746b
feat:[TS-4897] virtual table (#30098)
* feat: [TS-4897] Support create/drop/alter/show/describe vtable

* feat: [TS-4897] Support vtable's query

* feat: [TS-4897] Support create virtual supertable

* feat: [TS-4897] Support explain analyze / where / count(*) and only select ts of vtable.

* feat: [TS-4897] Add create test and fix bugs

* feat: [TS-4897] Add alter/drop test and fix bugs

* feat: [TS-4897] Add describe/show test and fix bugs

* feat: [TS-4897] Add auth test and fix bugs

* feat: [TS-4897] Fix meta/catalog/cache bugs

* feat: [TS-4897] Support select tag from virtual child table

* feat: [TS-4897] Add select test and fix plenty of bugs

* feat: [TS-4897] Add optimize rule for vtable scan / support create vtable cross database / remove enterprise constraint / fix bugs.

* feat: [TS-4897] Fix 'schema is old'

* feat: [TS-4897] Support virtual stable query

* feat: [TS-4897] Add tests and Fix bugs

* feat: [TS-4897] resolve conflict.
2025-03-15 14:10:46 +08:00
liuyao
2de6736160
feat(stream): add new trigger continuous_window_close (#30125)
* opt stream build twa result

* opt force window close memory

* feat(stream):optimize new interval and scan operator

* adj log

* opt code

* opt code

* fill history

* fix issue for fill history

* add ci

* feat(stream): add new stream nonblock interval operator

* adjust code

* use new scan operator

* use new scan operator

* add log

* fix issue

* recover stream scan next function

* fix issue

* fix issue

* fix issue

* ignore disorder data

* fix issue

* fix issue for interval sliding

* fix issue

* fix ci issue

* fix ci issue

* fix ci issue

* add semi && final nonblock interval operator

* fix issue

* fix issue

* fix issue

* fix issue

* fix issue

* fix issue

* fix issue

* fix issue

* refactor(stream): track the msgId for each upstream tasks.

* fix(stream): fix race condition.

* fix(stream): update the task last msgId when putting into input queue succ.

* fix issue

* fix issue

* put recalculate data to rocksdb

* fix issue

* fix issue

* enh(query)[TD-33071]: add support for saving and restoring tsdbReader scan progress

- Implement functionality to save scan progress during tsdbReader operations
- Enable resuming scans from the last saved position

* fix issue

* fix issue

* fix issue

* fix issue

* fix issue

* add rec interval check

* enh(stream):add recalculate tasks.

* enh(stream): support the re-calculate the tasks.

* fix issue && do refactor

* do refactor

* fix issue

* fix issue

* update backend opt

* add new interface

* add new session operator

* support blob

* add new session operator

* fix issue

* add rec state for rec task

* fix invalid read

* add new session window

* enh(stream): update the stream tasks backend.

* new session operator

* add pull data

* fix(stream): fix error in expand stream backend.

* fix issue

* fix issue

* fix issue

* merge code

* fix issue

* fix(stream): check for null ptr.

* fix(stream): add more check.

* fix issue

* fix issue

* fix issue

* add debug code

* fix issue

* fix issue

* fix issue

* set rec end flag

* fix(stream): opt re-calculate stream tasks.

* fix issue

* fix issue

* add new operator

* enh(stream): dispatch recalculate block to agg tasks.

* fix issue

* fix issue

* fix(stream): adjust filter.

* fix issue

* refactor

* refactor(stream): adjust the recalculate end block.

* fix issue

* fix(stream): set correct create trigger block.

* fix issue

* fix(stream): fix error in build recalculate end block.

* fix(stream): check null ptr.

* add stream client && fix issue

* fix mem leak

* fix(stream): free msg.

* add stream client

* fix(stream): fix error.

* add stream client && fix issue

* add stream client

* refactor(stream): set the recalculate task complete.

* add wend and group_id for session window dest table

* feat(stream): refactor and set the recalcul agg task complete.

* add cfg for adapter

* fix issue

* add state && event operator

* feat(stream): support fill-history task.

* add new fill operator

* fix(stream): set correct backend when updating fill-history task to recalculate task.

* add new fill operator

* fix(stream): s2025-03-06 11:10:31.272
et ops always open in scan tsdb

* fix(stream):set the correct taskType for sink task.

* new fill operator

* adj stream fill operator

* fix issue

* fix issue

* fix issue

* fix issue

* fix issue

* fix issue

* fix issue

* fix issue

* adj test

* fix issue

* fix(stream): fix issue

* fix(steam): fix issue

* fix(steam): fix issue

* fix(steam): fix issue

* fix(steam): fix issue

* fix(stream): fix issue

* fix(stream): fix issue

* fix(stream): fix issue

* fix: ut com error

* fix(stream): fix mem leak and adjust operator type check rule

* fix(stream): fix mem leak and adjust test case

* refactor code

* fix(stream): free items.

* fix(stream): free fix memory leak.

* fix(stream): fix syntax error.

* fix: ignore unexpect block

* fix: adjust op type

---------

Co-authored-by: Haojun Liao <hjliao@taosdata.com>
Co-authored-by: Jinqing Kuang <kuangjinqingcn@gmail.com>
Co-authored-by: yihaoDeng <luomoxyz@126.com>
2025-03-14 20:14:01 +08:00
Simon Guan
6826edf5e0 Merge branch 'main' into merge/mainto3.0 2025-03-03 14:01:43 +08:00
Haojun Liao
bf1f162aa0 fix(stream): drop task in synch model in write thread. 2025-02-27 00:15:36 +08:00
Haojun Liao
c14ba08d7c refactor(stream): drop task in async ways 2025-02-27 00:12:11 +08:00
Shengliang Guan
3a739beaa5 Merge branch 'main' into merge/mainto3.0 2025-02-22 20:07:40 +08:00
Haojun Liao
ecc667c22b
Revert "refactor(stream): drop task in async ways" 2025-02-21 01:10:18 +08:00
Shengliang Guan
2bee60a41a Merge branch 'main' into merge/mainto3.0 2025-02-20 18:12:02 +08:00
Haojun Liao
c4cd6dd62d fix(stream): drop task in synch model in write thread. 2025-02-17 10:06:09 +08:00
Jinqing Kuang
fc2d2672b9 feat(stream)[TS-5469]: add more stream event notifications
- Add event notifications for all window types
- Implement window invalidation notifications
- Add statistics for event notification delivery
2025-02-16 11:07:16 +08:00
Haojun Liao
fd96b096ea refactor(stream): drop task in async ways 2025-02-13 23:41:06 +08:00
Jinqing Kuang
eb5d463490 feat(stream)[TS-5469]. add support for window event notifications in stream processing
- Introduce new syntax to specify notification type and destination address
- Collect relevant event information during window computations
- Implement websocket-based notification delivery to the specified address
2025-01-21 17:39:46 +08:00
dapan1121
d77147d296 Merge remote-tracking branch 'origin/3.0' into feat/TD-30268 2024-12-06 17:43:19 +08:00