Commit graph

27 commits

Author SHA1 Message Date
Jinqing Kuang
ac23a4fc55
feat(stream): support stream processing for virtual tables (#30144)
* enh: add client processing

* enh: add mnode vtables processing

* enh: add mnode vtable processing

* enh: add normal child vtable support

* fix: compile issues

* fix: compile issues

* fix: create stream issues

* fix: multi stream scan issue

* fix: remove debug info

* fix: agg task and task level issues

* fix: correct task output type

* fix: split vtablescan from agg

* fix: memory leak issues

* fix: add limitations

* Update 09-error-code.md

* Update 09-error-code.md

* fix: remove usless case

* feat(stream): extract original table data in source scan task

Implemented functionality in the source task to extract data
corresponding to the virtual table from the original table using WAL.
The extracted data is then sent to the downstream merge task for further
processing.

* feat(stream): multi-way merge using loser tree in virtual merge task

Implemented multi-way merge in the merge task using a loser tree to
combine data from multiple original table into a single virtual table.
The merged virtual table data is then pushed downstream for further
processing.  Introduced memory limit handling during the merge process
with configurable behavior when the memory limit is reached.

* fix(test): remove useless cases

---------

Co-authored-by: dapan1121 <wpan@taosdata.com>
Co-authored-by: Pan Wei <72057773+dapan1121@users.noreply.github.com>
2025-03-15 17:06:48 +08:00
wangjiaming0909
d35bdba180 fix parser/planner/nodes ret check ci 2024-07-26 17:57:29 +08:00
wangjiaming0909
4d5cd9cd1a planner/nodes/parser ret check 2024-07-26 17:57:27 +08:00
Xiaoyu Wang
df0d200f59 feat: the compact command adds 'start with end with' clause 2023-03-07 15:24:04 +08:00
Xiaoyu Wang
ca43c0ab55 feat: support writing streams to existing tables 2023-01-17 16:47:43 +08:00
Xiaoyu Wang
80dc658dbb Merge remote-tracking branch 'origin/3.0' into enh/3.0_planner_optimize 2022-11-04 18:03:21 +08:00
Xiaoyu Wang
8513dc1067 enh: insert optimize 2022-11-04 15:21:38 +08:00
Xiaoyu Wang
02ac6fa552 enh: add unit test and delete useless code 2022-10-26 17:01:55 +08:00
Xiaoyu Wang
18fa48ae57 feat: support batch loading of csv files 2022-10-11 15:54:01 +08:00
Xiaoyu Wang
8a010a58f3 enh: added memory allocators for parser and planner 2022-09-20 16:13:08 +08:00
Xiaoyu Wang
eef595318d feat: system table visible permission 2022-08-24 19:08:50 +08:00
Xiaoyu Wang
b420bf4c3d enh: insert client optimize 2022-08-15 14:45:59 +08:00
Xiaoyu Wang
555a575c58 enh: insert client optimize 2022-08-12 10:46:55 +08:00
Xiaoyu Wang
8fc86a31d0 enh: asynchronous catalog interface optimization 2022-08-04 20:53:43 +08:00
Xiaoyu Wang
37ec05a715 fix: memory leak problems of parser and planner 2022-07-20 17:20:29 +08:00
Xiaoyu Wang
5988fc0e4a feat: sql command 'show tags from table_name' 2022-07-19 15:31:44 +08:00
Xiaoyu Wang
ae076af4f1 feat: sql system info function 'database()', 'client_version()', 'server_version()', 'server_status()', 'current_user()' and 'user()' 2022-06-29 11:41:32 +08:00
Xiaoyu Wang
1676bfe5cf fix: some problems of parser 2022-06-14 15:43:56 +08:00
Xiaoyu Wang
380a071551 fix: handle the memory leak of parser 2022-06-13 14:54:38 +08:00
Xiaoyu Wang
d1e0327d19 fix: handle the memory leak of parser 2022-06-11 21:32:29 +08:00
afwerar
73724363f5 fix(os): add win ctest 2022-05-28 21:38:35 +08:00
Xiaoyu Wang
768beb6482 feat: parser adapts asynchronous interface 2022-05-28 16:45:49 +08:00
Xiaoyu Wang
b5db96b717 feat: parser adapts asynchronous interface 2022-05-28 15:11:37 +08:00
Xiaoyu Wang
d0c11c5bba feat: parser adapts asynchronous interface 2022-05-27 17:00:37 +08:00
Xiaoyu Wang
2745113a77 fix: some problems of parser 2022-05-10 20:59:10 +08:00
Xiaoyu Wang
3ec702605b enh: refactor unit test of parser and planner 2022-04-28 21:02:11 +08:00
Xiaoyu Wang
6d449c8224 enh: refactor db and table options 2022-04-27 18:18:37 +08:00