Commit graph

4669 commits

Author SHA1 Message Date
liuyao
f7beaeebfe
ci: adjust test && fix issue
ci: adjust test && fix issue
2025-03-20 17:54:20 +08:00
liuyao
fac36002c1
ci(stream): add ci and fix issue (#30270)
* ci(stream): add ci and fix issue

* ci: adj test

* ci: add test and fix issue

* ci: adj test

* ci(stream): add test for fill operator and fix issue

* fix: fix issue

* fix: fix issue

* fix: adjust defalut rec value
2025-03-20 10:34:31 +08:00
WANG MINGMING
3fafecb242
feat[TS-6137]: support sliding in force_window_close (#30203)
* feat[TS-6137]: support sliding in force_window_close

* feat[TS-6137]: support sliding in force_window_close

* feat(stream): force window close support interval sliding

* feat[TS-6137]: support sliding in force_window_close

* feat[TS-6137]: support sliding in force_window_close

* feat[TS-6137]: support sliding in force_window_close

---------

Co-authored-by: 54liuyao <54liuyao@163.com>
2025-03-19 10:22:06 +08:00
xinsheng Ren
789f36edda
docs: udf function, extract_avg (#30244) 2025-03-18 17:52:21 +08:00
Simon Guan
ddc05f168e
refactor: adjust the log to track the data writing process on the server via QID (#30195)
* refactor: adjust wal log infos

* refactor: adjust some log's level

* refactor: adjust some log's level

* fix: add traceId for sync module

* refactor: adjust log level

* refactor: adjust log level

* refactor: adjust log levels

* refactor: adjust log levels

* refactor: adjust log levels

* refactor: adjust log levels

* refactor: adjust log levels

* refactor: adjust log levels

* refactor: adjust log levels

* fix: compile errors in windows

* refactor: minor changes
2025-03-18 16:21:44 +08:00
Jinqing Kuang
ac23a4fc55
feat(stream): support stream processing for virtual tables (#30144)
* enh: add client processing

* enh: add mnode vtables processing

* enh: add mnode vtable processing

* enh: add normal child vtable support

* fix: compile issues

* fix: compile issues

* fix: create stream issues

* fix: multi stream scan issue

* fix: remove debug info

* fix: agg task and task level issues

* fix: correct task output type

* fix: split vtablescan from agg

* fix: memory leak issues

* fix: add limitations

* Update 09-error-code.md

* Update 09-error-code.md

* fix: remove usless case

* feat(stream): extract original table data in source scan task

Implemented functionality in the source task to extract data
corresponding to the virtual table from the original table using WAL.
The extracted data is then sent to the downstream merge task for further
processing.

* feat(stream): multi-way merge using loser tree in virtual merge task

Implemented multi-way merge in the merge task using a loser tree to
combine data from multiple original table into a single virtual table.
The merged virtual table data is then pushed downstream for further
processing.  Introduced memory limit handling during the merge process
with configurable behavior when the memory limit is reached.

* fix(test): remove useless cases

---------

Co-authored-by: dapan1121 <wpan@taosdata.com>
Co-authored-by: Pan Wei <72057773+dapan1121@users.noreply.github.com>
2025-03-15 17:06:48 +08:00
liuyao
2de6736160
feat(stream): add new trigger continuous_window_close (#30125)
* opt stream build twa result

* opt force window close memory

* feat(stream):optimize new interval and scan operator

* adj log

* opt code

* opt code

* fill history

* fix issue for fill history

* add ci

* feat(stream): add new stream nonblock interval operator

* adjust code

* use new scan operator

* use new scan operator

* add log

* fix issue

* recover stream scan next function

* fix issue

* fix issue

* fix issue

* ignore disorder data

* fix issue

* fix issue for interval sliding

* fix issue

* fix ci issue

* fix ci issue

* fix ci issue

* add semi && final nonblock interval operator

* fix issue

* fix issue

* fix issue

* fix issue

* fix issue

* fix issue

* fix issue

* fix issue

* refactor(stream): track the msgId for each upstream tasks.

* fix(stream): fix race condition.

* fix(stream): update the task last msgId when putting into input queue succ.

* fix issue

* fix issue

* put recalculate data to rocksdb

* fix issue

* fix issue

* enh(query)[TD-33071]: add support for saving and restoring tsdbReader scan progress

- Implement functionality to save scan progress during tsdbReader operations
- Enable resuming scans from the last saved position

* fix issue

* fix issue

* fix issue

* fix issue

* fix issue

* add rec interval check

* enh(stream):add recalculate tasks.

* enh(stream): support the re-calculate the tasks.

* fix issue && do refactor

* do refactor

* fix issue

* fix issue

* update backend opt

* add new interface

* add new session operator

* support blob

* add new session operator

* fix issue

* add rec state for rec task

* fix invalid read

* add new session window

* enh(stream): update the stream tasks backend.

* new session operator

* add pull data

* fix(stream): fix error in expand stream backend.

* fix issue

* fix issue

* fix issue

* merge code

* fix issue

* fix(stream): check for null ptr.

* fix(stream): add more check.

* fix issue

* fix issue

* fix issue

* add debug code

* fix issue

* fix issue

* fix issue

* set rec end flag

* fix(stream): opt re-calculate stream tasks.

* fix issue

* fix issue

* add new operator

* enh(stream): dispatch recalculate block to agg tasks.

* fix issue

* fix issue

* fix(stream): adjust filter.

* fix issue

* refactor

* refactor(stream): adjust the recalculate end block.

* fix issue

* fix(stream): set correct create trigger block.

* fix issue

* fix(stream): fix error in build recalculate end block.

* fix(stream): check null ptr.

* add stream client && fix issue

* fix mem leak

* fix(stream): free msg.

* add stream client

* fix(stream): fix error.

* add stream client && fix issue

* add stream client

* refactor(stream): set the recalculate task complete.

* add wend and group_id for session window dest table

* feat(stream): refactor and set the recalcul agg task complete.

* add cfg for adapter

* fix issue

* add state && event operator

* feat(stream): support fill-history task.

* add new fill operator

* fix(stream): set correct backend when updating fill-history task to recalculate task.

* add new fill operator

* fix(stream): s2025-03-06 11:10:31.272
et ops always open in scan tsdb

* fix(stream):set the correct taskType for sink task.

* new fill operator

* adj stream fill operator

* fix issue

* fix issue

* fix issue

* fix issue

* fix issue

* fix issue

* fix issue

* fix issue

* adj test

* fix issue

* fix(stream): fix issue

* fix(steam): fix issue

* fix(steam): fix issue

* fix(steam): fix issue

* fix(steam): fix issue

* fix(stream): fix issue

* fix(stream): fix issue

* fix(stream): fix issue

* fix: ut com error

* fix(stream): fix mem leak and adjust operator type check rule

* fix(stream): fix mem leak and adjust test case

* refactor code

* fix(stream): free items.

* fix(stream): free fix memory leak.

* fix(stream): fix syntax error.

* fix: ignore unexpect block

* fix: adjust op type

---------

Co-authored-by: Haojun Liao <hjliao@taosdata.com>
Co-authored-by: Jinqing Kuang <kuangjinqingcn@gmail.com>
Co-authored-by: yihaoDeng <luomoxyz@126.com>
2025-03-14 20:14:01 +08:00
Mario Peng
f5a339f03e
enh(stmt2): [TD-33660] interlace mode support auto create table (#30084)
* enh: remove interlace autocreate tb limit

* enh: autocreate tb with single ctb

* enh: multi-insert in one ctb

* enh: Function realization

* fix: some unit test error

fix: stmt1 unit test

* fix: review and autoCreateTb & interlace support insert into stb syntax

* fix: insert into tags fixed value auto create table

* fix: remove a wrong test

* fix: adapter segment fault

* fix: adaptor core

* fix: get fields make interlace error

* fix: adapter core problem 2

* fix: multi bind exec invalid message problem
2025-03-14 14:13:26 +08:00
WANG MINGMING
3d053e2c9b
feat(stream): optimize client logic of creating stream if with history (#30059)
* feat:[TS-5617]use last_row cache mode if creating stream in fill_history

* feat(stream): optimize client logic of creating stream if with history

* feat(stream): optimize client logic of creating stream if with history

* feat(stream): optimize client logic of creating stream if with history

* feat(stream): optimize client logic of creating stream if with history

* feat(stream): optimize client logic of creating stream if with history

* feat(stream): optimize client logic of creating stream if with history

* feat(stream): optimize client logic of creating stream if with history

* feat(stream): optimize client logic of creating stream if with history

* feat(stream): optimize client logic of creating stream if with history

* feat(stream): optimize client logic of creating stream if with history

* feat(stream): optimize client logic of creating stream if with history

* fix: heap use after free

* feat: add log

* fix: ci case error

* fix: compile error in windows

* fix: ci case error

* fix: heap user after free

* fix: memory leak

* fix: ci case error

* fix: ci case error

* fix: ci case error

---------

Co-authored-by: yihaoDeng <yhdeng@taosdata.com>
2025-03-14 13:55:52 +08:00
Simon Guan
abaf48ae15 Merge branch '3.0' into merge/mainto3.0 2025-03-10 11:07:48 +08:00
Simon Guan
1170955b95 Merge branch 'main' into merge/mainto3.0 2025-03-10 11:05:52 +08:00
Simon Guan
6c88048f77
Merge pull request #30002 from taosdata/refactor/TD-33926
refactor: do not compile test-related files when BUILD_TEST=OFF
2025-03-06 18:43:46 +08:00
Simon Guan
c00d772492
Merge pull request #29649 from taosdata/feat/TS-5927-long-password
feat:[TS-5927]long-password
2025-03-06 17:25:48 +08:00
Simon Guan
e321a3be12 refactor: cmake files 2025-03-06 13:49:31 +08:00
Simon Guan
34120a38c5 Merge branch '3.0' into merge/mainto3.0 2025-03-05 18:12:55 +08:00
Simon Guan
292b0cb317 Merge branch 'main' into merge/mainto3.0 2025-03-05 18:10:31 +08:00
Haojun Liao
24882459e3 chore: merge main 2025-03-05 13:40:44 +08:00
Haojun Liao
04c2ecce69 fix(analytics): revise the test cases. 2025-03-05 13:10:41 +08:00
dongming chen
e5fb75797a
Merge branch '3.0' into feat/TS-5927-long-password 2025-03-05 12:15:12 +08:00
Haojun Liao
2c92cca603 test(analytics): add test case and set the unit to be second. 2025-03-04 23:31:02 +08:00
Simon Guan
1a9f75e663 Merge branch '3.0' into fix/internal 2025-03-04 13:47:03 +08:00
Haojun Liao
f563a35b3f fix(stream):check the maximum allowed forecast rows, add one test case, and do some internal refactor. 2025-03-03 17:27:05 +08:00
Simon Guan
6826edf5e0 Merge branch 'main' into merge/mainto3.0 2025-03-03 14:01:43 +08:00
Simon Guan
43ca6d5f9e refactor: adjust some logs 2025-02-28 16:49:46 +08:00
Simon Guan
24fae6adbb
Merge pull request #29900 from taosdata/fix/checkfh
refactor(stream): scan wal is driven by time, instead of insert events.
2025-02-28 13:42:43 +08:00
Haojun Liao
d279aaa22c test(stream): sleep a little bit longer. 2025-02-27 11:44:50 +08:00
Shengliang Guan
8fccf318ab
Merge pull request #29902 from taosdata/merge/mainto3.0
merge: from main to 3.0 branch
2025-02-24 15:47:21 +08:00
Shengliang Guan
3a1d06f4d7
Merge pull request #29354 from taosdata/enh/TS-5255/colsFunc2
enh: ts5255/cols func2
2025-02-24 14:19:37 +08:00
Shengliang Guan
3a739beaa5 Merge branch 'main' into merge/mainto3.0 2025-02-22 20:07:40 +08:00
WANG Xu
100136fa4e
Merge pull request #29890 from taosdata/test/new
Test/new
2025-02-21 21:07:40 +08:00
facetosea
fa61c13a75 fix: _group_key not select 2025-02-21 00:40:11 +08:00
Shengliang Guan
f16ae7abe1
Merge pull request #29809 from taosdata/merge/mainto3.0
merge: from main to 3.0 branch
2025-02-20 10:27:20 +08:00
huohong
b1f409be37 new case arch demo 2025-02-20 09:32:15 +08:00
facetosea
00473a9c64 fix: aliasName too long 2025-02-18 13:06:13 +08:00
xiao-77
25a1af2b88 Fix ci problems. 2025-02-18 12:28:07 +08:00
Shengliang Guan
63fc5dea34 Merge branch '3.0' into merge/mainto3.0 2025-02-18 10:29:16 +08:00
dmchen
e6e1d3ff1c feat/TS-5927-long-password-fix-case 2025-02-17 16:35:08 +08:00
dmchen
c51c64196f retry case 2025-02-17 05:20:35 +00:00
dmchen
9ce37c3ec8 retry case 2025-02-17 04:54:49 +00:00
dmchen
2dcddb024f retry case 2025-02-17 03:26:51 +00:00
dmchen
6dc6a59b9b retry case 2025-02-17 02:53:32 +00:00
xiao-77
9b87d982a4 Fix modify s3 bucket name from global to local. 2025-02-17 09:26:26 +08:00
dmchen
9364bb63f9 retry case 2025-02-17 01:24:55 +00:00
dmchen
49173c6018 retry case 2025-02-14 09:06:59 +00:00
dmchen
1272524f7f retry case 2025-02-14 09:05:31 +00:00
dmchen
cb8f132077 feat/TS-5927-long-password-retry-fail-case 2025-02-14 08:43:39 +00:00
dmchen
6d3d2b787a feat/TS-5927-long-password-retry-cases-fail 2025-02-14 08:18:34 +00:00
Shengliang Guan
d6bd52ece7
Merge pull request #29750 from taosdata/enh/streamqueue
refactor(stream): add more operation handled in ctrl queue, and limit the stream input q size
2025-02-14 15:02:23 +08:00
dmchen
8f2370f3fa feat/TS-5927-long-password-fix-case 2025-02-14 01:12:09 +00:00
xiao-77
f09dd39f89 Fix ci problems. 2025-02-13 18:24:14 +08:00