Commit graph

6076 commits

Author SHA1 Message Date
Haojun Liao
40c6c8a2bc
feat(query): add high priority queries process queue in mnode to avoid normal queries blocking critical processing. (#33088) 2025-09-29 10:01:33 +08:00
Kaili Xu
a407446dc8
fix: move tsdb retention monitor to community (#33094) 2025-09-27 12:26:10 +08:00
Kaili Xu
13ae5350b7
feat: rollup sma (#33063) 2025-09-26 17:32:32 +08:00
WANG MINGMING
558928cbe1
feat(stream): optimize stream logic (#33027)
* fix: remove debug log

* fix: remove assert

* fix: delete unused code

* enh: [TD-37251] Support expr in state window.

* feat(stream): support expr in state window trigger

* enh: [TD-37251] Fix SCL_IS_CONST_CALC condition.

* fix(stream): set ver in wal

* fix: print code

* fix: increase runner replica num

* fix: trigger mem error

* fix: sliding _tnext_ts value

* fix(stream): disable tagFilterCache in stream reader trigger

* fix: crash

* Revert "fix(stream): fix history calc finish check"

This reverts commit f93d17f1d2.

* Revert "fix(stream): fix calc request allocation in trigger"

This reverts commit c5410f6da0.

* fix(stream): fix calc request allocation in trigger

* enh: [TD-37251] External window support more placeholder.

* fix(stream): modify size of return from 1000000->4096

* enh: [TD-37251] Modify error msg when stream query do not have from clause.

* fix(stream): add log for group not found

* fix(stream): do not return gid=0 in walMetaData interface

* enh: [TD-37251] Fix missing ts column in vtable query.

* fix: test case build failed

* fix: invalid read issue

* fix(stream): add vtable logic

* fix(stream): encode error in wal

* fix(stream): add vtable logic

* fix(stream): add log

* fix: diff funcition crash

* Revert "Merge branch 'enh/TD-37251-3.0-dropoutput' into enh/TD-37251-3.0"

This reverts commit e93cbd6fd4, reversing
changes made to dc3230591d.

* Revert "Merge branch 'enh/TD-37251-3.0-vtable' into enh/TD-37251-3.0"

This reverts commit dc3230591d, reversing
changes made to 085e086782.

* fix(stream): fix block data len is too large if data type is vchar

* fix: drop output table

* feat: runner delete output table

* process pDropBlock in trigger task.

* fix(stream): opti log level

* fix(stream): build block for drop table

* fix(stream): set gid for normal table

* fix(stream): set gid for normal table

* feat: Support delete output.

* fix(stream): rows error

* fix(stream): memory leak

* enh: [TD-37251] Fix external window wrong ts column.

* fix(stream): fix calc time check in batch mode

* fix: merge aligned external window issue

* Revert "fix(stream): fix calc time check in batch mode"

This reverts commit d895b7f577.

* fix(stream): add test case

* fix(stream): add insert drop table logic

* fix: external window end issue

* fix(stream): add test case

* fix(stream): fix trigger pull data

* fix(stream): fix history calc request

* enh: drop table on snode

* fix(stream): adjust hash index if data is filtered in wal

* fix(stream): rollback

* enh: add merge aligned extwin window row idx

* fix: drop output table

* fix: compile issue

* enh: [TD-37251] Add flag to identify interval window is overlapped

* fix: overlap

* fix(stream): set gid=-1 for initialized

* fix(stream): modify log level

* fix: trigger slow issue

* fix(stream): add basic test for obj pool

* fix(stream): fix metadata clear in trigger

* fix(stream): fix idle runner allocation in trigger

* fix: handle agg output on externalWin

* fix: test case

* fix(stream): adjust log

* fix: reset pCtx pOutput

* fix: memory leak

* fix: search first win for tsCol

* fix(stream): add test case for schema change

* fix: mem leak

* fix: mem leak

* Reapply "Merge branch 'enh/TD-37251-3.0-vtable' into enh/TD-37251-3.0"

This reverts commit b508e66958.

* fix(stream): fix virtual table data pull

* fix(stream): fix set table request

* fix(stream): process empty uidlist

* fix(stream): fix set table request

* fix(stream): fix data new request in trigger

* fix(stream): tablelist error for vtable

* fix(stream): block ver is null

* fix(stream): remove version limition for wal

* fix(stream): block rows error

* fix(stream): fix pending calc param in batch mode

* fix(stream): auto create table

* fix(stream): fix stream vtable data merge

* fix: _tcurrentts

* fix(stream): destroy hash

* fix(stream): fix trigger status

* fix(stream): colId error in vtable

* fix(stream): update nrows of vtable data block

* fix(stream): fix trigger status

* fix(stream): enable low latency calc for period trigger

* fix: test case file path

* fix: test case file path

* fix: string to node in reader

* enh: add test log

* fix(stream): increase wait time of non-low-latency mode

* fix(stream): fix column capacity in scalar calculation

* fix(stream): fix column capacity in trigger expr calculation

* fix: get origTableInfos

* fix(stream): fix calc data pull in trigger

* fix(stream): fix calc data cache write in trigger

* enh: [TD-37251] Add flag to identify interval wind

* fix: external window memory usage issue

* fix(stream): fix epxr result column in trigger

* fix(stream): add metaCache for calc plan

* fix(stream): fix stream obj list clear

* fix(stream): rollback

* enh: [TD-37251] Add flag to identify interval wind

* fix: mem free

* fix(stream): add metaCache for calc plan

* fix(stream): fix calc data cache write in trigger

* fix(stream): fix calc data cache write in trigger

* enh: optimize external result block memory

* fix(stream): modify logic of judge table for create table

* fix(stream): fix event window check in trigger

* fix(stream): fix count window check in trigger

* fix(stream): colSize=0 while encoding block because pDataCol->hasNull is false in secode time & reload table list if create table

* enh: optimize stream memory

* fix(stream): trigger tag error

* fix: add log

* fix(stream): fix calc data write in trigger

* fix: drop output table

* fix(stream): fix max delay in trigger

* fix: drop output table

* fix(stream): fix max delay in trigger

* fix(build): handle return value of function

* fix(stream): read gid error if it is child table in stream

* fix(stream): fix calc param of history calculation

* fix(stream): fix recalc of delete data

* fix(stream): fix calc data of period trigger

* fix(stream): fix cache read check

* fix(stream): filter error in calc plan

* fix: reset externla window expr

* fix(stream): fix calc parm of max delay

* fix: time range and case issues

* fix(stream): fix crash in trigger

* fix: case issues

* fix: fix window node mem leak.

* fix(stream): fix meta data clear in trigger

* fix(stream): table schema is old in TsdbDataRequest for vtable

* fix: fix slingding window place holder check condition.

* Revert "fix: fix slingding window place holder check condition."

This reverts commit ad864a1dc1.

* fix(stream): null pointer error

* fix: case issue

* fix(stream): calc data error for vtable

* fix(stream): fix data sorter in trigger

* fix(stream): add log for delete data

* fix(stream): fix start version of realtime calculation

* fix(stream): fix cache data merger of vtable

* fix: case issues

* fix(ci): upgrade stream cases in test_cols_function

* fix(stream): gid not found if change tag value

* fix: fix slingding window place holder check condition.

* fix(stream): fix virt table info request in trigger

* fix(stream): set gid = uid if stream table type != SUPER table

* fix: clean cache data by group

* fix: add block info and case issues

* fix: fix heap-buffer-overflow.

* fix(stream): fix state window with extend param

* fix: fix access null pointer.

* fix: case issues

* fix: case issue

* fix(stream): fix ignore_nodata_trigger option for period trigger

* fix(stream): fix pseudo col fetch for calc data

* fix: Extend checking time to avoid timeout.

* fix: case issues

* fix(stream): fix group col fetch for virtual tables

* fix(stream): tag is NULL for non vtable

* fix(stream): fix sliding check of virtual table

* test(stream): check stream status after create all streams

* fix: add log

* fix(stream): fix wal meta truncate when ignore disorder

* fix(stream): gid not found for child table

* fix: fix place holder condition pushdown error.

* fix(stream): suid not equal when delete data for child table

* fix: id issue

* fix(stream): disable recalc for count trigger

* fix: cast result rowSize error in project

* fix(stream): add log for tsdb meta

* fix(stream): fix gid in tsdb meta request

* fix(stream): fix wend of unclosed windows

* fix(stream): fix ignore_nodata_trigger option for period trigger

* fix: case issues

* fix(stream): fix calc data pull for empty interval window

* fix: fix ext window condition.

* fix(ci): smaBasic performance check affectd by debug level log

* fix(stream): add suid when set table list for vtable

* fix: forbid using prefilter when using %%trows an trigger table is virtual table.

* fix: forbid prefilter %%trows cases.

* fix(stream): sort cid in tsdbVirtalDataReq

* fix: forbid prefilter %%trows cases.

* fix(stream): add log for virtual table tsdb data

* fix(stream): get delete msg for vtable

* fix: winRowIndex

* Revert "fix: winRowIndex"

This reverts commit e08b41cf96.

* fix(stream): fix data merge in trigger

* test(stream): fix case ans

* fix(stream): fix empty calc data pull for period trigger

* fix(stream): col index error for tsdbVirtalDataReq

* fix(stream): pTableList is NULL for vtable

* test(stream): fix case ans

* fix(stream): fix notification in trigger

* fix(stream): memory leak

* fix(stream): fix virtual data pull in trigger

* test(stream): fix case ans

* fix(stream): col index error for tsdbVirtalDataReq

* fix(stream): pTableList is NULL for vtable

* test(stream): fix case ans

* fix(stream): fix notification in trigger

* fix(stream): memory leak

* fix(stream): fix virtual data pull in trigger

* test(stream): fix case ans

* fix: case issue

* fix: crash issue

* fix: forbid prefilter %%trows cases.

* fix(stream): session case

* fix(stream): fix data pull for virtual tables

* fix(stream): add log

* fix(stream): fix calc req send in batch mode

* fix: fix stream UT

* fix(stream): tablelist is null for non vtable

* fix: mem leak

* fix(stream): fix compile error in trigger

* fix: return code issue

---------

Co-authored-by: dapan1121 <wpan@taosdata.com>
Co-authored-by: facetosea <285808407@qq.com>
Co-authored-by: Jing Sima <simondominic9997@outlook.com>
Co-authored-by: Jinqing Kuang <kuangjinqingcn@gmail.com>
Co-authored-by: Tony Zhang <tonyzhang@taosdata.com>
2025-09-25 15:48:14 +08:00
dongming chen
270ee9a686
enh[TS-6480]: auto weight balance leader (#33030) 2025-09-24 16:49:09 +08:00
Hongze Cheng
0e08c947c3
Merge pull request #33026 from taosdata/fix/TS-6413
fix: test bug
2025-09-22 14:32:57 +08:00
Tony Zhang
e4777c94b8
feat: support extend option in state window (#33003) 2025-09-22 13:52:52 +08:00
Hongze Cheng
b4f5be97c7 fix: return code check error 2025-09-22 10:24:21 +08:00
Hongze Cheng
82983ebaf4 fix: test bug 2025-09-19 17:00:45 +08:00
Hongze Cheng
e61fc3c3a6
feat: impl vnode scan (#32942)
* feat: support SQL commands

* feat: impl `scan database` client part code

* feat: impl `scan db.vgroups` client part code

* feat: impl `show scans` client codes

* feat: impl `show scan <id>` client code

* feat: impl `kill scan <id>` client code

* feat: impl mnode part code

* more code

* feat: impl mndScanDb

* feat: Impl all command mnode code

* feat: impl vnode part code

* fix: ci return code check

* fix: return code ci

* doc: change document

* fix: forever loop

* fix: fix some test cases

* add more codes

* fix: compile error

* fix: CI cases

* fix: some scan issues
2025-09-19 13:41:41 +08:00
Jing Sima
a656416e2e
feat: [TD-37209] Support TSMA. (#33005) 2025-09-19 12:09:11 +08:00
Simon Guan
4dd6b9a7e4 Merge branch 'main' into merge/mainto3.0 2025-09-17 17:25:39 +08:00
Simon Guan
75492f7346 Merge branch 'merge/tmp' into merge/3.3.6tomain 2025-09-16 09:35:54 +08:00
dongming chen
0d6d5265fd
fix: TS-7219-update-in-sdb (#32968) 2025-09-15 15:29:18 +08:00
Bomin Zhang
67bd099881
enh[TD-37448][TD-37449]: implement show ssmigrate and kill ssmigrate command (#32888) 2025-09-12 15:07:57 +08:00
Simon Guan
c7efdf1a2b Merge branch '3.0' into merge/mainto3.0 2025-09-12 09:32:06 +08:00
Simon Guan
d94b75564d Merge branch 'main' into merge/mainto3.0 2025-09-12 09:32:02 +08:00
dmchen
000d746f7f Merge branch '3.3.6' into merge/3.3.6tomain 2025-09-11 10:32:07 +08:00
dongming chen
433bd8f91b
fix: TD-37959-add-log-for-drop-stable (#32940) 2025-09-11 10:16:11 +08:00
Jing Sima
72d3189367
feat: [TD-37052] Handle origin table's vnode change in stream query. (#32931) 2025-09-10 16:12:11 +08:00
Simon Guan
f8af487d92 Merge branch 'main' into merge/mainto3.0 2025-09-09 08:34:17 +08:00
Haojun Liao
3cdc1604f3
feat(gpt): support imputation function (#32910) 2025-09-09 08:31:20 +08:00
Simon Guan
56e980f440 Merge branch 'main' into merge/mainto3.0 2025-09-08 12:45:03 +08:00
dmchen
486aca36a0 Merge branch '3.3.6' into merge/3.3.6tomain 2025-09-08 10:09:21 +08:00
dongming chen
624577ea8c
enh/TD-37682-set-sync-timeout (#32762)
* enh/TD-37682-set-sync-timeout

* add reset vnode sync timeout

* add 4 parameter

* change default

* rename parameter

* refactor: replace strcmp with taosStrncasecmp for syncTimeout checks

* feat: add support for additional sync interval configuration options

* feat: update sync timeout configuration handling in dmProcessConfigReq

* fix: adjust sync interval calculations in dmProcessConfigReq for accuracy

* fix: correct parameter names for sync vnode and mnode election intervals

* fix: update expected row count for 'show variables' in test_show_basic

* fix: update expected row count for 'show variables' in test_valgrind_checkerror1

* feat: add millisecond precision for status intervals and timeouts

* fix: update expected row count for 'show variables' in test_show_basic and test_valgrind_checkerror1

* fix: reduce sleep duration in management threads for improved responsiveness

* feat: add internal parameters for debugging synchronization module in taosd documentation
2025-09-08 09:02:28 +08:00
wangmm0220
256116de59 fix(stream): conflicts from main 2025-09-04 20:02:22 +08:00
WANG MINGMING
a9f758bdb8
fix(tmq): set pointer = NULL to avoid double free (#32727) 2025-09-04 18:38:50 +08:00
Bomin Zhang
026f8b7658
fix: alter database cannot set ss_keeplocal to 1440m (#32827) 2025-09-03 09:18:00 +08:00
Haojun Liao
8808fbb82c
fix(gpt): fix race-condition in pareparing files for forecast. (#32815) 2025-08-29 18:10:29 +08:00
Haojun Liao
5975c363cd
fix(gpt): fix race-condition in preparing tmp files (#32800) 2025-08-29 14:36:36 +08:00
Simon Guan
a23f9eabab Merge branch 'main' into merge/mainto3.0 2025-08-27 09:44:00 +08:00
Bomin Zhang
ea42ece29b
enh[TS-6242]: retry if migrate triggered compact (#32412) 2025-08-26 17:35:57 +08:00
Bomin Zhang
050c20aeda
fix[TD-37628]: fix bnode creation failure on windows (#32739) 2025-08-26 15:47:55 +08:00
Simon Guan
a32607ba3d Merge branch 'main' into merge/mainto3.0 2025-08-22 17:51:28 +08:00
Simon Guan
1a5455b1aa Merge branch '3.3.6' into merge/3.3.6tomain 2025-08-22 14:56:06 +08:00
WANG MINGMING
4615c1c862
fix(tmq): [TD-37592]double free if subplan string decode error (#32707) 2025-08-22 14:52:45 +08:00
Simon Guan
0279c7a33a Merge branch 'main' into merge/mainto3.0 2025-08-21 19:28:39 +08:00
Simon Guan
71a377f0af Merge branch '3.0' into merge/mainto3.0 2025-08-21 19:28:36 +08:00
Simon Guan
f96283a526 Merge branch '3.3.6' into merge/3.3.6tomain 2025-08-21 19:06:33 +08:00
dongming chen
dd21bac16b
enh[TS-6867]: support for REDISTRIBUTE command on dual-replica databases (#32468) 2025-08-21 16:49:31 +08:00
dongming chen
975ad17d5a
enh[TD-37558]: add operation name for transaction (#32657) 2025-08-21 16:42:51 +08:00
dongming chen
253175ffd3
enh[TD-3757]: only block createdb for normal user (#32676) 2025-08-21 15:53:00 +08:00
dongming chen
b22d2acf84
fix[TD-37161]: check createdb conflict (#32627) 2025-08-21 09:40:47 +08:00
dongming chen
6c3521e978
fix: TD-37161-create-db-in-global (#32406) 2025-08-13 22:13:13 +08:00
xinsheng Ren
c2533b5703
fix[TS-6995]: bool value and other (#32507) 2025-08-12 13:51:20 +08:00
Pan Wei
9a1694b41a
fix: memory leak and trigger redeploy issue (#32398)
* enh: add operator reset func

* fix: merge join reset issue

* fix: memory issues

* fix: add debug assert

* fix: memory issues

* fix: memory leak

* fix: memory issues

* fix taos log miss

* fix: case issue

* fix: case issue

* fix: case issues

* fix: drop dnode issue

* fix: memory issues

* fix: memory issues

* fix: memory leak issues

* fix: recalculate time range issue

* fix: add debug log

* fix: memory issues

* fix: enable case asan

* Update streamlist_for_ci.task

* fix: case asan issue

* fix: stream name issue

* fix: external window compile issues

* fix: deploy memory issue

* fix: ahandle issue

* fix: ahandle issue

* fix: ahandle issue

* fix: virtual table reader list issue

* fix: log info

* fix: msg error

* fix: virtual table addr list issue

* fix: memory issues

* fix: memory leak issue

* fix: memory issues

* fix: memory free issues

* fix: memory issues

* fix: snode deploy issue

* fix: mnode reader issue

* fix: memory issues

* fix: add debug test

* enh: add ignore nodata trigger

* fix: memory leaks

* fix: configuration issue

* fix: memory issue

* fix: external window issue

* fix: external window issues

* fix: external window placeholder issue

* fix: placeholder function init issues

* fix: memory leak issue

* fix: add debug log

* fix: compile issues

* fix: double free issue

* fix: runner addr update issue

* fix: msg rsp issue

* fix: external window reset issue

* fix: configuration issue

* fix: deploy msg issue

* fix: compile issue

* fix: external window idx issue

* fix: ci issues

* fix: ci case issues

* fix: drop dnode issue

* fix: add debug log

* fix: conflict

* fix: create stream if not exists issue

* fix: ahandle memory leak

* fix: case issue

* fix: exchange issues

* fix: crash issue

* fix: exchange prefetch issue

* fix: snode quit issue

* enh: support indef rows func

* fix: crash issues

* Fix external window collect vector function

* fix: external window indef rows issues

* fix: external window issue

* enh: support count always return value in external window

* fix: force output when has more result block

* fix: runner block retrieve issue

* fix: crash issue

* fix: count cases issue

* fix: reader deploy message issue

* fix: task deploy issue

* fix: external window scalar issue

* fix: compile issue

* fix: group cache reset issue

* fix: add protection check

* fix: add grant check

* fix: add disableStream config

* fix: notify free issue

* fix: case issue

* fix: grant issues

* fix: memory leak issue

* fix: memory leak issue

* fix: memory leak issue

* fix: stbJoin issue

* fix: rpc send issue

* fix: rsp stream group id issue

* fix: redeploy stream issue

* fix: cases issues

* fix: memory leak issue

* fix: snode quit issue

* fix: invalid read issue

* fix: memory leak issue

* fix: split dead loop issue

* fix: crash issue

* fix: acquire task issue

* fix: memory leak and trigger redeploy issue

* fix: taosd crash issue

* fix: return code issues

* fix: return code issues

* fix: return code issue

* fix: return code issues

* fix: return code issue

* fix: mnode crash and dead lock issue

* fix: memory and crash issues

---------

Co-authored-by: huohong <sallyhuo@taosdata.com>
Co-authored-by: Jing Sima <simondominic9997@outlook.com>
Co-authored-by: facetosea <285808407@qq.com>
2025-08-12 08:39:14 +08:00
kailixu
09110e5cdf fix: update dnode state from offline to online 2025-08-01 10:21:29 +08:00
kailixu
d3ff5348ad fix: update dnode state from offline to online 2025-08-01 10:20:09 +08:00
Simon Guan
fe1658b79c
merge: from 3.3.6 to main branch (#32389)
* enh: grant support for data source ORC (#32378)

* fix: wrong col_id in ins_columns (#32358)

* test: remove un checked case (#32388)

* fix(parser): subquery use last_row can't found the colname (#32353)

* add col_id info in ins_columns

* fix: test case

---------

Co-authored-by: Kaili Xu <klxu@taosdata.com>
Co-authored-by: Tony Zhang <34825804+Tony2h@users.noreply.github.com>
Co-authored-by: hongzhenliu <wluckyjob@gmail.com>
Co-authored-by: Tony Zhang <tonyzhang@taosdata.com>
2025-08-01 10:04:15 +08:00
Tony Zhang
cc68ac81a1
fix: wrong col_id in ins_columns (#32358) 2025-07-30 14:14:29 +08:00