Commit graph

39830 commits

Author SHA1 Message Date
liuyao
f7beaeebfe
ci: adjust test && fix issue
ci: adjust test && fix issue
2025-03-20 17:54:20 +08:00
Simon Guan
9b7434d0ab
merge: from main to 3.0 branch (#30296)
* chore(ci): statistics ci report

* chore(ci): statistics ci report never fail

* chore(ci): statistics ci report sort by size desc

* Update .github/workflows/taosd-ci-build.yml

* docs: upgrade the node.js connector version (#30264)

* feat: change the max value of minFreeDiskSize to 2TB (#30250)

* feat: change the max value of minFreeDiskSize to 2TB

* fix: CI test case change

* enh: remove useless comment

* docs(query): enhance time window clause descriptions in the documentation

* docs: fix image position in DST document (#30266)

* fix node.js example error (#30287)

* test: node example case

* fix: node.js example error

* fix: restore test case validation

* fix(stream): fix race condition in send msg. (#30277)

Co-authored-by: 54liuyao <54liuyao@163.com>
Co-authored-by: Jinqing Kuang <kuangjinqingcn@gmail.com>
Co-authored-by: wangmm0220 <wangmm0220@gmail.com>
Co-authored-by: yihaoDeng <luomoxyz@126.com>

* add c stmt2 example to doc (#30286)

* add: stmt2 insert example

* fix: free all heap

* format code

* add c stmt2 example

* update en doc desc

* mod makefile

---------

Co-authored-by: pengrongkun94@qq.com <pengrongkun94@qq.com>

---------

Co-authored-by: freemine <freemine@yeah.net>
Co-authored-by: WANG Xu <feici02@outlook.com>
Co-authored-by: kevin men <men_shi_bin@163.com>
Co-authored-by: Hongze Cheng <hzcheng@taosdata.com>
Co-authored-by: Jinqing Kuang <kuangjinqingcn@gmail.com>
Co-authored-by: Linhe Huo <linhehuo@gmail.com>
Co-authored-by: Haojun Liao <hjxilinx@users.noreply.github.com>
Co-authored-by: 54liuyao <54liuyao@163.com>
Co-authored-by: wangmm0220 <wangmm0220@gmail.com>
Co-authored-by: yihaoDeng <luomoxyz@126.com>
Co-authored-by: She Yanjie <57549981+sheyanjie-qq@users.noreply.github.com>
Co-authored-by: pengrongkun94@qq.com <pengrongkun94@qq.com>
2025-03-20 16:54:39 +08:00
Zhixiao Bao
0b8bff92c6
enh: add keep option while show stable. (#30294) 2025-03-20 15:30:22 +08:00
Haojun Liao
cd10173454
fix(gpt): fix error in gpt (#30275)
* fix(stream): support packaging enterprise edition.

* feat(gpt): support lstm and do some internal refactor, add sample autoencoder model.

* feat(gpt): support lstm and do some internal refactor, add sample autoencoder model.

* test(gpt): disable model case.

* test(gpt): disable model case.

* doc: fix title error in doc.

* doc: add mlp doc.

* fix(gpt): add gpt

* fix(gpt): update the test cases.

* fix(gpt): update the test cases.

* test(gpt): disable gpt test in github action.

* fix(gpt): check the grant, check the rsp code.

* fix(gpt): limit the forecast rows.

* fix(gpt): use default gpt service

* fix(gpt): update error code.
2025-03-20 15:18:58 +08:00
Simon Guan
2c27c10fa8 Merge branch 'main' into merge/mainto3.0 2025-03-20 14:05:45 +08:00
xinsheng Ren
59f45345fb
fix: memleak while stop udfd
* fix: use address sanitizer to detect buffer overflow in udfTest.py

* fix: improve UDF process termination in udfTest.py

* fix: udf test case, stack overflow

---------

Co-authored-by: chenhaoran <haoran920c@163.com>
2025-03-20 13:45:33 +08:00
liuyao
fac36002c1
ci(stream): add ci and fix issue (#30270)
* ci(stream): add ci and fix issue

* ci: adj test

* ci: add test and fix issue

* ci: adj test

* ci(stream): add test for fill operator and fix issue

* fix: fix issue

* fix: fix issue

* fix: adjust defalut rec value
2025-03-20 10:34:31 +08:00
Haojun Liao
db57fc4ba8
fix(stream): fix race condition in send msg. (#30277)
Co-authored-by: 54liuyao <54liuyao@163.com>
Co-authored-by: Jinqing Kuang <kuangjinqingcn@gmail.com>
Co-authored-by: wangmm0220 <wangmm0220@gmail.com>
Co-authored-by: yihaoDeng <luomoxyz@126.com>
2025-03-19 23:15:15 +08:00
Simon Guan
0a96294056
fix: adjust raft logs (#30258) 2025-03-19 21:44:39 +08:00
Jing Sima
ccaa0b91ae
feat: [TS-4897] Add more test and fix bugs. (#30208)
* feat: [TS-4897] Fix state window wrong res when all state col is NULL.

* fix: [TD-34074] Forbid decimal type in virtual table.
2025-03-19 17:15:17 +08:00
wangjiaming
77f9707f89
feat: add more test cases for decimal
* add decimal tests

* decimal: add more decimal tests

* test(decimal): add more tests for decimal

* fix(decimal): fix decimal.py

* test(decimal): fix decimal test

* fix(decimal): fix decimal3.py test case
2025-03-19 17:13:56 +08:00
Pan Wei
0cb0e109a7
enh: add more cases for constant join (#30225)
* fix: add more cases and fix issues

* enh: add more test cases

* enh: add more test cases

* fix: memory leak issue

* enh: add view cases and fix order by issue

* enh: add document description
2025-03-19 16:50:44 +08:00
Hongze Cheng
93825d49b1
feat: change the max value of minFreeDiskSize to 2TB (#30250)
* feat: change the max value of minFreeDiskSize to 2TB

* fix: CI test case change

* enh: remove useless comment
2025-03-19 14:03:37 +08:00
Simon Guan
1099d91280
fix: update error codes while balance leader (#30261) 2025-03-19 10:52:02 +08:00
WANG MINGMING
3fafecb242
feat[TS-6137]: support sliding in force_window_close (#30203)
* feat[TS-6137]: support sliding in force_window_close

* feat[TS-6137]: support sliding in force_window_close

* feat(stream): force window close support interval sliding

* feat[TS-6137]: support sliding in force_window_close

* feat[TS-6137]: support sliding in force_window_close

* feat[TS-6137]: support sliding in force_window_close

---------

Co-authored-by: 54liuyao <54liuyao@163.com>
2025-03-19 10:22:06 +08:00
Mario Peng
fd09c267a9
fix(stmt2):fix init db is NULL problem (#30256)
* fix: stmt2 interlace mode db is NULL
* fix: add test and fix one interlace no pk error
2025-03-19 10:09:00 +08:00
dongming chen
fd813aa62d
fix: TS-6126-change-error-msg (#30186) 2025-03-19 09:09:16 +08:00
Simon Guan
aef00a615b
fix: ci errors (#30254)
* refactor: adjust wal log infos

* refactor: adjust some log's level

* refactor: adjust some log's level

* fix: add traceId for sync module

* refactor: adjust log level

* refactor: adjust log level

* refactor: adjust log levels

* refactor: adjust log levels

* refactor: adjust log levels

* refactor: adjust log levels

* refactor: adjust log levels

* refactor: adjust log levels

* refactor: adjust log levels

* docs: update keywords

* fix: compile errors in windows

* docs: minor changes

* docs: minor changes

* refactor: minor changes

* docs: fminro changes

* docs: format

* refactr: remove uncessory logs

* docs: format doc

* fix: ci errors

* fix: ci errors
2025-03-18 18:55:30 +08:00
Simon Guan
d02196a231
refactor: adjust some logs (#30245)
* refactor: adjust wal log infos

* refactor: adjust some log's level

* refactor: adjust some log's level

* fix: add traceId for sync module

* refactor: adjust log level

* refactor: adjust log level

* refactor: adjust log levels

* refactor: adjust log levels

* refactor: adjust log levels

* refactor: adjust log levels

* refactor: adjust log levels

* refactor: adjust log levels

* refactor: adjust log levels

* docs: update keywords

* fix: compile errors in windows

* docs: minor changes

* docs: minor changes

* refactor: minor changes

* docs: fminro changes

* docs: format

* refactr: remove uncessory logs

* docs: format doc
2025-03-18 17:55:36 +08:00
Minglei Jin
0c1db8feb2
fix(tdb/test): remove unstable cases (#30238) 2025-03-18 16:40:54 +08:00
Simon Guan
ddc05f168e
refactor: adjust the log to track the data writing process on the server via QID (#30195)
* refactor: adjust wal log infos

* refactor: adjust some log's level

* refactor: adjust some log's level

* fix: add traceId for sync module

* refactor: adjust log level

* refactor: adjust log level

* refactor: adjust log levels

* refactor: adjust log levels

* refactor: adjust log levels

* refactor: adjust log levels

* refactor: adjust log levels

* refactor: adjust log levels

* refactor: adjust log levels

* fix: compile errors in windows

* refactor: minor changes
2025-03-18 16:21:44 +08:00
Simon Guan
564a60fe25
Merge pull request #30237 from taosdata/merge/mainto3.0
merge: from main to 3.0 branch
2025-03-18 16:17:49 +08:00
xinsheng Ren
0fb373ef03
fix: scalarTest on mac (#30233)
Co-authored-by: facetosea <25808407@qq.com>
2025-03-18 15:11:10 +08:00
Simon Guan
718d455a97 Merge branch 'main' into merge/mainto3.0 2025-03-18 14:35:54 +08:00
WANG MINGMING
3bc6635de3
feat[TS-6115]: support deletion of topics with active consumers. (#30232)
* feat[TS-6115]: drop topic & consumer group in force

* feat[TS-6115]: drop topic & consumer group in force

* feat[TS-6115]: drop topic & consumer group in force
2025-03-18 14:11:00 +08:00
Zhixiao Bao
2523e60d3b
fix: remove meaningless error logs when fetching child table metadata. (#30201)
* Enh: remove useless error logs.

* Fix: solving performance issues.

* fix: format code

---------

Co-authored-by: Simon Guan <slguan@taosdata.com>
2025-03-18 14:01:06 +08:00
xinsheng Ren
8453726080
fix: when querying super table, using ORDER BY and LIMIT together may return an unexpected record.
* fix: limit error

* fix: limit error, test case.
2025-03-18 13:58:30 +08:00
Mario Peng
a3b042c078
fix: on windows, closing a JDBC PreparedStatement while writing data may cause the JVM to crash. 2025-03-18 13:45:20 +08:00
WANG MINGMING
930b57b770
feat: support fill_history 1 async when creating stream (#30185)
* feat:[TS-5617]use last_row cache mode if creating stream in fill_history

* feat(stream): optimize client logic of creating stream if with history

* feat(stream): optimize client logic of creating stream if with history

* feat(stream): optimize client logic of creating stream if with history

* feat(stream): optimize client logic of creating stream if with history

* feat(stream): optimize client logic of creating stream if with history

* feat(stream): optimize client logic of creating stream if with history

* feat(stream): optimize client logic of creating stream if with history

* feat(stream): optimize client logic of creating stream if with history

* feat(stream): optimize client logic of creating stream if with history

* feat(stream): optimize client logic of creating stream if with history

* feat(stream): optimize client logic of creating stream if with history

* fix: heap use after free

* feat: add log

* fix: ci case error

* fix: compile error in windows

* fix: ci case error

* fix: heap user after free

* fix: memory leak

* fix: ci case error

* fix: ci case error

* fix: ci case error

* feat: support fill_history 1 async when creating stream

* feat: support fill_history 1 async when creating stream

* fix: compile error

* feat[TS-5617]: support fill_history 1 async when creating stream

---------

Co-authored-by: yihaoDeng <yhdeng@taosdata.com>
2025-03-18 10:01:33 +08:00
Simon Guan
a59170ccc3
Merge pull request #30207 from taosdata/merge/mainto3.0
merge: from main to 3.0 branch
2025-03-18 09:04:49 +08:00
Simon Guan
7edd97a321 fix: compile errors 2025-03-17 22:57:06 +08:00
Simon Guan
87fbdd24ad fix: compile errors 2025-03-17 22:54:12 +08:00
Kaili Xu
094836daea
fix: log output for tsim and other apps (#30210) 2025-03-17 22:38:49 +08:00
Simon Guan
cee5ab4297 Merge branch '3.0' into merge/mainto3.0 2025-03-17 22:28:22 +08:00
Jing Sima
a2fd1ee68f
fix: [TD-34074] fix compile error. (#30221) 2025-03-17 22:27:56 +08:00
Simon Guan
cf8d7cd0ac Merge branch 'main' into merge/mainto3.0 2025-03-17 21:42:42 +08:00
Yihao Deng
2151c5cae0
update telemInterval (#29703) 2025-03-17 17:42:24 +08:00
Jing Sima
379a75a429
test: reopen mac unit tests. (#29648) 2025-03-17 17:41:46 +08:00
dongming chen
03fe3e5430
fix: [TD-34000] remove lock when send heartbeat reply main (#30123)
* fix(stream): reduce the consensus checkpoint id trans.

* refactor(stream): add some logs.

* refactor(stream): set the max checkpoint exec time 30min.

* refactor(stream): add checkpoint-consensus trans conflict check.

* refactor(stream): remove unused local variables.

* fix(stream): fix syntax error.

* fix(stream): 1. fix free memory error 2. continue if put result into dst hashmap failed.

* fix issue

* fix issue

* fix(mnd): follower mnode not processes the timer event.

* fix(stream): print correct error msg.

* fix(stream): add some logs.

* fix(stream): add some logs.

* fix(stream): add some logs.

* fix(stream): add some logs.

* fix(stream): add some logs.

* fix(stream): add some logs.

* fix(stream): add some logs.

* fix(stream): add some logs.

* fix(stream): truncate long subtable name

* fix(stream): add buffer len.

* refactor(stream): update some logs.

* fix issue

* refactor(stream): update some logs.

* refactor(stream): update some logs.

* fix(stream): check return value.

* fix(stream): fix syntax error.

* fix(stream): check return value.

* fix(stream): update the timer check in mnode.

* fix: TD-34000-remove-lock-when-send-heartbeat-reply

* fix: TD-34000-remove-lock-when-send-heartbeat-reply

---------

Co-authored-by: Haojun Liao <hjliao@taosdata.com>
Co-authored-by: 54liuyao <54liuyao@163.com>
Co-authored-by: Jinqing Kuang <kuangjinqingcn@gmail.com>
2025-03-17 17:13:54 +08:00
Simon Guan
9164bf815c Merge branch '3.0' into merge/mainto3.0 2025-03-17 15:16:59 +08:00
Haojun Liao
dfe32225de chore: merge main. 2025-03-17 14:51:11 +08:00
wangjiaming
40d32e16b0
fix(TS-6117): fix drop table name with star crash (#30204) 2025-03-17 14:13:03 +08:00
Simon Guan
60498503b2
tests: add tdb ut (#30093)
* fix: compile warnings

* tests: add tdb ut

* test(tdb): fix return code

* test: recover ut

* fix: minor changes

* fix: enable test

* fix: ut errors

---------

Co-authored-by: Minglei Jin <mljin@taosdata.com>
2025-03-17 13:39:00 +08:00
Bomin Zhang
3606a1af3a
enh: improve performance of buildChildTableName (#30159) 2025-03-17 10:39:47 +08:00
Haojun Liao
ab92886820
fix(stream): reduce the consensus checkpoint id trans. (#30105)
* fix(stream): reduce the consensus checkpoint id trans.

* refactor(stream): add some logs.

* refactor(stream): set the max checkpoint exec time 30min.

* refactor(stream): add checkpoint-consensus trans conflict check.

* refactor(stream): remove unused local variables.

* fix(stream): fix syntax error.

* fix(stream): 1. fix free memory error 2. continue if put result into dst hashmap failed.

* fix issue

* fix issue

* fix(mnd): follower mnode not processes the timer event.

* fix(stream): print correct error msg.

* fix(stream): add some logs.

* fix(stream): add some logs.

* fix(stream): add some logs.

* fix(stream): add some logs.

* fix(stream): add some logs.

* fix(stream): add some logs.

* fix(stream): add some logs.

* fix(stream): add some logs.

* fix(stream): truncate long subtable name

* fix(stream): add buffer len.

* refactor(stream): update some logs.

* fix issue

* refactor(stream): update some logs.

* refactor(stream): update some logs.

* fix(stream): check return value.

* fix(stream): fix syntax error.

* fix(stream): check return value.

* fix(stream): update the timer check in mnode.

* fix(stream): add restart stage tracking.

* fix(stream): track the start task stage for meta.

* fix(stream): fix error in log.

* refactor(stream): adjust log info.

* fix mem issue

* fix(stream): check the number of required tasks for consensus checkpointId.

* fix(stream): lock the whole start procedure.

* fix(stream): add lock during start all tasks.

* fix(stream): update logs.

* fix(stream): update logs.

* fix(stream): update logs.

* fix(stream): fix dead-lock.

* fix(stream): fix syntax error.

* fix(stream): not drop the scan-history task.

* fix(stream): fix syntax error.

* fix(stream): wait for executor stop before restarting.

* fix(stream): add some logs.

* fix(stream): add some logs.

* fix(stream): add some logs.

* fix(stream): disable some logs.

* fix(stream): reset the start info if no task left.

---------

Co-authored-by: 54liuyao <54liuyao@163.com>
Co-authored-by: Jinqing Kuang <kuangjinqingcn@gmail.com>
2025-03-17 10:20:17 +08:00
Jinqing Kuang
ac23a4fc55
feat(stream): support stream processing for virtual tables (#30144)
* enh: add client processing

* enh: add mnode vtables processing

* enh: add mnode vtable processing

* enh: add normal child vtable support

* fix: compile issues

* fix: compile issues

* fix: create stream issues

* fix: multi stream scan issue

* fix: remove debug info

* fix: agg task and task level issues

* fix: correct task output type

* fix: split vtablescan from agg

* fix: memory leak issues

* fix: add limitations

* Update 09-error-code.md

* Update 09-error-code.md

* fix: remove usless case

* feat(stream): extract original table data in source scan task

Implemented functionality in the source task to extract data
corresponding to the virtual table from the original table using WAL.
The extracted data is then sent to the downstream merge task for further
processing.

* feat(stream): multi-way merge using loser tree in virtual merge task

Implemented multi-way merge in the merge task using a loser tree to
combine data from multiple original table into a single virtual table.
The merged virtual table data is then pushed downstream for further
processing.  Introduced memory limit handling during the merge process
with configurable behavior when the memory limit is reached.

* fix(test): remove useless cases

---------

Co-authored-by: dapan1121 <wpan@taosdata.com>
Co-authored-by: Pan Wei <72057773+dapan1121@users.noreply.github.com>
2025-03-15 17:06:48 +08:00
Simon Guan
6913d271d9
fix: compile error in the release build (#30199)
* fix: compile error in the release build

* fix: compile error in the release build
2025-03-15 16:35:02 +08:00
Jing Sima
410324746b
feat:[TS-4897] virtual table (#30098)
* feat: [TS-4897] Support create/drop/alter/show/describe vtable

* feat: [TS-4897] Support vtable's query

* feat: [TS-4897] Support create virtual supertable

* feat: [TS-4897] Support explain analyze / where / count(*) and only select ts of vtable.

* feat: [TS-4897] Add create test and fix bugs

* feat: [TS-4897] Add alter/drop test and fix bugs

* feat: [TS-4897] Add describe/show test and fix bugs

* feat: [TS-4897] Add auth test and fix bugs

* feat: [TS-4897] Fix meta/catalog/cache bugs

* feat: [TS-4897] Support select tag from virtual child table

* feat: [TS-4897] Add select test and fix plenty of bugs

* feat: [TS-4897] Add optimize rule for vtable scan / support create vtable cross database / remove enterprise constraint / fix bugs.

* feat: [TS-4897] Fix 'schema is old'

* feat: [TS-4897] Support virtual stable query

* feat: [TS-4897] Add tests and Fix bugs

* feat: [TS-4897] resolve conflict.
2025-03-15 14:10:46 +08:00
liuyao
2de6736160
feat(stream): add new trigger continuous_window_close (#30125)
* opt stream build twa result

* opt force window close memory

* feat(stream):optimize new interval and scan operator

* adj log

* opt code

* opt code

* fill history

* fix issue for fill history

* add ci

* feat(stream): add new stream nonblock interval operator

* adjust code

* use new scan operator

* use new scan operator

* add log

* fix issue

* recover stream scan next function

* fix issue

* fix issue

* fix issue

* ignore disorder data

* fix issue

* fix issue for interval sliding

* fix issue

* fix ci issue

* fix ci issue

* fix ci issue

* add semi && final nonblock interval operator

* fix issue

* fix issue

* fix issue

* fix issue

* fix issue

* fix issue

* fix issue

* fix issue

* refactor(stream): track the msgId for each upstream tasks.

* fix(stream): fix race condition.

* fix(stream): update the task last msgId when putting into input queue succ.

* fix issue

* fix issue

* put recalculate data to rocksdb

* fix issue

* fix issue

* enh(query)[TD-33071]: add support for saving and restoring tsdbReader scan progress

- Implement functionality to save scan progress during tsdbReader operations
- Enable resuming scans from the last saved position

* fix issue

* fix issue

* fix issue

* fix issue

* fix issue

* add rec interval check

* enh(stream):add recalculate tasks.

* enh(stream): support the re-calculate the tasks.

* fix issue && do refactor

* do refactor

* fix issue

* fix issue

* update backend opt

* add new interface

* add new session operator

* support blob

* add new session operator

* fix issue

* add rec state for rec task

* fix invalid read

* add new session window

* enh(stream): update the stream tasks backend.

* new session operator

* add pull data

* fix(stream): fix error in expand stream backend.

* fix issue

* fix issue

* fix issue

* merge code

* fix issue

* fix(stream): check for null ptr.

* fix(stream): add more check.

* fix issue

* fix issue

* fix issue

* add debug code

* fix issue

* fix issue

* fix issue

* set rec end flag

* fix(stream): opt re-calculate stream tasks.

* fix issue

* fix issue

* add new operator

* enh(stream): dispatch recalculate block to agg tasks.

* fix issue

* fix issue

* fix(stream): adjust filter.

* fix issue

* refactor

* refactor(stream): adjust the recalculate end block.

* fix issue

* fix(stream): set correct create trigger block.

* fix issue

* fix(stream): fix error in build recalculate end block.

* fix(stream): check null ptr.

* add stream client && fix issue

* fix mem leak

* fix(stream): free msg.

* add stream client

* fix(stream): fix error.

* add stream client && fix issue

* add stream client

* refactor(stream): set the recalculate task complete.

* add wend and group_id for session window dest table

* feat(stream): refactor and set the recalcul agg task complete.

* add cfg for adapter

* fix issue

* add state && event operator

* feat(stream): support fill-history task.

* add new fill operator

* fix(stream): set correct backend when updating fill-history task to recalculate task.

* add new fill operator

* fix(stream): s2025-03-06 11:10:31.272
et ops always open in scan tsdb

* fix(stream):set the correct taskType for sink task.

* new fill operator

* adj stream fill operator

* fix issue

* fix issue

* fix issue

* fix issue

* fix issue

* fix issue

* fix issue

* fix issue

* adj test

* fix issue

* fix(stream): fix issue

* fix(steam): fix issue

* fix(steam): fix issue

* fix(steam): fix issue

* fix(steam): fix issue

* fix(stream): fix issue

* fix(stream): fix issue

* fix(stream): fix issue

* fix: ut com error

* fix(stream): fix mem leak and adjust operator type check rule

* fix(stream): fix mem leak and adjust test case

* refactor code

* fix(stream): free items.

* fix(stream): free fix memory leak.

* fix(stream): fix syntax error.

* fix: ignore unexpect block

* fix: adjust op type

---------

Co-authored-by: Haojun Liao <hjliao@taosdata.com>
Co-authored-by: Jinqing Kuang <kuangjinqingcn@gmail.com>
Co-authored-by: yihaoDeng <luomoxyz@126.com>
2025-03-14 20:14:01 +08:00
Yihao Deng
0806cca09e
fix(query): fix ins_usage error on multi storage (#30134)
* fix calc disk usage error

* fix calc disk usage error

* fix calc disk usage error

* fix calc disk usage error

* fix calc disk usage error

* add test case

* add test case

* add test case

* add test case

* add test case

* add test case

* refactor code

* refactor code

* refactor code

* refactor code

* refactor code

* refactor code
2025-03-14 18:11:39 +08:00