Commit graph

237 commits

Author SHA1 Message Date
WANG MINGMING
3bc6635de3
feat[TS-6115]: support deletion of topics with active consumers. (#30232)
* feat[TS-6115]: drop topic & consumer group in force

* feat[TS-6115]: drop topic & consumer group in force

* feat[TS-6115]: drop topic & consumer group in force
2025-03-18 14:11:00 +08:00
WANG MINGMING
930b57b770
feat: support fill_history 1 async when creating stream (#30185)
* feat:[TS-5617]use last_row cache mode if creating stream in fill_history

* feat(stream): optimize client logic of creating stream if with history

* feat(stream): optimize client logic of creating stream if with history

* feat(stream): optimize client logic of creating stream if with history

* feat(stream): optimize client logic of creating stream if with history

* feat(stream): optimize client logic of creating stream if with history

* feat(stream): optimize client logic of creating stream if with history

* feat(stream): optimize client logic of creating stream if with history

* feat(stream): optimize client logic of creating stream if with history

* feat(stream): optimize client logic of creating stream if with history

* feat(stream): optimize client logic of creating stream if with history

* feat(stream): optimize client logic of creating stream if with history

* fix: heap use after free

* feat: add log

* fix: ci case error

* fix: compile error in windows

* fix: ci case error

* fix: heap user after free

* fix: memory leak

* fix: ci case error

* fix: ci case error

* fix: ci case error

* feat: support fill_history 1 async when creating stream

* feat: support fill_history 1 async when creating stream

* fix: compile error

* feat[TS-5617]: support fill_history 1 async when creating stream

---------

Co-authored-by: yihaoDeng <yhdeng@taosdata.com>
2025-03-18 10:01:33 +08:00
Jing Sima
410324746b
feat:[TS-4897] virtual table (#30098)
* feat: [TS-4897] Support create/drop/alter/show/describe vtable

* feat: [TS-4897] Support vtable's query

* feat: [TS-4897] Support create virtual supertable

* feat: [TS-4897] Support explain analyze / where / count(*) and only select ts of vtable.

* feat: [TS-4897] Add create test and fix bugs

* feat: [TS-4897] Add alter/drop test and fix bugs

* feat: [TS-4897] Add describe/show test and fix bugs

* feat: [TS-4897] Add auth test and fix bugs

* feat: [TS-4897] Fix meta/catalog/cache bugs

* feat: [TS-4897] Support select tag from virtual child table

* feat: [TS-4897] Add select test and fix plenty of bugs

* feat: [TS-4897] Add optimize rule for vtable scan / support create vtable cross database / remove enterprise constraint / fix bugs.

* feat: [TS-4897] Fix 'schema is old'

* feat: [TS-4897] Support virtual stable query

* feat: [TS-4897] Add tests and Fix bugs

* feat: [TS-4897] resolve conflict.
2025-03-15 14:10:46 +08:00
liuyao
2de6736160
feat(stream): add new trigger continuous_window_close (#30125)
* opt stream build twa result

* opt force window close memory

* feat(stream):optimize new interval and scan operator

* adj log

* opt code

* opt code

* fill history

* fix issue for fill history

* add ci

* feat(stream): add new stream nonblock interval operator

* adjust code

* use new scan operator

* use new scan operator

* add log

* fix issue

* recover stream scan next function

* fix issue

* fix issue

* fix issue

* ignore disorder data

* fix issue

* fix issue for interval sliding

* fix issue

* fix ci issue

* fix ci issue

* fix ci issue

* add semi && final nonblock interval operator

* fix issue

* fix issue

* fix issue

* fix issue

* fix issue

* fix issue

* fix issue

* fix issue

* refactor(stream): track the msgId for each upstream tasks.

* fix(stream): fix race condition.

* fix(stream): update the task last msgId when putting into input queue succ.

* fix issue

* fix issue

* put recalculate data to rocksdb

* fix issue

* fix issue

* enh(query)[TD-33071]: add support for saving and restoring tsdbReader scan progress

- Implement functionality to save scan progress during tsdbReader operations
- Enable resuming scans from the last saved position

* fix issue

* fix issue

* fix issue

* fix issue

* fix issue

* add rec interval check

* enh(stream):add recalculate tasks.

* enh(stream): support the re-calculate the tasks.

* fix issue && do refactor

* do refactor

* fix issue

* fix issue

* update backend opt

* add new interface

* add new session operator

* support blob

* add new session operator

* fix issue

* add rec state for rec task

* fix invalid read

* add new session window

* enh(stream): update the stream tasks backend.

* new session operator

* add pull data

* fix(stream): fix error in expand stream backend.

* fix issue

* fix issue

* fix issue

* merge code

* fix issue

* fix(stream): check for null ptr.

* fix(stream): add more check.

* fix issue

* fix issue

* fix issue

* add debug code

* fix issue

* fix issue

* fix issue

* set rec end flag

* fix(stream): opt re-calculate stream tasks.

* fix issue

* fix issue

* add new operator

* enh(stream): dispatch recalculate block to agg tasks.

* fix issue

* fix issue

* fix(stream): adjust filter.

* fix issue

* refactor

* refactor(stream): adjust the recalculate end block.

* fix issue

* fix(stream): set correct create trigger block.

* fix issue

* fix(stream): fix error in build recalculate end block.

* fix(stream): check null ptr.

* add stream client && fix issue

* fix mem leak

* fix(stream): free msg.

* add stream client

* fix(stream): fix error.

* add stream client && fix issue

* add stream client

* refactor(stream): set the recalculate task complete.

* add wend and group_id for session window dest table

* feat(stream): refactor and set the recalcul agg task complete.

* add cfg for adapter

* fix issue

* add state && event operator

* feat(stream): support fill-history task.

* add new fill operator

* fix(stream): set correct backend when updating fill-history task to recalculate task.

* add new fill operator

* fix(stream): s2025-03-06 11:10:31.272
et ops always open in scan tsdb

* fix(stream):set the correct taskType for sink task.

* new fill operator

* adj stream fill operator

* fix issue

* fix issue

* fix issue

* fix issue

* fix issue

* fix issue

* fix issue

* fix issue

* adj test

* fix issue

* fix(stream): fix issue

* fix(steam): fix issue

* fix(steam): fix issue

* fix(steam): fix issue

* fix(steam): fix issue

* fix(stream): fix issue

* fix(stream): fix issue

* fix(stream): fix issue

* fix: ut com error

* fix(stream): fix mem leak and adjust operator type check rule

* fix(stream): fix mem leak and adjust test case

* refactor code

* fix(stream): free items.

* fix(stream): free fix memory leak.

* fix(stream): fix syntax error.

* fix: ignore unexpect block

* fix: adjust op type

---------

Co-authored-by: Haojun Liao <hjliao@taosdata.com>
Co-authored-by: Jinqing Kuang <kuangjinqingcn@gmail.com>
Co-authored-by: yihaoDeng <luomoxyz@126.com>
2025-03-14 20:14:01 +08:00
Zhixiao Bao
af7e34e189
feat(keep): support keep on super table level. (#30097)
* Feat: support use keep while create super table.

* Test(keep): add test for create super table with keep option.

* Feat(keep): Add tmsg for create keep.

* Feat(keep): support alter table option keep.

* Fix(keep): Add baisc test for alter table option.

* Fix(keep): memory leek.

* Feat(keep): add keep to metaEntry&metaCache and fix earliestTs with stn keep.

* Test(keep): add some cases for select with stb keep.

* Fix: fix ci core while alter stb.

* Feat(keep): delete expired data in super table level.

* Feat: remove get stb keep while query.

* Fix : build error.

* Revert "Fix : build error."

This reverts commit 0ed66e4e84.

* Revert "Feat(keep): delete expired data in super table level."

This reverts commit 36330f6b4c.

* Fix : build errors.

* Feat : support restart taosd.

* Fix : alter table comment problems.

* Test : add tests for super table keep.

* Fix: change sdb stb reserve size.

* Test: add more tests.

* Feat: Disable normal tables and sub tables from setting the keep parameter

* Fix: add more checks to avoid unknown address.

* Docs: Add docs for stable keep.

* Fix: some review changes.

* Fix: review errors.
2025-03-14 16:10:13 +08:00
dongming chen
e5fb75797a
Merge branch '3.0' into feat/TS-5927-long-password 2025-03-05 12:15:12 +08:00
Hongze Cheng
0d2ca243b1 Merge branch '3.0' of https://github.com/taosdata/TDengine into enh/TS-5445-3.0 2025-02-27 17:18:33 +08:00
dongming chen
0ce087f2b3
Merge branch '3.0' into feat/TS-5805-force-assign-leader 2025-02-26 13:57:36 +08:00
Hongze Cheng
d48547ed64 more code 2025-02-25 16:48:43 +08:00
Jinqing Kuang
fc2d2672b9 feat(stream)[TS-5469]: add more stream event notifications
- Add event notifications for all window types
- Implement window invalidation notifications
- Add statistics for event notification delivery
2025-02-16 11:07:16 +08:00
dmchen
36caaf7aab feat/TS-5927-long-password-length 2025-02-12 09:50:56 +00:00
dongming chen
0350f46ba4
Merge pull request #29756 from taosdata/3.0
3.0
2025-02-12 14:00:02 +08:00
Shengliang Guan
cd52401303 Merge branch 'main' into merge/mainto3.02 2025-02-11 15:15:35 +08:00
kailixu
27963d0978 fix: keep_time_offset format of show create database 2025-02-10 14:21:44 +08:00
dmchen
d9401fd75d feat/TS-5805-add-sql-command 2025-01-23 09:37:07 +00:00
Jinqing Kuang
eb5d463490 feat(stream)[TS-5469]. add support for window event notifications in stream processing
- Introduce new syntax to specify notification type and destination address
- Collect relevant event information during window computations
- Implement websocket-based notification delivery to the specified address
2025-01-21 17:39:46 +08:00
Shengliang Guan
fcca5e7c2e
Merge pull request #29381 from taosdata/fix/liaohj
test: add test cases and fix related bugs.
2024-12-31 11:10:34 +08:00
kailixu
b0fa490dd6 test: add test case for auto compact 2024-12-29 15:13:38 +08:00
kailixu
078367d29f test: add test case for auto compact 2024-12-28 20:52:22 +08:00
54liuyao
0bf9fb1e31 add reset stream 2024-12-27 15:11:19 +08:00
dongming chen
20cc54aaaa
Merge branch 'enh/TD-29974-improve-trans' into dmchen/trans-improve 2024-12-16 10:34:11 +08:00
Hongze Cheng
37c1be4818 Merge branch '3.0' of https://github.com/taosdata/TDengine into feat/TS-4994-3.0 2024-12-14 23:51:25 +08:00
kailixu
03e051ee63 feat: dispatch auto compact 2024-12-05 17:15:23 +08:00
xiao-77
dbaa4349ea Merge branch '3.0' into enh/3.0/TS-5007 2024-12-04 19:34:14 +08:00
kailixu
fb029944a4 feat: auto compact options for db 2024-12-04 15:46:42 +08:00
kailixu
3e816658af chore: resolve conflicts 2024-12-03 15:27:55 +08:00
Hongze Cheng
25b9de5d51 Merge branch '3.0' of https://github.com/taosdata/TDengine into feat/TS-4994-3.0 2024-12-03 14:32:03 +08:00
Hongze Cheng
9f6f82ff98 Merge branch '3.0' of https://github.com/taosdata/TDengine into feat/TS-4994-3.0 2024-11-29 15:10:27 +08:00
yihaoDeng
e495d39220 Merge remote-tracking branch 'origin/3.0' into enh/addDiskUsage 2024-11-29 13:50:09 +08:00
xiao-77
bcabfa3382 Merge branch '3.0' into enh/3.0/TS-5007 2024-11-29 11:33:38 +08:00
kailixu
67924ee55b feat: support auto compact 2024-11-26 19:46:48 +08:00
xiao-77
d778ada8f9 Add show 2024-11-26 16:08:16 +08:00
yihaoDeng
4d857b0149 fix mem leak 2024-11-25 15:22:53 +08:00
Hongze Cheng
496d9bb405 Merge branch '3.0' of https://github.com/taosdata/TDengine into feat/TS-4994-3.0 2024-11-25 13:39:47 +08:00
yihaoDeng
3027e377fa support update multi tag 2024-11-23 22:18:30 +08:00
Haojun Liao
af1c0c98bb fix(analytics): do some internal refactor and fix the error in parse the option string. 2024-11-21 15:14:37 +08:00
Hongze Cheng
e97750608e Merge branch '3.0' of https://github.com/taosdata/TDengine into feat/TS-4994-3.0 2024-11-18 14:33:43 +08:00
yihaoDeng
2b2377869f Merge remote-tracking branch 'origin/3.0' into enh/addDiskUsage 2024-11-15 15:36:05 +08:00
yihaoDeng
42065bf2e8 Merge remote-tracking branch 'origin/3.0' into enh/addDiskUsage 2024-11-14 17:44:29 +08:00
Hongze Cheng
be743733a5 feat: support database auto compact parameters 2024-11-14 17:01:15 +08:00
Shungang Li
67da7b1036 feat: alter systable variablesSchema 2024-11-13 16:51:40 +08:00
yihaoDeng
eddc7918d3 add quick cmd 2024-11-12 21:10:17 +08:00
Hongze Cheng
318ac32ca2 more code 2024-11-07 15:43:54 +08:00
dmchen
623017b16d dmchen/trans-improve-show 2024-11-07 02:32:44 +00:00
Hongze Cheng
598843f699 some code change 2024-11-05 10:55:06 +08:00
Shengliang Guan
0cae5f178c enh: increase synatx parser 2024-10-26 21:16:28 +08:00
Shengliang Guan
3003e7286e feat: data analysis 2024-10-15 10:00:38 +08:00
Shengliang Guan
9f05ca5ee8 revert: data analysis 2024-10-14 23:27:11 +08:00
Shengliang Guan
7ba8a6a680 fix: manual testing errors 2024-10-11 12:11:59 +08:00
Shengliang Guan
4dc0c2f47e feat: data analysis 2024-10-09 17:09:43 +08:00