Commit graph

1635 commits

Author SHA1 Message Date
Jing Sima
410324746b
feat:[TS-4897] virtual table (#30098)
* feat: [TS-4897] Support create/drop/alter/show/describe vtable

* feat: [TS-4897] Support vtable's query

* feat: [TS-4897] Support create virtual supertable

* feat: [TS-4897] Support explain analyze / where / count(*) and only select ts of vtable.

* feat: [TS-4897] Add create test and fix bugs

* feat: [TS-4897] Add alter/drop test and fix bugs

* feat: [TS-4897] Add describe/show test and fix bugs

* feat: [TS-4897] Add auth test and fix bugs

* feat: [TS-4897] Fix meta/catalog/cache bugs

* feat: [TS-4897] Support select tag from virtual child table

* feat: [TS-4897] Add select test and fix plenty of bugs

* feat: [TS-4897] Add optimize rule for vtable scan / support create vtable cross database / remove enterprise constraint / fix bugs.

* feat: [TS-4897] Fix 'schema is old'

* feat: [TS-4897] Support virtual stable query

* feat: [TS-4897] Add tests and Fix bugs

* feat: [TS-4897] resolve conflict.
2025-03-15 14:10:46 +08:00
liuyao
2de6736160
feat(stream): add new trigger continuous_window_close (#30125)
* opt stream build twa result

* opt force window close memory

* feat(stream):optimize new interval and scan operator

* adj log

* opt code

* opt code

* fill history

* fix issue for fill history

* add ci

* feat(stream): add new stream nonblock interval operator

* adjust code

* use new scan operator

* use new scan operator

* add log

* fix issue

* recover stream scan next function

* fix issue

* fix issue

* fix issue

* ignore disorder data

* fix issue

* fix issue for interval sliding

* fix issue

* fix ci issue

* fix ci issue

* fix ci issue

* add semi && final nonblock interval operator

* fix issue

* fix issue

* fix issue

* fix issue

* fix issue

* fix issue

* fix issue

* fix issue

* refactor(stream): track the msgId for each upstream tasks.

* fix(stream): fix race condition.

* fix(stream): update the task last msgId when putting into input queue succ.

* fix issue

* fix issue

* put recalculate data to rocksdb

* fix issue

* fix issue

* enh(query)[TD-33071]: add support for saving and restoring tsdbReader scan progress

- Implement functionality to save scan progress during tsdbReader operations
- Enable resuming scans from the last saved position

* fix issue

* fix issue

* fix issue

* fix issue

* fix issue

* add rec interval check

* enh(stream):add recalculate tasks.

* enh(stream): support the re-calculate the tasks.

* fix issue && do refactor

* do refactor

* fix issue

* fix issue

* update backend opt

* add new interface

* add new session operator

* support blob

* add new session operator

* fix issue

* add rec state for rec task

* fix invalid read

* add new session window

* enh(stream): update the stream tasks backend.

* new session operator

* add pull data

* fix(stream): fix error in expand stream backend.

* fix issue

* fix issue

* fix issue

* merge code

* fix issue

* fix(stream): check for null ptr.

* fix(stream): add more check.

* fix issue

* fix issue

* fix issue

* add debug code

* fix issue

* fix issue

* fix issue

* set rec end flag

* fix(stream): opt re-calculate stream tasks.

* fix issue

* fix issue

* add new operator

* enh(stream): dispatch recalculate block to agg tasks.

* fix issue

* fix issue

* fix(stream): adjust filter.

* fix issue

* refactor

* refactor(stream): adjust the recalculate end block.

* fix issue

* fix(stream): set correct create trigger block.

* fix issue

* fix(stream): fix error in build recalculate end block.

* fix(stream): check null ptr.

* add stream client && fix issue

* fix mem leak

* fix(stream): free msg.

* add stream client

* fix(stream): fix error.

* add stream client && fix issue

* add stream client

* refactor(stream): set the recalculate task complete.

* add wend and group_id for session window dest table

* feat(stream): refactor and set the recalcul agg task complete.

* add cfg for adapter

* fix issue

* add state && event operator

* feat(stream): support fill-history task.

* add new fill operator

* fix(stream): set correct backend when updating fill-history task to recalculate task.

* add new fill operator

* fix(stream): s2025-03-06 11:10:31.272
et ops always open in scan tsdb

* fix(stream):set the correct taskType for sink task.

* new fill operator

* adj stream fill operator

* fix issue

* fix issue

* fix issue

* fix issue

* fix issue

* fix issue

* fix issue

* fix issue

* adj test

* fix issue

* fix(stream): fix issue

* fix(steam): fix issue

* fix(steam): fix issue

* fix(steam): fix issue

* fix(steam): fix issue

* fix(stream): fix issue

* fix(stream): fix issue

* fix(stream): fix issue

* fix: ut com error

* fix(stream): fix mem leak and adjust operator type check rule

* fix(stream): fix mem leak and adjust test case

* refactor code

* fix(stream): free items.

* fix(stream): free fix memory leak.

* fix(stream): fix syntax error.

* fix: ignore unexpect block

* fix: adjust op type

---------

Co-authored-by: Haojun Liao <hjliao@taosdata.com>
Co-authored-by: Jinqing Kuang <kuangjinqingcn@gmail.com>
Co-authored-by: yihaoDeng <luomoxyz@126.com>
2025-03-14 20:14:01 +08:00
wangjiaming
f50fcb85b8
feat(decimal): support decimal data type (#30060)
* decimal: create table

* decimal: add test case decimal.py

* decimal: add decimal.c

* support input decimal

* decimal test

* refactor svalue

* fix test cases

* add decimal unit test

* add decimal test cmake

* support insert and query decimal type

* define wide integer, support decimal128

* support decimal128 divide

* set decimal type expr res types

* scalar decimal

* convert to decimal

* fix decimal64/128 from str and to str

* fix decimal from str and decimal to str

* decimal simple conversion

* unit test for decimal

* decimal conversion and unit tests

* decimal + - * /

* decimal scalar ops and comparision

* start to refactor GET_TYPED_DATA

* support decimal max func, cast func

* refactor GET_TYPED_DATA interface

* decimal scalar comparision

* start to implement sum for decimal

* support sum and avg for decimal type

* decimal tests

* add decimal test

* decimal add test cases

* decimal use int256/int128

* decimal testing

* fix decimal table meta and add tests for decimal col streams

* fix create stream and create tsma

* test insert decimal values

* decimal from str

* test decimal input

* test parse decimal from string

* add taos_fetch_field_e api

* decimal insert tests

* test decimal operators

* decimal operator test

* feat:support decimal in raw block

* decimal operator tests

* decimal test

* feat:support decimal in raw block

* feat:support decimal in raw block

* feat:add schemaExt to SMqDataRsp

* feat:remove add schemaExt to SMqDataRsp

* feat:remove add schemaExt to SMqDataRsp

* feat:remove add schemaExt to SMqDataRsp

* decimal test operators

* decimal operator test

* test decimal operators

* test decimal compare operators

* decimal unary operator test

* decimal col with decimal col oper test

* test decimal col filtering

* fix decimal float operator test

* decimal test where filtering

* fix decimal filtering

* fix decimal order by

* fix decimal op test

* test decimal agg funcs

* test decimal functions

* remove assert

* fix ci build for ret check

* fix decimal windows build

* fix ci ret check

* skip decimal ret check

* skip decimal ret check

* fix decimal tests

* fix decimal ci test

* decimal test

* fix(tmq): heap user after free

* fix(tmq): double free

* fix(tmq): double free

* fix decimal tests

* fix(decimal): decimal test ci build

* fix(decimal): windows build

* fix(decimal): decimal test build

* fix(decimal): fix decimal build and tests

* fix(decimal): fix decimal tests

* fix(decimal): fix taos_fetch_fields_e api

* fix(decimal): fix decimal taos_fetch_fields_e api

* fix(decimal): rebase 3.0

* fix(decimal): fix decimal functions

* fix(decimal): fix decimal test case memory leak

* fix(decimal): fix decimal tests

* fix(decimal): fix decimal test case

* fix(decimal): fix decimal tests

* feat(decimal): fix unit tests

* feat(decimal): fix deicmal unit test

---------

Co-authored-by: wangmm0220 <wangmm0220@gmail.com>
Co-authored-by: yihaoDeng <yhdeng@taosdata.com>
2025-03-14 18:08:07 +08:00
Simon Guan
43ca6d5f9e refactor: adjust some logs 2025-02-28 16:49:46 +08:00
Simon Guan
76ad364b93 enh: adjust some logs 2025-02-27 09:55:11 +08:00
Shengliang Guan
2bee60a41a Merge branch 'main' into merge/mainto3.0 2025-02-20 18:12:02 +08:00
54liuyao
870fe1c071 fix(tsma):drop invalid state for tsma 2025-02-20 11:52:04 +08:00
wangmm0220
7c5c9f6245 fix:[TS-5776]add raw type from consumer 2025-01-22 16:38:37 +08:00
dapan1121
2bd493f77b fix: return code issues 2024-12-11 19:18:50 +08:00
Shengliang Guan
6db8a25c98 Merge branch 'main' into merge/mainto3.0 2024-11-27 22:31:17 +08:00
54liuyao
39564a41ae fix(stream):build create table request for force_window_close 2024-11-25 13:54:31 +08:00
Shengliang Guan
a607951115 Merge branch 'main' into merge/mainto3.0 2024-11-22 13:39:44 +08:00
Shengliang Guan
cfa7d275f2 Merge branch 'main' into merge/mainto3.0 2024-11-21 11:00:00 +08:00
54liuyao
6464077cec add log 2024-11-21 09:50:45 +08:00
54liuyao
53ae014354 fix issue for streamscan 2024-11-20 19:13:05 +08:00
wangjiaming0909
b589f9089c add test case 2024-11-18 09:33:43 +08:00
54liuyao
ea7c30505a fix(stream):fix bloom filter decode issue 2024-11-13 19:37:22 +08:00
wangjiaming0909
2ddd07142a remove logs 2024-11-08 09:50:32 +08:00
wangjiaming0909
de8576a6a1 refine comments and functions 2024-11-08 09:50:07 +08:00
wangjiaming0909
9a92c136ce fix drop child table with tsma 2024-11-08 09:47:47 +08:00
wangjiaming0909
65dffbda0c fix tsma drop ctb 2024-11-08 09:47:04 +08:00
wangjiaming0909
f2860b766a add ret check for deletePartName 2024-11-08 09:42:59 +08:00
54liuyao
2b924c68fd ignore delete result 2024-11-08 09:42:59 +08:00
54liuyao
f452ca47fe delete window state when drop table 2024-11-08 09:42:56 +08:00
Shengliang Guan
10df16b01a
Merge pull request #28667 from taosdata/merge/3.0tomain
merge: from 3.0 to main branch
2024-11-06 20:46:56 +08:00
54liuyao
719a3faa12 Merge branch '3.0' of https://github.com/taosdata/TDengine into fix/TD-30837 2024-11-06 11:13:15 +08:00
Shengliang Guan
0be118c0aa
Merge pull request #28408 from taosdata/enh/TD-32181/client
Enh/td 32181/client
2024-11-06 09:42:05 +08:00
54liuyao
d010323eb6 fix ci issue 2024-11-05 13:15:32 +08:00
54liuyao
ecf8f92ac8 fix ci issue 2024-11-05 11:11:23 +08:00
54liuyao
35a1d79b98 fix(stream):check scan range of stream operator 2024-11-05 10:37:39 +08:00
xsren
73be7ddbc6 fix: set tablefield of stream block to NULL 2024-10-30 19:38:08 +08:00
54liuyao
d065d71821 Merge branch '3.0' of https://github.com/taosdata/TDengine into fix/TD-30837 2024-10-30 10:09:48 +08:00
54liuyao
a0818d2a2d fix issue 2024-10-29 18:34:51 +08:00
xsren
85989212c6 fix: set scolumnInfo nodata flag where table scan notload 2024-10-28 19:38:56 +08:00
54liuyao
bb733ca52b fix issue 2024-10-28 11:18:31 +08:00
54liuyao
6b2d338a20 enh(executor):avoid use dangerous functions 2024-10-25 13:47:09 +08:00
54liuyao
7a3f1afe68 fix issue 2024-10-18 16:08:22 +08:00
54liuyao
e9550a6b2c feat(stream):save force window close scan range 2024-10-18 15:40:37 +08:00
54liuyao
3e49f40c74 stream twa 2024-10-16 12:16:59 +08:00
54liuyao
58d8b9f84e fix ci issue 2024-10-09 11:32:20 +08:00
54liuyao
1353614dc2 fix ci issue 2024-10-09 10:02:52 +08:00
Haojun Liao
09b729f97d
Merge pull request #28201 from taosdata/fix/3_liaohj
refactor: update the error logs.
2024-09-30 13:00:15 +08:00
54liuyao
d13819bbf4 Merge branch '3.0' of https://github.com/taosdata/TDengine into fix/TD-30837 2024-09-30 09:05:46 +08:00
Haojun Liao
84da5eab2e
Merge pull request #28187 from taosdata/fix/3_liaohj
fix(stream): use meta id instead of ptr.
2024-09-30 09:00:56 +08:00
Hongze Cheng
fe18c77809
Merge pull request #28163 from taosdata/fix/TD-32338-3.0
enh: add lrucache overwriter
2024-09-29 16:06:13 +08:00
Shungang Li
10bd0fed21 enh: add lrucache overwriter 2024-09-27 17:24:45 +08:00
Pan Wei
b9df867479
Merge pull request #28155 from taosdata/fix/ly_res
adjust log level
2024-09-27 15:48:04 +08:00
54liuyao
c52da5ab33 adjust log level 2024-09-27 14:21:51 +08:00
Jinqing Kuang
03c33e77aa fix(query)[TD-32353]. Fix error handling during tag scan
Correct the row count in the result data block when the tag scan
tolerates partial errors. This prevents upper-layer operators from
accessing invalid memory address.
2024-09-27 13:55:11 +08:00
Haojun Liao
09600132ec other: merge 3.0 2024-09-26 09:59:29 +08:00