Commit graph

401 commits

Author SHA1 Message Date
dapan1121
009e8aa41e Merge remote-tracking branch 'origin/3.0' into feat/TS-6100-3.0 2025-05-12 16:24:22 +08:00
pengrongkun94@qq.com
820a0101ae feat:add auto create table 2025-05-07 11:06:12 +08:00
pengrongkun94@qq.com
d0eb0c5962 fix:invalid but no error output 2025-05-07 11:05:52 +08:00
wangmm0220
5ce115cc23 feat(stream): fix compile error 2025-04-24 18:02:12 +08:00
Haojun Liao
a90b41e4ce
fix: tables with a composite primary key of type varchar may have incorrect counts of projected results when sorted. 2025-04-11 15:44:34 +08:00
liuyao
2de6736160
feat(stream): add new trigger continuous_window_close (#30125)
* opt stream build twa result

* opt force window close memory

* feat(stream):optimize new interval and scan operator

* adj log

* opt code

* opt code

* fill history

* fix issue for fill history

* add ci

* feat(stream): add new stream nonblock interval operator

* adjust code

* use new scan operator

* use new scan operator

* add log

* fix issue

* recover stream scan next function

* fix issue

* fix issue

* fix issue

* ignore disorder data

* fix issue

* fix issue for interval sliding

* fix issue

* fix ci issue

* fix ci issue

* fix ci issue

* add semi && final nonblock interval operator

* fix issue

* fix issue

* fix issue

* fix issue

* fix issue

* fix issue

* fix issue

* fix issue

* refactor(stream): track the msgId for each upstream tasks.

* fix(stream): fix race condition.

* fix(stream): update the task last msgId when putting into input queue succ.

* fix issue

* fix issue

* put recalculate data to rocksdb

* fix issue

* fix issue

* enh(query)[TD-33071]: add support for saving and restoring tsdbReader scan progress

- Implement functionality to save scan progress during tsdbReader operations
- Enable resuming scans from the last saved position

* fix issue

* fix issue

* fix issue

* fix issue

* fix issue

* add rec interval check

* enh(stream):add recalculate tasks.

* enh(stream): support the re-calculate the tasks.

* fix issue && do refactor

* do refactor

* fix issue

* fix issue

* update backend opt

* add new interface

* add new session operator

* support blob

* add new session operator

* fix issue

* add rec state for rec task

* fix invalid read

* add new session window

* enh(stream): update the stream tasks backend.

* new session operator

* add pull data

* fix(stream): fix error in expand stream backend.

* fix issue

* fix issue

* fix issue

* merge code

* fix issue

* fix(stream): check for null ptr.

* fix(stream): add more check.

* fix issue

* fix issue

* fix issue

* add debug code

* fix issue

* fix issue

* fix issue

* set rec end flag

* fix(stream): opt re-calculate stream tasks.

* fix issue

* fix issue

* add new operator

* enh(stream): dispatch recalculate block to agg tasks.

* fix issue

* fix issue

* fix(stream): adjust filter.

* fix issue

* refactor

* refactor(stream): adjust the recalculate end block.

* fix issue

* fix(stream): set correct create trigger block.

* fix issue

* fix(stream): fix error in build recalculate end block.

* fix(stream): check null ptr.

* add stream client && fix issue

* fix mem leak

* fix(stream): free msg.

* add stream client

* fix(stream): fix error.

* add stream client && fix issue

* add stream client

* refactor(stream): set the recalculate task complete.

* add wend and group_id for session window dest table

* feat(stream): refactor and set the recalcul agg task complete.

* add cfg for adapter

* fix issue

* add state && event operator

* feat(stream): support fill-history task.

* add new fill operator

* fix(stream): set correct backend when updating fill-history task to recalculate task.

* add new fill operator

* fix(stream): s2025-03-06 11:10:31.272
et ops always open in scan tsdb

* fix(stream):set the correct taskType for sink task.

* new fill operator

* adj stream fill operator

* fix issue

* fix issue

* fix issue

* fix issue

* fix issue

* fix issue

* fix issue

* fix issue

* adj test

* fix issue

* fix(stream): fix issue

* fix(steam): fix issue

* fix(steam): fix issue

* fix(steam): fix issue

* fix(steam): fix issue

* fix(stream): fix issue

* fix(stream): fix issue

* fix(stream): fix issue

* fix: ut com error

* fix(stream): fix mem leak and adjust operator type check rule

* fix(stream): fix mem leak and adjust test case

* refactor code

* fix(stream): free items.

* fix(stream): free fix memory leak.

* fix(stream): fix syntax error.

* fix: ignore unexpect block

* fix: adjust op type

---------

Co-authored-by: Haojun Liao <hjliao@taosdata.com>
Co-authored-by: Jinqing Kuang <kuangjinqingcn@gmail.com>
Co-authored-by: yihaoDeng <luomoxyz@126.com>
2025-03-14 20:14:01 +08:00
wangjiaming
f50fcb85b8
feat(decimal): support decimal data type (#30060)
* decimal: create table

* decimal: add test case decimal.py

* decimal: add decimal.c

* support input decimal

* decimal test

* refactor svalue

* fix test cases

* add decimal unit test

* add decimal test cmake

* support insert and query decimal type

* define wide integer, support decimal128

* support decimal128 divide

* set decimal type expr res types

* scalar decimal

* convert to decimal

* fix decimal64/128 from str and to str

* fix decimal from str and decimal to str

* decimal simple conversion

* unit test for decimal

* decimal conversion and unit tests

* decimal + - * /

* decimal scalar ops and comparision

* start to refactor GET_TYPED_DATA

* support decimal max func, cast func

* refactor GET_TYPED_DATA interface

* decimal scalar comparision

* start to implement sum for decimal

* support sum and avg for decimal type

* decimal tests

* add decimal test

* decimal add test cases

* decimal use int256/int128

* decimal testing

* fix decimal table meta and add tests for decimal col streams

* fix create stream and create tsma

* test insert decimal values

* decimal from str

* test decimal input

* test parse decimal from string

* add taos_fetch_field_e api

* decimal insert tests

* test decimal operators

* decimal operator test

* feat:support decimal in raw block

* decimal operator tests

* decimal test

* feat:support decimal in raw block

* feat:support decimal in raw block

* feat:add schemaExt to SMqDataRsp

* feat:remove add schemaExt to SMqDataRsp

* feat:remove add schemaExt to SMqDataRsp

* feat:remove add schemaExt to SMqDataRsp

* decimal test operators

* decimal operator test

* test decimal operators

* test decimal compare operators

* decimal unary operator test

* decimal col with decimal col oper test

* test decimal col filtering

* fix decimal float operator test

* decimal test where filtering

* fix decimal filtering

* fix decimal order by

* fix decimal op test

* test decimal agg funcs

* test decimal functions

* remove assert

* fix ci build for ret check

* fix decimal windows build

* fix ci ret check

* skip decimal ret check

* skip decimal ret check

* fix decimal tests

* fix decimal ci test

* decimal test

* fix(tmq): heap user after free

* fix(tmq): double free

* fix(tmq): double free

* fix decimal tests

* fix(decimal): decimal test ci build

* fix(decimal): windows build

* fix(decimal): decimal test build

* fix(decimal): fix decimal build and tests

* fix(decimal): fix decimal tests

* fix(decimal): fix taos_fetch_fields_e api

* fix(decimal): fix decimal taos_fetch_fields_e api

* fix(decimal): rebase 3.0

* fix(decimal): fix decimal functions

* fix(decimal): fix decimal test case memory leak

* fix(decimal): fix decimal tests

* fix(decimal): fix decimal test case

* fix(decimal): fix decimal tests

* feat(decimal): fix unit tests

* feat(decimal): fix deicmal unit test

---------

Co-authored-by: wangmm0220 <wangmm0220@gmail.com>
Co-authored-by: yihaoDeng <yhdeng@taosdata.com>
2025-03-14 18:08:07 +08:00
Simon Guan
1170955b95 Merge branch 'main' into merge/mainto3.0 2025-03-10 11:05:52 +08:00
Jinqing Kuang
7a7aacd6cc fix(query): change error log to warning when failed to acquire tsdb reader
Jira: https://jira.taosdata.com:18080/browse/TS-6066
2025-03-06 14:23:00 +08:00
Simon Guan
76ad364b93 enh: adjust some logs 2025-02-27 09:55:11 +08:00
Jinqing Kuang
d32cf8bcb8 fix(query)[TS-5984]. correct data block filtering logic in tsdbReader
When scanning a table with a primary key, tsdbReader could load
irrelevant data blocks outside the query time range, causing errors
when returning results.

This fix improves filtering logic to ensure only valid blocks are
read and adds test verification.
2025-02-08 17:32:25 +08:00
Shengliang Guan
bc17e0aa9a
Merge pull request #29328 from taosdata/fix/TS-5812-main
fix[TS-5812]:add params check
2024-12-26 11:35:06 +08:00
wangmm0220
a95304a857 fix[TS-5812]:add params check 2024-12-25 11:59:48 +08:00
Jinqing Kuang
4f71d4a3f5 enh(query)[TD-33268]. add unit tests to increase test coverage 2024-12-22 14:23:05 +08:00
Jinqing Kuang
ab7badb0c0 fix(query)[TD-33006]. resolve wild pointer release issue in tsdbCreateReader
Initialize pointer member variables in tsdbCreateReader to prevent
random memory errors. It addresses a bug where uninitialized pointers
are freed during error cleanup.
2024-11-25 15:01:49 +08:00
Jinqing Kuang
fbc77d1e71 fix(query)[TD-33006]. resolve wild pointer release issue in tsdbCreateReader
Initialize pointer member variables in tsdbCreateReader to prevent
random memory errors. It addresses a bug where uninitialized pointers
are freed during error cleanup.
2024-11-25 14:58:42 +08:00
Shengliang Guan
e8d663809d
Merge pull request #28756 from taosdata/fix/main/tsbs_perf_optimization
fix tsbs perf issue
2024-11-21 10:40:47 +08:00
wangjiaming0909
9314dfb80c fix not setting step in tsdbread 2024-11-20 14:29:27 +08:00
Shengliang Guan
99bebacd8e
Merge pull request #28784 from taosdata/fix/TD-32861-main
fix(query)[TD-32861]. fix that WHERE condition not work in statements with INTERP function
2024-11-19 09:40:28 +08:00
Jinqing Kuang
f5c66cec19 fix(query)[TD-32861]. fix that WHERE condition not work with INTERP function
Previously, a statement with INTERP would force underlying TABLE SCAN
operator to scan all data within the RANGE clause, causing the time
range constraint in the WHERE condition to be ignored. This fix ensures
that the TABLE SCAN respects both the RANGE clause and WHERE condition,
improving query accuracy and performance.
2024-11-18 16:08:27 +08:00
wangjiaming0909
6c18289de0 fix tsbs perf issue 2024-11-18 14:11:21 +08:00
wangjiaming0909
2be0ca15d8 fix group_partition test 2024-11-15 18:10:44 +08:00
Jinqing Kuang
b4e0f9a922 enh(query)[TD-32127]: improve code quality of tsdbRead module
- Add parameter validation for functions to ensure robustness
- Nullify pointers when freeing memory to prevent dangling pointers
- Initialize local variables to avoid undefined behavior
- Add error handling and logging for better debugging
2024-11-11 11:53:28 +08:00
Haojun Liao
27c087e9ae refactor: do some internal refactor. 2024-10-16 18:36:49 +08:00
Haojun Liao
5e94ed5e25 Merge branch '3.0' into fix/syntax 2024-10-10 14:53:37 +08:00
Haojun Liao
5dc933f5f1 refactor: add some logs. 2024-10-10 14:52:51 +08:00
lyh250-666
676f8759e2 enh:modify error code passing 2024-10-09 15:37:00 +08:00
Haojun Liao
e2e1490f78 refactor: do some internal refactor. 2024-09-30 15:11:35 +08:00
Haojun Liao
e8ad3f4251 fix(query): return correct error code. 2024-09-30 15:07:16 +08:00
lyh250-666
5f6802e0a1 enh:modify addede rror code passing 2024-09-24 16:29:28 +08:00
Haojun Liao
60308ae844 fix(tsdb): return value if failed. 2024-09-23 14:05:36 +08:00
Haojun Liao
72d44d4207 other: merge 3.0 2024-09-23 11:05:31 +08:00
Haojun Liao
b8f06a06db fix(tsdb): check the return value. 2024-09-23 10:57:19 +08:00
Pan Wei
fa9f8f257b
Merge pull request #27984 from taosdata/enh/TD-31691
enh:modify error code passing
2024-09-20 08:44:07 +08:00
lyh250-666
ae7dde1aa2 enh:modify error code passing 2024-09-19 19:51:51 +08:00
Jinqing Kuang
4b46c4c1fc enh(query)[TD-32130]. Handle return values of memory allocation functions 2024-09-19 09:06:01 +08:00
Haojun Liao
1a90e9612d refactor: check return value. 2024-09-13 23:04:41 +08:00
Pan Wei
c35ec8e07c
Merge pull request #27820 from taosdata/fix/TD-32027-3.0
fix(query)[TD-32027]. Fix null pointer access in exceptional cases
2024-09-13 09:07:08 +08:00
Jinqing Kuang
7b4116b28e fix(query)[TD-32027]. Fix null pointer access in exceptional cases 2024-09-12 19:47:02 +08:00
Haojun Liao
4319cacdc3 fix(query):fix invalid read. 2024-09-11 19:09:17 +08:00
Hongze Cheng
37c12b6c3a enh: error handling 2024-09-10 18:55:02 +08:00
Hongze Cheng
63b64a90c0 enh: remove void with error code handle 2024-09-10 13:12:05 +08:00
Haojun Liao
3540ec9d44
Merge pull request #27742 from taosdata/fix/syntax
refactor: remove void.
2024-09-10 12:57:14 +08:00
Jinqing Kuang
1ef8574429 fix(query)[TD-31971]. Fix potential wild pointer access in tsdbReaderClose
Resolved an issue in tsdbReaderClose where wild pointer access could
occur in cases of insufficient memory or other exceptions.
2024-09-09 18:11:52 +08:00
Pan Wei
07424b7a06
Merge pull request #27719 from taosdata/fix/TD-31790/errCheck
fix: distinguish the calling scenarios of the metaGetTbTSchema function
2024-09-09 14:10:20 +08:00
Haojun Liao
31ab46f210 other: merge 3.0 2024-09-08 22:52:17 +08:00
Haojun Liao
06f42443cb refactor: remove void. 2024-09-08 22:49:05 +08:00
Haojun Liao
0236db51f6 fix(query): fix memory leak and do some internal refactor. 2024-09-06 15:44:25 +08:00
xsren
3fb89fb814 fix: distinguish the calling scenarios of the metaGetTbTSchema function 2024-09-06 14:16:58 +08:00
Jinqing Kuang
ad5aa4b032 fix(query)[TD-31929]. Fix mem-leak caused by redundant allocations 2024-09-05 16:58:20 +08:00