Commit graph

136 commits

Author SHA1 Message Date
dapan1121
009e8aa41e Merge remote-tracking branch 'origin/3.0' into feat/TS-6100-3.0 2025-05-12 16:24:22 +08:00
Jing Sima
6ae805bb7a feat: [TS-6100] Process SCMCreateStreamReq ver-4
support placeHolderBitmap and interval offset in stream trigger.
remove useless code.
2025-04-29 15:19:30 +08:00
Jing Sima
f84f04398d
fix: [TS-4897] Support query stables cross databases and fix potential deadlock. (#30507) 2025-03-28 10:07:08 +08:00
Jinqing Kuang
ac23a4fc55
feat(stream): support stream processing for virtual tables (#30144)
* enh: add client processing

* enh: add mnode vtables processing

* enh: add mnode vtable processing

* enh: add normal child vtable support

* fix: compile issues

* fix: compile issues

* fix: create stream issues

* fix: multi stream scan issue

* fix: remove debug info

* fix: agg task and task level issues

* fix: correct task output type

* fix: split vtablescan from agg

* fix: memory leak issues

* fix: add limitations

* Update 09-error-code.md

* Update 09-error-code.md

* fix: remove usless case

* feat(stream): extract original table data in source scan task

Implemented functionality in the source task to extract data
corresponding to the virtual table from the original table using WAL.
The extracted data is then sent to the downstream merge task for further
processing.

* feat(stream): multi-way merge using loser tree in virtual merge task

Implemented multi-way merge in the merge task using a loser tree to
combine data from multiple original table into a single virtual table.
The merged virtual table data is then pushed downstream for further
processing.  Introduced memory limit handling during the merge process
with configurable behavior when the memory limit is reached.

* fix(test): remove useless cases

---------

Co-authored-by: dapan1121 <wpan@taosdata.com>
Co-authored-by: Pan Wei <72057773+dapan1121@users.noreply.github.com>
2025-03-15 17:06:48 +08:00
wangjiaming
f50fcb85b8
feat(decimal): support decimal data type (#30060)
* decimal: create table

* decimal: add test case decimal.py

* decimal: add decimal.c

* support input decimal

* decimal test

* refactor svalue

* fix test cases

* add decimal unit test

* add decimal test cmake

* support insert and query decimal type

* define wide integer, support decimal128

* support decimal128 divide

* set decimal type expr res types

* scalar decimal

* convert to decimal

* fix decimal64/128 from str and to str

* fix decimal from str and decimal to str

* decimal simple conversion

* unit test for decimal

* decimal conversion and unit tests

* decimal + - * /

* decimal scalar ops and comparision

* start to refactor GET_TYPED_DATA

* support decimal max func, cast func

* refactor GET_TYPED_DATA interface

* decimal scalar comparision

* start to implement sum for decimal

* support sum and avg for decimal type

* decimal tests

* add decimal test

* decimal add test cases

* decimal use int256/int128

* decimal testing

* fix decimal table meta and add tests for decimal col streams

* fix create stream and create tsma

* test insert decimal values

* decimal from str

* test decimal input

* test parse decimal from string

* add taos_fetch_field_e api

* decimal insert tests

* test decimal operators

* decimal operator test

* feat:support decimal in raw block

* decimal operator tests

* decimal test

* feat:support decimal in raw block

* feat:support decimal in raw block

* feat:add schemaExt to SMqDataRsp

* feat:remove add schemaExt to SMqDataRsp

* feat:remove add schemaExt to SMqDataRsp

* feat:remove add schemaExt to SMqDataRsp

* decimal test operators

* decimal operator test

* test decimal operators

* test decimal compare operators

* decimal unary operator test

* decimal col with decimal col oper test

* test decimal col filtering

* fix decimal float operator test

* decimal test where filtering

* fix decimal filtering

* fix decimal order by

* fix decimal op test

* test decimal agg funcs

* test decimal functions

* remove assert

* fix ci build for ret check

* fix decimal windows build

* fix ci ret check

* skip decimal ret check

* skip decimal ret check

* fix decimal tests

* fix decimal ci test

* decimal test

* fix(tmq): heap user after free

* fix(tmq): double free

* fix(tmq): double free

* fix decimal tests

* fix(decimal): decimal test ci build

* fix(decimal): windows build

* fix(decimal): decimal test build

* fix(decimal): fix decimal build and tests

* fix(decimal): fix decimal tests

* fix(decimal): fix taos_fetch_fields_e api

* fix(decimal): fix decimal taos_fetch_fields_e api

* fix(decimal): rebase 3.0

* fix(decimal): fix decimal functions

* fix(decimal): fix decimal test case memory leak

* fix(decimal): fix decimal tests

* fix(decimal): fix decimal test case

* fix(decimal): fix decimal tests

* feat(decimal): fix unit tests

* feat(decimal): fix deicmal unit test

---------

Co-authored-by: wangmm0220 <wangmm0220@gmail.com>
Co-authored-by: yihaoDeng <yhdeng@taosdata.com>
2025-03-14 18:08:07 +08:00
WANG MINGMING
4a3eda51e0
Merge branch '3.0' into feat/TD-32642 2024-12-10 14:11:47 +08:00
Shengliang Guan
e99b888654 Merge branch 'main' into merge/mainto3.0 2024-12-09 19:06:06 +08:00
wangjiaming0909
2bf3da7415 fix tests 2024-12-04 14:51:10 +08:00
wangjiaming0909
9a01b42bc2 replace unsafe funcs 2024-12-02 16:20:40 +08:00
wangmm0220
0e2a16d70f feat:[TD-32642] add charset for connection support 2024-11-27 14:25:34 +08:00
Pengrongkun
6b2846f941 fix segmentation fault 2024-11-19 22:25:46 +08:00
wangjiaming0909
9a92c136ce fix drop child table with tsma 2024-11-08 09:47:47 +08:00
wangjiaming0909
5955d121b3 revert first/last fetch 2024-11-08 09:42:59 +08:00
wangjiaming0909
7f501130a2 client add query fisrt/last/count for drop table 2024-11-08 09:42:58 +08:00
kailixu
207c7aded0 enh: support drop table with uid 2024-09-18 19:38:57 +08:00
xsren
78e9bb95ac osString 2024-09-12 14:02:19 +08:00
lyh250-666
01bae36e82 enh:modify error code passing in libs/ 2024-09-10 18:42:00 +08:00
Minglei Jin
298c4b4812 feat(stmt2): initial commit for stmt2 2024-08-15 09:28:32 +08:00
wangjiaming0909
4d5cd9cd1a planner/nodes/parser ret check 2024-07-26 17:57:27 +08:00
wangjiaming0909
1994c3339b nodes add ret check 2024-07-26 17:55:48 +08:00
Shungang Li
942599fe04 enh: 'create table' parses csv file per tsMaxInsertBatchRows 2024-07-11 16:14:31 +08:00
xjzhou
42361fcd45 fix: failed to identify dbName when only ? exists 2024-06-26 16:38:21 +08:00
xjzhou
b444e18d90 enh: stmt supports 'insert into db.?' 2024-06-25 16:16:32 +08:00
wangjiaming0909
d8200b2f63 fetch lastts vnode version before create tsma 2024-04-16 14:18:10 +08:00
wangjiaming0909
70ca830378 support recursive tsma creation 2024-04-16 14:17:12 +08:00
wangjiaming0909
ef47fd57d6 support tsma catalog cache 2024-04-16 14:14:34 +08:00
Haojun Liao
cb2ea4a721 fix(stream): take ver snapshot for all vgroups before launching stream with fill-history option opened. 2024-02-01 11:37:57 +08:00
dapan1121
099f72e84a fix: memory issues 2023-10-27 11:36:17 +08:00
dapan1121
5b639a172d feat: support query from view 2023-10-08 18:36:06 +08:00
wangmm0220
34c7832850 feat:support stmt/tmq json in varbinary type 2023-08-29 15:24:12 +08:00
wangjiaming0909
1f71ce9409 feature: get last timestamp before create sma index 2023-07-06 10:33:22 +08:00
dapan1121
170182fecd feat: support fill history with sub request 2023-06-12 16:10:54 +08:00
Dingle Zhang
984211f3b4
Feature/3.0 geometry (#21037)
* Add GEOMETRY data type and make sql.c able to parse it. The GEMETRY works like BINARY so far.

* add GEOMETRY type into gConvertTypes to fix some issues like DELETE calling

* change some test cases to make sure no same timestamp is inserted, and add my smoketest.sh

* Add a function MakePoint() and introduce a lib geometry

* implement sql functions GeomFromText() and AsText()

* Use GEOS *_r funcions instead for thread safety

* Handle with TSDB_DATA_TYPE_GEOMETRY when INSERT geometry data by converting WKT.
Add geosWrapper to wrap the basic GEOS functions for TDEngine.

* refactor AsText and MakePoint functions to be like GeomFromText

* Show WKT when print geometry data in screen
Dump hex data when dump geometry data in a file

* define TYPE_BYTES item for TSDB_DATA_TYPE_GEOMETRY, which casued some strange issues.

* set number of decimals of WKT to 6

* Implement SQL function Intersects()

* refactor geometry sql functions

* Add geosErrMsgeHandler() to get the GEOS error detail

* use threadlocal to instantiate SGeosContext
call destroyGeosContext() only if the thread exists

* remove SGeosContext *context param for all geometry functions since we use thread local one,
so that all caller do not need to know the context.

* Modify Intersects() to call PreparedIntersects() when one of param is a constant, which has higher performance.

* rename prepareFn() to initCtxFn() to avoid confusion with PreparedFn

* Add prefix "ST_" for all geometry functions

* move getThreadLocalGeosCtx() and destroyThreadLocalGeosCtx() into util,
so that all unit test tools can compile

* Add unit test for geometry lib, only test MakePoint so far

* refactor and enhance existing cases in geomFuncTest

* implement NULL type and NULL value test for geomFuncTest

* add test on geomFromText()

* add unit test on AsText() in geomFuncTest

* combine some makePointFunction test items

* add intersectsFunctionTwoColumns test
refactor on callGeomFromTextWrapper functions

* enhance intersectsFunction test to add cases like input constant , NULL type, NULL value, or wrong content

* add more cases into intersectsFunction test

* Add basic test on geometry in system test

* Add ST_GeomFromText and ST_AsText function test in system test on geometry

* add ST_Intersects function test in system test on geometry

* support to check expectedErrno in system test on geometry

* adjust geomTest unit test and geometry system test

* add geometry data type and functions in doc english version

* implement touchesFunction() in geometry lib
refactor geometry relation functions model

* separate gemFuncTest into several src files

* add unit test on touchesFunction

* support sql function ST_Touches()
add system test on ST_Touches

* add docs for ST_Touches()

* Add ST_Contains()

* Add ST_Covers()

* Add ST_Equals()

* add swapAllowed param for geomRelationFunction()
read geom2 earlier intead of at doGeosRelation()

* Add ST_ContainsProperly()

* build on windows

* Merge from 3.0 to 3.0_geometry

* change macro definition TSDB_DATA_TYPE_GEOMETRY as the last one for compatibility

* change '\\NULL' to 'NULL' back in shellDumpFieldToFile()

* add /usr/local/include into include directory

* add /usr/local/inlcude and /usr/local/lib in cmake.platform for DARWIN
2023-05-24 15:36:46 +08:00
Xiaoyu Wang
115160bbd0 fix: insert syntax error 2023-02-05 10:11:26 +08:00
Xiaoyu Wang
8513dc1067 enh: insert optimize 2022-11-04 15:21:38 +08:00
Xiaoyu Wang
0563eb2475 feat: support batch loading of csv files 2022-10-11 17:28:58 +08:00
Xiaoyu Wang
18fa48ae57 feat: support batch loading of csv files 2022-10-11 15:54:01 +08:00
Xiaoyu Wang
876303be56 enh: added memory allocators for parser and planner 2022-09-21 17:42:03 +08:00
Xiaoyu Wang
5116e0a01b enh: added memory allocators for parser and planner 2022-09-21 17:04:12 +08:00
dapan1121
205299e532 fix: fix taosd stmt query crash issue 2022-08-29 17:46:34 +08:00
Xiaoyu Wang
4290582249 enh: insert optimize 2022-08-10 17:47:28 +08:00
Xiaoyu Wang
8fc86a31d0 enh: asynchronous catalog interface optimization 2022-08-04 20:53:43 +08:00
Xiaoyu Wang
dad14a28c9 merge 3.0 2022-07-21 19:24:14 +08:00
Xiaoyu Wang
c54daccf2d fix: plan problem caused by project eliminate optimize 2022-07-21 14:35:36 +08:00
Xiaoyu Wang
37ec05a715 fix: memory leak problems of parser and planner 2022-07-20 17:20:29 +08:00
dapan1121
1444054437 fix: fix stmt memory leak 2022-07-18 18:57:47 +08:00
Xiaoyu Wang
6a845b746c fix: some problems of parser 2022-07-11 15:15:39 +08:00
Xiaoyu Wang
72738040a9 fix: some problem of parser and planner 2022-07-10 16:44:50 +08:00
dapan1121
f3c8bcb953 feat: insert from query 2022-07-07 09:42:20 +08:00
Xiaoyu Wang
331dca1f58 feat: sql command 'insert ... select' 2022-07-06 10:02:20 +08:00