Commit graph

5040 commits

Author SHA1 Message Date
Kaili Xu
f1e480ef53
enh: support revoke sign in show grants logs (#30929) 2025-04-29 14:46:27 +08:00
Kaili Xu
ee289615e8
enh: behavior of negative logKeepDays identical to positive values (#30939) 2025-04-29 14:13:34 +08:00
Kaili Xu
b88ec2f1a8
fix: timeseries statis for virtual table (#30911) 2025-04-27 18:03:20 +08:00
Pan Wei
00e0c4ef25
Merge pull request #30876 from taosdata/fix/TS-6321
fix(tmq): add log
2025-04-25 17:13:56 +08:00
wangmm0220
24be8c8cfa fix: ci case error in sometime 2025-04-24 19:55:27 +08:00
WANG MINGMING
ddd32e23d9
fix(tmq): init tmq handle error if snapshotVer is out of range (#30859) 2025-04-24 09:50:51 +08:00
WANG MINGMING
88eec0cddd
fix: (TS-6376)jump virtual table if snapshot is true in tmq (#30835) 2025-04-19 17:39:36 +08:00
Simon Guan
75df845c05
fix: show create table and desc table statements return composite key instead of primary key (#30830) 2025-04-19 13:32:34 +08:00
Zhixiao Bao
e25da1cd35
fix: taosd fails to start after modifying the timezone configuration parameter. 2025-04-17 19:19:05 +08:00
wangjiaming
7ba3fefde2
fix fill range check for natural units (#30789) 2025-04-16 15:59:55 +08:00
WANG Xu
26e0e619df
Merge pull request #30709 from taosdata/enh/wait_checkpoint_ready_main
enh: wait checkpoint ready before pause stream
2025-04-11 08:58:02 +08:00
Simon Guan
50d3705bea
enh[TS-6318]: changed the keyword for composite primary keys from "primary key" to "composite key" (#30741) 2025-04-10 20:31:35 +08:00
haoranchen
b730f310b3
refactor: remove unused import of 'insert' from numpy in multiple test files (#30744) 2025-04-10 18:05:15 +08:00
Zhixiao Bao
65d5df68db
test: fix test_dismatch_config.py. (#30737) 2025-04-10 15:49:34 +08:00
WANG MINGMING
135a1f06be
fix(stream): disable stream check in tmq interface (#30705) 2025-04-10 10:36:44 +08:00
Zhixiao Bao
34a41bf54c
fix: crash while upgrade taosd to 3362. (#30712) 2025-04-09 18:53:47 +08:00
jiajingbin
5cab78f295 enh: wait checkpoint ready before pause stream 2025-04-09 11:16:29 +08:00
wangjiaming
2ca729532b
fix(fill2): fix fill2 test case (#30658) 2025-04-07 18:46:51 +08:00
wangjiaming
b3b6c3d3ff
fix(fill): fix fill next/prev got different result with asc/desc order 2025-04-03 21:24:39 +08:00
xinsheng Ren
df3b482c9e
fix: test case don't check debugFlag value (#30613) 2025-04-02 19:27:57 +08:00
xinsheng Ren
463430ab15
enh: [TS-6195] support cols in stream (#30580) 2025-04-02 11:31:08 +08:00
haoranchen
55c5f8190d
test: add python script for windows cases to replace the bat (#30386)
* test: add python scripts for  test windows cases  replace the old bat scripts

* refactor: update color_echo function to use log_file instead of error_file

* ci: update workflow to use fixed branch for Windows test runs

* chore: add empty line for better readability in tdengine-test.yml

* test:fix  from branch fix/xftan/taos-shell-configdir

* fix: replace global variant configDir with local variant on taos

* fix: remove check cfgdir exist

* fix: restore no input msg

* fix: move dumpConfig after taos_options

* fix: dump Config on windows failed

* fix: move configDirShell variant to shellArgument.c

* fix: get last char pos is -1

* ci: update Windows test workflow to use main branch and enhance test output

* ci: remove unnecessary line from tdengine-test workflow

---------

Co-authored-by: Alex Duan <417921451@qq.com>
2025-03-24 21:19:30 +08:00
Linhe Huo
e273a943ea
fix: add show connMode string (#30323)
* fix: add show connMode string

* fix: add stdbool.h to pub.h

* fix: remove trash file army/output.txt

* fix: caseBase.py modify syntax error

* fix: restore -R option for taosdump

* fix: taosdumpCommandline.py case

* fix: native stmt write normal table failed

* fix: taosdumpCommandline.py case passed

* fix: restore test.py from main branch

* fix: taosCli.py check default conn mode

* fix: commandline-sml.py case pass

* fix: websiteCase.py case passed

* fix: connMode.py case

* fix: modify default port is 0

* fix: taos_options with config dir not work

* fix: websocket.py delete -D timeout options

* fix: default_tmq_json.py context move to default_json.py, so delete

* fix python kafka bug

* chore: improve taos_init in wrapper

* chore: add installation path preparation in build workflow

* fix connMode bug

* fix: fix tmq conf/consumer new error in wrapperFunc.c

* fix: correct the spelling toss -> taosGetInstall...

* chore: fix compile error in wrapperFunc.c

* fix: createConnect fix memory leak

* fix: tsim forbid CHECK  ODR

* modify userOperTest uuse static lib

* reverse userOperTest use static lib

---------

Co-authored-by: Alex Duan <417921451@qq.com>
Co-authored-by: taos-support <it@taosdata.com>
Co-authored-by: “chris <“zk662144@163.com”>
Co-authored-by: t_max <1172915550@qq.com>
Co-authored-by: sheyanjie-qq <249478495@qq.com>
2025-03-22 20:44:07 +08:00
Alex Duan
c133489917
new-libtaos-with-ws (#29971)
* enh: rename libtaos.so to libtaosinternal.so

* enh: let python system-test work

* enh: shell for libinternal.so

* enh: refact script

* enh: wrapper for libtaosinternal

* enh: rename some files

* enh: let shell support internal driver

* enh: minor changes

* enh: minor changes

* add intenal.h

* enh: remove unused codes

* enh: minor changs

* feat: mac os issues

* enh: let python test use internal connection

* enh: let nettest work

* enh: let taosc -C work

* enh: test shell in mac os

* enh: minor changes

* enh: let libtaosinternal.so work in linux

* enh: update install script

* enh: for ci

* enh: for windows compile

* enh: minor changes

* enh: compile in windows

* enh: minor changes

* enh: taosinternal work in windows

* enh: add taos_internal_static

* enh: rollback os_for_wrapper

* fix: conflicts

* fix: conflicts

* fix: compile errors

* fix: conflicts

* enh: rename taosinternal to taosnative

* enh: rename taosinternal to taosnative

* fix: compile error

* enh: remove taosnative.h

* fix: compile errors

* enh: set default shell options

* fix: compile errors

* fix: compile errors

* debug: switch taosws branch to feat/new-libtaos-with-ws

* enh: taosBenchmark remove WEBSOCKET finished

* fix: CMakeLists.txt remove WEBSOCKET

* fix: compile errors

* fix: taosBenchmark delete restful code

* .gitignore remove taos-tools folder

* fix: remove taosdump WEBSOCKET MACRO

* fix: build error for taosdump

* fix: build project passed

* fix: support -Z for connect mode

* fix: build error

* fix: build error fixed

* fix: add -Z options

* fix: native and websocket with string

* fix: -Z option core

* fix: build unit test

* fix: build error

* fix: add colon for array

* fix: unit test can not include fun

* fix: CTest pointer null is nullptr

* fix: include pub.h

* fix: taosdump add -Z --driver options

* fix: taosdump support -Z with pub.h pub.c

* fix: toolsGetTimeDay move to pub.c

* fix: restore pub.c only simple fun

* fix: add error tips

* fix: tmfree remove from taosdump

* fix: remove double declare taos variant

* fix: taos support -Z option

* enh: configDir set with main fun

* fix: add g_arguments.configDir

* fix: show tips add cfgdir

* fix: g_argument is pointer

* fix: configDir is global var

* fix: declare g_configDir in benchMain.c

* fix: taos version not show on websocket

* stmt2 websocket prepare with supertable

* stmt2 websocket prepare build

* fix: stmt and stmt2 prepare add db name

* fix: remove -R --restful test case

* fix: taosBenchmark adjust priority cmd > json > evn

* fix: add connect mode test case connMode.py

* fix: clear evn and add host port check

* fix: army/test.py modify start taosAdapter is default

* change: system-test/test.py default start taosAdapter

* fix: add taosAdapter variant to system-test/test.py

* fix: -y must put behind other options

* fix: remove rest sml and add -N normal case

* fix: add -c work cfg test case

* del: queryMain.py remove rest query

* fix: remove groups have a blank append bug

* fix: caseBase.py add db in sql query

* fix: query_json.py remove rest test

* fix: taosBenchmark uniform dsn describe

* fix: add pub.h pub.c to tools public

* fix: case remove restful interface

* fix: build error

* fix: taosDriverCleanup no arg call

* fix: taosdump add test conn mode case taosdumpCommandline.py

* fix: modify insertFullType.json db name

* fix: taosdump connMode test case taosDumpCommandline.py passed

* enh: priority cmd > env > json

* fix: taosCli remove is_native variant

* fix: password is not pointer

* fix: comment -o test case

* fix: solve fun return code check

* fix: windows unresolved strcasecmp

* fix: remove coverHostToServAddr

* fix: forbid check ODR violation

* ci: trigger pr run

* fix: build error not del >>>>>3.0

* fix: support json dbinfo->vgroups have blank space

* fix: solve conflict with merge

* fix: with TrimCaseCmp fun to compare vgroups

* fix: del vgroups support blank code

* feat: wrapper add taos_fetch_fields_e

* fix: move jni from libtaosnative.so to libtaos.so

* fix: move clientTmqConnector.c for jni

* fix: fix assert check with del comment code

* fix: taosBenchmark add prepare stmt debug log

* fix: stmt2 already have debug log on preapare

* fix:  (double)LLONG_MAX force covert

* fix: stmt insert normal table with websocket

* fix: normal table with stmt too many ?

* fix: taosdump stmt_prepare with child tbname

* fix: build error add int code

* feat: stmtPrepare with child table to do

* taos -a options move to native test case

* fix:unit test benchmarkTest passed

* fix: query_json-with-sqlfile.py case no check results

* fix: queryMain.py case lost, put in tasks again

---------

Co-authored-by: Shengliang Guan <slguan@taosdata.com>
2025-03-20 22:54:29 +08:00
Jing Sima
7cffc7aff2
fix: [TD-34074] Forbid virtual table in tq and stmt. 2025-03-20 22:42:27 +08:00
liuyao
f7beaeebfe
ci: adjust test && fix issue
ci: adjust test && fix issue
2025-03-20 17:54:20 +08:00
Simon Guan
2c27c10fa8 Merge branch 'main' into merge/mainto3.0 2025-03-20 14:05:45 +08:00
xinsheng Ren
59f45345fb
fix: memleak while stop udfd
* fix: use address sanitizer to detect buffer overflow in udfTest.py

* fix: improve UDF process termination in udfTest.py

* fix: udf test case, stack overflow

---------

Co-authored-by: chenhaoran <haoran920c@163.com>
2025-03-20 13:45:33 +08:00
liuyao
fac36002c1
ci(stream): add ci and fix issue (#30270)
* ci(stream): add ci and fix issue

* ci: adj test

* ci: add test and fix issue

* ci: adj test

* ci(stream): add test for fill operator and fix issue

* fix: fix issue

* fix: fix issue

* fix: adjust defalut rec value
2025-03-20 10:34:31 +08:00
wangjiaming
77f9707f89
feat: add more test cases for decimal
* add decimal tests

* decimal: add more decimal tests

* test(decimal): add more tests for decimal

* fix(decimal): fix decimal.py

* test(decimal): fix decimal test

* fix(decimal): fix decimal3.py test case
2025-03-19 17:13:56 +08:00
Hongze Cheng
93825d49b1
feat: change the max value of minFreeDiskSize to 2TB (#30250)
* feat: change the max value of minFreeDiskSize to 2TB

* fix: CI test case change

* enh: remove useless comment
2025-03-19 14:03:37 +08:00
WANG MINGMING
3fafecb242
feat[TS-6137]: support sliding in force_window_close (#30203)
* feat[TS-6137]: support sliding in force_window_close

* feat[TS-6137]: support sliding in force_window_close

* feat(stream): force window close support interval sliding

* feat[TS-6137]: support sliding in force_window_close

* feat[TS-6137]: support sliding in force_window_close

* feat[TS-6137]: support sliding in force_window_close

---------

Co-authored-by: 54liuyao <54liuyao@163.com>
2025-03-19 10:22:06 +08:00
Simon Guan
718d455a97 Merge branch 'main' into merge/mainto3.0 2025-03-18 14:35:54 +08:00
WANG MINGMING
3bc6635de3
feat[TS-6115]: support deletion of topics with active consumers. (#30232)
* feat[TS-6115]: drop topic & consumer group in force

* feat[TS-6115]: drop topic & consumer group in force

* feat[TS-6115]: drop topic & consumer group in force
2025-03-18 14:11:00 +08:00
xinsheng Ren
8453726080
fix: when querying super table, using ORDER BY and LIMIT together may return an unexpected record.
* fix: limit error

* fix: limit error, test case.
2025-03-18 13:58:30 +08:00
WANG MINGMING
930b57b770
feat: support fill_history 1 async when creating stream (#30185)
* feat:[TS-5617]use last_row cache mode if creating stream in fill_history

* feat(stream): optimize client logic of creating stream if with history

* feat(stream): optimize client logic of creating stream if with history

* feat(stream): optimize client logic of creating stream if with history

* feat(stream): optimize client logic of creating stream if with history

* feat(stream): optimize client logic of creating stream if with history

* feat(stream): optimize client logic of creating stream if with history

* feat(stream): optimize client logic of creating stream if with history

* feat(stream): optimize client logic of creating stream if with history

* feat(stream): optimize client logic of creating stream if with history

* feat(stream): optimize client logic of creating stream if with history

* feat(stream): optimize client logic of creating stream if with history

* fix: heap use after free

* feat: add log

* fix: ci case error

* fix: compile error in windows

* fix: ci case error

* fix: heap user after free

* fix: memory leak

* fix: ci case error

* fix: ci case error

* fix: ci case error

* feat: support fill_history 1 async when creating stream

* feat: support fill_history 1 async when creating stream

* fix: compile error

* feat[TS-5617]: support fill_history 1 async when creating stream

---------

Co-authored-by: yihaoDeng <yhdeng@taosdata.com>
2025-03-18 10:01:33 +08:00
Haojun Liao
dfe32225de chore: merge main. 2025-03-17 14:51:11 +08:00
wangjiaming
40d32e16b0
fix(TS-6117): fix drop table name with star crash (#30204) 2025-03-17 14:13:03 +08:00
Haojun Liao
ab92886820
fix(stream): reduce the consensus checkpoint id trans. (#30105)
* fix(stream): reduce the consensus checkpoint id trans.

* refactor(stream): add some logs.

* refactor(stream): set the max checkpoint exec time 30min.

* refactor(stream): add checkpoint-consensus trans conflict check.

* refactor(stream): remove unused local variables.

* fix(stream): fix syntax error.

* fix(stream): 1. fix free memory error 2. continue if put result into dst hashmap failed.

* fix issue

* fix issue

* fix(mnd): follower mnode not processes the timer event.

* fix(stream): print correct error msg.

* fix(stream): add some logs.

* fix(stream): add some logs.

* fix(stream): add some logs.

* fix(stream): add some logs.

* fix(stream): add some logs.

* fix(stream): add some logs.

* fix(stream): add some logs.

* fix(stream): add some logs.

* fix(stream): truncate long subtable name

* fix(stream): add buffer len.

* refactor(stream): update some logs.

* fix issue

* refactor(stream): update some logs.

* refactor(stream): update some logs.

* fix(stream): check return value.

* fix(stream): fix syntax error.

* fix(stream): check return value.

* fix(stream): update the timer check in mnode.

* fix(stream): add restart stage tracking.

* fix(stream): track the start task stage for meta.

* fix(stream): fix error in log.

* refactor(stream): adjust log info.

* fix mem issue

* fix(stream): check the number of required tasks for consensus checkpointId.

* fix(stream): lock the whole start procedure.

* fix(stream): add lock during start all tasks.

* fix(stream): update logs.

* fix(stream): update logs.

* fix(stream): update logs.

* fix(stream): fix dead-lock.

* fix(stream): fix syntax error.

* fix(stream): not drop the scan-history task.

* fix(stream): fix syntax error.

* fix(stream): wait for executor stop before restarting.

* fix(stream): add some logs.

* fix(stream): add some logs.

* fix(stream): add some logs.

* fix(stream): disable some logs.

* fix(stream): reset the start info if no task left.

---------

Co-authored-by: 54liuyao <54liuyao@163.com>
Co-authored-by: Jinqing Kuang <kuangjinqingcn@gmail.com>
2025-03-17 10:20:17 +08:00
Jinqing Kuang
ac23a4fc55
feat(stream): support stream processing for virtual tables (#30144)
* enh: add client processing

* enh: add mnode vtables processing

* enh: add mnode vtable processing

* enh: add normal child vtable support

* fix: compile issues

* fix: compile issues

* fix: create stream issues

* fix: multi stream scan issue

* fix: remove debug info

* fix: agg task and task level issues

* fix: correct task output type

* fix: split vtablescan from agg

* fix: memory leak issues

* fix: add limitations

* Update 09-error-code.md

* Update 09-error-code.md

* fix: remove usless case

* feat(stream): extract original table data in source scan task

Implemented functionality in the source task to extract data
corresponding to the virtual table from the original table using WAL.
The extracted data is then sent to the downstream merge task for further
processing.

* feat(stream): multi-way merge using loser tree in virtual merge task

Implemented multi-way merge in the merge task using a loser tree to
combine data from multiple original table into a single virtual table.
The merged virtual table data is then pushed downstream for further
processing.  Introduced memory limit handling during the merge process
with configurable behavior when the memory limit is reached.

* fix(test): remove useless cases

---------

Co-authored-by: dapan1121 <wpan@taosdata.com>
Co-authored-by: Pan Wei <72057773+dapan1121@users.noreply.github.com>
2025-03-15 17:06:48 +08:00
Jing Sima
410324746b
feat:[TS-4897] virtual table (#30098)
* feat: [TS-4897] Support create/drop/alter/show/describe vtable

* feat: [TS-4897] Support vtable's query

* feat: [TS-4897] Support create virtual supertable

* feat: [TS-4897] Support explain analyze / where / count(*) and only select ts of vtable.

* feat: [TS-4897] Add create test and fix bugs

* feat: [TS-4897] Add alter/drop test and fix bugs

* feat: [TS-4897] Add describe/show test and fix bugs

* feat: [TS-4897] Add auth test and fix bugs

* feat: [TS-4897] Fix meta/catalog/cache bugs

* feat: [TS-4897] Support select tag from virtual child table

* feat: [TS-4897] Add select test and fix plenty of bugs

* feat: [TS-4897] Add optimize rule for vtable scan / support create vtable cross database / remove enterprise constraint / fix bugs.

* feat: [TS-4897] Fix 'schema is old'

* feat: [TS-4897] Support virtual stable query

* feat: [TS-4897] Add tests and Fix bugs

* feat: [TS-4897] resolve conflict.
2025-03-15 14:10:46 +08:00
Yihao Deng
0806cca09e
fix(query): fix ins_usage error on multi storage (#30134)
* fix calc disk usage error

* fix calc disk usage error

* fix calc disk usage error

* fix calc disk usage error

* fix calc disk usage error

* add test case

* add test case

* add test case

* add test case

* add test case

* add test case

* refactor code

* refactor code

* refactor code

* refactor code

* refactor code

* refactor code
2025-03-14 18:11:39 +08:00
wangjiaming
f50fcb85b8
feat(decimal): support decimal data type (#30060)
* decimal: create table

* decimal: add test case decimal.py

* decimal: add decimal.c

* support input decimal

* decimal test

* refactor svalue

* fix test cases

* add decimal unit test

* add decimal test cmake

* support insert and query decimal type

* define wide integer, support decimal128

* support decimal128 divide

* set decimal type expr res types

* scalar decimal

* convert to decimal

* fix decimal64/128 from str and to str

* fix decimal from str and decimal to str

* decimal simple conversion

* unit test for decimal

* decimal conversion and unit tests

* decimal + - * /

* decimal scalar ops and comparision

* start to refactor GET_TYPED_DATA

* support decimal max func, cast func

* refactor GET_TYPED_DATA interface

* decimal scalar comparision

* start to implement sum for decimal

* support sum and avg for decimal type

* decimal tests

* add decimal test

* decimal add test cases

* decimal use int256/int128

* decimal testing

* fix decimal table meta and add tests for decimal col streams

* fix create stream and create tsma

* test insert decimal values

* decimal from str

* test decimal input

* test parse decimal from string

* add taos_fetch_field_e api

* decimal insert tests

* test decimal operators

* decimal operator test

* feat:support decimal in raw block

* decimal operator tests

* decimal test

* feat:support decimal in raw block

* feat:support decimal in raw block

* feat:add schemaExt to SMqDataRsp

* feat:remove add schemaExt to SMqDataRsp

* feat:remove add schemaExt to SMqDataRsp

* feat:remove add schemaExt to SMqDataRsp

* decimal test operators

* decimal operator test

* test decimal operators

* test decimal compare operators

* decimal unary operator test

* decimal col with decimal col oper test

* test decimal col filtering

* fix decimal float operator test

* decimal test where filtering

* fix decimal filtering

* fix decimal order by

* fix decimal op test

* test decimal agg funcs

* test decimal functions

* remove assert

* fix ci build for ret check

* fix decimal windows build

* fix ci ret check

* skip decimal ret check

* skip decimal ret check

* fix decimal tests

* fix decimal ci test

* decimal test

* fix(tmq): heap user after free

* fix(tmq): double free

* fix(tmq): double free

* fix decimal tests

* fix(decimal): decimal test ci build

* fix(decimal): windows build

* fix(decimal): decimal test build

* fix(decimal): fix decimal build and tests

* fix(decimal): fix decimal tests

* fix(decimal): fix taos_fetch_fields_e api

* fix(decimal): fix decimal taos_fetch_fields_e api

* fix(decimal): rebase 3.0

* fix(decimal): fix decimal functions

* fix(decimal): fix decimal test case memory leak

* fix(decimal): fix decimal tests

* fix(decimal): fix decimal test case

* fix(decimal): fix decimal tests

* feat(decimal): fix unit tests

* feat(decimal): fix deicmal unit test

---------

Co-authored-by: wangmm0220 <wangmm0220@gmail.com>
Co-authored-by: yihaoDeng <yhdeng@taosdata.com>
2025-03-14 18:08:07 +08:00
Simon Guan
1a6becbd2b Merge branch '3.0' into merge/mainto3.0 2025-03-14 14:54:02 +08:00
WANG MINGMING
3d053e2c9b
feat(stream): optimize client logic of creating stream if with history (#30059)
* feat:[TS-5617]use last_row cache mode if creating stream in fill_history

* feat(stream): optimize client logic of creating stream if with history

* feat(stream): optimize client logic of creating stream if with history

* feat(stream): optimize client logic of creating stream if with history

* feat(stream): optimize client logic of creating stream if with history

* feat(stream): optimize client logic of creating stream if with history

* feat(stream): optimize client logic of creating stream if with history

* feat(stream): optimize client logic of creating stream if with history

* feat(stream): optimize client logic of creating stream if with history

* feat(stream): optimize client logic of creating stream if with history

* feat(stream): optimize client logic of creating stream if with history

* feat(stream): optimize client logic of creating stream if with history

* fix: heap use after free

* feat: add log

* fix: ci case error

* fix: compile error in windows

* fix: ci case error

* fix: heap user after free

* fix: memory leak

* fix: ci case error

* fix: ci case error

* fix: ci case error

---------

Co-authored-by: yihaoDeng <yhdeng@taosdata.com>
2025-03-14 13:55:52 +08:00
Simon Guan
724facd134 Merge branch 'main' into merge/mainto3.0 2025-03-14 11:49:02 +08:00
Haojun Liao
83cc432bf9 test(stream): add status check before pause/resume streams. 2025-03-13 14:55:22 +08:00
facetosea
ff85dc876c fix: test case 2025-03-12 15:10:19 +08:00
facetosea
8fbd28eb6e fix: state col precal 2025-03-12 14:33:18 +08:00
Simon Guan
81e6f49399 Merge branch '3.0' into merge/mainto3.0 2025-03-12 14:20:43 +08:00