Commit graph

6101 commits

Author SHA1 Message Date
Kaili Xu
b88ec2f1a8
fix: timeseries statis for virtual table (#30911) 2025-04-27 18:03:20 +08:00
Jing Sima
227c4c81d5
fix: [TD-34638] Fix schema is old error in vtable queries. (#30891) 2025-04-27 16:06:52 +08:00
Jinqing Kuang
6921e31b62
fix(stream): fix notification for stream with project operator (#30879) 2025-04-25 17:24:15 +08:00
wangmm0220
b9239dac57 fix: ci case error in sometime 2025-04-25 09:06:14 +08:00
wangmm0220
ffffefb802 fix(tmq): add log 2025-04-24 11:45:37 +08:00
freemine
6267763e85
Freemine.cmake.3.0 (#30663)
* chore(ci): with cmake verbose, temporarily

* chore(cmake): prefer Threads::Threads

* fix(cmake): debug building gtest on ubuntu 20.04 for the moment

* fix(cmake): try adding Threads::Threads

* fix(cmake): debug building gtest on ubuntu 20.04 for the moment

* fix(cmake): revert taosd-ci-build.yml

* fix(cmake): debug building on ubuntu 20.04 for the moment

* fix(cmake): debug building on ubuntu 20.04 for the moment

* fix(cmake): debug building on ubuntu 20.04 for the moment, adding dl right after DEP_ext_libuv

* fix(cmake): debug building on ubuntu 20.04 for the moment, try full run

* fix(cmake): fix building on ubuntu 20.04

* fix(cmake): dl is only valid in non-windows platform

* fix(cmake): dl order

* refactor(cmake): adjust option/cache variables for external dependencies

* refactor(cmake): mv patch files from contrib/ to cmake/

* refactor(cmake): typo

* refactor(cmake): adding cache on ci

* refactor(cmake):
1. ext_lzma2 is only used by Linux
2. fix bug in taosd-ci-build.yml

* enh: adding functionalities of build.sh

* fix(ci): `cmake -B` shall maintain consistency

* refactor(cmake): LOCAL_REPO/LOCAL_URL as cache variable

* refactor(cmake): taosadapter to be built in main build directory rather than .externals

* refactor(cmake): adding rpath for taosadapter, linux now

* refactor(cmake): adding rpath for taosadapter, macos now

* chore(cmake): remove some comments

* debug(ci): bypass TD_EXTERNALS_USE_ONLY

* debug(ci): bypass .externals caching for the moment

* fix(cmake): temporarily use taosadapter/main branch because this branch(freemine.cmake.3.0) has not include taos_fetch_fields_e from upstream

* refactor(cmake): clean GIT_TAG/GIT_SHALLOW pairs

* enh(ci): use build.sh rather than hard-coded in taosd-ci-build.yml, linux for now

* debug(ci): undefined __isoc23_strtoull

* debug(ci): benchmarkTest

* debug(ci): what if use_only = off

* debug(ci): remove .externals to make a refresh build

* debug(ci): back to cache, and ls files of gtest

* debug(ci): remove find coz {} not escaped

* debug(ci): cache key seems missing "matrix.os"

* refactor(ci): fixed cache key and reopen macos/windows

* chore(ci): literally -j4

* chore(ci): Trigger GitHub Actions re-run

* chore(ci): Trigger GitHub Actions re-run

* fix(ci): typo

* chore(ci): JOBS calculated by build.sh itself

* refactor(ci): enable WEBSOCKET in build.sh by default

* refactor(ci): enable WEBSOCKET in build.sh by default, macOS

* chore(cmake): remove obsoletes

* refactor(ci): stop service before running taosd

* chore(ci): `./build.sh install` just install TDengine, you will have to start taosd by `./build.sh start`

* refactor(cmake): adding source code found on http://emfisis.physics.uiowa.edu/Software/C/libargp/

* refactor(cmake): moving jansson/snappy/avro to .externals, and porting BUILD_TEST/WEBSOCKET on Windows

* refactor(cmake): porting linux

* refactor(cmake): porting macos
1. check argp.h
2. re-enable building taosdump

* refactor(cmake): porting linux
1. adjust CMAKE_C/CXX_FLAGS

* refactor(cmake): porting windows, adjust CMAKE_C/CXX_FLAGS

* refactor(cmake): use cmake_parse_arguments in macro INIT_EXT

* refactor(cmake): format ci cache directives

* refactor(cmake): adding comments for ci externals-cache directives

* refactor(cmake): adding check_env.cmake to checking libraries installed on the system

* refactor(cmake): porting linux, make homebrew-installed-libs available

* refactor(cmake): remove BYPRODUCTS from INIT_EXT macro

* refactor(cmake): INC_DIR is oneValueArg

* refactor(cmake): taosdump depends on ext_xz(LZMA)

* refactor(cmake): add copy_pthreadVC3 target on windows

* refactor(cmake): mv taosd-ci-build.yml to tdengine-build.yml

* chore(ci): try to fix yml syntax error

* refactor(cmake): use brew --prefix to get HomeBrew's install path

* refactor(cmake): typo

* debug(ci): build and cache externals first

* debug(ci): adding -Wno-stringop-overread for the moment

* debug(ci): make Threads::Threads public from os target

* debug(ci): make os public

* debug(ci): re-enable start and test

* debug(ci): let it go even if taosBenchmark failed

* debug(ci): copy logic from TDengine/main

* debug(ci): try ./build.sh start/test

* debug(ci): revert to original simple test logic in ci, taosBenchark/taos

* chore(cmake): code-convention and ci-logic

* debug(ci): show versions

* debug(ci): forcefully use specific GIT_TAG by commit-id for taos-connector-rust.git and taosadapter.git

* refactor(cmake): remove empty lines

* debug(ci): typo

* debug(ci): set supportVnodes to 256 before starting taosd

* debug(ci): fix: sudo and pipe

* debug(ci): cat /etc/taos/taos.cfg to check if supportVnodes effective

* debug(ci): cat forceReadConfig to taos.cfg

* debug(ci): adding taos -s "show dnodes"

* debug(ci): print more info

* debug(ci):
1. git log .... in an independant step
2. sleep 3 deliberately

* build: release ver-3.3.6.0

* chore(ci): comment and remove unnecessary code

* refactor(ci): cleanup tools/CMakeLists.txt
1. remove unnecessary or uneffective codes
2. better approach to pass compile flags to go

* chore(cmake): remove unnecessary code

* merge(cmake): porting linux

* merge(cmake): porting macOS

* merge(cmake): porting windows

* merge(cmake): pitfall fix: use literal format string instead

* merge(cmake): ver-3.3.6.0 for both taosadatper and taosws

* merge(cmake): adjust ci for debug purposes

* merge(cmake): add dependenices to make sure native built

* merge(main): link os

* merge(main): revert to main

* chore(cmake): remove freemine'd comments

* chore(cmake): make github ci to run with empty change

* refactor(cmake): remove unnecessaries

* refactor(cmake): remove unnecessaries

* chore(cmake):
1. remove taosudf in build.sh::do_purge
2. packaging taosBenchmark

* chore(cmake):
1. typo corrections
2. add -DCMAKE_POLICY_VERSION_MINIMUM=3.5 to bypass compatibility check imposed by latest cmake since 4.0

* chore(cmake): show cmake version

* chore(cmake): add -DCMAKE_POLICY_VERSION_MINIMUM=3.5 to bypass compatibility check imposed by latest cmake since 4.0

* chore(cmake): typo

* chore(cmake): remove unnecessaries

* cmake: debug enterprise compilation

* refactor: preparatory work before merging the freemin PR

* refactor(cmake): revert to d91205594a

* refactor(cmake): optimize by adjusting cache key

* chore(cmake): make github ci to run with empty change

* refactor(cmake): move libs3 and azure to external.cmake

* refactor(cmake): porting macOS

* refactor(cmake): move cos to external.cmake

* refactor(cmake): bypass s3/azure/cos on windows for the moment

* refactor(cmake): try find_package(Threads)

* refactor(cmake): add dl/pthread in libs3

* refactor(cmake): enable s3/azure for the moment, hard-coded

* refactor(cmake): adjust BUILD_WITH_ANALYSIS option

* refactor(cmake):
1. enum AVROTYPE => enum enAVROTYPE
2. remove INSTALL(CODE ...) from tools/taos-tools/CMakeLists.txt

* refactor(cmake): portint macOS and typo

* refactor(cmake): ticking external cache key to force refresh external building

* refactor(cmake): remove obsoletes

* refactor(cmake): remove obsoletes

* refactor(cmake): remove obsoletes, contrib/deps-download related

* refactor(cmake): adding build.bat, and use build.bat in .github/workflows/tdengine-build.yml

* refactor(cmake):
1. call .\build.bat
2. remove POST_BUILD which is not valid in this signature of add_custom_command

* refactor(ci): use 'NMake Makefiles JOM'

* refactor(ci): typo

* refactor(ci): debug

* refactor(ci): debug

* refactor(ci): debug

* refactor(ci): use call on windows

* refactor(cmake): move utils/TSZ to contrib/TSZ

* chore(cmake): make github ci to run with empty change

* refactor(cmake): mv cmake.* -> *.cmake and related in CMakeLists.txt

* refactor(cmake): debugging integrating community

* refactor(cmake): debugging integrating community

* chore(cmake): make github ci to run with empty change

* refactor(cmake): revert geos.git back to 3.12.0

* refactor(cmake): revert doc/** to main branch

* refactor(cmake): revert tests/** to main branch, except tests/taosc_test/CMakeLists.txt

* refactor(cmake): revert test/cases/** to main branch

* refactor(cmake): revert test/docs/** to main branch

* refactor(cmake): revert tools/taos-tools/deps/{avro,jansson,libargp,snappy,zlib-1.2.13} to main branch, which need to be removed later

* refactor(ci): revert .github/workflows/{deploy-case-docs.yml,tdengine-doc-build.yml} to main branch

* refactor(cmake): revert tcommon.h and related to main branch

* refactor(cmake): revert test/* stuffs back to main branch

* refactor(cmake): revert libuv.git back to v1.49.2

* refactor(cmake): revert .h/.c and related to main branch

* refactor(cmake): typo format string

* refactor(cmake): typo correction, incorrect use of macro `assert`

* refactor(cmake): adding /wd4028 to bypass thttp.c under enterprise build for the moment

* refactor(cmake):
1. adding some rules to .gitignore for the last resort to save
2. adding source files manually to source/libs/parser/CMakeLists.txt to exclude potential sql.c generated by old build system

* refactor(cmake): add -Wno-unused-result for the moment

* refactor(cmake): adjust target dependencies

* refactor(cmake): tweaking building of mxml

* refactor(cmake): adjust externals CMAKE_ARGS to save compilation time

* refactor(cmake): temporarily use tomchon-patch-3

* refactor(cmake): temporarily adding dumpbin directives

* refactor(cmake): adding NODEFAULTLIB temporarily for debugging purpose

* refactor(cmake): adding NODEFAULTLIB:libcpmt temporarily for debugging purpose

* refactor(cmake): adding NODEFAULTLIB:cpmt temporarily for debugging purpose

* chore(cmake): make github ci to run with empty change

* chore(cmake): make github ci to run with empty change

* refactor(ci): use freemine.debug branch for the moment

* refactor(ci): revert back to use run-tests-on-windows.yml of tomchon-patch-3 branch

* chore(cmake): make github ci to run with empty change

* refactor(cmake): remove temporary debug directives

* refactor(cmake): move cmake_policy(..CMP0135..) to the root of this project

* refactor(cmake): Remove temporary debugging conditional and restore original logic

* chore(cmake): adding -B in army tests

* chore(cmake): make github ci to run with empty change

* chore(cmake): revert: adding -B in army tests

* refactor(cmake):
1. use ver-3.3.6.3 of taosadapter
2. pitfall fix: do not link with taosws for taos shell

* refactor(cmake): revert taosadapter to ver-3.3.6.0

* chore(cmake): adding logging message in more detail

* chore(cmake): adding logging message in more detail

* chore(ci): debugging by using ls -l

* chore(ci): typo

* chore(ci): typo

* refactor(cmake): revert to remove debug code, and taosadapter to ver-3.3.6.3

* refactor(cmake): remove DEP_ext_taosws

* chore(cmake): make github ci to run with empty change

* chore(cmake): make github ci to run with empty change

* chore(cmake): make github ci to run with empty change

* chore(cmake): make github ci to run with empty change

* chore(cmake): typo correction: utils/test/c/sml_test.c

* debug(cmake): test cases debug

* chore(cmake): make github ci to run with empty change

* chore(ci): using tomchon-patch-3 branch for run-tests-on-linux for the moment

* chore(cmake): make github ci to run with empty change

* debug(test): revert utils/test/c/sml_test.c back to tdengine/main branch

* debug(test): adding VERBOSE=1 and gcc --version

* debug(test): revert tests/parallel_test/cases.task back to tdengine/main branch

* debug: add log in run_case.sh

* fix: test log

* fix: docker run privileged

* tes: add debug test in  cases.task

* refactor(cmake): remove ref to \contrib\<dep-name> and ~/.cos-local-{1,2} from CMakeLists.txt(s)

* refactor(cmake): make taosudf link with libtaos.so

* chore(cmake): fix compiler warning as error problem

* refactor(cmake): revert 7d7f03832d

* chore(cmake): only udf.sim and udfpy.sim for the moment

* debug(ci): debug udf.sim for the moment

* debug(ci): debug udf.sim for the moment

* debug(ci): debug udf.sim for the moment

* debug(ci): debug only udf.sim for the moment

* debug(ci): debug only udf.sim for the moment, using tee

* debug(ci): debug only udf.sim for the moment, revert using tee

* debug(ci): debug only udf.sim for the moment, adding TAOS_LIB again

* debug(cmake): adding -rdynamic for the moment

* debug(cmake): re-adding udfpy.sim

* debug(cmake): do NOT run with Sanitizer for udfpy.sim for the moment

* chore(cmake): revert cases.task, and remove ubuntu-20.04 from tdengine-build.yml

* refactor(cmake): using ENABLE_EXPORTS to export symbols in taosudf

* chore(cmake): make github ci to run with empty change

* chore(cmake): remove stuffs for debug purpose and revert

---------

Co-authored-by: support-platform <it@taosdata.com>
Co-authored-by: Simon Guan <slguan@taosdata.com>
Co-authored-by: Simon Guan <guanshengliang@qq.com>
Co-authored-by: haoranchen <haoran920c@163.com>
2025-04-19 14:03:29 +08:00
WANG MINGMING
ca724865b9
fix(stream): parName is null (#30592) 2025-04-19 10:26:32 +08:00
Haojun Liao
7a724c84f4
feat(gpt): support covariate forecasting (#30806) 2025-04-18 14:01:54 +08:00
freemine
9897d85f86
enh(test): adding TD_ALWAYS_ASSERT macro for test cases to use (#30739) 2025-04-10 16:52:55 +08:00
Haojun Liao
311cb53509
feat(tdgpt): auto load tsfm from amazon/google/salesforce (#30634) 2025-04-08 15:01:12 +08:00
Jing Sima
ffd0a5eb57
fix: [TD-34205] Fix vtable mem leak (#30608) 2025-04-07 18:49:49 +08:00
wangjiaming
b3b6c3d3ff
fix(fill): fix fill next/prev got different result with asc/desc order 2025-04-03 21:24:39 +08:00
WANG MINGMING
f398baff66
fix(stream): add null pointer check (#30614) 2025-04-03 10:20:54 +08:00
xinsheng Ren
463430ab15
enh: [TS-6195] support cols in stream (#30580) 2025-04-02 11:31:08 +08:00
Haojun Liao
7f797bd439 fix(gpt): fix memory leak and errors. 2025-04-01 13:52:57 +08:00
Haojun Liao
865e0aec79
Merge branch 'main' into doc/analysis 2025-03-31 09:37:51 +08:00
WANG MINGMING
1c4507ab75
fix(stream): row size error (#30330) 2025-03-28 18:10:57 +08:00
Hongze Cheng
6e6224eeba
fix: typo in documents and codes (#30544) 2025-03-28 13:33:38 +08:00
Jing Sima
f84f04398d
fix: [TS-4897] Support query stables cross databases and fix potential deadlock. (#30507) 2025-03-28 10:07:08 +08:00
Haojun Liao
494e6d1565 fix(gpt): check the lower bounding for data input analysis. 2025-03-27 10:05:04 +08:00
Haojun Liao
82ffeb82a9
fix(gpt): fix the bug for algo name case sensitive (#30362) 2025-03-23 13:20:24 +08:00
Linhe Huo
b008ef17af Merge branch '3.0' into dev/merge-3.0-0322
* 3.0: (24 commits)
  doc: update docs. (#30347)
  doc: update docs for analysis (#30352)
  docs: fix virtual table err code (#30348)
  fix(stmt2):stmt2 get fields return wrong when tag is value (#30283)
  docs: [TS-4897] Add docs for virtual table. (#30325)
  fix: add show connMode string (#30323)
  fix: add requests pkg in tdgpt install.sh
  ci: add release build
  Update remove.sh
  refactor: streamline library removal and linking in installation scripts
  doc: add error code information about tdgpt (#30338)
  Update 09-error-code.md (#30327)
  chore: update taosadapter depends (#30326)
  chore: use taosws 3.0 branch in TDengine 3.0
  fix: show connMode is ok for taos
  fix: remove old setConnMode
  fix: build error
  fix: get default port rule
  chore: use taosadapter 3.0 branch in 3.0
  ci: fix pthread_tryjoin_np compile error
  ...
2025-03-22 22:44:07 +08:00
Haojun Liao
f2e9193a95 fix(gpt): fix error. 2025-03-21 17:31:15 +08:00
Haojun Liao
b4e5aafac2 Merge branch '3.3.6' into fix/mergegpt 2025-03-21 16:36:17 +08:00
Haojun Liao
47004861c7 fix(gpt): 1. fix error in anomalywindow count; 2. update gpt model name; 3. update the expired test case. 4. support the user specified valid_code 2025-03-21 12:52:41 +08:00
Alex Duan
c133489917
new-libtaos-with-ws (#29971)
* enh: rename libtaos.so to libtaosinternal.so

* enh: let python system-test work

* enh: shell for libinternal.so

* enh: refact script

* enh: wrapper for libtaosinternal

* enh: rename some files

* enh: let shell support internal driver

* enh: minor changes

* enh: minor changes

* add intenal.h

* enh: remove unused codes

* enh: minor changs

* feat: mac os issues

* enh: let python test use internal connection

* enh: let nettest work

* enh: let taosc -C work

* enh: test shell in mac os

* enh: minor changes

* enh: let libtaosinternal.so work in linux

* enh: update install script

* enh: for ci

* enh: for windows compile

* enh: minor changes

* enh: compile in windows

* enh: minor changes

* enh: taosinternal work in windows

* enh: add taos_internal_static

* enh: rollback os_for_wrapper

* fix: conflicts

* fix: conflicts

* fix: compile errors

* fix: conflicts

* enh: rename taosinternal to taosnative

* enh: rename taosinternal to taosnative

* fix: compile error

* enh: remove taosnative.h

* fix: compile errors

* enh: set default shell options

* fix: compile errors

* fix: compile errors

* debug: switch taosws branch to feat/new-libtaos-with-ws

* enh: taosBenchmark remove WEBSOCKET finished

* fix: CMakeLists.txt remove WEBSOCKET

* fix: compile errors

* fix: taosBenchmark delete restful code

* .gitignore remove taos-tools folder

* fix: remove taosdump WEBSOCKET MACRO

* fix: build error for taosdump

* fix: build project passed

* fix: support -Z for connect mode

* fix: build error

* fix: build error fixed

* fix: add -Z options

* fix: native and websocket with string

* fix: -Z option core

* fix: build unit test

* fix: build error

* fix: add colon for array

* fix: unit test can not include fun

* fix: CTest pointer null is nullptr

* fix: include pub.h

* fix: taosdump add -Z --driver options

* fix: taosdump support -Z with pub.h pub.c

* fix: toolsGetTimeDay move to pub.c

* fix: restore pub.c only simple fun

* fix: add error tips

* fix: tmfree remove from taosdump

* fix: remove double declare taos variant

* fix: taos support -Z option

* enh: configDir set with main fun

* fix: add g_arguments.configDir

* fix: show tips add cfgdir

* fix: g_argument is pointer

* fix: configDir is global var

* fix: declare g_configDir in benchMain.c

* fix: taos version not show on websocket

* stmt2 websocket prepare with supertable

* stmt2 websocket prepare build

* fix: stmt and stmt2 prepare add db name

* fix: remove -R --restful test case

* fix: taosBenchmark adjust priority cmd > json > evn

* fix: add connect mode test case connMode.py

* fix: clear evn and add host port check

* fix: army/test.py modify start taosAdapter is default

* change: system-test/test.py default start taosAdapter

* fix: add taosAdapter variant to system-test/test.py

* fix: -y must put behind other options

* fix: remove rest sml and add -N normal case

* fix: add -c work cfg test case

* del: queryMain.py remove rest query

* fix: remove groups have a blank append bug

* fix: caseBase.py add db in sql query

* fix: query_json.py remove rest test

* fix: taosBenchmark uniform dsn describe

* fix: add pub.h pub.c to tools public

* fix: case remove restful interface

* fix: build error

* fix: taosDriverCleanup no arg call

* fix: taosdump add test conn mode case taosdumpCommandline.py

* fix: modify insertFullType.json db name

* fix: taosdump connMode test case taosDumpCommandline.py passed

* enh: priority cmd > env > json

* fix: taosCli remove is_native variant

* fix: password is not pointer

* fix: comment -o test case

* fix: solve fun return code check

* fix: windows unresolved strcasecmp

* fix: remove coverHostToServAddr

* fix: forbid check ODR violation

* ci: trigger pr run

* fix: build error not del >>>>>3.0

* fix: support json dbinfo->vgroups have blank space

* fix: solve conflict with merge

* fix: with TrimCaseCmp fun to compare vgroups

* fix: del vgroups support blank code

* feat: wrapper add taos_fetch_fields_e

* fix: move jni from libtaosnative.so to libtaos.so

* fix: move clientTmqConnector.c for jni

* fix: fix assert check with del comment code

* fix: taosBenchmark add prepare stmt debug log

* fix: stmt2 already have debug log on preapare

* fix:  (double)LLONG_MAX force covert

* fix: stmt insert normal table with websocket

* fix: normal table with stmt too many ?

* fix: taosdump stmt_prepare with child tbname

* fix: build error add int code

* feat: stmtPrepare with child table to do

* taos -a options move to native test case

* fix:unit test benchmarkTest passed

* fix: query_json-with-sqlfile.py case no check results

* fix: queryMain.py case lost, put in tasks again

---------

Co-authored-by: Shengliang Guan <slguan@taosdata.com>
2025-03-20 22:54:29 +08:00
Jing Sima
7cffc7aff2
fix: [TD-34074] Forbid virtual table in tq and stmt. 2025-03-20 22:42:27 +08:00
Jing Sima
818bea5568
fix: [TD-34074] Forbid virtual table in tq and stmt and forbid supertable query's origin table from different databases.
fix: [TD-34074] Forbid virtual table in tq and stmt and forbid supertable query's origin table from different databases.
2025-03-20 22:42:10 +08:00
Simon Guan
eff763dc91
fix(stream): fix stream processing for virtual tables 2025-03-20 22:31:29 +08:00
Simon Guan
e5a1539c6d
Merge pull request #30307 from taosdata/fix/TS-5467-3.0
fix(stream): fix stream processing for virtual tables
2025-03-20 22:24:45 +08:00
Jinqing Kuang
4d97d691cf fix(stream): add test cases for vtable stream processing 2025-03-20 18:57:58 +08:00
Simon Guan
9b7434d0ab
merge: from main to 3.0 branch (#30296)
* chore(ci): statistics ci report

* chore(ci): statistics ci report never fail

* chore(ci): statistics ci report sort by size desc

* Update .github/workflows/taosd-ci-build.yml

* docs: upgrade the node.js connector version (#30264)

* feat: change the max value of minFreeDiskSize to 2TB (#30250)

* feat: change the max value of minFreeDiskSize to 2TB

* fix: CI test case change

* enh: remove useless comment

* docs(query): enhance time window clause descriptions in the documentation

* docs: fix image position in DST document (#30266)

* fix node.js example error (#30287)

* test: node example case

* fix: node.js example error

* fix: restore test case validation

* fix(stream): fix race condition in send msg. (#30277)

Co-authored-by: 54liuyao <54liuyao@163.com>
Co-authored-by: Jinqing Kuang <kuangjinqingcn@gmail.com>
Co-authored-by: wangmm0220 <wangmm0220@gmail.com>
Co-authored-by: yihaoDeng <luomoxyz@126.com>

* add c stmt2 example to doc (#30286)

* add: stmt2 insert example

* fix: free all heap

* format code

* add c stmt2 example

* update en doc desc

* mod makefile

---------

Co-authored-by: pengrongkun94@qq.com <pengrongkun94@qq.com>

---------

Co-authored-by: freemine <freemine@yeah.net>
Co-authored-by: WANG Xu <feici02@outlook.com>
Co-authored-by: kevin men <men_shi_bin@163.com>
Co-authored-by: Hongze Cheng <hzcheng@taosdata.com>
Co-authored-by: Jinqing Kuang <kuangjinqingcn@gmail.com>
Co-authored-by: Linhe Huo <linhehuo@gmail.com>
Co-authored-by: Haojun Liao <hjxilinx@users.noreply.github.com>
Co-authored-by: 54liuyao <54liuyao@163.com>
Co-authored-by: wangmm0220 <wangmm0220@gmail.com>
Co-authored-by: yihaoDeng <luomoxyz@126.com>
Co-authored-by: She Yanjie <57549981+sheyanjie-qq@users.noreply.github.com>
Co-authored-by: pengrongkun94@qq.com <pengrongkun94@qq.com>
2025-03-20 16:54:39 +08:00
Haojun Liao
cd10173454
fix(gpt): fix error in gpt (#30275)
* fix(stream): support packaging enterprise edition.

* feat(gpt): support lstm and do some internal refactor, add sample autoencoder model.

* feat(gpt): support lstm and do some internal refactor, add sample autoencoder model.

* test(gpt): disable model case.

* test(gpt): disable model case.

* doc: fix title error in doc.

* doc: add mlp doc.

* fix(gpt): add gpt

* fix(gpt): update the test cases.

* fix(gpt): update the test cases.

* test(gpt): disable gpt test in github action.

* fix(gpt): check the grant, check the rsp code.

* fix(gpt): limit the forecast rows.

* fix(gpt): use default gpt service

* fix(gpt): update error code.
2025-03-20 15:18:58 +08:00
Simon Guan
2c27c10fa8 Merge branch 'main' into merge/mainto3.0 2025-03-20 14:05:45 +08:00
Haojun Liao
c52df0bb24 fix(gpt): update error code. 2025-03-20 11:54:43 +08:00
Haojun Liao
f42ce412ae fix(gpt): limit the forecast rows. 2025-03-20 11:23:19 +08:00
liuyao
fac36002c1
ci(stream): add ci and fix issue (#30270)
* ci(stream): add ci and fix issue

* ci: adj test

* ci: add test and fix issue

* ci: adj test

* ci(stream): add test for fill operator and fix issue

* fix: fix issue

* fix: fix issue

* fix: adjust defalut rec value
2025-03-20 10:34:31 +08:00
Jinqing Kuang
c82398e175 fix(stream): fix stream processing for virtual tables
- Fixed status check in virtual table stream processing
- Fixed multi-way merge logic for super virtual tables
- Fixed generation of target subtable names during merge
2025-03-20 01:15:54 +08:00
Haojun Liao
db57fc4ba8
fix(stream): fix race condition in send msg. (#30277)
Co-authored-by: 54liuyao <54liuyao@163.com>
Co-authored-by: Jinqing Kuang <kuangjinqingcn@gmail.com>
Co-authored-by: wangmm0220 <wangmm0220@gmail.com>
Co-authored-by: yihaoDeng <luomoxyz@126.com>
2025-03-19 23:15:15 +08:00
Jing Sima
ccaa0b91ae
feat: [TS-4897] Add more test and fix bugs. (#30208)
* feat: [TS-4897] Fix state window wrong res when all state col is NULL.

* fix: [TD-34074] Forbid decimal type in virtual table.
2025-03-19 17:15:17 +08:00
WANG MINGMING
3fafecb242
feat[TS-6137]: support sliding in force_window_close (#30203)
* feat[TS-6137]: support sliding in force_window_close

* feat[TS-6137]: support sliding in force_window_close

* feat(stream): force window close support interval sliding

* feat[TS-6137]: support sliding in force_window_close

* feat[TS-6137]: support sliding in force_window_close

* feat[TS-6137]: support sliding in force_window_close

---------

Co-authored-by: 54liuyao <54liuyao@163.com>
2025-03-19 10:22:06 +08:00
Simon Guan
d02196a231
refactor: adjust some logs (#30245)
* refactor: adjust wal log infos

* refactor: adjust some log's level

* refactor: adjust some log's level

* fix: add traceId for sync module

* refactor: adjust log level

* refactor: adjust log level

* refactor: adjust log levels

* refactor: adjust log levels

* refactor: adjust log levels

* refactor: adjust log levels

* refactor: adjust log levels

* refactor: adjust log levels

* refactor: adjust log levels

* docs: update keywords

* fix: compile errors in windows

* docs: minor changes

* docs: minor changes

* refactor: minor changes

* docs: fminro changes

* docs: format

* refactr: remove uncessory logs

* docs: format doc
2025-03-18 17:55:36 +08:00
Simon Guan
718d455a97 Merge branch 'main' into merge/mainto3.0 2025-03-18 14:35:54 +08:00
xinsheng Ren
8453726080
fix: when querying super table, using ORDER BY and LIMIT together may return an unexpected record.
* fix: limit error

* fix: limit error, test case.
2025-03-18 13:58:30 +08:00
Haojun Liao
dfe32225de chore: merge main. 2025-03-17 14:51:11 +08:00
Haojun Liao
ab92886820
fix(stream): reduce the consensus checkpoint id trans. (#30105)
* fix(stream): reduce the consensus checkpoint id trans.

* refactor(stream): add some logs.

* refactor(stream): set the max checkpoint exec time 30min.

* refactor(stream): add checkpoint-consensus trans conflict check.

* refactor(stream): remove unused local variables.

* fix(stream): fix syntax error.

* fix(stream): 1. fix free memory error 2. continue if put result into dst hashmap failed.

* fix issue

* fix issue

* fix(mnd): follower mnode not processes the timer event.

* fix(stream): print correct error msg.

* fix(stream): add some logs.

* fix(stream): add some logs.

* fix(stream): add some logs.

* fix(stream): add some logs.

* fix(stream): add some logs.

* fix(stream): add some logs.

* fix(stream): add some logs.

* fix(stream): add some logs.

* fix(stream): truncate long subtable name

* fix(stream): add buffer len.

* refactor(stream): update some logs.

* fix issue

* refactor(stream): update some logs.

* refactor(stream): update some logs.

* fix(stream): check return value.

* fix(stream): fix syntax error.

* fix(stream): check return value.

* fix(stream): update the timer check in mnode.

* fix(stream): add restart stage tracking.

* fix(stream): track the start task stage for meta.

* fix(stream): fix error in log.

* refactor(stream): adjust log info.

* fix mem issue

* fix(stream): check the number of required tasks for consensus checkpointId.

* fix(stream): lock the whole start procedure.

* fix(stream): add lock during start all tasks.

* fix(stream): update logs.

* fix(stream): update logs.

* fix(stream): update logs.

* fix(stream): fix dead-lock.

* fix(stream): fix syntax error.

* fix(stream): not drop the scan-history task.

* fix(stream): fix syntax error.

* fix(stream): wait for executor stop before restarting.

* fix(stream): add some logs.

* fix(stream): add some logs.

* fix(stream): add some logs.

* fix(stream): disable some logs.

* fix(stream): reset the start info if no task left.

---------

Co-authored-by: 54liuyao <54liuyao@163.com>
Co-authored-by: Jinqing Kuang <kuangjinqingcn@gmail.com>
2025-03-17 10:20:17 +08:00
Jinqing Kuang
ac23a4fc55
feat(stream): support stream processing for virtual tables (#30144)
* enh: add client processing

* enh: add mnode vtables processing

* enh: add mnode vtable processing

* enh: add normal child vtable support

* fix: compile issues

* fix: compile issues

* fix: create stream issues

* fix: multi stream scan issue

* fix: remove debug info

* fix: agg task and task level issues

* fix: correct task output type

* fix: split vtablescan from agg

* fix: memory leak issues

* fix: add limitations

* Update 09-error-code.md

* Update 09-error-code.md

* fix: remove usless case

* feat(stream): extract original table data in source scan task

Implemented functionality in the source task to extract data
corresponding to the virtual table from the original table using WAL.
The extracted data is then sent to the downstream merge task for further
processing.

* feat(stream): multi-way merge using loser tree in virtual merge task

Implemented multi-way merge in the merge task using a loser tree to
combine data from multiple original table into a single virtual table.
The merged virtual table data is then pushed downstream for further
processing.  Introduced memory limit handling during the merge process
with configurable behavior when the memory limit is reached.

* fix(test): remove useless cases

---------

Co-authored-by: dapan1121 <wpan@taosdata.com>
Co-authored-by: Pan Wei <72057773+dapan1121@users.noreply.github.com>
2025-03-15 17:06:48 +08:00
Jing Sima
410324746b
feat:[TS-4897] virtual table (#30098)
* feat: [TS-4897] Support create/drop/alter/show/describe vtable

* feat: [TS-4897] Support vtable's query

* feat: [TS-4897] Support create virtual supertable

* feat: [TS-4897] Support explain analyze / where / count(*) and only select ts of vtable.

* feat: [TS-4897] Add create test and fix bugs

* feat: [TS-4897] Add alter/drop test and fix bugs

* feat: [TS-4897] Add describe/show test and fix bugs

* feat: [TS-4897] Add auth test and fix bugs

* feat: [TS-4897] Fix meta/catalog/cache bugs

* feat: [TS-4897] Support select tag from virtual child table

* feat: [TS-4897] Add select test and fix plenty of bugs

* feat: [TS-4897] Add optimize rule for vtable scan / support create vtable cross database / remove enterprise constraint / fix bugs.

* feat: [TS-4897] Fix 'schema is old'

* feat: [TS-4897] Support virtual stable query

* feat: [TS-4897] Add tests and Fix bugs

* feat: [TS-4897] resolve conflict.
2025-03-15 14:10:46 +08:00
liuyao
2de6736160
feat(stream): add new trigger continuous_window_close (#30125)
* opt stream build twa result

* opt force window close memory

* feat(stream):optimize new interval and scan operator

* adj log

* opt code

* opt code

* fill history

* fix issue for fill history

* add ci

* feat(stream): add new stream nonblock interval operator

* adjust code

* use new scan operator

* use new scan operator

* add log

* fix issue

* recover stream scan next function

* fix issue

* fix issue

* fix issue

* ignore disorder data

* fix issue

* fix issue for interval sliding

* fix issue

* fix ci issue

* fix ci issue

* fix ci issue

* add semi && final nonblock interval operator

* fix issue

* fix issue

* fix issue

* fix issue

* fix issue

* fix issue

* fix issue

* fix issue

* refactor(stream): track the msgId for each upstream tasks.

* fix(stream): fix race condition.

* fix(stream): update the task last msgId when putting into input queue succ.

* fix issue

* fix issue

* put recalculate data to rocksdb

* fix issue

* fix issue

* enh(query)[TD-33071]: add support for saving and restoring tsdbReader scan progress

- Implement functionality to save scan progress during tsdbReader operations
- Enable resuming scans from the last saved position

* fix issue

* fix issue

* fix issue

* fix issue

* fix issue

* add rec interval check

* enh(stream):add recalculate tasks.

* enh(stream): support the re-calculate the tasks.

* fix issue && do refactor

* do refactor

* fix issue

* fix issue

* update backend opt

* add new interface

* add new session operator

* support blob

* add new session operator

* fix issue

* add rec state for rec task

* fix invalid read

* add new session window

* enh(stream): update the stream tasks backend.

* new session operator

* add pull data

* fix(stream): fix error in expand stream backend.

* fix issue

* fix issue

* fix issue

* merge code

* fix issue

* fix(stream): check for null ptr.

* fix(stream): add more check.

* fix issue

* fix issue

* fix issue

* add debug code

* fix issue

* fix issue

* fix issue

* set rec end flag

* fix(stream): opt re-calculate stream tasks.

* fix issue

* fix issue

* add new operator

* enh(stream): dispatch recalculate block to agg tasks.

* fix issue

* fix issue

* fix(stream): adjust filter.

* fix issue

* refactor

* refactor(stream): adjust the recalculate end block.

* fix issue

* fix(stream): set correct create trigger block.

* fix issue

* fix(stream): fix error in build recalculate end block.

* fix(stream): check null ptr.

* add stream client && fix issue

* fix mem leak

* fix(stream): free msg.

* add stream client

* fix(stream): fix error.

* add stream client && fix issue

* add stream client

* refactor(stream): set the recalculate task complete.

* add wend and group_id for session window dest table

* feat(stream): refactor and set the recalcul agg task complete.

* add cfg for adapter

* fix issue

* add state && event operator

* feat(stream): support fill-history task.

* add new fill operator

* fix(stream): set correct backend when updating fill-history task to recalculate task.

* add new fill operator

* fix(stream): s2025-03-06 11:10:31.272
et ops always open in scan tsdb

* fix(stream):set the correct taskType for sink task.

* new fill operator

* adj stream fill operator

* fix issue

* fix issue

* fix issue

* fix issue

* fix issue

* fix issue

* fix issue

* fix issue

* adj test

* fix issue

* fix(stream): fix issue

* fix(steam): fix issue

* fix(steam): fix issue

* fix(steam): fix issue

* fix(steam): fix issue

* fix(stream): fix issue

* fix(stream): fix issue

* fix(stream): fix issue

* fix: ut com error

* fix(stream): fix mem leak and adjust operator type check rule

* fix(stream): fix mem leak and adjust test case

* refactor code

* fix(stream): free items.

* fix(stream): free fix memory leak.

* fix(stream): fix syntax error.

* fix: ignore unexpect block

* fix: adjust op type

---------

Co-authored-by: Haojun Liao <hjliao@taosdata.com>
Co-authored-by: Jinqing Kuang <kuangjinqingcn@gmail.com>
Co-authored-by: yihaoDeng <luomoxyz@126.com>
2025-03-14 20:14:01 +08:00
wangjiaming
f50fcb85b8
feat(decimal): support decimal data type (#30060)
* decimal: create table

* decimal: add test case decimal.py

* decimal: add decimal.c

* support input decimal

* decimal test

* refactor svalue

* fix test cases

* add decimal unit test

* add decimal test cmake

* support insert and query decimal type

* define wide integer, support decimal128

* support decimal128 divide

* set decimal type expr res types

* scalar decimal

* convert to decimal

* fix decimal64/128 from str and to str

* fix decimal from str and decimal to str

* decimal simple conversion

* unit test for decimal

* decimal conversion and unit tests

* decimal + - * /

* decimal scalar ops and comparision

* start to refactor GET_TYPED_DATA

* support decimal max func, cast func

* refactor GET_TYPED_DATA interface

* decimal scalar comparision

* start to implement sum for decimal

* support sum and avg for decimal type

* decimal tests

* add decimal test

* decimal add test cases

* decimal use int256/int128

* decimal testing

* fix decimal table meta and add tests for decimal col streams

* fix create stream and create tsma

* test insert decimal values

* decimal from str

* test decimal input

* test parse decimal from string

* add taos_fetch_field_e api

* decimal insert tests

* test decimal operators

* decimal operator test

* feat:support decimal in raw block

* decimal operator tests

* decimal test

* feat:support decimal in raw block

* feat:support decimal in raw block

* feat:add schemaExt to SMqDataRsp

* feat:remove add schemaExt to SMqDataRsp

* feat:remove add schemaExt to SMqDataRsp

* feat:remove add schemaExt to SMqDataRsp

* decimal test operators

* decimal operator test

* test decimal operators

* test decimal compare operators

* decimal unary operator test

* decimal col with decimal col oper test

* test decimal col filtering

* fix decimal float operator test

* decimal test where filtering

* fix decimal filtering

* fix decimal order by

* fix decimal op test

* test decimal agg funcs

* test decimal functions

* remove assert

* fix ci build for ret check

* fix decimal windows build

* fix ci ret check

* skip decimal ret check

* skip decimal ret check

* fix decimal tests

* fix decimal ci test

* decimal test

* fix(tmq): heap user after free

* fix(tmq): double free

* fix(tmq): double free

* fix decimal tests

* fix(decimal): decimal test ci build

* fix(decimal): windows build

* fix(decimal): decimal test build

* fix(decimal): fix decimal build and tests

* fix(decimal): fix decimal tests

* fix(decimal): fix taos_fetch_fields_e api

* fix(decimal): fix decimal taos_fetch_fields_e api

* fix(decimal): rebase 3.0

* fix(decimal): fix decimal functions

* fix(decimal): fix decimal test case memory leak

* fix(decimal): fix decimal tests

* fix(decimal): fix decimal test case

* fix(decimal): fix decimal tests

* feat(decimal): fix unit tests

* feat(decimal): fix deicmal unit test

---------

Co-authored-by: wangmm0220 <wangmm0220@gmail.com>
Co-authored-by: yihaoDeng <yhdeng@taosdata.com>
2025-03-14 18:08:07 +08:00