Commit graph

10 commits

Author SHA1 Message Date
Bomin Zhang
876979b5f5
feat[ts-6107]: shared storage (#31552)
* add API to use s3 as shared storage

* support using local file system as shared storage

* upload file to shared storage

* support read, compact and drop

* finish basic mnode & vnode msg processing

* follower sync migration state

* implement mnode transaction, and improve log

* send migration progress msg to dnode to avoid deadlock

* implement following migration

* remove mcount

* avoid redo migration on startup

* avoid follower deadlock when leader is down

* trigger migrate by timer,  avoid compact after migration

* comment out the usage of 'tcs' functions in stream

* change config item prefix from s3 to ss

* change db option prefix from s3 to ss

* rename s3 data struct, function, file to ss

* rename s3 macro to ss

* update s3 sql to ss

* rename remaining s3 items to ss

* check ss configruation, improve s3 retry

* grant object storage -> shared storage,  check ssEnabled

* fix memory leaks

* update build options

* omit sensitive information when dump config

* fix backward compatibility issue

* fix issues found in ci-checks

* fix some failed test cases

* avoid follower timeout and improve log

* fix: follower timeout because migration status not updated

* refuse migration if there's an in progress one

* fix ss test case

* remove garbage files and other minor improvement

* fix failed test cases

* update unit test

* fix failed test case

* fix failed test case

* update document

* update document and fix failed test cases

* fix minor issues in code, test and document

* check new commit after migration task is scheduled

* fix several issus

1. migrate information cannot be dropped sometimes because progress response was put into read queue.
2. memory leak in rare cases
3. data corruption in rare cases
4. failed test case

* add shared storage upgrade tool

* fix compile error
2025-07-14 16:33:53 +08:00
Alex Duan
c133489917
new-libtaos-with-ws (#29971)
* enh: rename libtaos.so to libtaosinternal.so

* enh: let python system-test work

* enh: shell for libinternal.so

* enh: refact script

* enh: wrapper for libtaosinternal

* enh: rename some files

* enh: let shell support internal driver

* enh: minor changes

* enh: minor changes

* add intenal.h

* enh: remove unused codes

* enh: minor changs

* feat: mac os issues

* enh: let python test use internal connection

* enh: let nettest work

* enh: let taosc -C work

* enh: test shell in mac os

* enh: minor changes

* enh: let libtaosinternal.so work in linux

* enh: update install script

* enh: for ci

* enh: for windows compile

* enh: minor changes

* enh: compile in windows

* enh: minor changes

* enh: taosinternal work in windows

* enh: add taos_internal_static

* enh: rollback os_for_wrapper

* fix: conflicts

* fix: conflicts

* fix: compile errors

* fix: conflicts

* enh: rename taosinternal to taosnative

* enh: rename taosinternal to taosnative

* fix: compile error

* enh: remove taosnative.h

* fix: compile errors

* enh: set default shell options

* fix: compile errors

* fix: compile errors

* debug: switch taosws branch to feat/new-libtaos-with-ws

* enh: taosBenchmark remove WEBSOCKET finished

* fix: CMakeLists.txt remove WEBSOCKET

* fix: compile errors

* fix: taosBenchmark delete restful code

* .gitignore remove taos-tools folder

* fix: remove taosdump WEBSOCKET MACRO

* fix: build error for taosdump

* fix: build project passed

* fix: support -Z for connect mode

* fix: build error

* fix: build error fixed

* fix: add -Z options

* fix: native and websocket with string

* fix: -Z option core

* fix: build unit test

* fix: build error

* fix: add colon for array

* fix: unit test can not include fun

* fix: CTest pointer null is nullptr

* fix: include pub.h

* fix: taosdump add -Z --driver options

* fix: taosdump support -Z with pub.h pub.c

* fix: toolsGetTimeDay move to pub.c

* fix: restore pub.c only simple fun

* fix: add error tips

* fix: tmfree remove from taosdump

* fix: remove double declare taos variant

* fix: taos support -Z option

* enh: configDir set with main fun

* fix: add g_arguments.configDir

* fix: show tips add cfgdir

* fix: g_argument is pointer

* fix: configDir is global var

* fix: declare g_configDir in benchMain.c

* fix: taos version not show on websocket

* stmt2 websocket prepare with supertable

* stmt2 websocket prepare build

* fix: stmt and stmt2 prepare add db name

* fix: remove -R --restful test case

* fix: taosBenchmark adjust priority cmd > json > evn

* fix: add connect mode test case connMode.py

* fix: clear evn and add host port check

* fix: army/test.py modify start taosAdapter is default

* change: system-test/test.py default start taosAdapter

* fix: add taosAdapter variant to system-test/test.py

* fix: -y must put behind other options

* fix: remove rest sml and add -N normal case

* fix: add -c work cfg test case

* del: queryMain.py remove rest query

* fix: remove groups have a blank append bug

* fix: caseBase.py add db in sql query

* fix: query_json.py remove rest test

* fix: taosBenchmark uniform dsn describe

* fix: add pub.h pub.c to tools public

* fix: case remove restful interface

* fix: build error

* fix: taosDriverCleanup no arg call

* fix: taosdump add test conn mode case taosdumpCommandline.py

* fix: modify insertFullType.json db name

* fix: taosdump connMode test case taosDumpCommandline.py passed

* enh: priority cmd > env > json

* fix: taosCli remove is_native variant

* fix: password is not pointer

* fix: comment -o test case

* fix: solve fun return code check

* fix: windows unresolved strcasecmp

* fix: remove coverHostToServAddr

* fix: forbid check ODR violation

* ci: trigger pr run

* fix: build error not del >>>>>3.0

* fix: support json dbinfo->vgroups have blank space

* fix: solve conflict with merge

* fix: with TrimCaseCmp fun to compare vgroups

* fix: del vgroups support blank code

* feat: wrapper add taos_fetch_fields_e

* fix: move jni from libtaosnative.so to libtaos.so

* fix: move clientTmqConnector.c for jni

* fix: fix assert check with del comment code

* fix: taosBenchmark add prepare stmt debug log

* fix: stmt2 already have debug log on preapare

* fix:  (double)LLONG_MAX force covert

* fix: stmt insert normal table with websocket

* fix: normal table with stmt too many ?

* fix: taosdump stmt_prepare with child tbname

* fix: build error add int code

* feat: stmtPrepare with child table to do

* taos -a options move to native test case

* fix:unit test benchmarkTest passed

* fix: query_json-with-sqlfile.py case no check results

* fix: queryMain.py case lost, put in tasks again

---------

Co-authored-by: Shengliang Guan <slguan@taosdata.com>
2025-03-20 22:54:29 +08:00
Shengliang Guan
4ed932aa81
Merge pull request #29845 from taosdata/enh/TD-33783-3.0
enh: grant support TDgpt
2025-02-24 16:56:11 +08:00
Jinqing Kuang
51ffff2079 feat(query)[TS-5470]: add syntax to specify minimum duration for event and state windows
Introduce the `true for` syntax to allow users to specify the minimum
duration for event and state windows. Add corresponding tests to
validate the feature. Updated the user manual with usage instructions.
2025-02-24 08:02:48 +08:00
kailixu
80183ad8f3 enh: grant support TDgpt 2025-02-20 11:11:27 +08:00
Shengliang Guan
2dda013646 enh: skil password check if create user with is_import 2024-12-20 10:13:53 +08:00
Pengrongkun
7af56a3120 Add TSDB_CODE_TSC_STMT_BIND_NUMBER_ERROR, Clear error message 2024-11-07 14:49:46 +08:00
kailixu
edabeb3a98 fix: compatibility for msg and error code 2024-06-21 11:22:32 +08:00
kailixu
79f8ab4596 fix: compatibility for msg and error code 2024-06-21 08:07:18 +08:00
cris.pei
26c9b757de Add the command-line argument '--output-config' for generating configuration in test cases that increase the compatibility of message types and error codes 2024-06-17 20:13:00 +08:00