Commit graph

24 commits

Author SHA1 Message Date
Bomin Zhang
876979b5f5
feat[ts-6107]: shared storage (#31552)
* add API to use s3 as shared storage

* support using local file system as shared storage

* upload file to shared storage

* support read, compact and drop

* finish basic mnode & vnode msg processing

* follower sync migration state

* implement mnode transaction, and improve log

* send migration progress msg to dnode to avoid deadlock

* implement following migration

* remove mcount

* avoid redo migration on startup

* avoid follower deadlock when leader is down

* trigger migrate by timer,  avoid compact after migration

* comment out the usage of 'tcs' functions in stream

* change config item prefix from s3 to ss

* change db option prefix from s3 to ss

* rename s3 data struct, function, file to ss

* rename s3 macro to ss

* update s3 sql to ss

* rename remaining s3 items to ss

* check ss configruation, improve s3 retry

* grant object storage -> shared storage,  check ssEnabled

* fix memory leaks

* update build options

* omit sensitive information when dump config

* fix backward compatibility issue

* fix issues found in ci-checks

* fix some failed test cases

* avoid follower timeout and improve log

* fix: follower timeout because migration status not updated

* refuse migration if there's an in progress one

* fix ss test case

* remove garbage files and other minor improvement

* fix failed test cases

* update unit test

* fix failed test case

* fix failed test case

* update document

* update document and fix failed test cases

* fix minor issues in code, test and document

* check new commit after migration task is scheduled

* fix several issus

1. migrate information cannot be dropped sometimes because progress response was put into read queue.
2. memory leak in rare cases
3. data corruption in rare cases
4. failed test case

* add shared storage upgrade tool

* fix compile error
2025-07-14 16:33:53 +08:00
yihaoDeng
6a4aea14ca add test case 2024-11-22 16:43:09 +08:00
yihaoDeng
ebcdad9666 add test case 2024-11-22 08:25:24 +08:00
54liuyao
5cfe7b8ec9 fix(query):fix the issue of return db error when the timestamp is 0 2024-09-11 14:58:06 +08:00
Haojun Liao
74f753606b fix(query): add regression test case. 2023-05-26 10:37:36 +08:00
Haojun Liao
e6b0560d47 test(query): update the sim. 2023-02-14 11:55:21 +08:00
shenglian zhou
3f577c5019 fix: show table distributed only works on super table, child table, and normal table 2023-02-02 11:57:22 +08:00
dapan1121
294a437657 fix: fix percentile + interval res error 2022-12-20 19:50:09 +08:00
Shengliang Guan
058788b2eb test: update asan case 2022-11-16 13:28:29 +08:00
Xiaoyu Wang
b5e177f151 fix: some problems of parser 2022-08-29 18:52:00 +08:00
Ganlin Zhao
9d4c19ac48 fix(query): fix test cases 2022-08-15 14:14:28 +08:00
Xiaoyu Wang
9a9aef985e enh: show command optimize 2022-08-11 20:26:40 +08:00
Haojun Liao
7df0a65e75 fix(query): set correct meta size in handling json. 2022-08-09 20:57:34 +08:00
Xiaoyu Wang
32418f161b enh: cumulative functions support mixed use 2022-07-29 20:41:16 +08:00
Shengliang Guan
cb1b679429 test: valgrind case 2022-07-16 20:27:41 +08:00
Shengliang Guan
697ab1c716 test: restore some cases 2022-07-15 09:52:57 +08:00
Shengliang Guan
c36c9bca1f test: restore some 2.0 case 2022-07-14 20:38:59 +08:00
Haojun Liao
159d6c47c8
Merge pull request #14911 from taosdata/feature/3_liaohj
fix(query): fix memory leak.
2022-07-14 20:36:30 +08:00
Haojun Liao
707f0a837a fix(query): set correct fill output object. 2022-07-14 20:11:29 +08:00
Shengliang Guan
31dbb07d8a test: restore some 2.0 case 2022-07-14 19:32:34 +08:00
Shengliang Guan
9b5f662275 test: restore some 2.0 case 2022-07-14 15:16:18 +08:00
Shengliang Guan
7d1c2fdbe5 test: restore some 2.0 case 2022-07-14 15:16:09 +08:00
Shengliang Guan
2f7d682807 test: restore some 2.0 case 2022-07-13 20:00:07 +08:00
Shengliang Guan
0ba9d22593 test: restore some 2.0 case 2022-07-13 17:59:25 +08:00