Commit graph

67 commits

Author SHA1 Message Date
Kaili Xu
348907e4b1
enh: rsma docs and task monitor (#33219) 2025-10-13 17:00:00 +08:00
Kaili Xu
13ae5350b7
feat: rollup sma (#33063) 2025-09-26 17:32:32 +08:00
Bomin Zhang
876979b5f5
feat[ts-6107]: shared storage (#31552)
* add API to use s3 as shared storage

* support using local file system as shared storage

* upload file to shared storage

* support read, compact and drop

* finish basic mnode & vnode msg processing

* follower sync migration state

* implement mnode transaction, and improve log

* send migration progress msg to dnode to avoid deadlock

* implement following migration

* remove mcount

* avoid redo migration on startup

* avoid follower deadlock when leader is down

* trigger migrate by timer,  avoid compact after migration

* comment out the usage of 'tcs' functions in stream

* change config item prefix from s3 to ss

* change db option prefix from s3 to ss

* rename s3 data struct, function, file to ss

* rename s3 macro to ss

* update s3 sql to ss

* rename remaining s3 items to ss

* check ss configruation, improve s3 retry

* grant object storage -> shared storage,  check ssEnabled

* fix memory leaks

* update build options

* omit sensitive information when dump config

* fix backward compatibility issue

* fix issues found in ci-checks

* fix some failed test cases

* avoid follower timeout and improve log

* fix: follower timeout because migration status not updated

* refuse migration if there's an in progress one

* fix ss test case

* remove garbage files and other minor improvement

* fix failed test cases

* update unit test

* fix failed test case

* fix failed test case

* update document

* update document and fix failed test cases

* fix minor issues in code, test and document

* check new commit after migration task is scheduled

* fix several issus

1. migrate information cannot be dropped sometimes because progress response was put into read queue.
2. memory leak in rare cases
3. data corruption in rare cases
4. failed test case

* add shared storage upgrade tool

* fix compile error
2025-07-14 16:33:53 +08:00
Kaili Xu
dc2b021778
feat: support mount data path (#31677) 2025-07-12 15:39:29 +08:00
Hongze Cheng
d48547ed64 more code 2025-02-25 16:48:43 +08:00
dmchen
9715bb59fe fix/TD-33294-compact-coverage-fix-case 2024-12-20 11:33:26 +08:00
Shengliang Guan
4c1499ebeb
Merge pull request #29122 from taosdata/enh/3.0/TD-31709
Replace unsafe memory functions with safe versions in wal&sync&mnode.
2024-12-16 21:37:33 +08:00
kailixu
2fa755d49c fix: compile problem of community 2024-12-16 10:49:58 +08:00
xiao-77
c4ed102c11 Merge branch '3.0' into enh/3.0/TD-31709 2024-12-16 10:00:02 +08:00
Kaili Xu
60fdd0fe64
enh: calculation of compact time offset 2024-12-13 17:05:46 +08:00
xiao-77
c66a834d1e Replace unsafe memory functions with safe versions in wal&sync. 2024-12-13 15:19:29 +08:00
Hongze Cheng
8bf5f3f62b Merge branch '3.0' of https://github.com/taosdata/TDengine into feat/TS-4994-3.0 2024-12-13 13:54:45 +08:00
kailixu
f57765564d feat: remove obsolete codes 2024-12-05 17:50:10 +08:00
kailixu
03e051ee63 feat: dispatch auto compact 2024-12-05 17:15:23 +08:00
yihaoDeng
9e7da45d37 use safe sys func 2024-12-05 16:54:10 +08:00
yihaoDeng
c5c56606c4 use safe sys func 2024-12-05 16:35:45 +08:00
kailixu
a6e86619b0 feat: database auto compact 2024-12-05 13:06:53 +08:00
kailixu
615f7f395f enh: default compact start and end time if compact interval is not zero 2024-12-05 10:52:11 +08:00
kailixu
a3dc026c0d feat: db auto compact 2024-12-05 10:33:51 +08:00
kailixu
f4fc423f34 feat: dispatch db auto compact 2024-12-04 20:09:04 +08:00
kailixu
4707149369 feat: dispatch auto compact req 2024-12-04 17:50:17 +08:00
kailixu
67924ee55b feat: support auto compact 2024-11-26 19:46:48 +08:00
Hongze Cheng
9a59689fba more code 2024-11-13 16:17:16 +08:00
xsren
e03ccdc6e2 replace snprintf 2024-10-10 10:35:48 +08:00
dmchen
e5fe2d33a4 fix/TD-31891-remove-void-mnode3 2024-09-19 11:18:09 +00:00
dongming chen
994cbf0585
Merge branch '3.0' into fix/TD-31891-remove-void-mnode1 2024-09-12 16:23:38 +08:00
dmchen
8393565d52 fix/TD-31891-remove-void-mnode1-fix-case 2024-09-12 08:08:08 +00:00
Hongze Cheng
a260a5f2f7 enh: error handle in message encode and decode 2024-09-09 11:49:44 +08:00
dmchen
7a26566362 fix/TD-31891-remove-void-mnode1 2024-09-05 11:39:49 +00:00
dmchen
6640860749 fix/TD-31686-remove-void-arrayput-mnode 2024-08-28 07:54:24 +00:00
Hongze Cheng
45efa44f3b
Merge pull request #26947 from taosdata/fix/TD-30742-3.0
fix/TD-30742
2024-08-06 15:45:41 +08:00
dmchen
0bff078c95 fix/TD-30742 2024-08-02 04:46:30 +00:00
dmchen
7289008869 fix/TD-31176-error-str 2024-08-02 02:51:29 +00:00
dmchen
ac706ee37a fix/TD-31176 2024-08-02 02:11:59 +00:00
dmchen
345fce614a fix case 2024-07-27 06:22:58 +00:00
dmchen
fce5174d10 fix/TD-30989 2024-07-25 08:19:01 +00:00
kailixu
280c5d4b3d enh: return error code 2024-07-22 16:33:42 +08:00
dmchen
013121e1df fix case 2024-07-18 06:14:14 +00:00
dmchen
f727668ebb fix/TD-30989 2024-07-17 11:41:15 +00:00
wangmm0220
51a66a684a fix:[TS-5067] memory leak 2024-06-21 18:25:35 +08:00
Hongze Cheng
805cc682d9 format code 2024-06-19 16:01:55 +08:00
dmchen
de19557abe fix/TD-30625 2024-06-18 09:36:19 +00:00
Hongze Cheng
22e52d6d82
Merge pull request #24958 from taosdata/fix/TD-28920-3.0
fix/TD-28920
2024-03-01 09:21:07 +08:00
dmchen
4e67da7017 fix/TD-28920 2024-02-29 07:24:29 +00:00
dmchen
f4c4465c0e fix/TD-28490 2024-02-29 06:54:24 +00:00
dmchen
85b78b0947 fix/TD-28910 2024-02-29 02:43:15 +00:00
dmchen
00d6611ba7 fix/TD-28895 2024-02-28 08:31:45 +00:00
Benguang Zhao
871c898920 enh: change redo log of trans compact-db as prepare log 2024-02-27 17:05:36 +08:00
dmchen
917a6eb5a7 fix/TD-28503 2024-01-30 09:47:20 +00:00
dmchen
8ddff2eecb fix/TD-28073 2024-01-05 07:02:16 +00:00