Commit graph

43 commits

Author SHA1 Message Date
Kaili Xu
13ae5350b7
feat: rollup sma (#33063) 2025-09-26 17:32:32 +08:00
Kaili Xu
fccf3f6a62
enh: destroy mutex of file object (#32852) 2025-09-03 11:25:28 +08:00
Bomin Zhang
876979b5f5
feat[ts-6107]: shared storage (#31552)
* add API to use s3 as shared storage

* support using local file system as shared storage

* upload file to shared storage

* support read, compact and drop

* finish basic mnode & vnode msg processing

* follower sync migration state

* implement mnode transaction, and improve log

* send migration progress msg to dnode to avoid deadlock

* implement following migration

* remove mcount

* avoid redo migration on startup

* avoid follower deadlock when leader is down

* trigger migrate by timer,  avoid compact after migration

* comment out the usage of 'tcs' functions in stream

* change config item prefix from s3 to ss

* change db option prefix from s3 to ss

* rename s3 data struct, function, file to ss

* rename s3 macro to ss

* update s3 sql to ss

* rename remaining s3 items to ss

* check ss configruation, improve s3 retry

* grant object storage -> shared storage,  check ssEnabled

* fix memory leaks

* update build options

* omit sensitive information when dump config

* fix backward compatibility issue

* fix issues found in ci-checks

* fix some failed test cases

* avoid follower timeout and improve log

* fix: follower timeout because migration status not updated

* refuse migration if there's an in progress one

* fix ss test case

* remove garbage files and other minor improvement

* fix failed test cases

* update unit test

* fix failed test case

* fix failed test case

* update document

* update document and fix failed test cases

* fix minor issues in code, test and document

* check new commit after migration task is scheduled

* fix several issus

1. migrate information cannot be dropped sometimes because progress response was put into read queue.
2. memory leak in rare cases
3. data corruption in rare cases
4. failed test case

* add shared storage upgrade tool

* fix compile error
2025-07-14 16:33:53 +08:00
Hongze Cheng
a72c46a7fc more cases 2024-12-27 17:44:05 +08:00
Hongze Cheng
288dc33f97 enh: seperate tsdb async tasks to different thread pools 2024-12-19 17:48:11 +08:00
Jinqing Kuang
de70067436 fix(query)[TD-32564]. Fix memory leak in exceptional cases
In function tsdbTFileSetInitRef, clear all FileObj stored in
the variable lvl when an error occurs, and release the memory
allocated for lvl itself.
2024-10-16 10:48:41 +08:00
Hongze Cheng
7cdb54033b enh: make CI happy 2024-09-29 11:38:43 +08:00
Hongze Cheng
f8be93571b fix: memory leak when alloc error occurs 2024-09-29 10:46:38 +08:00
Hongze Cheng
9b3cd3fb8d enh: error code handle 2024-09-11 10:56:40 +08:00
Hongze Cheng
f18c1d8710 enh: error code handle 2024-09-10 14:35:43 +08:00
Hongze Cheng
63b64a90c0 enh: remove void with error code handle 2024-09-10 13:12:05 +08:00
Hongze Cheng
44c19e9da8 fix more 2024-09-06 15:36:57 +08:00
Minglei Jin
0614bc47a8 fix(tsdb/file2): free fobj if appending failed 2024-09-06 14:47:50 +08:00
Haojun Liao
f510f2961b fix(query): fix memory leak. 2024-08-27 18:45:20 +08:00
Haojun Liao
d3f6892345 fix(tsdb): fix memory leak. 2024-08-27 17:57:16 +08:00
Hongze Cheng
02c7474167 enh: remove some useless asserts 2024-08-22 09:47:44 +08:00
Hongze Cheng
8ef31bc186 enh: refactor return code 2024-07-29 20:14:15 +08:00
Hongze Cheng
8bfd9adabd ehn: refactor return code 2024-07-29 14:04:46 +08:00
Hongze Cheng
69fa2a187a enh: refactor return code 2024-07-24 13:58:17 +08:00
Hongze Cheng
69e79966b6 enh: check return code 2024-07-18 11:32:13 +08:00
Hongze Cheng
f729072044 enh: reduce compact block write time 2024-05-23 17:57:07 +08:00
Minglei Jin
e14d4e172d cos/multi-writing: fix remove with different version 2024-04-15 09:30:24 +08:00
Minglei Jin
d5e9169769 cos/multi-write: dnode part including mnode/vnode/tsdb 2024-04-08 10:31:49 +08:00
Benguang Zhao
962febef02 refact: rename tsdb snap range to tsdb fset range 2023-12-07 15:28:14 +08:00
Benguang Zhao
e429526789 refact: rename STSnapRange to STFileSetRange 2023-12-07 15:20:55 +08:00
Hongze Cheng
cea4be6f3c fix: bg task memory waste 2023-12-04 13:40:28 +08:00
Hongze Cheng
65ccc4aa48 feat: backgroud task priority 2023-11-29 18:01:31 +08:00
Hongze Cheng
faea18c38f fix: invalid read when file is deleted 2023-11-02 16:59:11 +08:00
Hongze Cheng
cdb1d8296d feat: concurrency on fileset 2023-10-27 17:48:19 +08:00
Benguang Zhao
eb4e2aa58f feat: restore incomplete fsm state with maxVerValid via snapshot replication 2023-10-19 11:04:36 +08:00
Benguang Zhao
0c41fa56dc feat: filter pExclude in tsdbFSCreateCopyRangedSnapshot 2023-10-19 11:00:44 +08:00
Benguang Zhao
1be2835fc1 enh: exclude a list of snap ranges in tsdbSnapReaderOpen 2023-10-19 10:17:34 +08:00
Benguang Zhao
573c28fd06 enh: iterate over a list of snap ranges for STsdbSnapReader 2023-10-19 10:17:28 +08:00
Hongze Cheng
03456ce443 fix: upgrade error when has tomb data 2023-09-26 09:23:22 +08:00
Hongze Cheng
cd88c8ec31 fix: little test 2023-09-06 14:45:53 +08:00
Hongze Cheng
8dbdeccc8f fix: coverity scan 2023-09-05 11:41:36 +08:00
Hongze Cheng
8582e0c71a fix: important fix for compact file corruption 2023-08-17 18:13:37 +08:00
Hongze Cheng
2e88ad2c14 fix: upgrade bug 2023-07-04 14:00:44 +08:00
Hongze Cheng
1556dedb85 fix file remove by edit 2023-06-29 14:03:01 +08:00
Hongze Cheng
d23902877e make it compile on win 2023-06-27 14:33:55 +08:00
Hongze Cheng
cc0727aee3 more code 2023-06-25 17:13:12 +08:00
Hongze Cheng
b6a567f92e more code 2023-06-15 14:43:14 +08:00
Hongze Cheng
4138361839 refact code 2023-06-14 14:05:41 +08:00
Renamed from source/dnode/vnode/src/tsdb/dev/tsdbFSet2.c (Browse further)