Commit graph

7700 commits

Author SHA1 Message Date
Haojun Liao
5564eb215c fix(stream): fix invalid check. 2023-10-03 00:38:24 +08:00
Haojun Liao
5718158fc0 refactor: do some internal refactor. 2023-10-01 22:27:29 +08:00
Haojun Liao
ff44539468 fix(stream): drop related fill-history task when dropping stream tasks. 2023-09-28 23:00:29 +08:00
Haojun Liao
31acbc6fe2 refactor: do some internal refactor (discard the checkpoint source msg during restoring). 2023-09-28 22:44:13 +08:00
Haojun Liao
f2d8431619 fix(stream): enable new time range for stream task. 2023-09-28 18:26:52 +08:00
Haojun Liao
0047d1e1a6 Merge branch 'main' into fix/liaohj 2023-09-28 14:35:14 +08:00
Haojun Liao
0a8f4530f7 fix(tsdb): replace destroy with reset. 2023-09-28 11:42:19 +08:00
kailixu
b1a0912a61 enh: exclude systables in log/audit db 2023-09-28 10:15:34 +08:00
Haojun Liao
22e485d132 fix(stream): opt perf. 2023-09-28 09:25:23 +08:00
Haojun Liao
cc9b9d1054
Merge pull request #23102 from taosdata/fix/liaohj
fix(stream): fix several bugs and do some internal refactor for stream processing.
2023-09-28 09:12:39 +08:00
Minglei Jin
cd811577f3 fix(vnode/cos): use EIO as object put errno 2023-09-27 15:10:34 +08:00
Minglei Jin
5ce1dd6284 fix(tsdb/readerwriter): fix fd writing 2023-09-27 14:02:37 +08:00
Haojun Liao
e13efd5b04 refactor: optimize the fill-history task launch policy, and do some other internal refactor. 2023-09-27 14:00:48 +08:00
Minglei Jin
e6287c1a89 fix(tsdb/readerwriter): fix var shadowing 2023-09-26 19:18:10 +08:00
liuyao
ebf4283e91 Merge branch '3.0' of https://github.com/taosdata/TDengine into feat/TD-26174 2023-09-26 17:19:41 +08:00
kailixu
e1cd54a1b3 fix: rwlock reentrant problem on windows 2023-09-26 15:44:35 +08:00
kailixu
c680180894 fix: rwlock reentrant problem on windows 2023-09-26 15:37:29 +08:00
wade zhang
f6134b42d7
Merge pull request #23036 from taosdata/feat/TS-4011-3.0
feat/TS-4011: adjust config 'keep_time_offset' level from dnode to db
2023-09-26 14:25:54 +08:00
Haojun Liao
890d658aea refactor: do some internal refactor. 2023-09-26 14:11:45 +08:00
Haojun Liao
525ee369d3
Merge pull request #23032 from taosdata/fix/liaohj
fix(stream): fix several stability-related bugs in stream framework.
2023-09-26 14:10:58 +08:00
Haojun Liao
f3f18b0fa3 fix(stream): add new status for meta. 2023-09-26 12:15:12 +08:00
liuyao
ca2325f886 Merge branch '3.0' of https://github.com/taosdata/TDengine into feat/TD-26174 2023-09-26 10:41:31 +08:00
Hongze Cheng
03456ce443 fix: upgrade error when has tomb data 2023-09-26 09:23:22 +08:00
wade zhang
92f5803303
Merge pull request #23050 from taosdata/fix/TD-26478
fix(cos): use EIO as failed code
2023-09-26 09:02:59 +08:00
Haojun Liao
6cc3618ed9 fix(stream): fix error in pause scan-history data. 2023-09-26 01:09:55 +08:00
Haojun Liao
5a4a0aa0e3 fix(stream): fix invalid read and memory leak. 2023-09-25 23:10:33 +08:00
Minglei Jin
7fd7902dda fix(cos): use EIO as failed code 2023-09-25 18:38:55 +08:00
Benguang Zhao
5d588cb7d2 enh: trigger failover if tsdbMerge failure 2023-09-25 17:22:38 +08:00
kailixu
126a371170 chore: more code 2023-09-25 16:24:03 +08:00
kailixu
0632fb838a chore: code refactor 2023-09-25 16:17:57 +08:00
Haojun Liao
966ce4f30e Merge remote-tracking branch 'origin/fix/liaohj' into fix/liaohj 2023-09-25 16:00:02 +08:00
Haojun Liao
fcf4cc7169 fix(stream): fix the invalid counter for ready tasks in vnode, fix an invalid read. 2023-09-25 15:58:24 +08:00
kailixu
b0b4d782fc chore: code refactor 2023-09-25 15:57:54 +08:00
Shungang Li
743a5b0640 feat: add db config item keep_time_offset 2023-09-25 15:46:08 +08:00
Haojun Liao
dbc53069db
Merge branch 'main' into fix/liaohj 2023-09-25 10:41:18 +08:00
Haojun Liao
eb4078b7f3 fix(stream): record the tasks started info, and do some internal refactor. 2023-09-25 09:59:05 +08:00
Haojun Liao
c987f61300 fix(stream): fix race condition. 2023-09-25 01:37:27 +08:00
kailixu
986788d4fe enh: timeseries calculation 2023-09-24 09:14:52 +08:00
kailixu
ff3e36e679 enh: timeseries calculation 2023-09-24 08:41:45 +08:00
kailixu
53a9459c20 enh: update timeseries 2023-09-24 07:52:27 +08:00
Haojun Liao
4ea737571d refactor: do some internal refactor. 2023-09-24 01:26:51 +08:00
kailixu
bc8ebf1562 Merge branch '3.0' into enh/TS-3812-3.0 2023-09-23 18:19:08 +08:00
Haojun Liao
b58ec1ee57 fix(stream): update logs. 2023-09-23 15:22:48 +08:00
Haojun Liao
acd3321494 fix(stream): rsp when checkpoint source failed. 2023-09-23 14:13:33 +08:00
Haojun Liao
7d5f76c6f1 fix(stream): disable auto rsp for checkpoint source. 2023-09-23 11:31:14 +08:00
Haojun Liao
5155519559 fix(stream): fix invalid assert. 2023-09-22 23:06:48 +08:00
Haojun Liao
cf6c255f22 refactor: do some internal refactor. 2023-09-22 22:50:21 +08:00
kailixu
8f828e052b enh: timeseries calculation optimization 2023-09-22 20:07:51 +08:00
liuyao
0d983e28ef Merge branch '3.0' of https://github.com/taosdata/TDengine into feat/TD-26174 2023-09-22 19:25:41 +08:00
Haojun Liao
5b12e276cf fix(stream): add msg identifier check. 2023-09-22 18:13:18 +08:00
wade zhang
849b05a09b
Merge pull request #22927 from taosdata/fix/TD-26189
s3/block_cache: new tsdb block cache
2023-09-22 16:15:07 +08:00
liuyao
4f8e6d7795 Merge branch '3.0' of https://github.com/taosdata/TDengine into feat/TD-26174 2023-09-22 15:00:52 +08:00
Haojun Liao
898aea5cfb refactor(stream): refactor send msg. 2023-09-22 14:52:18 +08:00
Minglei Jin
4c151f2031 tsdb/readerwriter: remove size with download 2023-09-22 14:06:06 +08:00
Haojun Liao
7e4410f094
Merge pull request #22992 from taosdata/feat/TD-26052
feat:support vnode split in tmq
2023-09-22 13:57:21 +08:00
Haojun Liao
87ea745861 fix(stream): add some logs, and remove invalid assert. 2023-09-22 10:53:03 +08:00
Haojun Liao
f7bcff862d fix(stream): add check info detailed information. 2023-09-22 10:23:56 +08:00
wangmm0220
19806cbcab fix:memory leak 2023-09-22 10:12:11 +08:00
wangmm0220
5555371fbc fix:memory leak 2023-09-22 10:04:53 +08:00
wangmm0220
86a4bfe28d fix:memory leak 2023-09-22 10:02:16 +08:00
liuyao
dc2f7872f8 Merge branch '3.0' of https://github.com/taosdata/TDengine into feat/TD-26174 2023-09-22 09:41:15 +08:00
wangmm0220
803b17a2db fix:memory leak 2023-09-22 09:31:58 +08:00
Haojun Liao
b1ba716299 refactor: do some internal refactor. 2023-09-22 09:24:36 +08:00
Haojun Liao
b075b4438b fix(stream): handle dispatch checkpoint-trigger failure. 2023-09-21 23:15:18 +08:00
Haojun Liao
3c921ba66e
Merge pull request #23006 from taosdata/fix/3_liaohj
fix(tsdb): set the correct scaninfo to load the tomb data.
2023-09-21 22:13:16 +08:00
Haojun Liao
f93df639c1 fix(tsdb): set the correct scaninfo to load the tomb data. 2023-09-21 17:32:21 +08:00
Minglei Jin
ce1f866416 tsdb/reader: use pTsdb handle instead of pgsz to open file 2023-09-21 16:36:17 +08:00
kailixu
7445b9ff50 Merge branch '3.0' into enh/TS-3812-3.0 2023-09-21 16:25:06 +08:00
Haojun Liao
2efd155adf fix(stream): wait for the task checkpoint before stop. 2023-09-21 15:53:21 +08:00
Haojun Liao
0f97a21163 fix(tsdb): set the correct scaninfo to load the tomb data. 2023-09-21 15:21:01 +08:00
Haojun Liao
e4073bd9a8 fix(stream): update logs. 2023-09-21 14:03:14 +08:00
Haojun Liao
09accc71d2 log(stream): update log. 2023-09-21 14:01:41 +08:00
Haojun Liao
17586af0ce refactor: do refactor the sink task. 2023-09-21 13:32:57 +08:00
liuyao
3b3c5d7cb8 Merge branch '3.0' of https://github.com/taosdata/TDengine into feat/TD-26174 2023-09-21 10:11:36 +08:00
wangmm0220
f9c5aa01af fix:enable split if there are topics & fix memory leak 2023-09-21 09:20:34 +08:00
Haojun Liao
6633fb971b fix(stream): update logs. 2023-09-20 18:18:31 +08:00
Minglei Jin
f206fa07d0 s3/put: use put2 2023-09-20 15:59:58 +08:00
Haojun Liao
2418f509ea fix(stream): avoid invalid read. 2023-09-20 15:47:09 +08:00
Haojun Liao
1d1d9a1b0b fix(stream): unlock when returning. 2023-09-20 15:45:15 +08:00
Haojun Liao
c8ab761d2a fix(stream): return if failed. 2023-09-20 15:38:49 +08:00
Haojun Liao
2a781c509f fix(stream): add task status check. 2023-09-20 15:32:15 +08:00
Minglei Jin
d155cfcf0f tsdb/readerwriter: use tsdb's s3 block cache to read file page 2023-09-20 15:24:54 +08:00
liuyao
1cebda368a session&state operator buff 2023-09-20 15:06:08 +08:00
Haojun Liao
af487c9048 fix(stream): fix obj. 2023-09-20 14:50:43 +08:00
Haojun Liao
361cde6497 fix(stream): fix deadlock caused by streamTaskHalt function. 2023-09-20 14:49:09 +08:00
Haojun Liao
55d4a9686c log(tsdb): remove some logs. 2023-09-20 13:18:42 +08:00
dm chen
4497a01168
Merge branch 'main' into feat/TD-26127-audit-sql 2023-09-20 11:13:34 +08:00
Haojun Liao
392aaeb592 log(stream): remove logs. 2023-09-20 10:49:18 +08:00
Haojun Liao
779c0c5f19
Merge pull request #22970 from taosdata/fix/liaohj
fix(stream): keep the status entry in hash table, instead entry index.
2023-09-20 10:33:05 +08:00
wangmm0220
2881a2504c fix:conflicts from 3.0 2023-09-19 18:39:40 +08:00
dmchen
1a7bffb079 ci break 2023-09-19 18:01:38 +08:00
Haojun Liao
b99232fd7a fix(stream): limit the max scan times. 2023-09-19 17:45:28 +08:00
Haojun Liao
271e492188 refactor: update the log level. 2023-09-19 17:40:26 +08:00
wangmm0220
766652a88a fix:vnode tranform support in tmq 2023-09-19 17:27:19 +08:00
dm chen
ee913cbfbc
Merge branch 'main' into feat/TD-26127-audit-sql 2023-09-19 14:39:20 +08:00
dmchen
e36ff1282c revert dependency 2023-09-19 13:42:02 +08:00
wangmm0220
ad70dc178f fix:conflicts from 3.0 2023-09-19 11:56:52 +08:00
wangmm0220
edb42aa2aa fix:move from main to 3.0 2023-09-19 11:51:52 +08:00
Haojun Liao
e76bab7122 fix(stream): not handle the check msg for follower tasks. 2023-09-19 09:43:14 +08:00
slzhou
900aacd428 fix: fix sanitizer error 2023-09-19 09:40:00 +08:00
kailixu
6698861c8b chore: timeseries distribute 2023-09-19 08:29:51 +08:00
Hongze Cheng
b1797834c1 fix: set snapshot writer compression 2023-09-18 20:58:20 +08:00
wangmm0220
1e859084b7 fix:conflicts from 3.0 2023-09-18 19:57:33 +08:00
wangmm0220
4b74135350 fix:vnode tranform support in tmq 2023-09-18 19:47:28 +08:00
Haojun Liao
101161c52e
Merge pull request #22954 from taosdata/fix/TD-26323
fix:core in race condition for pTq->pExecStore & return if poll too l…
2023-09-18 17:41:01 +08:00
Haojun Liao
c33ef4ce88 fix(stream): add null check 2023-09-18 15:14:51 +08:00
wangmm0220
c3498fbfe8 fix:core in race condition for pTq->pExecStore & return if poll too long & fix test cases if submit empty 2023-09-18 14:35:07 +08:00
wangmm0220
7869b693f2 add test case for tmq in vnode transform & fix core in race condition for pTq->pExecStore 2023-09-18 14:09:28 +08:00
dmchen
b473122dac break dependency 2023-09-18 12:29:08 +08:00
dm chen
fda32dcc51
Merge branch 'main' into feat/TD-26127-audit-sql 2023-09-18 09:41:47 +08:00
dmchen
c674e025e2 skip auto create tb 2023-09-18 09:29:51 +08:00
kailixu
8111c18cc4 chore: merge 3.0 2023-09-18 09:20:43 +08:00
Haojun Liao
8aa9960760
Merge pull request #22944 from taosdata/fix/liaohj
refactor: do some internal refactor, and fix some bugs.
2023-09-17 21:25:33 +08:00
Haojun Liao
01144c58a7 fix(stream): fix stream task id error. 2023-09-17 01:19:59 +08:00
Haojun Liao
4846194bbb refactor: do some internal refactor, remove the related fill-history task by sending mesg to replay it in follower node. 2023-09-16 21:37:30 +08:00
Haojun Liao
caa7542cea
Merge pull request #22913 from taosdata/fix/liaohj
fix(stream): fix error in drop task.
2023-09-16 21:36:38 +08:00
Haojun Liao
91e3d70796 fix(stream): remove fill-history sink task. 2023-09-15 17:46:17 +08:00
Minglei Jin
0181bb24d3 tsdb/readerwriter: use PRId64 instead of lld 2023-09-15 16:40:04 +08:00
Alex Duan
47f2c38a56
Merge pull request #22910 from taosdata/fix/TS-3987
fix: delete data error
2023-09-15 16:19:29 +08:00
Minglei Jin
318281356b s3/block_cache: new tsdb block cache 2023-09-15 16:19:21 +08:00
kailixu
03aea454d7 chore: more code 2023-09-15 14:33:35 +08:00
wade zhang
b123dd7df3
Merge pull request #22904 from taosdata/FIX/TD-26292-main
enh: ignore compact req during restoring
2023-09-15 11:05:18 +08:00
Haojun Liao
2bfd6e3355 refactor: do some internal refactor. 2023-09-15 09:25:22 +08:00
Haojun Liao
117de7ab7d fix(stream): fix error in drop task. 2023-09-15 00:51:13 +08:00
Haojun Liao
500fcad36c
Merge pull request #22892 from taosdata/fix/liaohj
refactor: do some internal refactor.
2023-09-15 00:46:46 +08:00
Haojun Liao
73c2782d78 fix(stream): update some logs. 2023-09-14 23:17:23 +08:00
Haojun Liao
5f55f489d8 fix(stream): fix memory leak. 2023-09-14 22:59:11 +08:00
Haojun Liao
303e8caf0c fix(stream): set the correct guard flag. 2023-09-14 22:44:16 +08:00
kailixu
449092c5dc enh: timeseries update optimize 2023-09-14 20:16:02 +08:00
Haojun Liao
27f626769e fix(stream): fix memory leak. 2023-09-14 19:33:05 +08:00
Haojun Liao
f2c3e0a6e8 fix(stream): disable follower send hb to mnode. and do some internal refactor. 2023-09-14 17:22:31 +08:00
Hongze Cheng
925cc8f55f fix: delete data error 2023-09-14 17:20:10 +08:00
Benguang Zhao
bbcbe460d4 enh: ignore compact req during restoring 2023-09-14 16:17:17 +08:00
Benguang Zhao
2886913fd8 enh: ignore compact req during restoring 2023-09-14 16:09:00 +08:00
Haojun Liao
fce5be0eb4 fix(stream): fix dead lock. 2023-09-14 14:09:05 +08:00
Haojun Liao
fd6344b319 fix(tsdb): check return code. 2023-09-14 09:25:35 +08:00
Haojun Liao
9b70c0c457 fix:fix coverity issue and remove unnecessary files. 2023-09-14 09:06:10 +08:00
wangmm0220
39fb5f4334 fix:add vnode snaphot for tmq 2023-09-13 18:09:47 +08:00
Haojun Liao
7305c8dbe1 refactor: do some internal refactor about the sched status set 2023-09-13 14:00:34 +08:00
wangmm0220
1dbd322fa0 fix:remove useless code 2023-09-13 11:42:09 +08:00
Haojun Liao
99477babca
Merge pull request #22872 from taosdata/fix/coverityScanDyh
fix coverity scan problem
2023-09-13 09:18:27 +08:00
Haojun Liao
09a15bf340 refactor: do some internal refactor. 2023-09-12 22:07:52 +08:00
Haojun Liao
60f5056ab7 other: merge 3.0 2023-09-12 22:00:40 +08:00
Haojun Liao
f6a097d96f refactor: do some internal refactor. 2023-09-12 21:58:22 +08:00
Haojun Liao
4325a0a433
Merge pull request #22866 from taosdata/fix/3_liaohj
refactor: do  some internal refactor.
2023-09-12 21:56:44 +08:00
yihaoDeng
8fc33dd74d fix coverity scan problem 2023-09-12 18:52:16 +08:00
Haojun Liao
e457d94ac6 refactor: do some internal refactor. 2023-09-12 18:09:02 +08:00
yihaoDeng
8e064cdd03 fix merge error 2023-09-12 16:41:14 +08:00
Haojun Liao
4d33387e80
Merge pull request #22814 from taosdata/fix/fixTransferCrash2
fix merge error
2023-09-12 16:10:07 +08:00
Haojun Liao
52931b530d
Merge pull request #22855 from taosdata/fix/3_liaohj
fix(stream):check error when vnode is closing.
2023-09-12 15:58:02 +08:00