Elgato_dark/TDengine
1
0
Fork 0
mirror of https://github.com/taosdata/TDengine synced 2026-05-24 10:09:01 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 4
67094 commits 554 branches 514 tags 1.7 GiB
Commit graph

2864 commits

Author SHA1 Message Date
Haojun Liao
32ce4b6a4c fix(stream): fix race condition. 2023-12-20 23:12:59 +08:00
Haojun Liao
64eee4bbd2 fix(stream): add conflict check for nodeUpdate 2023-12-20 19:06:32 +08:00
Haojun Liao
849aaf8d4f fix(stream): avoid dead lock. 2023-12-20 18:07:53 +08:00
yihaoDeng
d6297e26ce init backend mutex 2023-12-20 16:56:12 +08:00
Haojun Liao
43fb7cf14a other: add log info. 2023-12-20 16:04:20 +08:00
wangmm0220
de8ce3f7ec fix:send stream retrieve msg to source task 2023-12-20 16:01:17 +08:00
Haojun Liao
1a8583887f fix(stream): remove clear backendcache. 2023-12-20 15:35:43 +08:00
Haojun Liao
94b260a1f9 refactor: do some internal refactor. 2023-12-20 10:29:04 +08:00
Haojun Liao
17e9e84aa9 Merge branch '3.0' into fix/3_liaohj 2023-12-19 16:43:08 +08:00
Haojun Liao
fee198f9ad refactor: do some internal refactor. 2023-12-19 16:21:11 +08:00
wangmm0220
81b5f59fa5 Merge branch 'feat/ly_multi_marks' of https://github.com/taosdata/TDengine into feat/ly_multi_marks 2023-12-19 10:54:12 +08:00
wangmm0220
3a0af1ea49 fix:aggCnt must bigger than 1 & move msg code to the end to be compatible with old version 2023-12-19 10:53:46 +08:00
Haojun Liao
ec3183b6a8 other: merge 3.0 2023-12-18 23:16:37 +08:00
Haojun Liao
612b50c4cd
Merge pull request #24097 from taosdata/fix/TD-27834 
fix stream backend crash when transfer
 2023-12-18 23:13:08 +08:00
Haojun Liao
e734569de0 fix(stream): fix dead lock. 2023-12-18 18:42:57 +08:00
Haojun Liao
bba5a4f08b other: merge main into 3.0 2023-12-18 18:26:42 +08:00
Haojun Liao
ea6e78cbaa fix(stream): add async call restart, instead of sync wait. 2023-12-18 16:33:44 +08:00
Haojun Liao
79672f8e5f
Merge pull request #24107 from taosdata/fix/3_liaohj 
fix(stream): add check for null ptr and leader .
 2023-12-18 16:24:07 +08:00
yihaoDeng
1dea66d4f5 fix stream backend crash when transfer 2023-12-18 15:48:01 +08:00
Haojun Liao
b4a1907fb7 fix(stream): add check for null ptr and leader . 2023-12-18 11:52:59 +08:00
yihaoDeng
0840e6a0d8 fix stream backend crash when transfer 2023-12-18 11:19:15 +08:00
Haojun Liao
ef0cab18cb
Merge pull request #24098 from taosdata/fix/3_liaohj 
fix(stream): send checkpoint complete msg to mnode when dropping it.
 2023-12-17 11:03:04 +08:00
Haojun Liao
4f852704f5 fix(stream): send checkpoint complete msg to mnode when dropping it. 2023-12-17 02:06:01 +08:00
Haojun Liao
9bd80cf0f8
Merge pull request #24094 from taosdata/fix/TD-27905 
Fix/td 27905
 2023-12-16 18:07:17 +08:00
yihaoDeng
5aa575acfa fix stream backend crash when transfer 2023-12-16 18:03:32 +08:00
yihaoDeng
6f4fb506a0 fix stream backend crash when transfer 2023-12-16 17:43:41 +08:00
yihaoDeng
8045d2bda7 fix stream backend crash when transfer 2023-12-16 15:20:38 +08:00
yihaoDeng
547d797a26 fix stream backend crash when transfer 2023-12-16 15:05:27 +08:00
Haojun Liao
9978d8cbfd
Merge pull request #24089 from taosdata/fix/TD-27880 
Fix/td 27880
 2023-12-16 13:13:48 +08:00
yihaoDeng
7ab7a64eb3 Merge branch '3.0' into fix/snodeBackendCrash 2023-12-15 18:16:14 +08:00
yihaoDeng
73c7e17033 fix error case 2023-12-15 18:02:14 +08:00
yihaoDeng
d8093d9f21 not do chkp when pbackend not inited 2023-12-15 11:47:44 +08:00
yihaoDeng
5189f3d15b not do chkp when pbackend not inited 2023-12-15 09:18:37 +08:00
Haojun Liao
05ca8f1784
Merge pull request #24056 from taosdata/fix/3_liaohj 
refactor: do some internal refactor.
 2023-12-14 13:52:04 +08:00
Haojun Liao
81f96603b0 refactor: do some internal refactor. 2023-12-14 09:33:51 +08:00
Minglei Jin
2dbb718101 Merge branch '3.0' into enh/TD-27119 2023-12-08 14:48:17 +08:00
yihaoDeng
a82856dff7 fix mem leak 2023-12-05 20:28:12 +08:00
yihaoDeng
c692f8c21c add stream backend copy 2023-12-05 18:03:58 +08:00
yihaoDeng
87c78919a9 add stream backend copy 2023-12-05 17:09:03 +08:00
54liuyao
38900d672e Merge branch '3.0' of https://github.com/taosdata/TDengine into feat/TD-27419 2023-12-04 09:12:53 +08:00
Haojun Liao
48af3a9559
Merge pull request #23909 from taosdata/fix/TD-27455 
fix:[TD-27644]heartbeat closed before snode close leading to snode is…
 2023-12-04 09:10:00 +08:00
wangmm0220
f06117dbc9 fix:[TD-27644]heartbeat closed before snode close leading to snode is hanged 2023-12-01 17:32:29 +08:00
Haojun Liao
d7a19a9179 fix(stream): fix invalid decode hbmsg and memory leaks caused by invalid hb msg. 2023-12-01 14:45:18 +08:00
Haojun Liao
4e5853d9fe refactor: do some internal refactor. 2023-12-01 12:09:49 +08:00
Haojun Liao
dba58f5e38 fix(stream): fix syntax error. 2023-12-01 11:54:13 +08:00
Haojun Liao
eac3cbf564 fix(stream): merge 3.0 2023-12-01 11:12:57 +08:00
Haojun Liao
e753d631af fix(stream): make sure open operator only once for scan-history task. 2023-12-01 10:21:07 +08:00
Haojun Liao
1df29b5d6c fix(stream): disable the time slice usage limitation for stream task. 2023-12-01 10:20:15 +08:00
Haojun Liao
57b8388dad fix(stream): fix error. 2023-12-01 10:20:00 +08:00
Haojun Liao
468094c86a fix(stream):set failure for the related fill-history task. 2023-12-01 10:19:45 +08:00
Haojun Liao
9d735e7f8b fix(stream): reduce the sink throughput. and fix the bug in status check. 2023-12-01 10:18:25 +08:00
Haojun Liao
2b03d64850 fix(stream): drop the related fill-history task & set the task to be ready, if the drop fill-history task msg is missing due to the vnode restarting. 2023-12-01 10:18:15 +08:00
Haojun Liao
f54fe2d6ef refactor: do some internal refactor. 2023-12-01 10:17:15 +08:00
Haojun Liao
fee9d751d4 fix(stream): fix the race condition in dispatching data. 2023-12-01 10:17:14 +08:00
Minglei Jin
49ecb1093f cos/put: new arg withcp to control resumable uploading 2023-11-30 17:20:21 +08:00
Haojun Liao
30d78c6be1 fix(stream): make sure open operator only once for scan-history task. 2023-11-30 16:10:09 +08:00
54liuyao
271311a770 Merge branch '3.0' of https://github.com/taosdata/TDengine into feat/TD-27419 2023-11-30 14:33:48 +08:00
yihaoDeng
5c32680cd8 Merge branch '3.0' into enh/refactorBackend 2023-11-30 10:31:27 +08:00
Haojun Liao
130614c938
Merge pull request #23831 from taosdata/fix/3_liaohj 
enh(stream/tsdb): do multiple optimization and fix some bugs.
 2023-11-30 09:20:15 +08:00
yihaoDeng
08cc976975 fix crash 2023-11-29 19:57:30 +08:00
yihaoDeng
03b698d8ae change default config 2023-11-29 14:50:07 +08:00
54liuyao
ae386c6e34 opt expired data 2023-11-29 11:16:56 +08:00
yihaoDeng
cc1eb2769f fix crash while quit 2023-11-28 20:29:42 +08:00
yihaoDeng
08eeee0794 Merge branch '3.0' into enh/refactorBackend 2023-11-28 11:52:49 +08:00
Haojun Liao
c5aba60899 fix(stream): disable the time slice usage limitation for stream task. 2023-11-28 09:28:43 +08:00
yihaoDeng
a4d475b582 fix rsma 2023-11-27 20:27:53 +08:00
wangmm0220
06e2fbfb1c fix:conflicts from 3.0 2023-11-27 17:53:53 +08:00
wangmm0220
70c5d36103 opti:extract common logic from snode & vnode to tqCommon 2023-11-27 17:49:09 +08:00
Haojun Liao
6111138567 fix(stream): fix error. 2023-11-27 17:23:34 +08:00
yihaoDeng
e9bec34c44 Merge branch '3.0' into enh/refactorBackend 2023-11-27 10:01:12 +08:00
yihaoDeng
6416a6153a Merge branch '3.0' into enh/refactorBackend 2023-11-24 16:29:09 +08:00
Haojun Liao
3deab3cfbf fix(stream):set failure for the related fill-history task. 2023-11-24 15:37:30 +08:00
yihaoDeng
23ae62d268 Merge branch '3.0' into enh/refactorBackend 2023-11-24 11:10:39 +08:00
yihaoDeng
50f996bbf7 Merge branch '3.0' into enh/refactorBackend 2023-11-23 21:07:58 +08:00
wangmm0220
d249892b2a fix:conflicts from 3.0 2023-11-23 20:29:46 +08:00
Haojun Liao
2e922fefc6 fix(stream): reduce the sink throughput. and fix the bug in status check. 2023-11-23 18:04:09 +08:00
yihaoDeng
6129ebc294 Merge branch '3.0' into enh/refactorBackend 2023-11-23 17:41:37 +08:00
Haojun Liao
0a90c33b4d fix(stream): drop the related fill-history task & set the task to be ready, if the drop fill-history task msg is missing due to the vnode restarting. 2023-11-23 15:49:07 +08:00
Haojun Liao
de47ea6c81 other: merge 3.0. 2023-11-23 15:42:40 +08:00
Haojun Liao
1a1319908d refactor: do some internal refactor. 2023-11-23 14:25:52 +08:00
yihaoDeng
6a0501967d Merge branch '3.0' into fix/fixChkpRespMemleak 2023-11-23 14:06:04 +08:00
Haojun Liao
8ad071e69a other: merge main. 2023-11-23 10:25:51 +08:00
Haojun Liao
f3355000a3 refactor: do some internal refactor. 2023-11-23 09:34:20 +08:00
Haojun Liao
00f8103262 fix(stream): fix error in generating token in bucket. 2023-11-23 09:34:19 +08:00
Haojun Liao
b7347a1d64 fix(stream): remove invalid assert 2023-11-23 09:33:41 +08:00
Haojun Liao
466c80e370 fix(stream): fix bug in delay send. 2023-11-23 09:33:38 +08:00
Haojun Liao
957ce07dc4 fix(stream): fix syntax error. 2023-11-23 09:33:38 +08:00
Haojun Liao
298755cc4e fix(stream): wait for task to be normal and then send data block. 2023-11-23 09:33:37 +08:00
Haojun Liao
7b1f586d4f
Merge pull request #23762 from taosdata/fix/3_liaohj 
fix(stream): fix error in generating token in bucket.
 2023-11-23 09:32:14 +08:00
wangmm0220
15d75a0497 fix:spelling errors 2023-11-23 09:17:54 +08:00
Haojun Liao
4ea49fe285 fix(stream): fix the race condition in dispatching data. 2023-11-22 23:49:16 +08:00
wangmm0220
9a09e318ca opti:common message process logic in vnode and snode 2023-11-22 16:53:33 +08:00
Haojun Liao
91e7000433 Merge branch 'enh/refactorBackend' of github.com:taosdata/tdengine into enh/refactorBackend 2023-11-22 15:29:31 +08:00
Haojun Liao
c96f7cf182 fix(stream): scheduler the checkpoint generating. 2023-11-22 15:17:21 +08:00
yihaoDeng
0eba15b4a4 Merge branch '3.0' into enh/refactorBackend 2023-11-22 10:57:41 +08:00
yihaoDeng
0c0d9cdd2c Merge branch '3.0' into fix/fixChkpRespMemleak 2023-11-22 10:42:30 +08:00
Haojun Liao
81d3e5737e fix(stream): fix send msg error. 2023-11-22 08:41:55 +08:00
Haojun Liao
c70c54a099 fix(stream): add missing pushed data. 2023-11-22 08:41:35 +08:00
Haojun Liao
b712f5e7dc refactor: do some internal refactor. 2023-11-22 08:41:17 +08:00
Haojun Liao
d355b78884 fix(stream): fix invalid read. 2023-11-22 08:40:28 +08:00
Haojun Liao
75420bfefb fix(stream): fix the checkpoint msg failure. 2023-11-22 08:40:01 +08:00
yihaoDeng
947775155b fix transfer crash 2023-11-21 20:51:11 +08:00
Haojun Liao
785d91c17b
Merge pull request #23661 from taosdata/feat/TD-26060 
fix: errors in snode in conditions like fill history/checkpoint/restart
 2023-11-21 19:02:38 +08:00
yihaoDeng
a92602b667 fix mem leak 2023-11-21 15:23:28 +08:00
Haojun Liao
2b099b11dd refactor: do some internal refactor. 2023-11-21 10:09:59 +08:00
yihaoDeng
939aa7a3cc Merge branch '3.0' into enh/refactorBackend 2023-11-20 17:33:30 +08:00
yihaoDeng
0d724244dc Merge branch '3.0' into enh/refactorBackend 2023-11-20 16:55:14 +08:00
Haojun Liao
9f8a63099c fix(stream): fix error in generating token in bucket. 2023-11-20 15:26:50 +08:00
Haojun Liao
7cfa45e95e enh(stream): add sink_quota/scan-history-idle-duration column for stream tasks. 2023-11-20 11:59:49 +08:00
yihaoDeng
734bf07317 Merge branch '3.0' into enh/refactorBackend 2023-11-20 10:48:49 +08:00
Haojun Liao
1b4f187ded
Merge pull request #23743 from taosdata/fix/pause_stream 
fix(stream): remove invalid assert
 2023-11-17 23:48:47 +08:00
Haojun Liao
709502ba9e fix(stream): record the check downstream failure. 2023-11-17 22:05:36 +08:00
wangmm0220
3241f9bda5 fix:memory leak 2023-11-17 17:26:52 +08:00
yihaoDeng
20de2f3cb5 Merge branch '3.0' into enh/refactorBackend 2023-11-17 17:15:14 +08:00
yihaoDeng
e7830bfdfd fix stream backend convert 2023-11-17 16:50:45 +08:00
Haojun Liao
753716f1e6 fix(stream): remove invalid assert 2023-11-17 16:36:17 +08:00
yihaoDeng
286413abbb fix stream backend convert 2023-11-17 16:14:51 +08:00
Haojun Liao
b12e5050ea fix(stream): remove invalid assert 2023-11-17 15:57:46 +08:00
yihaoDeng
ccc74ef558 fix stream backend convert 2023-11-17 15:39:03 +08:00
yihaoDeng
1dbdb32dd7 fix stream backend convert 2023-11-17 11:56:22 +08:00
yihaoDeng
c7c1bdef6e fix stream backend convert 2023-11-17 11:44:21 +08:00
yihaoDeng
e455c4799d fix stream backend convert 2023-11-17 10:33:43 +08:00
yihaoDeng
b6994413d0 fix stream backend convert 2023-11-17 10:13:39 +08:00
Haojun Liao
27f3723cd3
Merge pull request #23723 from taosdata/fix/3_liaohj 
fix(stream): wait for task to be normal and then send data block.
 2023-11-16 22:58:50 +08:00
yihaoDeng
52d7a26c10 fix stream backend convert 2023-11-16 21:22:47 +08:00
wangmm0220
a5de1b032b fix:snode upport stream task 2023-11-16 21:19:58 +08:00
Haojun Liao
3100edae5d
Merge pull request #23727 from taosdata/fix/pause_stream 
fix(stream): wait for task to be normal and then send data block.
 2023-11-16 19:51:33 +08:00
Haojun Liao
bc779edda9 fix(stream): fix bug in delay send. 2023-11-16 17:57:13 +08:00
Haojun Liao
f0f4b988a1 fix(stream): fix syntax error. 2023-11-16 17:57:13 +08:00
Haojun Liao
a69b870260 fix(stream): wait for task to be normal and then send data block. 2023-11-16 17:57:13 +08:00
Haojun Liao
811d661d83 fix(stream): fix bug in delay send. 2023-11-16 17:34:47 +08:00
Haojun Liao
37d181bb9c fix(stream): fix syntax error. 2023-11-16 17:00:50 +08:00
Haojun Liao
c0d3c15528 fix(stream): wait for task to be normal and then send data block. 2023-11-16 16:34:47 +08:00
yihaoDeng
f5ef2e727f fix stream backend convert 2023-11-16 14:41:04 +08:00
yihaoDeng
cee33ffb95 fix stream backend convert 2023-11-16 11:22:18 +08:00
yihaoDeng
77261068e0 fix stream backend convert 2023-11-16 11:06:55 +08:00
wangmm0220
a16189d2a9 fix:confilct from 3.0 2023-11-15 19:14:27 +08:00
Haojun Liao
f7ee65d059 fix(stream): fix send msg error. 2023-11-15 16:31:07 +08:00
Haojun Liao
d7fb9c5b6e fix(stream): add missing pushed data. 2023-11-15 16:10:26 +08:00
Haojun Liao
9d2256ec9c refactor: do some internal refactor. 2023-11-15 16:03:23 +08:00
Haojun Liao
2ca663ce66
Merge pull request #23593 from taosdata/feat/TD-21898 
stream event window
 2023-11-15 14:18:07 +08:00
dapan1121
dfb3feb5dc Merge remote-tracking branch 'origin/main' into merge/m23 2023-11-15 11:39:18 +08:00
Haojun Liao
51897a679f fix(stream): fix invalid read. 2023-11-15 11:21:31 +08:00
Haojun Liao
4af844ee61 fix(stream): fix the checkpoint msg failure. 2023-11-14 23:35:09 +08:00
yihaoDeng
b74105727d fix mem leak 2023-11-14 20:58:35 +08:00
yihaoDeng
20e1bd9cff fix invalid read/write 2023-11-14 20:46:56 +08:00
yihaoDeng
f4d6cf7c74 Merge branch '3.0' into enh/refactorBackend 2023-11-14 18:12:48 +08:00
yihaoDeng
aa2c09ef5b rm dup data 2023-11-14 18:09:00 +08:00
yihaoDeng
4476e46a5f rm dup data 2023-11-14 17:43:13 +08:00
dapan1121
478c8ef87f
Merge pull request #23665 from taosdata/fix/TD-27281-3.0 
fix: extend buffer of trace
 2023-11-14 13:50:50 +08:00
Haojun Liao
436c7871a3
Merge pull request #23673 from taosdata/fix/3_liaohj 
fix(stream): fix deadlock and do some internal refactor.
 2023-11-14 11:26:54 +08:00
Haojun Liao
e46a13c788
Merge pull request #23669 from taosdata/enh/stream_delay 
opt max delay
 2023-11-14 09:31:17 +08:00
Haojun Liao
f99e22b91f fix(stream): fix deadlock. 2023-11-13 23:56:42 +08:00
yihaoDeng
68b6cc1d9d Merge branch '3.0' into enh/refactorBackend 2023-11-13 19:07:58 +08:00
wangmm0220
56381dea90 fix:conflict from 3.0 2023-11-13 18:42:59 +08:00
wangmm0220
1cbe568be7 fix:add logic when snode restart 2023-11-13 18:39:42 +08:00
Haojun Liao
aad8a2eb87 fix(stream): fix deadlock. 2023-11-13 18:20:30 +08:00
kailixu
dd663a7650 fix: extend buffer of trace 2023-11-13 18:16:48 +08:00
yihaoDeng
4104840cea Merge branch '3.0' into enh/refactorBackend 2023-11-13 18:07:38 +08:00
liuyao
37c102a115 Merge branch '3.0' of https://github.com/taosdata/TDengine into feat/TD-21898 2023-11-13 16:06:17 +08:00
yihaoDeng
1786f7077d Merge branch '3.0' into enh/refactorBackend 2023-11-13 14:35:23 +08:00
yihaoDeng
de7f2a2d1a Merge branch '3.0' into enh/refactorBackend 2023-11-13 11:11:28 +08:00
yihaoDeng
cf3e01e942 Merge branch '3.0' into enh/refactorBackend 2023-11-13 10:34:44 +08:00
Haojun Liao
4ff8907b58 fix: fix syntax error. 2023-11-11 23:05:20 +08:00
Haojun Liao
9ee9174608 other: merge 3.0 2023-11-11 15:48:31 +08:00
Haojun Liao
3bacd7516e fix(stream): extact stream nodes list if not initialized. 2023-11-11 15:40:35 +08:00
Haojun Liao
15430f4d50 refactor: 2023-11-11 15:40:33 +08:00
Haojun Liao
5fcef5bd89 refactor:add some logs. 2023-11-11 15:40:32 +08:00
Haojun Liao
7a23df4b1a fix(stream): set the correct updated nodeId. 2023-11-11 15:40:32 +08:00
Haojun Liao
8e5db21c3f fix(test): fix syntax error. 2023-11-11 15:40:13 +08:00
Haojun Liao
af08a189c1 fix(stream): disable concurrently restart stream tasks. 2023-11-11 15:40:10 +08:00
Haojun Liao
e1de1de421 fix(stream): fix the invalid free. 2023-11-11 15:40:07 +08:00
Haojun Liao
97772e9aab fix(stream): the checkpoint version can only be updated when generating checkpoint. 2023-11-11 15:40:05 +08:00
Haojun Liao
0463c0d755 refactor: wait for a while when no quota available. 2023-11-11 15:38:41 +08:00
Haojun Liao
22808ce1b7 fix(stream): update the fill-time for quota limitation. 2023-11-11 15:38:32 +08:00
yihaoDeng
1e8d46c405 Merge branch '3.0' into enh/refactorBackend 2023-11-10 18:37:25 +08:00
Haojun Liao
438c303b0d other: merge main. 2023-11-10 17:05:36 +08:00
Haojun Liao
08a9244230 fix(stream): extact stream nodes list if not initialized. 2023-11-10 16:24:20 +08:00
yihaoDeng
a3fdb6dd59 Merge branch '3.0' into enh/refactorBackend 2023-11-10 16:06:08 +08:00
Haojun Liao
d0307e5876 refactor: 2023-11-10 15:30:21 +08:00
Haojun Liao
0b36081158 refactor:add some logs. 2023-11-10 14:36:18 +08:00
wangmm0220
8a05bad899 fix:snode error 2023-11-10 14:14:22 +08:00
Haojun Liao
1f9a58361d fix(stream): set the correct updated nodeId. 2023-11-10 13:36:35 +08:00
Haojun Liao
5824caa8f5
Merge pull request #23631 from taosdata/fix/liaohj 
multiple fix
 2023-11-10 13:09:40 +08:00
yihaoDeng
ad119ea4c0 Merge branch '3.0' into enh/refactorBackend 2023-11-10 12:07:33 +08:00
Haojun Liao
ecf6c1c09d fix(test): fix syntax error. 2023-11-10 10:10:38 +08:00
yihaoDeng
54fe42d372 Merge branch '3.0' into enh/refactorBackend 2023-11-10 09:24:40 +08:00
Haojun Liao
9978667c48 fix(stream): disable concurrently restart stream tasks. 2023-11-10 09:24:08 +08:00
liuyao
6d2d5823dc Merge branch '3.0' of https://github.com/taosdata/TDengine into feat/TD-21898 2023-11-10 09:24:05 +08:00
yihaoDeng
0669037de0 fix mem leak 2023-11-09 20:37:08 +08:00
yihaoDeng
4a99bdc2b0 support s3 2023-11-09 20:27:30 +08:00
yihaoDeng
5f9b922a04 support s3 2023-11-09 20:15:08 +08:00
Haojun Liao
da83f27124
Merge pull request #23610 from taosdata/fix/TD-27216 
change chkp gen way
 2023-11-09 18:55:31 +08:00
Haojun Liao
7f3a50dfde fix(stream): fix the invalid free. 2023-11-09 18:14:27 +08:00
yihaoDeng
757425bff8 Merge branch '3.0' into enh/refactorBackend 2023-11-09 17:58:49 +08:00
yihaoDeng
dd2252598e upload by rsync 2023-11-09 17:39:34 +08:00
Yihao Deng
0453991cdf
Merge branch '3.0' into fix/TD-27216-3 2023-11-09 17:12:56 +08:00
Haojun Liao
13c6d1b23a fix(stream): the checkpoint version can only be updated when generating checkpoint. 2023-11-09 17:11:37 +08:00
dapan1121
8314cb6013
Merge pull request #23494 from taosdata/enh/TD-23769-3.0x 
enh: checkpoint for rsma
 2023-11-09 14:51:37 +08:00
yihaoDeng
f74edc7618 Merge branch '3.0' into enh/refactorBackend 2023-11-09 06:14:08 +00:00
liuyao
e56a3e8a1e Merge branch '3.0' of https://github.com/taosdata/TDengine into feat/TD-21898 2023-11-09 13:59:39 +08:00
kailixu
1a950afdb8 enh: test case for rsma snapshot 2023-11-09 12:09:27 +08:00
Haojun Liao
257ab4f270
Merge pull request #23526 from taosdata/feat/TD-26060 
feat:checkpoint backup function
 2023-11-09 11:46:28 +08:00
yihaoDeng
679155f771 change chkpid gen way 2023-11-09 11:25:52 +08:00
yihaoDeng
a215332e2f change chkpid gen way 2023-11-09 11:24:18 +08:00
yihaoDeng
50666987f0 change chkpid gen way 2023-11-09 11:23:17 +08:00
Haojun Liao
ef646873ef
Merge pull request #23596 from taosdata/fix/streamTransferError 
fix stream transfer err
 2023-11-09 10:51:22 +08:00
kailixu
edef4de7b2 chore: revert the code 2023-11-08 20:57:57 +08:00
kailixu
72ecb0431c enh: rsma checkpoint 2023-11-08 20:45:44 +08:00
kailixu
e72582a1fa Merge branch 'fix/streamTransferError' into enh/TD-23769-3.0x 2023-11-08 20:45:21 +08:00
yihaoDeng
3abf0ae4b2 Merge branch 'feat/TD-26060' into enh/refactorBackend 2023-11-08 17:31:16 +08:00
yihaoDeng
92fa1043b8 Merge branch 'enh/refactorBackend' of https://github.com/taosdata/TDengine into enh/refactorBackend 2023-11-08 17:12:32 +08:00
yihaoDeng
2de5c4a97d add bg thread 2023-11-08 17:11:11 +08:00
yihaoDeng
76fcc49c6d fix stream transfer error 2023-11-08 17:10:54 +08:00
Haojun Liao
f949333e8b refactor:do some internal refactor. 2023-11-08 16:57:55 +08:00
Haojun Liao
470f244032 Merge remote-tracking branch 'origin/enh/refactorBackend' into enh/refactorBackend 
# Conflicts:
#	include/libs/stream/tstream.h
 2023-11-08 15:21:21 +08:00
Haojun Liao
99dbbe61d7 refactor(stream): do some internal refactor. 2023-11-08 15:20:47 +08:00
yihaoDeng
6a82cd7fbf Merge branch 'main' of https://github.com/taosdata/TDengine into fix/fixStreamTransferErrMain 2023-11-08 15:14:31 +08:00
yihaoDeng
b3c056e6cb fix stream transfer err 2023-11-08 15:12:47 +08:00
wangmm0220
c4c1423243 fix:error 2023-11-08 15:01:04 +08:00
yihaoDeng
7e03fa6664 fix stream transfer err 2023-11-08 14:57:20 +08:00
yihaoDeng
2e7d041e0a fix stream transfer err 2023-11-08 14:48:28 +08:00
liuyao
340fb3e727 Merge branch '3.0' of https://github.com/taosdata/TDengine into feat/TD-21898 2023-11-08 14:16:59 +08:00
yihaoDeng
7b0891981e merge 3.0 2023-11-08 14:14:01 +08:00
wangmm0220
7d19fcf19f Merge branch 'feat/TD-26060' of https://github.com/taosdata/TDengine into feat/TD-26060 2023-11-08 11:59:03 +08:00
wangmm0220
047eb14f67 fix:error 2023-11-08 11:57:49 +08:00
yihaoDeng
fb77d811f9 Merge branch 'enh/refactorBackend' of https://github.com/taosdata/TDengine into enh/refactorBackend 2023-11-08 10:48:36 +08:00
Haojun Liao
17bd6badec refactor(stream): do checkpoint for each task. 2023-11-08 10:47:54 +08:00
yihaoDeng
7ada7f34ac fix:merge from 3.0 2023-11-08 10:45:15 +08:00
wangmm0220
07aa749fcd fix:adapt to windows 2023-11-08 10:28:45 +08:00
yihaoDeng
594da2d603 merge 3.0 2023-11-08 10:04:46 +08:00
kailixu
b9f389a49f Merge branch '3.0' into enh/TD-23769-3.0x 2023-11-08 08:36:35 +08:00
kailixu
f2d24306b1 enh: rsma tasks share one checkpoint and fix memory leak 2023-11-08 08:35:52 +08:00
Haojun Liao
e052d3cd62 fix(stream): check null. 2023-11-08 00:10:08 +08:00
kailixu
6b738884a4 chore: rsma checkpoint 2023-11-07 19:59:05 +08:00
wangmm0220
f0b2ed567c fix:upload checkpoint to s3 2023-11-07 18:49:12 +08:00
Haojun Liao
73d75aac25 fix(stream): add ref for task. 2023-11-07 18:43:35 +08:00
Haojun Liao
26b96b149d refactor: wait for a while when no quota available. 2023-11-07 18:43:35 +08:00
Haojun Liao
0979ff944b fix(stream): update the fill-time for quota limitation. 2023-11-07 18:43:34 +08:00
Haojun Liao
2bdde122fc refactor: wait a while when no token available. 2023-11-07 18:43:34 +08:00
Haojun Liao
a0d81734f1 fix(stream): add ref for task. 2023-11-07 18:18:58 +08:00
Haojun Liao
43b14ba8b4 refactor: wait for a while when no quota available. 2023-11-07 17:49:49 +08:00
Haojun Liao
06d02b862e fix(stream): update the fill-time for quota limitation. 2023-11-07 17:33:30 +08:00
Haojun Liao
0b59681158 refactor: wait a while when no token available. 2023-11-07 17:18:58 +08:00
Haojun Liao
884770dab1 fix(stream): fix syntax error. 2023-11-07 17:12:06 +08:00
Haojun Liao
344c3c5b13 fix(stream): fix syntax error. 2023-11-07 17:11:36 +08:00
Haojun Liao
9c49de69f8 refactor: do some internal refactor. 2023-11-07 17:05:20 +08:00
Haojun Liao
ed0c580d11 refactor: do some internal refactor. 2023-11-07 17:04:55 +08:00
Haojun Liao
1301b92844 fix(stream): fix errors in scan-history, introduced by refactor 2023-11-07 16:41:15 +08:00
Haojun Liao
c9a1cc50fb refactor: do some internal refactor. 2023-11-07 16:40:04 +08:00
Haojun Liao
9125fe44c6 fix(stream): fix error. 2023-11-07 16:40:04 +08:00
Haojun Liao
8c2f6bab0e fix(stream): exec scan-history in timer. 2023-11-07 16:40:03 +08:00
Haojun Liao
7370daa6e0 fix(stream): record the start failure tasks. 2023-11-07 16:39:56 +08:00
Haojun Liao
e38e526dda fix(stream):reset the starting flag for the followers. 2023-11-07 16:39:12 +08:00
Haojun Liao
207b53f3f6 fix(stream): fix errors in scan-history, introduced by refactor 2023-11-07 16:19:09 +08:00
yihaoDeng
4f53ffd76f refactor backend 2023-11-07 16:01:57 +08:00
yihaoDeng
1ee82f5e2c refactor backend 2023-11-07 15:50:42 +08:00
liuyao
467c27c758 recover flush mark 2023-11-07 15:45:38 +08:00
yihaoDeng
8f7ad2bd90 refactor backend 2023-11-07 15:20:58 +08:00
yihaoDeng
042522ac7a refactor backend 2023-11-07 15:18:24 +08:00
yihaoDeng
e015134d6d refactor backend 2023-11-07 14:46:52 +08:00
liuyao
e972ab16fe recover flush mark 2023-11-07 14:11:08 +08:00
yihaoDeng
e5449bf161 refactor backend 2023-11-07 11:49:56 +08:00
yihaoDeng
95cfc5eb46 refactor checkpoint 2023-11-07 09:55:55 +08:00
yihaoDeng
021859361f refactor checkpoint 2023-11-07 09:22:57 +08:00
Haojun Liao
96f9264906 fix(stream): add null ptr check. 2023-11-07 00:38:02 +08:00
Haojun Liao
1312b9ea36 fix(stream): fix error. 2023-11-06 23:44:01 +08:00
Haojun Liao
4d8548e938 refactor: do some internal refactor. 2023-11-06 23:42:16 +08:00
yihaoDeng
c9bf4f6507 refactor checkpoint 2023-11-06 20:19:11 +08:00
yihaoDeng
819be65db0 refactor checkpoint 2023-11-06 19:59:11 +08:00
kailixu
f5d796a081 chore: print stream state 2023-11-06 19:53:09 +08:00
Haojun Liao
312dbc1caa refactor: do some internal refactor. 2023-11-06 19:49:25 +08:00
yihaoDeng
9ca1151b67 refactor checkpoint 2023-11-06 19:47:47 +08:00
Haojun Liao
1f09d3a46b fix(stream): fix error. 2023-11-06 19:46:47 +08:00
Haojun Liao
d9474016fc fix(stream): exec scan-history in timer. 2023-11-06 18:55:53 +08:00
yihaoDeng
aee1b700a6 Merge branch 'enh/new3.0' into enh/refactorBackend 2023-11-06 18:08:03 +08:00
yihaoDeng
ba52769867 Merge branch 'enh/new3.0' into enh/refactorBackend 2023-11-06 17:56:09 +08:00
Haojun Liao
6b1889284b fix(stream): record the start failure tasks. 2023-11-06 15:45:44 +08:00
yihaoDeng
feeead50c5 Merge branch 'enh/new3.0' into enh/refactorBackend 2023-11-06 13:26:38 +08:00
Haojun Liao
69863febe6 fix(stream):reset the starting flag for the followers. 2023-11-06 13:17:13 +08:00
Haojun Liao
d41fd27189 fix(stream): restart tasks in stream threads, instead of write thread. 2023-11-06 11:57:21 +08:00
Haojun Liao
0e44950a37 enh(stream): execute scan history by using time slice 1s. 2023-11-06 09:21:33 +08:00
Haojun Liao
1f756f58f6 fix(stream): add need update node into update node list. 2023-11-06 09:21:33 +08:00
Haojun Liao
680eb5674e refactor: do some internal refactor. 2023-11-06 09:15:49 +08:00
Haojun Liao
d440da0d8c enh(stream): execute scan history by using time slice 1s. 2023-11-05 01:26:53 +08:00
Haojun Liao
c71f6713a7 fix(stream): add need update node into update node list. 2023-11-05 01:24:23 +08:00
Haojun Liao
e13f003b90 refactor: do some internal refactor. 2023-11-05 01:23:23 +08:00
kailixu
411151d671 fix: buffer overflow/buffer use after free/memory leak 2023-11-04 08:44:26 +08:00
yihaoDeng
f038f88c07 Merge branch 'feat/TD-26060' into enh/refactorBackend 2023-11-03 18:29:18 +08:00
wangmm0220
5a6c3f2b04 fix:add windows support 2023-11-03 17:45:37 +08:00
kailixu
77eadc59dc Merge branch '3.0' into enh/TD-23769-3.0x 2023-11-03 16:26:07 +08:00
kailixu
a48968e5e8 enh: rsma checkpoint 2023-11-03 14:33:45 +08:00
Haojun Liao
53b2158c54 refactor: do some internal refactor. 2023-11-03 13:51:17 +08:00
wangmm0220
0a1f1085fc Merge branch '3.0' of https://github.com/taosdata/TDengine into feat/TD-26060 2023-11-03 11:33:50 +08:00
wangmm0220
3ac11eba0e fix:modify upload checkpoint interface 2023-11-03 11:33:18 +08:00
yihaoDeng
d82d51cf8f Merge branch 'enh/new3.0' into enh/refactorBackend 2023-11-03 10:40:00 +08:00
yihaoDeng
24e961976f Merge branch 'enh/new3.0' into enh/refactorBackend 2023-11-03 09:31:57 +08:00
Haojun Liao
9d77351bb2 refactor: do some internal refactor. 2023-11-03 09:30:50 +08:00
yihaoDeng
069dd8de84 Merge branch 'enh/new3.0' into enh/refactorBackend 2023-11-03 09:28:30 +08:00
liuyao
31ad73f6d1 Merge branch '3.0' of https://github.com/taosdata/TDengine into feat/TD-21898 2023-11-03 09:12:17 +08:00
yihaoDeng
0a73832fd2 add checkpoint delta 2023-11-02 20:39:55 +08:00
kailixu
6c944bb192 enh: skip rsma_task during load stream tasks 2023-11-02 20:32:54 +08:00
wangmm0220
159873692c feat:checkpoint bakeup using rsync 2023-11-02 20:13:29 +08:00
Haojun Liao
b3d57e4a2d fix(stream): not transfer state if state is not appropriate. 2023-11-02 19:31:35 +08:00
Haojun Liao
7a32b3a209 fix(stream): not transfer state if state is not appropriate. 2023-11-02 19:12:36 +08:00
Haojun Liao
706f1e4744 fix(stream): fix deadlock in transfer state. 2023-11-02 18:18:04 +08:00
Haojun Liao
39175dc03b fix(stream): fix deadlock in transfer state. 2023-11-02 18:07:36 +08:00
Haojun Liao
c48c801e19 fix(stream): return error code. 2023-11-02 11:02:40 +08:00
Haojun Liao
95efa07e92 fix(stream): add full unsupported event filtering. 2023-11-02 11:02:39 +08:00
Haojun Liao
05b3d51803 fix(stream): add qualified status in assert 2023-11-02 11:02:39 +08:00
Haojun Liao
420825c62f fix(stream): return error code. 2023-11-02 10:22:03 +08:00
yihaoDeng
7e7a39f077 Merge branch 'enh/new3.0' into enh/refactorBackend 2023-11-02 10:15:34 +08:00
yihaoDeng
b911492270 Merge branch 'enh/new3.0' into enh/refactorBackend 2023-11-02 09:46:36 +08:00
Haojun Liao
351a31302f fix(stream): add full unsupported event filtering. 2023-11-02 09:43:48 +08:00
Haojun Liao
e14d42c01e fix(stream): add qualified status in assert 2023-11-02 08:46:18 +08:00
Haojun Liao
dc8a90c864 fix(stream): fix memory leak. 2023-11-02 08:13:11 +08:00
Haojun Liao
79e971385f fix(stream): fix memory leak. 2023-11-02 08:11:51 +08:00
Haojun Liao
df7633650e fix(stream): fix memory leak. 2023-11-01 20:35:45 +08:00
Haojun Liao
e1b3e00b3d fix(stream): fix memory leak. 2023-11-01 20:10:20 +08:00
Haojun Liao
6dc468c55f fix(stream): fix dead-lock 2023-11-01 19:38:13 +08:00
Haojun Liao
fa7ebf63ea refactor: do some internal refactor. 2023-11-01 19:38:13 +08:00
Haojun Liao
9156e8449d fix(stream): fix deference null ptr. 2023-11-01 19:38:13 +08:00
Haojun Liao
7cf90dde5c refactor: do some internal refactor. 2023-11-01 19:38:13 +08:00
Haojun Liao
00f029e44f refactor: add some logs. 2023-11-01 19:38:12 +08:00
Haojun Liao
385e699cc2 fix(stream): do some internal refactor. 2023-11-01 19:38:12 +08:00
Haojun Liao
bd12ae88e1 fix(stream):fix the bug when event in waiting list not fulfilled. 2023-11-01 19:38:12 +08:00
Haojun Liao
9a9c1d689d fix(stream): add lock log. 2023-11-01 19:38:12 +08:00
Haojun Liao
e7609d8e56 fix(stream): fix dead-lock 2023-11-01 19:09:21 +08:00
Haojun Liao
ae25f09c45 refactor: do some internal refactor. 2023-11-01 18:03:44 +08:00
Haojun Liao
899d13b4ae fix(stream): fix deference null ptr. 2023-11-01 17:25:59 +08:00
Haojun Liao
246ed4e022 refactor: do some internal refactor. 2023-11-01 17:19:21 +08:00
liuyao
505c44d9aa Merge branch '3.0' of https://github.com/taosdata/TDengine into feat/TD-21898 2023-11-01 16:50:46 +08:00
Haojun Liao
12bc4f32cf refactor: add some logs. 2023-11-01 16:12:15 +08:00
yihaoDeng
4c8edb5545 Merge branch 'enh/new3.0' into enh/refactorBackend 2023-11-01 16:07:49 +08:00
Haojun Liao
778c29b808 fix(stream): do some internal refactor. 2023-11-01 14:07:18 +08:00
Haojun Liao
e4aeea176b fix(stream):fix the bug when event in waiting list not fulfilled. 2023-11-01 13:59:51 +08:00
Haojun Liao
5ff89bc098 fix(stream): add lock log. 2023-11-01 13:19:53 +08:00
Haojun Liao
3f561fb8e4 fix(stream): remove invalid assert. 2023-11-01 11:43:02 +08:00
Haojun Liao
3e6c89d8eb fix(stream): add unsupport trans. 2023-11-01 11:43:02 +08:00
kailixu
1530fe0c49 Merge branch '3.0' into enh/TD-23769-3.0x 2023-11-01 09:52:16 +08:00
kailixu
ad1e6accd4 chore: build checkpoint for rsma 2023-11-01 09:45:58 +08:00
Haojun Liao
0fb4cfd94f fix(stream): remove invalid assert. 2023-11-01 00:33:17 +08:00
Haojun Liao
639d5cde47 fix(stream): add unsupport trans. 2023-10-31 23:14:02 +08:00
yihaoDeng
bb8252df16 fix mem leak 2023-10-31 20:59:24 +08:00
Haojun Liao
294c59150c fix(stream): add jump out loop by checking task status. 2023-10-31 19:31:28 +08:00
Haojun Liao
4694859325 fix(stream): add jump out loop by checking task status. 2023-10-31 19:31:00 +08:00
Haojun Liao
0b18192b23 fix(stream): fix error in finishing event handling. 2023-10-31 16:40:16 +08:00
Haojun Liao
762cfef498 fix(stream): adjust the stop condition for stream tasks. 2023-10-31 16:40:16 +08:00
Haojun Liao
4c0242a58d fix(stream): fix error in finishing event handling. 2023-10-31 16:39:14 +08:00
Haojun Liao
e01c7f3cfd fix(stream): adjust the stop condition for stream tasks. 2023-10-31 11:51:47 +08:00
Haojun Liao
88469568b8 fix(stream): handle failure during checkpoint 2023-10-31 10:45:14 +08:00
Haojun Liao
635243f01a refactor: do some internal refactor. 2023-10-31 10:45:12 +08:00
Haojun Liao
0c982093cd fix(stream): enable the set of fill-history task to be un-init. 2023-10-31 10:45:12 +08:00
Haojun Liao
86dfc52a6e fix(stream): add event check when handling the end of event. 2023-10-31 10:45:12 +08:00
Haojun Liao
72f3e4446d fix(stream): handle failure during checkpoint 2023-10-31 10:38:17 +08:00
yihaoDeng
7fad855247 fix mem leak 2023-10-30 18:16:12 +08:00
yihaoDeng
ee6b620597 fix mem leak 2023-10-30 17:29:32 +08:00
Haojun Liao
fdf3f210ac refactor: do some internal refactor. 2023-10-30 14:39:24 +08:00
Haojun Liao
93232a0e14 fix(stream): enable the set of fill-history task to be un-init. 2023-10-30 11:48:51 +08:00
Haojun Liao
19c5882cbe fix(stream): add event check when handling the end of event. 2023-10-30 10:22:24 +08:00
Haojun Liao
980f6fc46e fix(stream): pause the task when they are still in uninit status. 2023-10-30 09:32:40 +08:00
Haojun Liao
1bbad532a7 fix(stream): add more check. 2023-10-30 09:32:40 +08:00
Haojun Liao
076b533ba3 fix(stream): fix race condition when state transferring. 2023-10-30 09:32:40 +08:00
Haojun Liao
85befa10b9 fix(stream): avoid clear the restart flag . 2023-10-30 09:32:39 +08:00
Haojun Liao
f708ddb792 fix(stream): pause the task when they are still in uninit status. 2023-10-30 09:09:21 +08:00
Haojun Liao
790d869494 fix(stream): add more check. 2023-10-29 22:57:33 +08:00
Haojun Liao
3c07c3ec2f fix(stream): fix race condition when state transferring. 2023-10-29 02:27:27 +08:00
Haojun Liao
c7285c0ede fix(stream): add expired epset node list for stream tasks in hb to mnode. 2023-10-27 23:41:57 +08:00
Haojun Liao
4c267b538f fix(stream): add expired epset node list for stream tasks in hb to mnode. 2023-10-27 23:05:41 +08:00
Haojun Liao
62cc33715c fix(stream): fix deadlock. 2023-10-27 15:05:36 +08:00
Haojun Liao
0c2079a963 fix(stream): fix deadlock. 2023-10-27 15:03:21 +08:00
yihaoDeng
ebdf6d1cc6 fix mem leak 2023-10-27 14:18:56 +08:00
Haojun Liao
afe7fdf672 fix(stream): avoid clear the restart flag . 2023-10-27 14:12:06 +08:00
Haojun Liao
dcca6b0868 other: merge 3.0. 2023-10-27 11:41:28 +08:00
Haojun Liao
29b3a7a5e9 refactor: do some internal refactor. 2023-10-27 09:59:53 +08:00
Haojun Liao
9c9d02b181 fix(stream): disable follower nodes exec reset. 2023-10-27 09:59:52 +08:00
Haojun Liao
52d8e05c0c fix(stream): resolve the nodeUpdate msg compatible issue. 2023-10-27 09:59:52 +08:00
Haojun Liao
454624516c fix(stream): handle the repeatly issued nodeUpdate msg from dnode. 2023-10-27 09:59:51 +08:00
Haojun Liao
9c9cd08e2e fix(stream): not clear role flag. 2023-10-27 09:59:44 +08:00
Haojun Liao
33dc51cc86 fix(stream): add some logs. 2023-10-27 09:59:44 +08:00
Haojun Liao
1ccbb77981 fix(stream): fix dead lock when shutdown all tasks. 2023-10-27 09:59:37 +08:00
Haojun Liao
9a1a8feec2 fix(stream): do send check info if downstream is not leader. 2023-10-27 09:58:53 +08:00
Haojun Liao
2493c21ed3 fix(stream): fix the invalid write in sma 2023-10-27 09:58:52 +08:00
Haojun Liao
e8520997b0 fix(stream): disable continue check for downstream tasks. 2023-10-27 09:58:52 +08:00
Haojun Liao
3bcffcf1da other: merge stream refactor from main. 2023-10-27 09:56:02 +08:00
Haojun Liao
b0ef072556 fix(stream): clear flag for checkpoint. 2023-10-27 09:55:07 +08:00
Haojun Liao
f682ad0479 refactor: do some internal refactor. 2023-10-27 09:55:06 +08:00
Haojun Liao
efede8669f refactor: do some internal refactor. 2023-10-27 09:55:06 +08:00
Haojun Liao
9ebf806789 fix(stream): clear the flag. 2023-10-27 09:55:05 +08:00
Haojun Liao
48ff521968 fix(stream): update the nodeEp even during checkpoint trans. 2023-10-27 09:55:02 +08:00
Haojun Liao
0d4892f25b fix(stream): fix bugs caused by refactor sm. 2023-10-27 09:55:00 +08:00
Haojun Liao
a31613ae95 refactor: add ref in log. 2023-10-27 09:54:59 +08:00
Haojun Liao
226b6596d6 fix(stream): fix bugs caused by refactor. 2023-10-27 09:54:58 +08:00
Haojun Liao
091238e7be fix(stream): fix bugs caused by refactor sm. 2023-10-27 09:54:58 +08:00
Haojun Liao
e1f6043eed fix(stream): fix bugs caused by refactor sm. 2023-10-27 09:54:57 +08:00
Haojun Liao
e359da7b19 fix(stream): fix bugs caused by refactor. 2023-10-27 09:54:57 +08:00
Haojun Liao
8729bdac1e fix(stream): fix bugs caused by refactor. 2023-10-27 09:54:57 +08:00
Haojun Liao
1eac7f44c0 fix(stream): fix error in stream. 2023-10-27 09:54:56 +08:00
Haojun Liao
579dd8e6b2 refactor(stream): do some internal refactor. 2023-10-27 09:54:56 +08:00
Haojun Liao
6f3c0d17cc refactor(stream): add state machine to manage the state of stream tasks. 2023-10-27 09:54:49 +08:00
Haojun Liao
596eb073b7 refactor(stream): do some internal refactor. 2023-10-27 09:54:48 +08:00
Haojun Liao
91e3fd66e6 enh(stream): sm for stream tasks. 2023-10-27 09:54:44 +08:00
Haojun Liao
2409a17517 refactor(stream): do some internal refactor. 2023-10-27 09:54:44 +08:00
Haojun Liao
c9bdb56d79 refactor: do some internal refactor. 2023-10-27 09:54:09 +08:00
Haojun Liao
164159f293 enh(stream): add parameter to limit the stream sink task. 2023-10-27 09:54:07 +08:00
Haojun Liao
62c58ecd8b fix(stream):add more check for test cases. 2023-10-27 09:54:06 +08:00
Haojun Liao
a4f6042966 fix(stream): fix error in creating update node trans. 2023-10-27 09:52:14 +08:00
Haojun Liao
ffa798c4fa enh(stream): add some logs. 2023-10-27 09:52:14 +08:00
yihaoDeng
7e1e68d8f5 fix stream case error 2023-10-26 18:27:09 +08:00
yihaoDeng
87db93e72e fix stream case error 2023-10-26 11:36:29 +08:00
liuyao
d0e4a5d0f7 opt max delay 2023-10-25 20:29:25 +08:00
yihaoDeng
db0f6258fe fix stream case error 2023-10-25 19:38:22 +08:00
liuyao
f51c985914 stream event window 2023-10-25 17:50:16 +08:00
Haojun Liao
f36b24ed24 fix(stream): disable follower nodes exec reset. 2023-10-25 14:16:36 +08:00
Haojun Liao
985371e841 fix(stream): resolve the nodeUpdate msg compatible issue. 2023-10-25 10:01:20 +08:00
Haojun Liao
03165d5327 fix(stream): handle the repeatly issued nodeUpdate msg from dnode. 2023-10-25 09:55:02 +08:00
Haojun Liao
26aaa4a83c other: merge main. 2023-10-25 00:39:14 +08:00
Haojun Liao
5b42204dba fix(stream): not clear role flag. 2023-10-25 00:32:46 +08:00
Haojun Liao
13d979a2ee fix(stream): add some logs. 2023-10-25 00:03:46 +08:00
Haojun Liao
8cde39eebd fix(stream): fix dead lock when shutdown all tasks. 2023-10-24 19:11:24 +08:00
yihaoDeng
d43b3b4a32 refactor stream backend 2023-10-24 16:17:54 +08:00
yihaoDeng
b3c54a0f06 refactor stream backend 2023-10-24 16:04:43 +08:00
Haojun Liao
6ffb5ee476 fix(stream): do send check info if downstream is not leader. 2023-10-24 15:49:50 +08:00
Haojun Liao
f1498f8929 fix(stream): fix the invalid write in sma 2023-10-24 15:24:34 +08:00
Haojun Liao
0f9328330b fix(stream): disable continue check for downstream tasks. 2023-10-24 11:56:36 +08:00
yihaoDeng
d46cf878eb refactor stream backend 2023-10-24 11:53:52 +08:00
Haojun Liao
7a8b046a35 fix(sma): fix bugs in sma and do some internal refactor. 2023-10-24 09:40:38 +08:00
yihaoDeng
09192acf96 fix mem leak 2023-10-23 10:20:57 +08:00
Haojun Liao
059e94e428 fix(stream): clear flag for checkpoint. 2023-10-22 01:25:00 +08:00
Haojun Liao
4826b34a2b refactor: do some internal refactor. 2023-10-21 16:32:44 +08:00
Haojun Liao
06cd67e26e refactor: do some internal refactor. 2023-10-21 16:20:03 +08:00
Haojun Liao
21e98e358e fix(stream): clear the flag. 2023-10-21 02:41:03 +08:00
Haojun Liao
d1095e2c4c Merge remote-tracking branch 'origin/refact/streamsm' into refact/streamsm 2023-10-21 01:42:59 +08:00
Haojun Liao
49014d4b68 fix(stream): update the nodeEp even during checkpoint trans. 2023-10-21 01:42:43 +08:00
yihaoDeng
aaf7e070ee Merge branch 'refact/streamsm' of https://github.com/taosdata/TDengine into refact/streamsm 2023-10-20 20:49:17 +08:00
yihaoDeng
e95735eea4 fix mem leak 2023-10-20 20:49:01 +08:00
Haojun Liao
c68c0eda79 fix(stream): fix bugs caused by refactor sm. 2023-10-20 16:27:39 +08:00
Haojun Liao
4feebffb05 refactor: add ref in log. 2023-10-20 15:59:43 +08:00
Haojun Liao
c4268ca395 fix(stream): fix bugs caused by refactor. 2023-10-20 15:54:30 +08:00
yihaoDeng
1c3c58062d Merge branch 'enh/new3.0' into enh/refactorBackend 2023-10-20 15:40:47 +08:00
yihaoDeng
1a4cfd11a5 Merge branch 'enh/new3.0' into enh/refactorBackend 2023-10-20 15:28:10 +08:00
Haojun Liao
3b8c85f632 fix(stream): fix bugs caused by refactor sm. 2023-10-20 14:19:57 +08:00
Haojun Liao
270a847809 fix(stream): fix bugs caused by refactor sm. 2023-10-20 10:23:48 +08:00
wade zhang
628667f22c
Merge pull request #23231 from taosdata/FEAT/TS-3816-3.0 
feat: support restoring vnodes on replaced disks as needed
 2023-10-20 09:02:25 +08:00
Haojun Liao
4d5d078fbe fix(stream): fix bugs caused by refactor. 2023-10-19 18:00:54 +08:00
Haojun Liao
d5059826a8 fix(stream): fix bugs caused by refactor. 2023-10-19 17:59:04 +08:00
Haojun Liao
c1cebae6ba fix(stream): fix error in stream. 2023-10-19 16:20:27 +08:00
Haojun Liao
614bdd9fb9
Merge pull request #23220 from taosdata/feat/TD-26644 
opt bloom filter
 2023-10-19 15:34:43 +08:00
Haojun Liao
da8f6a4c2f
Merge pull request #23248 from taosdata/feat/TD-26647 
recover session buff state
 2023-10-19 15:34:03 +08:00
Haojun Liao
402aefd95d refactor(stream): do some internal refactor. 2023-10-19 15:09:47 +08:00
Benguang Zhao
c4e9069a66 fix: set nextProcessedVer properly in tqProcessTaskScanHistory 2023-10-19 11:18:57 +08:00
Haojun Liao
f4caeca24a refactor(stream): add state machine to manage the state of stream tasks. 2023-10-19 10:34:20 +08:00
Haojun Liao
54ec679b58 refactor(stream): do some internal refactor. 2023-10-18 15:34:53 +08:00
yihaoDeng
c6ba2ca205 fix transfer crash 2023-10-18 14:49:12 +08:00
yihaoDeng
d6840ae07f fix transfer crash 2023-10-18 14:22:52 +08:00
yihaoDeng
1b755dd4e7 fix transfer crash 2023-10-18 14:22:33 +08:00
yihaoDeng
e49b9b2276 fix transfer crash 2023-10-18 11:34:11 +08:00
Haojun Liao
2904cfd4d5 Merge branch 'main' into refact/streamsm 2023-10-18 10:29:09 +08:00
Haojun Liao
5115a1ac88
Merge pull request #23304 from taosdata/fix/TD-26771 
double free
 2023-10-18 10:09:17 +08:00
yihaoDeng
d67fdb5e46 fix transfer crash 2023-10-17 21:08:31 +08:00
yihaoDeng
2004c1a346 fix transfer crash 2023-10-17 18:19:52 +08:00
Haojun Liao
64ed7e49a2 enh(stream): sm for stream tasks. 2023-10-17 13:47:55 +08:00
yihaoDeng
77542cc461 fix stream snap deadlock 2023-10-17 11:23:39 +08:00
Haojun Liao
7b7ce10475 refactor(stream): do some internal refactor. 2023-10-17 11:23:04 +08:00
Haojun Liao
14035c763c refactor: do some internal refactor. 2023-10-17 09:57:13 +08:00
Haojun Liao
fdb6ec1fa3 enh(stream): add parameter to limit the stream sink task. 2023-10-16 22:52:39 +08:00
liuyao
990d36654d release buff 2023-10-16 19:49:45 +08:00
Haojun Liao
a1aa2c9e0c fix(stream):add more check for test cases. 2023-10-16 18:47:01 +08:00
Haojun Liao
67bee17e5e other: merge main. 2023-10-16 09:55:41 +08:00
Haojun Liao
f90cf54714 fix(stream): fix error in creating update node trans. 2023-10-15 21:56:49 +08:00
Haojun Liao
6611f216ac enh(stream): add some logs. 2023-10-13 19:45:29 +08:00
liuyao
7269dedb39 Merge branch '3.0' of https://github.com/taosdata/TDengine into 3.0 2023-10-13 11:35:01 +08:00
liuyao
429125be93 session state recover 2023-10-13 11:32:35 +08:00
Haojun Liao
33d9f772be other: merge 3.0 2023-10-13 09:06:13 +08:00
Haojun Liao
84c5359d5b other: merge 3.0 2023-10-13 08:58:26 +08:00
yihaoDeng
d6ff282006 refactor backend 2023-10-12 20:42:36 +08:00
yihaoDeng
8c1a9f7c58 refactor backend 2023-10-12 20:28:40 +08:00
yihaoDeng
0ba7e2278a refactor backend 2023-10-12 20:21:11 +08:00
yihaoDeng
a56d831a14 refactor backend 2023-10-12 20:16:41 +08:00
yihaoDeng
861d26e356 refactor backend 2023-10-12 15:24:50 +08:00
yihaoDeng
f704d4b82e refactor backend 2023-10-12 11:45:18 +08:00
yihaoDeng
bfd205b3f7 refactor backend 2023-10-12 11:31:39 +08:00
Haojun Liao
eb4449999f other: merge 3.0 2023-10-12 10:33:48 +08:00
yihaoDeng
4c94fbb44f refactor backend 2023-10-11 20:26:19 +08:00
yihaoDeng
f264ffdcd6 refactor backend 2023-10-11 20:12:07 +08:00
yihaoDeng
83b52a4c1d refactor backend 2023-10-11 20:02:40 +08:00
yihaoDeng
031506aef8 refactor backend 2023-10-11 19:46:35 +08:00
yihaoDeng
4f47162512 refactor backend 2023-10-11 16:36:43 +08:00
yihaoDeng
722d72659b refactor backend 2023-10-11 16:10:16 +08:00
yihaoDeng
0baef335d7 refactor backend 2023-10-11 16:03:04 +08:00
yihaoDeng
4cff121a38 refactor backend 2023-10-11 14:46:41 +08:00
liuyao
e9fc079d26 opt bloom filter 2023-10-11 10:14:11 +08:00
Haojun Liao
1d4f0c8599
Merge pull request #23201 from taosdata/enh/TD-26646 
Enh/td 26646
 2023-10-10 22:59:37 +08:00
yihaoDeng
28421cd35e refactor backend 2023-10-10 20:51:41 +08:00
Haojun Liao
05dfadd8f8 fix(stream): set initial version for fill-history task. disable reentry of scan-history task function. 2023-10-10 15:52:48 +08:00
Haojun Liao
5f9b858fc1 other: merge main. 2023-10-10 15:38:46 +08:00
yihaoDeng
b36163a197 add session iter 2023-10-10 15:12:21 +08:00
yihaoDeng
825341d90a add sess iter 2023-10-10 14:35:02 +08:00
Haojun Liao
adf6f65d09 fix(stream): set initial version for fill-history task. disable reentry of scan-history task function. 2023-10-10 14:00:30 +08:00
Alex Duan
9da96ebf63
Merge pull request #23017 from taosdata/case/TD-26408-MAIN 
fix: add case for lost data after  split vgroup (main)
 2023-10-10 00:54:34 -05:00
Alex Duan
e566027cef
Merge pull request #23019 from taosdata/case/TD-26408-3.0 
fix: add case for lost data after split 3.0
 2023-10-10 00:53:56 -05:00
yihaoDeng
6adf7532a8 add sess iter 2023-10-10 11:35:18 +08:00
yihaoDeng
7ac4ef5beb add sess iter 2023-10-10 10:41:46 +08:00
yihaoDeng
0fc6ffe6b6 change test case 2023-10-09 20:51:26 +08:00
Haojun Liao
a25d94ab6f fix(stream): retry when error occurs during timer. 2023-10-09 20:10:28 +08:00
Haojun Liao
7d39164133 fix(stream): disable retrieving results during checkpoint procedure 2023-10-09 20:10:27 +08:00
Haojun Liao
af4618f958 fix(stream): remove invalid clear of rspmsg list. 2023-10-09 20:10:27 +08:00
Haojun Liao
847baa8b71 fix(stream): fix the pause failure bug. 2023-10-09 20:10:27 +08:00
Haojun Liao
21fb502690 enh(stream): add trans to reset task status to avoid being frozen in checkpoint status due to doing checkpoint failure of partial tasks. 2023-10-09 20:10:26 +08:00
Haojun Liao
64bde93f85 fix(stream): not idle for sink task when no tokens are available. 2023-10-09 20:10:25 +08:00
Haojun Liao
9a7167aaba fix(stream): sleep for 10ms 2023-10-09 20:10:24 +08:00
Haojun Liao
9af8ce21ed fix(stream): wait for a while when no available token. 2023-10-09 20:10:23 +08:00
Haojun Liao
4780474877 fix(stream): continue 2023-10-09 20:10:23 +08:00
Haojun Liao
da833ccb30 refactor: inc the dump batch. 2023-10-09 20:10:23 +08:00
Haojun Liao
2936ac9b89 fix(stream): add missing refactor. 2023-10-09 20:10:22 +08:00
Haojun Liao
af2c0c95e9 refactor: do some internal refactor. 2023-10-09 20:10:22 +08:00
Haojun Liao
940d6a3cb4 fix(stream): check items in queue and qall, stead of only the queue items. 2023-10-09 20:10:21 +08:00
Haojun Liao
e95aea84ae fix(stream): fix syntax error. 2023-10-09 20:10:21 +08:00
Haojun Liao
59e284c332 fix(stream): quit from loop when input queue is full. 2023-10-09 20:10:21 +08:00
Haojun Liao
9d5a3b8d78 other: merge stream fix. 2023-10-09 20:10:13 +08:00
Haojun Liao
c5997bee5b fix(stream): update the current processing version. 2023-10-09 20:09:23 +08:00
Haojun Liao
28008f8263 enh(stream): add more info for sink task. 2023-10-09 20:09:22 +08:00
Haojun Liao
b3b9e89d30 fix(stream): fix the invalid check. 2023-10-09 20:09:22 +08:00
Haojun Liao
cf14772dd9 enh(stream): optimize the scan wal perf 2023-10-09 20:09:21 +08:00
Haojun Liao
d2160efa05 refactor: wait for 100ms before scan wal again. 2023-10-09 20:09:21 +08:00
Haojun Liao
932d93ab35 refactor: wait for more time for future data 2023-10-09 20:09:21 +08:00
Haojun Liao
7682c033ad fix(stream): add null check. 2023-10-09 20:09:21 +08:00
Haojun Liao
0ab5d4b342 refactor: do some internal refactor. 2023-10-09 20:09:20 +08:00
Haojun Liao
c295ec322a refactor: do some internal refactor. 2023-10-09 20:09:20 +08:00
Haojun Liao
d5abb21335 fix(stream): fix invalid check. 2023-10-09 20:09:20 +08:00
Haojun Liao
583ac1fc45 refactor: do some internal refactor. 2023-10-09 20:09:20 +08:00
Haojun Liao
83f84d92bc refactor: do some internal refactor (discard the checkpoint source msg during restoring). 2023-10-09 20:09:10 +08:00
Haojun Liao
bb4ba54f28 fix(stream): enable new time range for stream task. 2023-10-09 20:09:09 +08:00
Haojun Liao
c759d90bf6 enh(stream): optimize the stream status when creating without fill-history option. 2023-10-09 20:09:09 +08:00
Haojun Liao
db897fb03a fix(stream): opt perf. 2023-10-09 20:09:09 +08:00
Haojun Liao
0a2b70fd7f fix(stream): fix syntax error. 2023-10-09 20:09:06 +08:00
Haojun Liao
8477271240 fix(stream): fix dead lock. 2023-10-09 20:09:00 +08:00
Haojun Liao
12eabb0c2c log(stream): update logs. 2023-10-09 20:09:00 +08:00
Haojun Liao
b716f0d1fd fix(stream): fix dead lock. 2023-10-09 20:08:57 +08:00
Haojun Liao
60a720efbb log: update logs. 2023-10-09 20:08:53 +08:00
Haojun Liao
d10b07d23d refactor: do some internal refactor. 2023-10-09 20:08:53 +08:00
Haojun Liao
6d2dbd549b fix(stream): avoid repeatly free obj. 2023-10-09 20:08:53 +08:00
Haojun Liao
6b5770aab7 log(stream): update the log. 2023-10-09 20:08:52 +08:00
Haojun Liao
dfe8641c0c refactor: optimize the fill-history task launch policy, and do some other internal refactor. 2023-10-09 20:08:52 +08:00
Haojun Liao
38164435f9 refactor: do some internal refactor. 2023-10-09 20:08:52 +08:00
Haojun Liao
f2848de12a fix(stream): reduce the counter when remove stream tasks. 2023-10-09 20:08:52 +08:00
Haojun Liao
ecb3b44026 fix(stream): limit sink quota. 2023-10-09 20:08:52 +08:00
Haojun Liao
2c0e54f4f7 fix(stream): update the retry interval. 2023-10-09 20:08:13 +08:00
Haojun Liao
7f8da725bc enh(stream): add quota limitation for sink task. 2023-10-09 20:08:13 +08:00
Haojun Liao
a248f3d088 refactor: do some internal refactor. 2023-10-09 20:08:12 +08:00
Haojun Liao
0db357401d fix(stream): add new status for meta. 2023-10-09 20:08:12 +08:00
Haojun Liao
3912d80c02 fix(stream): do some internal refactor. 2023-10-09 20:08:11 +08:00
Haojun Liao
feb09c8361 fix(stream): fix error in pause scan-history data. 2023-10-09 20:08:11 +08:00
Haojun Liao
6a98b11bef fix(stream): fix invalid read and memory leak. 2023-10-09 20:08:11 +08:00
Haojun Liao
65f66f0986 fix(stream): transfer state when all downstream rsp already. 2023-10-09 20:08:11 +08:00
Haojun Liao
a32b56f381 fix(stream): fix memory leak. 2023-10-09 20:08:10 +08:00
Haojun Liao
4af7812f0b fix(stream): fix the invalid counter for ready tasks in vnode, fix an invalid read. 2023-10-09 20:08:10 +08:00
Haojun Liao
4fe1382cd9 fix(stream): adjust nodeUpdate check interval. 2023-10-09 20:08:09 +08:00
Haojun Liao
f4f55fcbce fix(stream): record the tasks started info, and do some internal refactor. 2023-10-09 20:08:09 +08:00
Haojun Liao
68b003c804 fix(stream): fix race condition. 2023-10-09 20:08:09 +08:00
Haojun Liao
fc96ec6bae fix(stream): do not wait for the checkpoint finish before stop tasks. 2023-10-09 20:08:09 +08:00
Haojun Liao
7167c3c5de fix(stream): add new timer for each task. 2023-10-09 20:08:08 +08:00
Haojun Liao
7336228c16 refactor: do some internal refactor. 2023-10-09 20:08:08 +08:00
Haojun Liao
477f57aa0e refactor: do some internal refactor. 2023-10-09 20:08:08 +08:00
Haojun Liao
52763ca2a3 refactor: do some internal refactor. 2023-10-09 20:08:07 +08:00
Haojun Liao
3baddbe67a fix(stream): update the checkpoint record start time. 2023-10-09 20:08:07 +08:00
Haojun Liao
a6ffb845c8 fix(stream): fix invalid checkpoint start time. 2023-10-09 20:08:07 +08:00
Haojun Liao
c05a8de3e8 fix(stream): rsp when checkpoint source failed. 2023-10-09 20:08:06 +08:00
Haojun Liao
b4bc4fac98 fix(stream): disable auto rsp for checkpoint source. 2023-10-09 20:08:06 +08:00
Haojun Liao
e671e87234 fix(stream): fix invalid assert. 2023-10-09 20:08:06 +08:00
Haojun Liao
d1a6b8bd22 refactor: do some internal refactor. 2023-10-09 20:08:06 +08:00
Haojun Liao
46b17aa9f1 fix(stream): fix race condition. 2023-10-09 20:08:06 +08:00
Haojun Liao
e49409a11e fix(stream): add some logs. 2023-10-09 20:08:05 +08:00
Haojun Liao
5eee76df65 fix(stream): add msg identifier check. 2023-10-09 20:08:05 +08:00
Haojun Liao
192c47a7c2 fix(stream): add some logs. 2023-10-09 20:08:05 +08:00
Haojun Liao
693254f602 fix(stream): remove invalid free. 2023-10-09 20:08:04 +08:00
Haojun Liao
f3bd5c411d fix(stream): remove the blocking flag after successfully retry sending msg. 2023-10-09 20:08:04 +08:00
Haojun Liao
ed9e035095 fix(stream): clear the failure vnode id list before send msg. 2023-10-09 20:08:04 +08:00
Haojun Liao
3726aa723e fix(stream): fix invalid read. 2023-10-09 20:08:04 +08:00
Haojun Liao
e0d19af000 fix(stream): refactor re-try dispatch msg for stream tasks. 2023-10-09 20:08:03 +08:00
Haojun Liao
2b16976902 refactor(stream): refactor send msg. 2023-10-09 20:08:03 +08:00
Haojun Liao
27e00b1f19 fix(stream): add some logs, and remove invalid assert. 2023-10-09 20:08:02 +08:00
Haojun Liao
12687eb287 fix(stream): add check info detailed information. 2023-10-09 20:08:02 +08:00
Haojun Liao
ab7c18f604 refactor: do some internal refactor. 2023-10-09 20:08:01 +08:00
Haojun Liao
e5535fa1f5 fix(stream): handle dispatch checkpoint-trigger failure. 2023-10-09 20:08:01 +08:00
Haojun Liao
65cc4040f3 fix(stream): update logs. 2023-10-09 20:08:00 +08:00
Haojun Liao
6d3a134077 fix(stream): ignore the dispatch failure, and set the correct rsp counter. 2023-10-09 20:07:54 +08:00
Haojun Liao
7f4d9b6e41 fix(stream): wait for the task checkpoint before stop. 2023-10-09 20:07:53 +08:00
Haojun Liao
ef9efc8a8e refactor: do refactor the sink task. 2023-10-09 20:07:53 +08:00
Haojun Liao
e4ac835554 fix(stream): disable token bucket. 2023-10-09 20:07:53 +08:00
Haojun Liao
a3d13d6634 fix(stream): remove invalid cached task info in buffer. 2023-10-09 20:07:08 +08:00
Haojun Liao
1f45bd82d2 other: merge stream fix in main. 2023-10-09 20:07:01 +08:00
Haojun Liao
d752fb1ed6 fix(stream): exec task in task queuee threads. 2023-10-09 20:06:11 +08:00
Haojun Liao
1ebafe4a1f fix(stream): add more check. 2023-10-09 20:06:11 +08:00
Haojun Liao
c403a15f36 fix(stream): check the status before exec stream task when handling the dispatch requests. 2023-10-09 20:06:11 +08:00
Haojun Liao
d76e3ac718 fix(stream): add task status check. 2023-10-09 20:06:10 +08:00
Haojun Liao
e27111ecd1 enh(stream): add node stage check. 2023-10-09 20:06:09 +08:00
Haojun Liao
ce85945583 fix(stream): set correct size for results generated by scan history stream tasks. 2023-10-09 20:06:05 +08:00
Haojun Liao
053643be19 fix(stream): add timestamp. 2023-10-09 20:06:05 +08:00
Haojun Liao
b38f9e0288 fix(stream): update logs. 2023-10-09 20:06:05 +08:00
Haojun Liao
5c0b8ea804 enh(stream): log the checkpoint time. 2023-10-09 20:06:04 +08:00
Haojun Liao
b95ad74c7f fix(stream): not handle the check msg for follower tasks. 2023-10-09 20:06:04 +08:00
Haojun Liao
e6fb9ee46c fix(stream): reset task counter. 2023-10-09 20:06:04 +08:00
Haojun Liao
c3a3cf21fb fix(stream): add some logs. 2023-10-09 20:05:39 +08:00
Haojun Liao
08b37dfc13 fix(stream): fix memory leak. 2023-10-09 20:05:38 +08:00
Haojun Liao
734f648891 fix(stream): set the output normal for stream dispatch when encountering the transfer state. 2023-10-09 20:05:38 +08:00
Haojun Liao
636eccc160 fix(stream): add null check 2023-10-09 20:05:37 +08:00
Haojun Liao
57bf1cadc4 fix(stream): fix error in decode stream task. 2023-10-09 20:05:37 +08:00
Haojun Liao
9d6199adf7 fix(stream): fix invalid read. 2023-10-09 20:05:37 +08:00
Haojun Liao
0459a4628c fix(stream): fix stream task id error. 2023-10-09 20:05:36 +08:00
Haojun Liao
7ccc273206 other: merge stream fix in main. 2023-10-09 20:05:29 +08:00
Haojun Liao
98ef566eb4 fix(stream): remove fill-history sink task. 2023-10-09 20:04:17 +08:00
Haojun Liao
e84eeee6b8 refactor: do some internal refactor. 2023-10-09 20:04:17 +08:00
Haojun Liao
fe7864b976 fix(stream): fix error in drop task. 2023-10-09 20:04:17 +08:00
Haojun Liao
c33e521aa1 fix(stream): update some logs. 2023-10-09 20:04:17 +08:00
Haojun Liao
39e0c57323 fix(stream): set the correct guard flag. 2023-10-09 20:04:16 +08:00
Haojun Liao
3d1f29ff21 fix(stream): fix memory leak. 2023-10-09 20:04:16 +08:00
yihaoDeng
c6e9134cea add sess iter 2023-10-09 20:02:23 +08:00
Haojun Liao
6fa54789cb fix(stream): disable follower send hb to mnode. and do some internal refactor. 2023-10-09 19:59:31 +08:00
Haojun Liao
1256eafddb other: merge stream fix. 2023-10-09 19:59:24 +08:00
Haojun Liao
283118bf68 fix(stream):fix missing unlock. 2023-10-09 19:56:17 +08:00
Haojun Liao
aaa12ed4f8 refactor: do some internal refactor about the sched status set 2023-10-09 19:56:17 +08:00
Haojun Liao
01d104706e refactor(stream): remove unnecessary streamschedTask function to improve the performance. 2023-10-09 19:56:17 +08:00
Haojun Liao
a678e722ce fix(stream): limit the token bucket capacity 2023-10-09 19:56:17 +08:00
Haojun Liao
b0ef091430 fix(stream): retry when error occurs during timer. 2023-10-09 16:40:05 +08:00
Haojun Liao
be0a2e4573 fix(stream): disable retrieving results during checkpoint procedure 2023-10-09 16:33:28 +08:00
yihaoDeng
2aa88dfe9b fix mem leak 2023-10-09 15:53:42 +08:00
Haojun Liao
8266c4ff77 fix(stream): remove invalid clear of rspmsg list. 2023-10-09 15:26:42 +08:00
Haojun Liao
d87db2f829 fix(stream): fix the pause failure bug. 2023-10-09 14:48:11 +08:00
yihaoDeng
28b546a4ad fix mem leak 2023-10-09 14:25:57 +08:00
yihaoDeng
be42143362 Merge branch '3.0' into case/TD-26408-3.0 2023-10-09 14:24:31 +08:00
yihaoDeng
3f9784f475 fix mem leak 2023-10-09 14:07:22 +08:00
yihaoDeng
0b92a6b1ea fix mem leak 2023-10-09 11:16:16 +08:00
Haojun Liao
74ff0c5b50 Merge branch 'main' into fix/liaohj 2023-10-09 10:14:53 +08:00
Haojun Liao
ee6516f3c4 enh(stream): add trans to reset task status to avoid being frozen in checkpoint status due to doing checkpoint failure of partial tasks. 2023-10-09 10:13:53 +08:00
Haojun Liao
4b8cd3bccd fix(stream): not idle for sink task when no tokens are available. 2023-10-09 10:07:19 +08:00
yihaoDeng
f49a2cbeb0 refact task backend 2023-10-08 18:14:17 +08:00
liuyao
1cbb8ef444 init bloomfilter array 2023-10-08 14:43:44 +08:00
yihaoDeng
b5994b31e2 fix mem leak 2023-10-08 13:33:02 +08:00
yihaoDeng
e3f18495e0 fix mem leak 2023-10-08 11:22:26 +08:00
yihaoDeng
2fa91341a7 refact task backend 2023-10-08 10:35:49 +08:00
Haojun Liao
23fb26798f fix(stream): sleep for 10ms 2023-10-07 19:47:49 +08:00
Haojun Liao
e4e4c5a36e fix(stream): wait for a while when no available token. 2023-10-07 19:41:49 +08:00
Haojun Liao
45bb3f50d7 fix(stream): continue 2023-10-07 19:25:40 +08:00
liuyao
95ffe6fa23 stream recover 2023-10-07 19:24:21 +08:00
Haojun Liao
62cdcbc7e3 refactor: inc the dump batch. 2023-10-07 19:20:48 +08:00
Haojun Liao
4d6f143146 fix(stream): add missing refactor. 2023-10-07 19:18:26 +08:00
Haojun Liao
ab17bfdf09 refactor: do some internal refactor. 2023-10-07 19:16:55 +08:00
Haojun Liao
0e8a8aabc1 fix(stream): check items in queue and qall, stead of only the queue items. 2023-10-07 19:16:25 +08:00
liuyao
1e323c0408 fix issue 2023-10-07 19:11:46 +08:00
yihaoDeng
98099ebbae fix mem leak 2023-10-07 18:41:27 +08:00
liuyao
19bbf10579 fix issue 2023-10-07 17:59:53 +08:00
yihaoDeng
97fdcbc60b fix mem leak 2023-10-07 17:48:24 +08:00
liuyao
1aaf873134 fix issue 2023-10-07 17:43:50 +08:00
yihaoDeng
bdc4afec83 refact task backend 2023-10-07 17:17:30 +08:00
yihaoDeng
e41da13d5b refact task backend 2023-10-07 17:07:26 +08:00
yihaoDeng
3ebc7eef5f refact task backend 2023-10-07 17:03:12 +08:00
yihaoDeng
c146327da8 refact task backend 2023-10-07 16:48:36 +08:00
Haojun Liao
d11c0a113c fix(stream): fix syntax error. 2023-10-07 14:56:09 +08:00
Haojun Liao
7ce3d2c200 fix(stream): quit from loop when input queue is full. 2023-10-07 14:54:11 +08:00
yihaoDeng
66315fe405 refact task backend 2023-10-07 13:53:44 +08:00
yihaoDeng
82e0242889 refact task backend 2023-10-07 13:43:52 +08:00
yihaoDeng
fa70221238 refact task backend 2023-10-07 12:17:55 +08:00
liuyao
460ee86fdf fix issue 2023-10-07 11:55:40 +08:00
liuyao
59d47e1e1f fix issuse 2023-10-07 10:34:35 +08:00
Haojun Liao
115a284ac0 enh(stream): update the checkpoint interval 2023-10-07 09:56:03 +08:00
Haojun Liao
011627d50a fix(stream): update the current processing version. 2023-10-07 09:07:44 +08:00
Haojun Liao
c20dd002fb enh(stream): add more info for sink task. 2023-10-07 01:14:00 +08:00
Haojun Liao
98659ad323 fix(stream): fix the invalid check. 2023-10-05 00:09:47 +08:00
Haojun Liao
41ef1a45c1 enh(stream): optimize the scan wal perf 2023-10-04 22:06:46 +08:00
Haojun Liao
45e413f8d5 refactor: wait for 100ms before scan wal again. 2023-10-03 20:57:19 +08:00
Haojun Liao
132fb8e82e refactor: wait for more time for future data 2023-10-03 15:42:00 +08:00
Haojun Liao
38aff641ee fix(stream): add null check. 2023-10-03 02:39:57 +08:00
Haojun Liao
58f410a9df refactor: do some internal refactor. 2023-10-03 02:37:52 +08:00
Haojun Liao
a543f4ca97 refactor: do some internal refactor. 2023-10-03 01:52:03 +08:00
Haojun Liao
5564eb215c fix(stream): fix invalid check. 2023-10-03 00:38:24 +08:00
Haojun Liao
5718158fc0 refactor: do some internal refactor. 2023-10-01 22:27:29 +08:00
yihaoDeng
b8e265e502 add cvt state 2023-09-30 09:05:28 +00:00
Haojun Liao
31acbc6fe2 refactor: do some internal refactor (discard the checkpoint source msg during restoring). 2023-09-28 22:44:13 +08:00
yihaoDeng
eac16b9d1b add cvt state 2023-09-28 11:59:53 +00:00
yihaoDeng
4f66dc538f add cvt state 2023-09-28 11:52:18 +00:00
Haojun Liao
f2d8431619 fix(stream): enable new time range for stream task. 2023-09-28 18:26:52 +08:00
yihaoDeng
c5bb37a5cb refact task backend 2023-09-28 17:30:49 +08:00
yihaoDeng
bb265887e6 refact task backend 2023-09-28 17:07:19 +08:00
yihaoDeng
54e3ac2c1e refact task backend 2023-09-28 15:46:12 +08:00
Haojun Liao
80ec77cbb6 enh(stream): optimize the stream status when creating without fill-history option. 2023-09-28 11:23:03 +08:00
Haojun Liao
22e485d132 fix(stream): opt perf. 2023-09-28 09:25:23 +08:00
Haojun Liao
f525778aef fix(stream): fix syntax error. 2023-09-28 00:53:36 +08:00
Haojun Liao
8c998673ad fix(stream): fix dead lock. 2023-09-27 17:15:49 +08:00
Haojun Liao
2dade996bb log(stream): update logs. 2023-09-27 15:37:12 +08:00
Haojun Liao
8f2d217834 log: update logs. 2023-09-27 14:48:53 +08:00
yihaoDeng
d4f87378fd avoid first tag index conflict 2023-09-27 14:44:09 +08:00
Haojun Liao
11641e1115 refactor: do some internal refactor. 2023-09-27 14:18:01 +08:00
Haojun Liao
34d547bfff fix(stream): avoid repeatly free obj. 2023-09-27 14:05:14 +08:00
Haojun Liao
ae020fcd2d log(stream): update the log. 2023-09-27 14:03:57 +08:00
Haojun Liao
e13efd5b04 refactor: optimize the fill-history task launch policy, and do some other internal refactor. 2023-09-27 14:00:48 +08:00
Haojun Liao
a7e6933ce7 refactor: do some internal refactor. 2023-09-27 09:46:15 +08:00
Haojun Liao
37bf387453 fix(stream): reduce the counter when remove stream tasks. 2023-09-27 09:36:20 +08:00
Haojun Liao
9ab9de0ad3 fix(stream): limit sink quota. 2023-09-27 00:30:38 +08:00
Haojun Liao
27dcbcb96f fix(stream): update the retry interval. 2023-09-26 23:42:02 +08:00
Haojun Liao
aec15ae39f enh(stream): add quota limitation for sink task. 2023-09-26 23:29:54 +08:00
liuyao
c95dc167b7 add ci 2023-09-26 17:19:21 +08:00
Haojun Liao
890d658aea refactor: do some internal refactor. 2023-09-26 14:11:45 +08:00
Haojun Liao
f3f18b0fa3 fix(stream): add new status for meta. 2023-09-26 12:15:12 +08:00
liuyao
82e18e32b0 add ci 2023-09-26 10:41:12 +08:00
Haojun Liao
d214a2f6bd fix(stream): do some internal refactor. 2023-09-26 09:32:57 +08:00
Haojun Liao
6cc3618ed9 fix(stream): fix error in pause scan-history data. 2023-09-26 01:09:55 +08:00
Haojun Liao
5a4a0aa0e3 fix(stream): fix invalid read and memory leak. 2023-09-25 23:10:33 +08:00
yihaoDeng
ed172a939c rename cfg default name 2023-09-25 18:09:35 +08:00
Haojun Liao
c9576d7a85 fix(stream): transfer state when all downstream rsp already. 2023-09-25 17:31:38 +08:00
Haojun Liao
c615c513e8 fix(stream): fix memory leak. 2023-09-25 16:01:17 +08:00
Haojun Liao
fcf4cc7169 fix(stream): fix the invalid counter for ready tasks in vnode, fix an invalid read. 2023-09-25 15:58:24 +08:00
Haojun Liao
1af95969b2 fix(stream): adjust nodeUpdate check interval. 2023-09-25 10:40:30 +08:00
Haojun Liao
eb4078b7f3 fix(stream): record the tasks started info, and do some internal refactor. 2023-09-25 09:59:05 +08:00
Haojun Liao
c987f61300 fix(stream): fix race condition. 2023-09-25 01:37:27 +08:00
Haojun Liao
da82a5f19e fix(stream): do not wait for the checkpoint finish before stop tasks. 2023-09-24 22:03:07 +08:00
Haojun Liao
a74b462ffb fix(stream): add new timer for each task. 2023-09-24 02:25:53 +08:00
Haojun Liao
b7dcd94c64 refactor: do some internal refactor. 2023-09-24 01:56:52 +08:00
Haojun Liao
3a75c21e66 refactor: do some internal refactor. 2023-09-24 01:55:59 +08:00
Haojun Liao
4ea737571d refactor: do some internal refactor. 2023-09-24 01:26:51 +08:00
Haojun Liao
7f265e181d fix(stream): update the checkpoint record start time. 2023-09-23 18:05:14 +08:00
Haojun Liao
6ef68fdf06 fix(stream): fix invalid checkpoint start time. 2023-09-23 17:57:09 +08:00
Haojun Liao
acd3321494 fix(stream): rsp when checkpoint source failed. 2023-09-23 14:13:33 +08:00
Haojun Liao
7d5f76c6f1 fix(stream): disable auto rsp for checkpoint source. 2023-09-23 11:31:14 +08:00
Haojun Liao
5155519559 fix(stream): fix invalid assert. 2023-09-22 23:06:48 +08:00
Haojun Liao
cf6c255f22 refactor: do some internal refactor. 2023-09-22 22:50:21 +08:00
yihaoDeng
8890f5c950 add cvt state 2023-09-22 12:51:29 +00:00
Haojun Liao
fcbd747afc fix(stream): fix race condition. 2023-09-22 19:35:12 +08:00
liuyao
5cc1ae1266 fix issue 2023-09-22 19:25:19 +08:00
Haojun Liao
1ca05fb3af fix(stream): add some logs. 2023-09-22 18:56:31 +08:00
Haojun Liao
5b12e276cf fix(stream): add msg identifier check. 2023-09-22 18:13:18 +08:00
Haojun Liao
bb1a2322ca fix(stream): add some logs. 2023-09-22 17:27:10 +08:00
yihaoDeng
11da631bb5 add cvt state 2023-09-22 09:08:42 +00:00
Haojun Liao
07ec6f38b8 fix(stream): remove invalid free. 2023-09-22 16:53:22 +08:00
Haojun Liao
5a695f632c fix(stream): remove the blocking flag after successfully retry sending msg. 2023-09-22 16:44:55 +08:00
yihaoDeng
970b1cb840 add cvt state 2023-09-22 08:38:41 +00:00
yihaoDeng
173604ee1a add cvt state 2023-09-22 08:32:14 +00:00
yihaoDeng
19e52c364e add cvt state 2023-09-22 08:26:03 +00:00
Haojun Liao
6c357104a5 fix(stream): clear the failure vnode id list before send msg. 2023-09-22 16:19:25 +08:00
Haojun Liao
0f5f21ba09 fix(stream): fix invalid read. 2023-09-22 16:05:12 +08:00
Haojun Liao
619a9f6d08 fix(stream): refactor re-try dispatch msg for stream tasks. 2023-09-22 15:57:55 +08:00
liuyao
4f8e6d7795 Merge branch '3.0' of https://github.com/taosdata/TDengine into feat/TD-26174 2023-09-22 15:00:52 +08:00
liuyao
410a177637 fix issue 2023-09-22 15:00:26 +08:00
Haojun Liao
898aea5cfb refactor(stream): refactor send msg. 2023-09-22 14:52:18 +08:00
Haojun Liao
87ea745861 fix(stream): add some logs, and remove invalid assert. 2023-09-22 10:53:03 +08:00
Haojun Liao
f7bcff862d fix(stream): add check info detailed information. 2023-09-22 10:23:56 +08:00
wangmm0220
5555371fbc fix:memory leak 2023-09-22 10:04:53 +08:00
liuyao
4979fc5283 fix issue 2023-09-22 09:40:08 +08:00
wangmm0220
803b17a2db fix:memory leak 2023-09-22 09:31:58 +08:00
Haojun Liao
b1ba716299 refactor: do some internal refactor. 2023-09-22 09:24:36 +08:00
Haojun Liao
b075b4438b fix(stream): handle dispatch checkpoint-trigger failure. 2023-09-21 23:15:18 +08:00
Haojun Liao
50d5fb6641 fix(stream): update logs. 2023-09-21 19:14:34 +08:00
yihaoDeng
470849982e add cvt func 2023-09-21 09:41:30 +00:00
Haojun Liao
4925b67d4a fix(stream): ignore the dispatch failure, and set the correct rsp counter. 2023-09-21 17:22:09 +08:00
Haojun Liao
2efd155adf fix(stream): wait for the task checkpoint before stop. 2023-09-21 15:53:21 +08:00
Haojun Liao
17586af0ce refactor: do refactor the sink task. 2023-09-21 13:32:57 +08:00
yihaoDeng
77ef632233 refactor code 2023-09-21 01:36:12 +00:00
Haojun Liao
9f9b712332 fix(stream): disable token bucket. 2023-09-21 01:19:58 +08:00
yihaoDeng
ee8c09667e refactor backend 2023-09-20 12:53:58 +00:00
Haojun Liao
92f18989df fix(stream): remove invalid cached task info in buffer. 2023-09-20 19:19:04 +08:00
Haojun Liao
6633fb971b fix(stream): update logs. 2023-09-20 18:18:31 +08:00
liuyao
b4be5fe895 fix mem leak 2023-09-20 17:35:11 +08:00
Haojun Liao
619b377dd9 fix(stream): exec task in task queuee threads. 2023-09-20 17:30:41 +08:00
Haojun Liao
49165f8c3c fix(stream): add more check. 2023-09-20 16:49:55 +08:00
Haojun Liao
8329ec79f2 fix(stream): check the status before exec stream task when handling the dispatch requests. 2023-09-20 16:32:27 +08:00
Haojun Liao
2a781c509f fix(stream): add task status check. 2023-09-20 15:32:15 +08:00
liuyao
1cebda368a session&state operator buff 2023-09-20 15:06:08 +08:00
Haojun Liao
b4419bda65 enh(stream): add node stage check. 2023-09-20 10:35:01 +08:00
Haojun Liao
1decaaee1e fix(stream): set correct size for results generated by scan history stream tasks. 2023-09-19 16:49:04 +08:00
Haojun Liao
309630eb11 fix(stream): add timestamp. 2023-09-19 14:22:05 +08:00
Haojun Liao
38b3a7c1bd fix(stream): update logs. 2023-09-19 11:05:49 +08:00
Haojun Liao
b7efedf4aa enh(stream): log the checkpoint time. 2023-09-19 09:53:33 +08:00
Haojun Liao
e76bab7122 fix(stream): not handle the check msg for follower tasks. 2023-09-19 09:43:14 +08:00
Haojun Liao
9a9fffa577 fix(stream): reset task counter. 2023-09-19 00:52:35 +08:00
Haojun Liao
377fdfacf6 fix(stream): add some logs. 2023-09-18 23:51:08 +08:00
Haojun Liao
2c9d54af37 fix(stream): fix memory leak. 2023-09-18 19:58:46 +08:00
Haojun Liao
c7b8d572f6 fix(stream): set the output normal for stream dispatch when encountering the transfer state. 2023-09-18 18:25:22 +08:00
Haojun Liao
c33ef4ce88 fix(stream): add null check 2023-09-18 15:14:51 +08:00
Haojun Liao
8aa9960760
Merge pull request #22944 from taosdata/fix/liaohj 
refactor: do some internal refactor, and fix some bugs.
 2023-09-17 21:25:33 +08:00
Haojun Liao
3dfdda3283 fix(stream): fix error in decode stream task. 2023-09-17 18:07:26 +08:00
Haojun Liao
301258784a fix(stream): fix invalid read. 2023-09-17 13:59:06 +08:00
Haojun Liao
01144c58a7 fix(stream): fix stream task id error. 2023-09-17 01:19:59 +08:00
Haojun Liao
4846194bbb refactor: do some internal refactor, remove the related fill-history task by sending mesg to replay it in follower node. 2023-09-16 21:37:30 +08:00
Haojun Liao
caa7542cea
Merge pull request #22913 from taosdata/fix/liaohj 
fix(stream): fix error in drop task.
 2023-09-16 21:36:38 +08:00
Haojun Liao
91e3d70796 fix(stream): remove fill-history sink task. 2023-09-15 17:46:17 +08:00
yihaoDeng
e958e18726 fix race crash and add log 2023-09-15 15:02:05 +08:00
yihaoDeng
6820fb0c5d fix race crash and add log 2023-09-15 14:59:44 +08:00
Haojun Liao
2bfd6e3355 refactor: do some internal refactor. 2023-09-15 09:25:22 +08:00
Haojun Liao
117de7ab7d fix(stream): fix error in drop task. 2023-09-15 00:51:13 +08:00
Haojun Liao
500fcad36c
Merge pull request #22892 from taosdata/fix/liaohj 
refactor: do some internal refactor.
 2023-09-15 00:46:46 +08:00
Haojun Liao
73c2782d78 fix(stream): update some logs. 2023-09-14 23:17:23 +08:00
Haojun Liao
303e8caf0c fix(stream): set the correct guard flag. 2023-09-14 22:44:16 +08:00
Haojun Liao
27f626769e fix(stream): fix memory leak. 2023-09-14 19:33:05 +08:00
Haojun Liao
f2c3e0a6e8 fix(stream): disable follower send hb to mnode. and do some internal refactor. 2023-09-14 17:22:31 +08:00
Haojun Liao
fce5be0eb4 fix(stream): fix dead lock. 2023-09-14 14:09:05 +08:00
Haojun Liao
d9f6cf3966 fix(stream):fix missing unlock. 2023-09-13 14:11:41 +08:00
Haojun Liao
7305c8dbe1 refactor: do some internal refactor about the sched status set 2023-09-13 14:00:34 +08:00
Haojun Liao
ca2fea3e1b refactor(stream): remove unnecessary streamschedTask function to improve the performance. 2023-09-13 11:02:38 +08:00
Haojun Liao
0943c1c5b1 fix(stream): limit the token bucket capacity 2023-09-13 09:19:59 +08:00
Haojun Liao
99477babca
Merge pull request #22872 from taosdata/fix/coverityScanDyh 
fix coverity scan problem
 2023-09-13 09:18:27 +08:00