Elgato_dark/TDengine
1
0
Fork 0
mirror of https://github.com/taosdata/TDengine synced 2026-05-24 10:09:01 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 4
67094 commits 554 branches 514 tags 1.7 GiB
Commit graph

817 commits

Author SHA1 Message Date
yihaoDeng
7b0891981e merge 3.0 2023-11-08 14:14:01 +08:00
kailixu
b9f389a49f Merge branch '3.0' into enh/TD-23769-3.0x 2023-11-08 08:36:35 +08:00
kailixu
6b738884a4 chore: rsma checkpoint 2023-11-07 19:59:05 +08:00
Haojun Liao
e558d8e497 fix: fix syntax error. 2023-11-07 18:53:21 +08:00
Haojun Liao
1301b92844 fix(stream): fix errors in scan-history, introduced by refactor 2023-11-07 16:41:15 +08:00
Haojun Liao
8c2f6bab0e fix(stream): exec scan-history in timer. 2023-11-07 16:40:03 +08:00
Haojun Liao
7370daa6e0 fix(stream): record the start failure tasks. 2023-11-07 16:39:56 +08:00
Haojun Liao
e38e526dda fix(stream):reset the starting flag for the followers. 2023-11-07 16:39:12 +08:00
Haojun Liao
207b53f3f6 fix(stream): fix errors in scan-history, introduced by refactor 2023-11-07 16:19:09 +08:00
Haojun Liao
4d8548e938 refactor: do some internal refactor. 2023-11-06 23:42:16 +08:00
Haojun Liao
d9474016fc fix(stream): exec scan-history in timer. 2023-11-06 18:55:53 +08:00
Haojun Liao
6b1889284b fix(stream): record the start failure tasks. 2023-11-06 15:45:44 +08:00
Haojun Liao
69863febe6 fix(stream):reset the starting flag for the followers. 2023-11-06 13:17:13 +08:00
Haojun Liao
d41fd27189 fix(stream): restart tasks in stream threads, instead of write thread. 2023-11-06 11:57:21 +08:00
Haojun Liao
0e44950a37 enh(stream): execute scan history by using time slice 1s. 2023-11-06 09:21:33 +08:00
Haojun Liao
d440da0d8c enh(stream): execute scan history by using time slice 1s. 2023-11-05 01:26:53 +08:00
kailixu
77eadc59dc Merge branch '3.0' into enh/TD-23769-3.0x 2023-11-03 16:26:07 +08:00
yihaoDeng
24e961976f Merge branch 'enh/new3.0' into enh/refactorBackend 2023-11-03 09:31:57 +08:00
liuyao
31ad73f6d1 Merge branch '3.0' of https://github.com/taosdata/TDengine into feat/TD-21898 2023-11-03 09:12:17 +08:00
Haojun Liao
7cf90dde5c refactor: do some internal refactor. 2023-11-01 19:38:13 +08:00
Haojun Liao
00f029e44f refactor: add some logs. 2023-11-01 19:38:12 +08:00
Haojun Liao
246ed4e022 refactor: do some internal refactor. 2023-11-01 17:19:21 +08:00
liuyao
505c44d9aa Merge branch '3.0' of https://github.com/taosdata/TDengine into feat/TD-21898 2023-11-01 16:50:46 +08:00
Haojun Liao
12bc4f32cf refactor: add some logs. 2023-11-01 16:12:15 +08:00
yihaoDeng
4c8edb5545 Merge branch 'enh/new3.0' into enh/refactorBackend 2023-11-01 16:07:49 +08:00
kailixu
1530fe0c49 Merge branch '3.0' into enh/TD-23769-3.0x 2023-11-01 09:52:16 +08:00
kailixu
ad1e6accd4 chore: build checkpoint for rsma 2023-11-01 09:45:58 +08:00
yihaoDeng
bb8252df16 fix mem leak 2023-10-31 20:59:24 +08:00
Haojun Liao
0b18192b23 fix(stream): fix error in finishing event handling. 2023-10-31 16:40:16 +08:00
Haojun Liao
4c0242a58d fix(stream): fix error in finishing event handling. 2023-10-31 16:39:14 +08:00
Haojun Liao
86dfc52a6e fix(stream): add event check when handling the end of event. 2023-10-31 10:45:12 +08:00
Haojun Liao
19c5882cbe fix(stream): add event check when handling the end of event. 2023-10-30 10:22:24 +08:00
Haojun Liao
c7285c0ede fix(stream): add expired epset node list for stream tasks in hb to mnode. 2023-10-27 23:41:57 +08:00
Haojun Liao
4c267b538f fix(stream): add expired epset node list for stream tasks in hb to mnode. 2023-10-27 23:05:41 +08:00
Haojun Liao
dcca6b0868 other: merge 3.0. 2023-10-27 11:41:28 +08:00
Haojun Liao
454624516c fix(stream): handle the repeatly issued nodeUpdate msg from dnode. 2023-10-27 09:59:51 +08:00
Haojun Liao
1ccbb77981 fix(stream): fix dead lock when shutdown all tasks. 2023-10-27 09:59:37 +08:00
Haojun Liao
b0ef072556 fix(stream): clear flag for checkpoint. 2023-10-27 09:55:07 +08:00
Haojun Liao
091238e7be fix(stream): fix bugs caused by refactor sm. 2023-10-27 09:54:58 +08:00
Haojun Liao
e1f6043eed fix(stream): fix bugs caused by refactor sm. 2023-10-27 09:54:57 +08:00
Haojun Liao
1eac7f44c0 fix(stream): fix error in stream. 2023-10-27 09:54:56 +08:00
Haojun Liao
579dd8e6b2 refactor(stream): do some internal refactor. 2023-10-27 09:54:56 +08:00
Haojun Liao
6f3c0d17cc refactor(stream): add state machine to manage the state of stream tasks. 2023-10-27 09:54:49 +08:00
Haojun Liao
596eb073b7 refactor(stream): do some internal refactor. 2023-10-27 09:54:48 +08:00
Haojun Liao
2409a17517 refactor(stream): do some internal refactor. 2023-10-27 09:54:44 +08:00
Haojun Liao
164159f293 enh(stream): add parameter to limit the stream sink task. 2023-10-27 09:54:07 +08:00
yihaoDeng
db0f6258fe fix stream case error 2023-10-25 19:38:22 +08:00
liuyao
f51c985914 stream event window 2023-10-25 17:50:16 +08:00
Haojun Liao
03165d5327 fix(stream): handle the repeatly issued nodeUpdate msg from dnode. 2023-10-25 09:55:02 +08:00
Haojun Liao
8cde39eebd fix(stream): fix dead lock when shutdown all tasks. 2023-10-24 19:11:24 +08:00
Haojun Liao
059e94e428 fix(stream): clear flag for checkpoint. 2023-10-22 01:25:00 +08:00
yihaoDeng
1a4cfd11a5 Merge branch 'enh/new3.0' into enh/refactorBackend 2023-10-20 15:28:10 +08:00
Haojun Liao
3b8c85f632 fix(stream): fix bugs caused by refactor sm. 2023-10-20 14:19:57 +08:00
Haojun Liao
270a847809 fix(stream): fix bugs caused by refactor sm. 2023-10-20 10:23:48 +08:00
Haojun Liao
c1cebae6ba fix(stream): fix error in stream. 2023-10-19 16:20:27 +08:00
Haojun Liao
402aefd95d refactor(stream): do some internal refactor. 2023-10-19 15:09:47 +08:00
Haojun Liao
f4caeca24a refactor(stream): add state machine to manage the state of stream tasks. 2023-10-19 10:34:20 +08:00
Haojun Liao
54ec679b58 refactor(stream): do some internal refactor. 2023-10-18 15:34:53 +08:00
Haojun Liao
2904cfd4d5 Merge branch 'main' into refact/streamsm 2023-10-18 10:29:09 +08:00
yihaoDeng
2004c1a346 fix transfer crash 2023-10-17 18:19:52 +08:00
Haojun Liao
7b7ce10475 refactor(stream): do some internal refactor. 2023-10-17 11:23:04 +08:00
Haojun Liao
fdb6ec1fa3 enh(stream): add parameter to limit the stream sink task. 2023-10-16 22:52:39 +08:00
liuyao
7269dedb39 Merge branch '3.0' of https://github.com/taosdata/TDengine into 3.0 2023-10-13 11:35:01 +08:00
liuyao
429125be93 session state recover 2023-10-13 11:32:35 +08:00
Haojun Liao
84c5359d5b other: merge 3.0 2023-10-13 08:58:26 +08:00
yihaoDeng
a56d831a14 refactor backend 2023-10-12 20:16:41 +08:00
Haojun Liao
eb4449999f other: merge 3.0 2023-10-12 10:33:48 +08:00
yihaoDeng
f264ffdcd6 refactor backend 2023-10-11 20:12:07 +08:00
yihaoDeng
031506aef8 refactor backend 2023-10-11 19:46:35 +08:00
Haojun Liao
05dfadd8f8 fix(stream): set initial version for fill-history task. disable reentry of scan-history task function. 2023-10-10 15:52:48 +08:00
Haojun Liao
adf6f65d09 fix(stream): set initial version for fill-history task. disable reentry of scan-history task function. 2023-10-10 14:00:30 +08:00
Haojun Liao
847baa8b71 fix(stream): fix the pause failure bug. 2023-10-09 20:10:27 +08:00
Haojun Liao
21fb502690 enh(stream): add trans to reset task status to avoid being frozen in checkpoint status due to doing checkpoint failure of partial tasks. 2023-10-09 20:10:26 +08:00
Haojun Liao
9a7167aaba fix(stream): sleep for 10ms 2023-10-09 20:10:24 +08:00
Haojun Liao
af2c0c95e9 refactor: do some internal refactor. 2023-10-09 20:10:22 +08:00
Haojun Liao
28008f8263 enh(stream): add more info for sink task. 2023-10-09 20:09:22 +08:00
Haojun Liao
cf14772dd9 enh(stream): optimize the scan wal perf 2023-10-09 20:09:21 +08:00
Haojun Liao
c295ec322a refactor: do some internal refactor. 2023-10-09 20:09:20 +08:00
Haojun Liao
583ac1fc45 refactor: do some internal refactor. 2023-10-09 20:09:20 +08:00
Haojun Liao
bb4ba54f28 fix(stream): enable new time range for stream task. 2023-10-09 20:09:09 +08:00
Haojun Liao
c759d90bf6 enh(stream): optimize the stream status when creating without fill-history option. 2023-10-09 20:09:09 +08:00
Haojun Liao
db897fb03a fix(stream): opt perf. 2023-10-09 20:09:09 +08:00
Haojun Liao
dfe8641c0c refactor: optimize the fill-history task launch policy, and do some other internal refactor. 2023-10-09 20:08:52 +08:00
Haojun Liao
a248f3d088 refactor: do some internal refactor. 2023-10-09 20:08:12 +08:00
Haojun Liao
0db357401d fix(stream): add new status for meta. 2023-10-09 20:08:12 +08:00
Haojun Liao
3912d80c02 fix(stream): do some internal refactor. 2023-10-09 20:08:11 +08:00
Haojun Liao
feb09c8361 fix(stream): fix error in pause scan-history data. 2023-10-09 20:08:11 +08:00
Haojun Liao
4af7812f0b fix(stream): fix the invalid counter for ready tasks in vnode, fix an invalid read. 2023-10-09 20:08:10 +08:00
Haojun Liao
f4f55fcbce fix(stream): record the tasks started info, and do some internal refactor. 2023-10-09 20:08:09 +08:00
Haojun Liao
68b003c804 fix(stream): fix race condition. 2023-10-09 20:08:09 +08:00
Haojun Liao
52763ca2a3 refactor: do some internal refactor. 2023-10-09 20:08:07 +08:00
Haojun Liao
c05a8de3e8 fix(stream): rsp when checkpoint source failed. 2023-10-09 20:08:06 +08:00
Haojun Liao
5eee76df65 fix(stream): add msg identifier check. 2023-10-09 20:08:05 +08:00
Haojun Liao
e0d19af000 fix(stream): refactor re-try dispatch msg for stream tasks. 2023-10-09 20:08:03 +08:00
Haojun Liao
2b16976902 refactor(stream): refactor send msg. 2023-10-09 20:08:03 +08:00
Haojun Liao
27e00b1f19 fix(stream): add some logs, and remove invalid assert. 2023-10-09 20:08:02 +08:00
Haojun Liao
12687eb287 fix(stream): add check info detailed information. 2023-10-09 20:08:02 +08:00
Haojun Liao
ef9efc8a8e refactor: do refactor the sink task. 2023-10-09 20:07:53 +08:00
Haojun Liao
1f45bd82d2 other: merge stream fix in main. 2023-10-09 20:07:01 +08:00
Haojun Liao
e27111ecd1 enh(stream): add node stage check. 2023-10-09 20:06:09 +08:00
Haojun Liao
5c0b8ea804 enh(stream): log the checkpoint time. 2023-10-09 20:06:04 +08:00
Haojun Liao
b95ad74c7f fix(stream): not handle the check msg for follower tasks. 2023-10-09 20:06:04 +08:00
Haojun Liao
636eccc160 fix(stream): add null check 2023-10-09 20:05:37 +08:00
Haojun Liao
0459a4628c fix(stream): fix stream task id error. 2023-10-09 20:05:36 +08:00
Haojun Liao
7ccc273206 other: merge stream fix in main. 2023-10-09 20:05:29 +08:00
Haojun Liao
98ef566eb4 fix(stream): remove fill-history sink task. 2023-10-09 20:04:17 +08:00
Haojun Liao
e84eeee6b8 refactor: do some internal refactor. 2023-10-09 20:04:17 +08:00
Haojun Liao
6fa54789cb fix(stream): disable follower send hb to mnode. and do some internal refactor. 2023-10-09 19:59:31 +08:00
Haojun Liao
1256eafddb other: merge stream fix. 2023-10-09 19:59:24 +08:00
Haojun Liao
aaa12ed4f8 refactor: do some internal refactor about the sched status set 2023-10-09 19:56:17 +08:00
Haojun Liao
d87db2f829 fix(stream): fix the pause failure bug. 2023-10-09 14:48:11 +08:00
Haojun Liao
ee6516f3c4 enh(stream): add trans to reset task status to avoid being frozen in checkpoint status due to doing checkpoint failure of partial tasks. 2023-10-09 10:13:53 +08:00
Haojun Liao
23fb26798f fix(stream): sleep for 10ms 2023-10-07 19:47:49 +08:00
Haojun Liao
ab17bfdf09 refactor: do some internal refactor. 2023-10-07 19:16:55 +08:00
liuyao
1aaf873134 fix issue 2023-10-07 17:43:50 +08:00
yihaoDeng
c146327da8 refact task backend 2023-10-07 16:48:36 +08:00
yihaoDeng
fa70221238 refact task backend 2023-10-07 12:17:55 +08:00
Haojun Liao
c20dd002fb enh(stream): add more info for sink task. 2023-10-07 01:14:00 +08:00
Haojun Liao
41ef1a45c1 enh(stream): optimize the scan wal perf 2023-10-04 22:06:46 +08:00
Haojun Liao
a543f4ca97 refactor: do some internal refactor. 2023-10-03 01:52:03 +08:00
Haojun Liao
5718158fc0 refactor: do some internal refactor. 2023-10-01 22:27:29 +08:00
Haojun Liao
f2d8431619 fix(stream): enable new time range for stream task. 2023-09-28 18:26:52 +08:00
yihaoDeng
54e3ac2c1e refact task backend 2023-09-28 15:46:12 +08:00
Haojun Liao
80ec77cbb6 enh(stream): optimize the stream status when creating without fill-history option. 2023-09-28 11:23:03 +08:00
Haojun Liao
22e485d132 fix(stream): opt perf. 2023-09-28 09:25:23 +08:00
Haojun Liao
e13efd5b04 refactor: optimize the fill-history task launch policy, and do some other internal refactor. 2023-09-27 14:00:48 +08:00
liuyao
c95dc167b7 add ci 2023-09-26 17:19:21 +08:00
Haojun Liao
890d658aea refactor: do some internal refactor. 2023-09-26 14:11:45 +08:00
Haojun Liao
f3f18b0fa3 fix(stream): add new status for meta. 2023-09-26 12:15:12 +08:00
liuyao
82e18e32b0 add ci 2023-09-26 10:41:12 +08:00
Haojun Liao
d214a2f6bd fix(stream): do some internal refactor. 2023-09-26 09:32:57 +08:00
Haojun Liao
6cc3618ed9 fix(stream): fix error in pause scan-history data. 2023-09-26 01:09:55 +08:00
Haojun Liao
fcf4cc7169 fix(stream): fix the invalid counter for ready tasks in vnode, fix an invalid read. 2023-09-25 15:58:24 +08:00
Haojun Liao
eb4078b7f3 fix(stream): record the tasks started info, and do some internal refactor. 2023-09-25 09:59:05 +08:00
Haojun Liao
c987f61300 fix(stream): fix race condition. 2023-09-25 01:37:27 +08:00
Haojun Liao
4ea737571d refactor: do some internal refactor. 2023-09-24 01:26:51 +08:00
Haojun Liao
acd3321494 fix(stream): rsp when checkpoint source failed. 2023-09-23 14:13:33 +08:00
liuyao
5cc1ae1266 fix issue 2023-09-22 19:25:19 +08:00
Haojun Liao
5b12e276cf fix(stream): add msg identifier check. 2023-09-22 18:13:18 +08:00
yihaoDeng
11da631bb5 add cvt state 2023-09-22 09:08:42 +00:00
Haojun Liao
619a9f6d08 fix(stream): refactor re-try dispatch msg for stream tasks. 2023-09-22 15:57:55 +08:00
liuyao
410a177637 fix issue 2023-09-22 15:00:26 +08:00
Haojun Liao
898aea5cfb refactor(stream): refactor send msg. 2023-09-22 14:52:18 +08:00
Haojun Liao
87ea745861 fix(stream): add some logs, and remove invalid assert. 2023-09-22 10:53:03 +08:00
Haojun Liao
f7bcff862d fix(stream): add check info detailed information. 2023-09-22 10:23:56 +08:00
liuyao
4979fc5283 fix issue 2023-09-22 09:40:08 +08:00
yihaoDeng
470849982e add cvt func 2023-09-21 09:41:30 +00:00
Haojun Liao
17586af0ce refactor: do refactor the sink task. 2023-09-21 13:32:57 +08:00
Haojun Liao
6633fb971b fix(stream): update logs. 2023-09-20 18:18:31 +08:00
liuyao
1cebda368a session&state operator buff 2023-09-20 15:06:08 +08:00
Haojun Liao
b4419bda65 enh(stream): add node stage check. 2023-09-20 10:35:01 +08:00
Haojun Liao
b7efedf4aa enh(stream): log the checkpoint time. 2023-09-19 09:53:33 +08:00
Haojun Liao
e76bab7122 fix(stream): not handle the check msg for follower tasks. 2023-09-19 09:43:14 +08:00
Haojun Liao
c33ef4ce88 fix(stream): add null check 2023-09-18 15:14:51 +08:00
Haojun Liao
01144c58a7 fix(stream): fix stream task id error. 2023-09-17 01:19:59 +08:00
Haojun Liao
4846194bbb refactor: do some internal refactor, remove the related fill-history task by sending mesg to replay it in follower node. 2023-09-16 21:37:30 +08:00
Haojun Liao
91e3d70796 fix(stream): remove fill-history sink task. 2023-09-15 17:46:17 +08:00
Haojun Liao
2bfd6e3355 refactor: do some internal refactor. 2023-09-15 09:25:22 +08:00
Haojun Liao
f2c3e0a6e8 fix(stream): disable follower send hb to mnode. and do some internal refactor. 2023-09-14 17:22:31 +08:00
Haojun Liao
fce5be0eb4 fix(stream): fix dead lock. 2023-09-14 14:09:05 +08:00
Haojun Liao
7305c8dbe1 refactor: do some internal refactor about the sched status set 2023-09-13 14:00:34 +08:00
Haojun Liao
09a15bf340 refactor: do some internal refactor. 2023-09-12 22:07:52 +08:00
Haojun Liao
60f5056ab7 other: merge 3.0 2023-09-12 22:00:40 +08:00
Haojun Liao
f6a097d96f refactor: do some internal refactor. 2023-09-12 21:58:22 +08:00
Haojun Liao
e457d94ac6 refactor: do some internal refactor. 2023-09-12 18:09:02 +08:00
Haojun Liao
85a5c45098 fix(stream): limit the already processed data. 2023-09-10 00:01:25 +08:00
Haojun Liao
b317c8ebff fix(stream): disable merge submit blocks. 2023-09-08 12:46:15 +08:00
Haojun Liao
b240e91c91 enh(stream): add token bucket to limit the rate of sink operation. 2023-09-07 09:58:48 +08:00
Haojun Liao
8edf86b3bb fix(stream): add some logs. 2023-09-06 19:08:09 +08:00
liuyao
d7caed1c44 opt stream state 2023-09-06 16:34:25 +08:00
Haojun Liao
8783ebcb02 enh(stream): add some logs and record the submit package performance. 2023-09-06 14:34:36 +08:00
Haojun Liao
aeea699c9c Merge branch '3.0' into fix/3_liaohj 2023-09-04 15:53:39 +08:00
Haojun Liao
037a232bfa refactor: do some internal refactor. 2023-09-01 17:05:36 +08:00
Haojun Liao
e816d0efa5
Merge pull request #22669 from taosdata/fix/TD-26069 
fix invalid insert
 2023-08-31 18:29:25 +08:00
yihaoDeng
1833be028f fix invalid insert 2023-08-31 15:28:31 +08:00
Haojun Liao
e2bb64eb18 refactor: do some internal refactor. 2023-08-31 14:23:46 +08:00
Haojun Liao
69f95f9324 other: merge 3.0 2023-08-31 13:56:34 +08:00
Haojun Liao
2285b834b0 refactor: do some internal refactor. 2023-08-31 13:53:12 +08:00
Haojun Liao
3fc3dafca6 refactor: do some internal refactor. 2023-08-31 13:46:48 +08:00
Haojun Liao
3ed78ddadd
Merge pull request #22662 from taosdata/fix/3_liaohj 
refactor: do some internal refactor.
 2023-08-31 13:46:14 +08:00
Haojun Liao
8645f763b5
Merge pull request #22661 from taosdata/enh/reserve 
stream change ver
 2023-08-31 13:35:23 +08:00
yihaoDeng
83a5e2be4c stream change ver 2023-08-31 11:42:12 +08:00
Haojun Liao
e53b5d4392 refactor: do some internal refactor. 2023-08-31 11:01:53 +08:00
yihaoDeng
05ca71d5de stream change ver 2023-08-31 09:28:43 +08:00
Haojun Liao
d99d8875e1 fix(stream): if stream task is running, do not recheck its status. 2023-08-30 17:24:13 +08:00
liuyao
0a72e576bd op stream pause 
pause data dispatch

opt stream pause
 2023-08-30 14:53:49 +08:00
Haojun Liao
3a7a220d43 fix(stream): fix error found by ci. 2023-08-30 10:14:01 +08:00
Haojun Liao
204fb97f68 fix(stream): fix error in snode, fix memory leaks, fix repeatly create bug in sink task. 2023-08-29 17:34:31 +08:00
Haojun Liao
391103bc5a fix(stream): do dome internal refactor. 2023-08-28 15:54:01 +08:00
Haojun Liao
b75ae217f6 fix(stream): notify close for snode. 2023-08-28 09:53:08 +08:00
yihaoDeng
998dac0ffb fix mem leak 2023-08-26 21:54:36 +08:00
yihaoDeng
7d237d4310 fix UAF 2023-08-26 17:37:09 +08:00
Haojun Liao
60d1df0e25
Merge branch '3.0' into enh/triggerCheckPoint2 2023-08-25 15:49:28 +08:00
Haojun Liao
ce1a3a379c fix(stream): handle the case when the task not in ready state, and the checkpoint msg arrived. 2023-08-25 10:12:12 +08:00
Haojun Liao
70396aa5a5 fix(stream): wakeup the hb timer every 200ms, to speeup the close db procedure. 2023-08-24 17:54:50 +08:00
yihaoDeng
3afea998ac refactor log 2023-08-24 10:15:15 +08:00
Haojun Liao
af80d94a88 refactor: do some internal refactor. 2023-08-22 15:43:50 +08:00
Haojun Liao
cc8b2263cb fix(stream): fix error in heartbeat from vnode. 2023-08-22 13:37:38 +08:00
Haojun Liao
def0877980 fix(stream): add timer to check stream task status. 2023-08-22 10:46:49 +08:00
Haojun Liao
b8101afbe2 fix(stream): refactor the tasks restart mechanism. 2023-08-18 22:36:38 +08:00
Haojun Liao
4b10393c81 fix(stream): fix memory leak and do some internal refactor. 2023-08-18 18:02:08 +08:00
Haojun Liao
6746339609 Merge remote-tracking branch 'origin/enh/triggerCheckPoint2' into enh/triggerCheckPoint2 
# Conflicts:
#	source/libs/stream/src/streamSnapshot.c
 2023-08-17 00:08:44 +08:00
Haojun Liao
ea6574b3c7 other: merge 3.0 2023-08-17 00:07:55 +08:00
yihaoDeng
ab7e2b3122 Merge branch 'enh/chkpTransfer' into enh/triggerCheckPoint2 2023-08-16 09:00:30 +00:00
Haojun Liao
d63758a1c5 fix(stream): fix error in execute task on a tranferred node. 2023-08-16 15:24:24 +08:00
Haojun Liao
39079ab64a fix(stream): load term from sync model to decide if the node is restart or not 2023-08-15 15:52:06 +08:00
Haojun Liao
b1d35f43ba enh(stream): using term from sync module to denote the upstream node is restarted/replica-modified/vnode-transferred or not. 2023-08-15 09:25:05 +08:00
Haojun Liao
e8c9a019a4 fix(stream): remove the invalid set of scheduler status. 2023-08-14 11:41:24 +08:00
Haojun Liao
0d38f389ab refactor: do some internal refactor. 2023-08-13 16:25:35 +08:00
Haojun Liao
afe9b848a5 fix(tsdb): check the 2023-08-12 18:54:39 +08:00
Haojun Liao
09da6c6840 refactor: do some internal refactor. 2023-08-12 18:51:20 +08:00
Haojun Liao
75e6fb0f16 refactor: do some internal refactor. 2023-08-12 18:47:01 +08:00
Haojun Liao
8f9de93cf0 refactor: do some internal refactor. 2023-08-12 18:37:36 +08:00
Haojun Liao
ce721a0146 fix(stream): fix error. 2023-08-12 16:32:05 +08:00
Haojun Liao
a89ce1a20b fix(stream): transfer state by using data block. 2023-08-12 01:24:07 +08:00
yihaoDeng
6e6eb05d5c Merge branch 'enh/triggerCheckPoint2' into enh/chkpTransfer 2023-08-11 08:03:02 +00:00
Haojun Liao
e984affe90 fix(stream): disable the stream in the followers nodes. 2023-08-11 15:58:15 +08:00
yihaoDeng
99be2836b2 Merge branch 'enh/triggerCheckPoint2' into enh/chkpTransfer 2023-08-11 02:50:12 +00:00
Haojun Liao
6a9826cf6e enh(stream): update the stream task epset. 2023-08-11 10:37:25 +08:00
yihaoDeng
ba4ce9db02 Merge branch 'enh/triggerCheckPoint2' into enh/chkpTransfer 2023-08-11 02:15:07 +00:00
Haojun Liao
f633c7fe2b other: merge 3.0. 2023-08-10 11:56:30 +08:00
yihaoDeng
ef247cdb1d support reopen stream state 2023-08-10 01:49:26 +00:00
Haojun Liao
1e8579e8c5 fix(stream): fix other cases. 2023-08-09 16:04:48 +08:00
Haojun Liao
4c92997328 fix(stream): use streamId&taskId to identify the stream task in the stream meta. 2023-08-09 15:52:23 +08:00
Haojun Liao
f3c5f20ee2 enh(stream): add timer to check the node changing. 2023-08-09 10:25:16 +08:00
Haojun Liao
af3a87560a refactor: do some refactor. 2023-08-08 09:11:41 +08:00
yihaoDeng
b6c991f896 support reopen stream state 2023-08-07 08:59:04 +00:00
yihaoDeng
eeb97351e8 support reopen stream state 2023-08-07 08:51:01 +00:00
Haojun Liao
225c0e61dc hen(stream): support the rollback and checkdown stream tasks. 2023-08-07 16:13:39 +08:00
Haojun Liao
8a11a0418f other: merge 3.0 2023-08-07 09:19:26 +08:00
yihaoDeng
6dc93a8dec add stream ver 2023-08-04 13:52:20 +08:00
yihaoDeng
f9d0874e45 support reopen stream state 2023-08-03 08:24:50 +00:00
Haojun Liao
13543f7daa feature(stream): handle task update in tq module. 2023-08-02 10:33:41 +08:00
Haojun Liao
ad4c5916e7 enh(stream): update the stream task. 2023-08-01 18:14:58 +08:00
Haojun Liao
164bfd5408 enh(stream): prepare update trans. 2023-08-01 16:12:35 +08:00
Haojun Liao
6365d5000b
Merge pull request #22271 from taosdata/fix/3_liaohj 
fix(stream): dump results to sink node before paused.
 2023-08-01 15:50:17 +08:00
Haojun Liao
75b1520be0 enh(stream): create task update info 2023-08-01 14:04:07 +08:00
Haojun Liao
f6ae9cf9bb fix(stream): fetch all data before paused and dump to sink node. 2023-08-01 10:16:14 +08:00
Haojun Liao
36ef9a537f
Merge pull request #22256 from taosdata/fix/TD-25457 
check update data
 2023-07-31 22:56:05 +08:00
Haojun Liao
d45596bb33 fix(stream): kill task when pause the stream execution. 2023-07-31 18:07:37 +08:00
liuyao
32b59af4b1 check update data 2023-07-31 17:32:24 +08:00
Haojun Liao
9981cecbed refactor: do some internal refactor. 2023-07-31 15:24:35 +08:00
Haojun Liao
e61aa83594 enh(stream): handle the stream hb. 2023-07-31 14:07:18 +08:00
Haojun Liao
8dfef8768a other: merge 3.0 2023-07-29 22:54:16 +08:00
Haojun Liao
936afeb5ac refactor: do internal refactor to simple the pause/resume in case of fill history exists. 2023-07-28 19:13:03 +08:00
Haojun Liao
d5974a8f25 refactor(stream): refactor the pause/resume for fill history execution. 2023-07-28 18:05:32 +08:00
Haojun Liao
9f943213ed enh(stream): update the hb info. 2023-07-28 14:23:53 +08:00
Haojun Liao
74834aea11 enh(stream): prepare hb to mnode from stream meta. 2023-07-28 13:07:16 +08:00
Haojun Liao
22a5a18fff fix(stream): 1. set correct timewindow after step2. 2. handle the case when the task is failed to be added into the meta store. 2023-07-27 18:57:36 +08:00
Haojun Liao
40fe3ef4f6 fix(stream): refactor the step2 wal scan. 2023-07-27 14:49:20 +08:00
Haojun Liao
e89f530db2 fix(stream): scan wal in step2 2023-07-27 10:55:59 +08:00
Haojun Liao
728112ed89 fix(stream): set correct upstream epinfo. 2023-07-26 10:26:59 +08:00
Haojun Liao
1c1bf63e1f other: merge 3.0 2023-07-25 18:17:11 +08:00
Haojun Liao
0bdccd2681 refactor: do some internal refactor. 2023-07-25 17:31:54 +08:00
Haojun Liao
a2694b9ce9 fix(stream): fix the invalid check of step2 . 2023-07-24 19:44:28 +08:00
Haojun Liao
1e34f84f78 other: merge 3.0 2023-07-24 15:23:03 +08:00
Haojun Liao
1367552f38 fix(stream): refactor the halt status to check more status. 2023-07-24 10:04:15 +08:00
Haojun Liao
4e0f7ffbf8 fix(stream): set the correct flag of pause/resume. 2023-07-23 18:49:56 +08:00
Haojun Liao
1b2636028a fix(stream): fix memory leak. 2023-07-21 23:05:42 +08:00
Haojun Liao
16d7707b90 fix(stream): align the scan real time data for stream task. 2023-07-21 19:33:43 +08:00
Haojun Liao
db46d31ca5 fix(stream): refactor the halt function. 2023-07-21 01:32:42 +08:00
Haojun Liao
aed99da6c1 refactor: refactor the stream task starting order. 2023-07-20 12:14:53 +08:00
yihaoDeng
6e5be0d30e fix stream state transfer 2023-07-20 03:49:32 +00:00
Haojun Liao
97c44e7c18 refactor: do some internal refactor. 2023-07-19 18:42:02 +08:00
Haojun Liao
2ae95217bd fix(stream): open inputQ by default for task deployed on snode. 2023-07-19 10:05:35 +08:00
Haojun Liao
d2c875e774 Merge remote-tracking branch 'origin/enh/triggerCheckPoint2' into enh/triggerCheckPoint2 2023-07-18 15:19:09 +08:00
Haojun Liao
901b7d8ecc fix(stream): if a source task is set to be non-normal, no data should be put into inputQ anymore. 2023-07-18 15:18:57 +08:00
yihaoDeng
40d59990d8 fix mem leak 2023-07-18 04:01:20 +00:00
Haojun Liao
08f43b9d00 fix(stream): fix checkpoint block not fetched bug. 2023-07-17 15:31:54 +08:00
Haojun Liao
b2a905bd91
Merge branch '3.0' into enh/triggerCheckPoint2 2023-07-17 11:50:12 +08:00
Haojun Liao
a9757637d9 Merge remote-tracking branch 'origin/enh/triggerCheckPoint2' into enh/triggerCheckPoint2 2023-07-16 12:47:24 +08:00
Haojun Liao
c6a322e0ef refactor(stream): back pressure is active, which is trigger by dispatch rsp. 2023-07-16 12:47:08 +08:00
yihaoDeng
e18c9ea672 add checkpoint 2023-07-15 04:26:34 +00:00
yihaoDeng
ed931c1100 Merge branch 'fix/newCheckpoint' into enh/triggerCheckPoint2 2023-07-14 10:11:20 +00:00
yihaoDeng
3c4631ef1f add checkpoint 2023-07-14 10:06:24 +00:00
Haojun Liao
2b85ea62aa refactor: do some internal refactor. 2023-07-14 17:48:47 +08:00
Haojun Liao
eb7e6152ae refactor(stream): do some internal refactor. 2023-07-14 17:21:20 +08:00
Haojun Liao
813f4cb363 refactor: do some internal refactor. 2023-07-13 16:32:25 +08:00
Haojun Liao
2f90a53548 fix(stream): set checkpoint id when expanding task. 2023-07-13 10:16:57 +08:00
liuyao
7d4bb1b932 add checkpoint id for recover 2023-07-12 10:48:58 +08:00
Haojun Liao
fed3803739 fix(stream): fix memory leak. 2023-07-11 19:29:52 +08:00
Haojun Liao
09b764494d fix(stream): set correct checkpoint version when starting stream tasks. 2023-07-11 14:03:43 +08:00
Haojun Liao
87bedef2cd fix(stream): fix error in checkpointing. 2023-07-10 19:20:05 +08:00
Haojun Liao
4f534ce6e9 fix(stream): fix error in generating checkpoint. 2023-07-08 10:29:09 +08:00
Haojun Liao
a5c19427e7 fix(stream): send correct rsp to mnode. 2023-07-08 09:39:44 +08:00
Haojun Liao
9c30abf95b fix(stream): fix error in generating checkpoint. 2023-07-07 21:38:53 +08:00
Haojun Liao
8298f30e56 fix(stream): fix bug in checkpoint. 2023-07-07 17:16:08 +08:00
Haojun Liao
47877898d0 enh(stream): generate the checkpoint framework. 2023-07-07 16:07:52 +08:00
Haojun Liao
d7101f7109 enh(stream): support dispatch checkpoint msg. 2023-07-06 18:26:22 +08:00
liuyao
0e914a19c6 stream operartor checkpoint 2023-07-06 14:28:34 +08:00
Haojun Liao
5a9461a69a other: merge 3.0 2023-07-06 11:52:21 +08:00
Haojun Liao
f4a3567935 other: merge 3.0 2023-07-06 11:24:30 +08:00
Haojun Liao
03dbcbf055 refactor: do some internal refactor. 2023-07-05 15:55:55 +08:00
yihaoDeng
7d30a6e27a vnode snapshot read 2023-06-30 11:48:46 +00:00
yihaoDeng
6b6e237079 add checkpoint 2023-06-28 09:57:08 +00:00
Haojun Liao
57fcd553f8 refactor: do some internal refactor. 2023-06-28 13:13:13 +08:00
Haojun Liao
a19e63fd23 fix(stream): fix error in handling fill history. 2023-06-27 23:00:14 +08:00
Haojun Liao
b73444b291 fix(stream): start stream task in case of scan history completing. 2023-06-27 09:44:51 +08:00
Haojun Liao
402c091df0 fix(stream): fix sma error. 2023-06-25 19:41:44 +08:00
yihaoDeng
0330f3b2db add checkpoint 2023-06-25 17:03:01 +08:00