Elgato_dark/TDengine
1
0
Fork 0
mirror of https://github.com/taosdata/TDengine synced 2026-05-24 10:09:01 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 4
67094 commits 554 branches 514 tags 1.7 GiB
Commit graph

284 commits

Author SHA1 Message Date
yihaoDeng
6129ebc294 Merge branch '3.0' into enh/refactorBackend 2023-11-23 17:41:37 +08:00
yihaoDeng
6a0501967d Merge branch '3.0' into fix/fixChkpRespMemleak 2023-11-23 14:06:04 +08:00
Haojun Liao
4ea49fe285 fix(stream): fix the race condition in dispatching data. 2023-11-22 23:49:16 +08:00
yihaoDeng
947775155b fix transfer crash 2023-11-21 20:51:11 +08:00
yihaoDeng
a92602b667 fix mem leak 2023-11-21 15:23:28 +08:00
yihaoDeng
939aa7a3cc Merge branch '3.0' into enh/refactorBackend 2023-11-20 17:33:30 +08:00
yihaoDeng
0d724244dc Merge branch '3.0' into enh/refactorBackend 2023-11-20 16:55:14 +08:00
Haojun Liao
7cfa45e95e enh(stream): add sink_quota/scan-history-idle-duration column for stream tasks. 2023-11-20 11:59:49 +08:00
yihaoDeng
aa2c09ef5b rm dup data 2023-11-14 18:09:00 +08:00
yihaoDeng
4476e46a5f rm dup data 2023-11-14 17:43:13 +08:00
yihaoDeng
4104840cea Merge branch '3.0' into enh/refactorBackend 2023-11-13 18:07:38 +08:00
yihaoDeng
de7f2a2d1a Merge branch '3.0' into enh/refactorBackend 2023-11-13 11:11:28 +08:00
Haojun Liao
9ee9174608 other: merge 3.0 2023-11-11 15:48:31 +08:00
Haojun Liao
97772e9aab fix(stream): the checkpoint version can only be updated when generating checkpoint. 2023-11-11 15:40:05 +08:00
yihaoDeng
1e8d46c405 Merge branch '3.0' into enh/refactorBackend 2023-11-10 18:37:25 +08:00
yihaoDeng
a3fdb6dd59 Merge branch '3.0' into enh/refactorBackend 2023-11-10 16:06:08 +08:00
yihaoDeng
5f9b922a04 support s3 2023-11-09 20:15:08 +08:00
yihaoDeng
dd2252598e upload by rsync 2023-11-09 17:39:34 +08:00
yihaoDeng
3abf0ae4b2 Merge branch 'feat/TD-26060' into enh/refactorBackend 2023-11-08 17:31:16 +08:00
yihaoDeng
92fa1043b8 Merge branch 'enh/refactorBackend' of https://github.com/taosdata/TDengine into enh/refactorBackend 2023-11-08 17:12:32 +08:00
yihaoDeng
2de5c4a97d add bg thread 2023-11-08 17:11:11 +08:00
yihaoDeng
76fcc49c6d fix stream transfer error 2023-11-08 17:10:54 +08:00
Haojun Liao
f949333e8b refactor:do some internal refactor. 2023-11-08 16:57:55 +08:00
Haojun Liao
99dbbe61d7 refactor(stream): do some internal refactor. 2023-11-08 15:20:47 +08:00
wangmm0220
c4c1423243 fix:error 2023-11-08 15:01:04 +08:00
wangmm0220
047eb14f67 fix:error 2023-11-08 11:57:49 +08:00
Haojun Liao
17bd6badec refactor(stream): do checkpoint for each task. 2023-11-08 10:47:54 +08:00
wangmm0220
07aa749fcd fix:adapt to windows 2023-11-08 10:28:45 +08:00
wangmm0220
f0b2ed567c fix:upload checkpoint to s3 2023-11-07 18:49:12 +08:00
yihaoDeng
f038f88c07 Merge branch 'feat/TD-26060' into enh/refactorBackend 2023-11-03 18:29:18 +08:00
wangmm0220
5a6c3f2b04 fix:add windows support 2023-11-03 17:45:37 +08:00
wangmm0220
0a1f1085fc Merge branch '3.0' of https://github.com/taosdata/TDengine into feat/TD-26060 2023-11-03 11:33:50 +08:00
wangmm0220
3ac11eba0e fix:modify upload checkpoint interface 2023-11-03 11:33:18 +08:00
yihaoDeng
24e961976f Merge branch 'enh/new3.0' into enh/refactorBackend 2023-11-03 09:31:57 +08:00
wangmm0220
159873692c feat:checkpoint bakeup using rsync 2023-11-02 20:13:29 +08:00
Haojun Liao
7cf90dde5c refactor: do some internal refactor. 2023-11-01 19:38:13 +08:00
Haojun Liao
00f029e44f refactor: add some logs. 2023-11-01 19:38:12 +08:00
yihaoDeng
4c8edb5545 Merge branch 'enh/new3.0' into enh/refactorBackend 2023-11-01 16:07:49 +08:00
Haojun Liao
88469568b8 fix(stream): handle failure during checkpoint 2023-10-31 10:45:14 +08:00
Haojun Liao
0d4892f25b fix(stream): fix bugs caused by refactor sm. 2023-10-27 09:55:00 +08:00
Haojun Liao
6f3c0d17cc refactor(stream): add state machine to manage the state of stream tasks. 2023-10-27 09:54:49 +08:00
yihaoDeng
1a4cfd11a5 Merge branch 'enh/new3.0' into enh/refactorBackend 2023-10-20 15:28:10 +08:00
Haojun Liao
af4618f958 fix(stream): remove invalid clear of rspmsg list. 2023-10-09 20:10:27 +08:00
Haojun Liao
21fb502690 enh(stream): add trans to reset task status to avoid being frozen in checkpoint status due to doing checkpoint failure of partial tasks. 2023-10-09 20:10:26 +08:00
Haojun Liao
af2c0c95e9 refactor: do some internal refactor. 2023-10-09 20:10:22 +08:00
Haojun Liao
8477271240 fix(stream): fix dead lock. 2023-10-09 20:09:00 +08:00
Haojun Liao
52763ca2a3 refactor: do some internal refactor. 2023-10-09 20:08:07 +08:00
Haojun Liao
3baddbe67a fix(stream): update the checkpoint record start time. 2023-10-09 20:08:07 +08:00
Haojun Liao
a6ffb845c8 fix(stream): fix invalid checkpoint start time. 2023-10-09 20:08:07 +08:00
Haojun Liao
e5535fa1f5 fix(stream): handle dispatch checkpoint-trigger failure. 2023-10-09 20:08:01 +08:00
Haojun Liao
65cc4040f3 fix(stream): update logs. 2023-10-09 20:08:00 +08:00
Haojun Liao
6d3a134077 fix(stream): ignore the dispatch failure, and set the correct rsp counter. 2023-10-09 20:07:54 +08:00
Haojun Liao
e4ac835554 fix(stream): disable token bucket. 2023-10-09 20:07:53 +08:00
Haojun Liao
a3d13d6634 fix(stream): remove invalid cached task info in buffer. 2023-10-09 20:07:08 +08:00
Haojun Liao
1f45bd82d2 other: merge stream fix in main. 2023-10-09 20:07:01 +08:00
Haojun Liao
d76e3ac718 fix(stream): add task status check. 2023-10-09 20:06:10 +08:00
Haojun Liao
053643be19 fix(stream): add timestamp. 2023-10-09 20:06:05 +08:00
Haojun Liao
5c0b8ea804 enh(stream): log the checkpoint time. 2023-10-09 20:06:04 +08:00
Haojun Liao
0459a4628c fix(stream): fix stream task id error. 2023-10-09 20:05:36 +08:00
Haojun Liao
7ccc273206 other: merge stream fix in main. 2023-10-09 20:05:29 +08:00
Haojun Liao
e84eeee6b8 refactor: do some internal refactor. 2023-10-09 20:04:17 +08:00
Haojun Liao
8266c4ff77 fix(stream): remove invalid clear of rspmsg list. 2023-10-09 15:26:42 +08:00
Haojun Liao
ee6516f3c4 enh(stream): add trans to reset task status to avoid being frozen in checkpoint status due to doing checkpoint failure of partial tasks. 2023-10-09 10:13:53 +08:00
Haojun Liao
ab17bfdf09 refactor: do some internal refactor. 2023-10-07 19:16:55 +08:00
yihaoDeng
e41da13d5b refact task backend 2023-10-07 17:07:26 +08:00
Haojun Liao
8c998673ad fix(stream): fix dead lock. 2023-09-27 17:15:49 +08:00
Haojun Liao
4ea737571d refactor: do some internal refactor. 2023-09-24 01:26:51 +08:00
Haojun Liao
7f265e181d fix(stream): update the checkpoint record start time. 2023-09-23 18:05:14 +08:00
Haojun Liao
6ef68fdf06 fix(stream): fix invalid checkpoint start time. 2023-09-23 17:57:09 +08:00
Haojun Liao
b075b4438b fix(stream): handle dispatch checkpoint-trigger failure. 2023-09-21 23:15:18 +08:00
Haojun Liao
50d5fb6641 fix(stream): update logs. 2023-09-21 19:14:34 +08:00
Haojun Liao
4925b67d4a fix(stream): ignore the dispatch failure, and set the correct rsp counter. 2023-09-21 17:22:09 +08:00
Haojun Liao
9f9b712332 fix(stream): disable token bucket. 2023-09-21 01:19:58 +08:00
Haojun Liao
92f18989df fix(stream): remove invalid cached task info in buffer. 2023-09-20 19:19:04 +08:00
Haojun Liao
6633fb971b fix(stream): update logs. 2023-09-20 18:18:31 +08:00
Haojun Liao
2a781c509f fix(stream): add task status check. 2023-09-20 15:32:15 +08:00
Haojun Liao
309630eb11 fix(stream): add timestamp. 2023-09-19 14:22:05 +08:00
Haojun Liao
b7efedf4aa enh(stream): log the checkpoint time. 2023-09-19 09:53:33 +08:00
Haojun Liao
01144c58a7 fix(stream): fix stream task id error. 2023-09-17 01:19:59 +08:00
Haojun Liao
4846194bbb refactor: do some internal refactor, remove the related fill-history task by sending mesg to replay it in follower node. 2023-09-16 21:37:30 +08:00
Haojun Liao
2bfd6e3355 refactor: do some internal refactor. 2023-09-15 09:25:22 +08:00
Haojun Liao
f602aa965f fix(stream): fix coverity scan issues. 2023-09-12 09:16:18 +08:00
Haojun Liao
85a5c45098 fix(stream): limit the already processed data. 2023-09-10 00:01:25 +08:00
Haojun Liao
2285b834b0 refactor: do some internal refactor. 2023-08-31 13:53:12 +08:00
Haojun Liao
3fc3dafca6 refactor: do some internal refactor. 2023-08-31 13:46:48 +08:00
Haojun Liao
e53b5d4392 refactor: do some internal refactor. 2023-08-31 11:01:53 +08:00
Haojun Liao
3a7a220d43 fix(stream): fix error found by ci. 2023-08-30 10:14:01 +08:00
Haojun Liao
391103bc5a fix(stream): do dome internal refactor. 2023-08-28 15:54:01 +08:00
Haojun Liao
46529d383c fix(stream): build checkpoint for sink tasks. 2023-08-27 00:18:41 +08:00
Haojun Liao
b7c36bbfe4 fix(stream): fix the status error. 2023-08-26 00:49:23 +08:00
Haojun Liao
ce1a3a379c fix(stream): handle the case when the task not in ready state, and the checkpoint msg arrived. 2023-08-25 10:12:12 +08:00
Haojun Liao
f7ea875af1 fix(stream): add some logs. 2023-08-24 23:27:48 +08:00
Haojun Liao
4afe603c6e fix(stream): fix error. 2023-08-11 14:17:09 +08:00
Haojun Liao
8dfef8768a other: merge 3.0 2023-07-29 22:54:16 +08:00
Haojun Liao
9f943213ed enh(stream): update the hb info. 2023-07-28 14:23:53 +08:00
Haojun Liao
1e34f84f78 other: merge 3.0 2023-07-24 15:23:03 +08:00
Haojun Liao
902f9d1f94 fix(stream): set not-ready initial value. 2023-07-18 18:29:35 +08:00
Haojun Liao
901b7d8ecc fix(stream): if a source task is set to be non-normal, no data should be put into inputQ anymore. 2023-07-18 15:18:57 +08:00
Haojun Liao
2f24f776fb fix(stream): fix error in generating checkpoint for state_window. 2023-07-18 09:04:28 +08:00
Haojun Liao
14b9d920ba fix(stream): fix memory leak. 2023-07-17 16:02:41 +08:00
Haojun Liao
c6a322e0ef refactor(stream): back pressure is active, which is trigger by dispatch rsp. 2023-07-16 12:47:08 +08:00
Haojun Liao
725db16af9 fix(stream): set save checkpoint ver. 2023-07-14 19:47:19 +08:00
Haojun Liao
035b199497 refactor(stream): do some internal refactor. 2023-07-14 18:46:38 +08:00
Haojun Liao
8c79157822 fix(stream): close the inputQ if checkpoint msg received. 2023-07-14 18:35:41 +08:00
Haojun Liao
eb7e6152ae refactor(stream): do some internal refactor. 2023-07-14 17:21:20 +08:00
Haojun Liao
813f4cb363 refactor: do some internal refactor. 2023-07-13 16:32:25 +08:00
Haojun Liao
2f90a53548 fix(stream): set checkpoint id when expanding task. 2023-07-13 10:16:57 +08:00
Haojun Liao
773ae39d33 Merge remote-tracking branch 'origin/enh/triggerCheckPoint2' into enh/triggerCheckPoint2 
# Conflicts:
#	source/libs/stream/src/streamCheckpoint.c
 2023-07-13 09:17:51 +08:00
Haojun Liao
08a4fb06ae fix(stream): dispatch checkpoint msg to downstream by puting message into input queue. 2023-07-13 09:17:20 +08:00
yihaoDeng
82468f162a Merge branch 'enh/triggerCheckPoint2' of https://github.com/taosdata/TDengine into enh/triggerCheckPoint2 2023-07-12 16:43:56 +08:00
Haojun Liao
d392bb60ba refactor(stream): set the initial version from the checkpoint. 2023-07-12 11:38:13 +08:00
Haojun Liao
6f2fc4fab3 fix(stream): commit task meta after do vnode-wide checkpoint. 2023-07-12 10:16:14 +08:00
Haojun Liao
09b764494d fix(stream): set correct checkpoint version when starting stream tasks. 2023-07-11 14:03:43 +08:00
Haojun Liao
62a763ead3 fix(stream): fix invalid assert. 2023-07-11 11:22:48 +08:00
Haojun Liao
cbda61c7f9 fix(stream): fix error in checkpointing. 2023-07-11 09:53:08 +08:00
Haojun Liao
ab5b2fe0a8 Merge remote-tracking branch 'origin/enh/triggerCheckPoint2' into enh/triggerCheckPoint2 2023-07-10 18:21:02 +08:00
Haojun Liao
385a5c9651 fix(stream): fix error in generating checkpoint. 2023-07-10 18:20:50 +08:00
yihaoDeng
e263d5f55f add checkpoint 2023-07-10 17:33:48 +08:00
Haojun Liao
f6515e2a2c fix(stream): fix error in checkpointing. 2023-07-10 11:43:37 +08:00
Haojun Liao
4f534ce6e9 fix(stream): fix error in generating checkpoint. 2023-07-08 10:29:09 +08:00
Haojun Liao
a5c19427e7 fix(stream): send correct rsp to mnode. 2023-07-08 09:39:44 +08:00
Haojun Liao
9c30abf95b fix(stream): fix error in generating checkpoint. 2023-07-07 21:38:53 +08:00
Haojun Liao
47877898d0 enh(stream): generate the checkpoint framework. 2023-07-07 16:07:52 +08:00
Haojun Liao
d7101f7109 enh(stream): support dispatch checkpoint msg. 2023-07-06 18:26:22 +08:00
liuyao
0e914a19c6 stream operartor checkpoint 2023-07-06 14:28:34 +08:00
Haojun Liao
f4a3567935 other: merge 3.0 2023-07-06 11:24:30 +08:00
yihaoDeng
a2030dc378 add trigger checkpoint 2023-06-09 12:12:51 +00:00
yihaoDeng
79b628101c add trigger checkpoint 2023-06-07 14:32:22 +00:00
Haojun Liao
06cf3588da refactor: do some internal refactor. 2023-06-07 15:03:04 +08:00
54liuyao
1d58f85a54 feat:coverity scan 2023-02-22 10:10:04 +08:00
Shengliang Guan
6d94afe48f
Revert "refact: adjust some assert cases" 2022-12-07 21:51:17 +08:00
Shengliang Guan
d5126d469a refact: replcase ASSERT with tAssert 2022-12-07 18:42:48 +08:00
Shengliang Guan
f69a188f7e refact: replcase ASSERT with tAssert 2022-12-07 18:37:40 +08:00
Liu Jicong
3424b184d3 refactor(stream): distributed checkpoint 2022-12-02 03:48:45 +08:00