Commit graph

7482 commits

Author SHA1 Message Date
Haojun Liao
77eed06bcd
Merge branch '3.0' into fix/3_liaohj 2023-09-04 18:51:07 +08:00
Haojun Liao
aeea699c9c Merge branch '3.0' into fix/3_liaohj 2023-09-04 15:53:39 +08:00
Haojun Liao
ed22dd5f26 fix(stream): fix deadlock caused during merging branch. 2023-09-04 14:09:39 +08:00
Haojun Liao
375d0abea5 fix(stream): merge main. 2023-09-04 13:12:53 +08:00
Haojun Liao
21dab8ceab other: merge 3.0 2023-09-04 13:08:19 +08:00
yihaoDeng
03cddfa7ce fix rc 2023-09-04 10:33:01 +08:00
Haojun Liao
4fee5ba59a fix(stream): even in ck, start stream tasks. 2023-09-01 14:40:30 +08:00
dapan1121
cc17848b78 fix: merge 3.0 to main 2023-09-01 13:24:47 +08:00
dapan1121
2141ac8460 Merge remote-tracking branch 'origin/main' into fix/m23.2 2023-09-01 11:21:38 +08:00
Hongze Cheng
113867081e enh: upgrade more efficiently 2023-09-01 10:16:10 +08:00
dapan1121
228e6c6acd Merge remote-tracking branch 'origin/main' into 3.0 2023-09-01 10:01:10 +08:00
Haojun Liao
e8294ed8dc fix(stream): fix bug in multi-replica vnode redistribute. 2023-09-01 03:10:12 +08:00
Haojun Liao
e816d0efa5
Merge pull request #22669 from taosdata/fix/TD-26069
fix invalid insert
2023-08-31 18:29:25 +08:00
wangmm0220
a02ab4ccfb fix:coredump if saved epoch bigger than request epoch & modify test cases 2023-08-31 17:35:41 +08:00
yihaoDeng
1833be028f fix invalid insert 2023-08-31 15:28:31 +08:00
yihaoDeng
9fd0bff673 fix invalid insert 2023-08-31 14:57:26 +08:00
Haojun Liao
92e258617a refactor: do some internal refactor. 2023-08-31 14:31:45 +08:00
Haojun Liao
e2bb64eb18 refactor: do some internal refactor. 2023-08-31 14:23:46 +08:00
Haojun Liao
3fc3dafca6 refactor: do some internal refactor. 2023-08-31 13:46:48 +08:00
Haojun Liao
e53b5d4392 refactor: do some internal refactor. 2023-08-31 11:01:53 +08:00
Haojun Liao
0facde6ddc
Merge pull request #22659 from taosdata/fix/TD-26035
fix(stream): do not restart the stream tasks when it is running.
2023-08-31 10:16:59 +08:00
Haojun Liao
0c599fa869
Merge pull request #22415 from taosdata/szhou/tsdb-neigh-block
tsdb neighbouring block search and set it to global block iterator
2023-08-31 09:46:37 +08:00
wangmm0220
7f2a9a7262 fix:cancel the limit time for consume 2023-08-31 09:07:40 +08:00
wade zhang
68e4a72021
Merge pull request #22643 from taosdata/feat/audit-user-typestr
alter user type to str
2023-08-31 07:59:31 +08:00
wangmm0220
4f20359f43 fix:modify log level to trace 2023-08-30 23:57:48 +08:00
Haojun Liao
a2c6e4306a fix(stream): reset the counter. 2023-08-30 20:20:20 +08:00
Haojun Liao
fd2f988cdb fix(stream): reset the counter. 2023-08-30 20:19:51 +08:00
Haojun Liao
5d23df100c fix(stream): ignore the fill-history tasks. 2023-08-30 19:07:06 +08:00
Haojun Liao
65eaa9b15f fix(stream): add task release. 2023-08-30 19:06:12 +08:00
wangmm0220
23c4d7eb0a fix:dead lock 2023-08-30 18:10:19 +08:00
dmchen
3d9274d713 create table db name 2023-08-30 17:28:03 +08:00
Haojun Liao
d99d8875e1 fix(stream): if stream task is running, do not recheck its status. 2023-08-30 17:24:13 +08:00
Haojun Liao
c5b279f045 Merge remote-tracking branch 'origin/enh/triggerCheckPoint2' into enh/triggerCheckPoint2 2023-08-30 16:01:08 +08:00
Haojun Liao
24b1a050fc other: merge 3.0 2023-08-30 16:00:14 +08:00
liuyao
0a72e576bd op stream pause
pause data dispatch

opt stream pause
2023-08-30 14:53:49 +08:00
dapan1121
f6321d0c4f Merge remote-tracking branch 'origin/main' into fix/m23.1 2023-08-30 11:03:33 +08:00
Haojun Liao
dec4d69529 fix(stream): set correct user-define table name in sink table info cache. 2023-08-30 10:52:39 +08:00
Haojun Liao
3a7a220d43 fix(stream): fix error found by ci. 2023-08-30 10:14:01 +08:00
Haojun Liao
610132e29a
Merge pull request #22633 from taosdata/fix/update_db_attr
fix(tsdb): add more check for block rows
2023-08-29 22:05:57 +08:00
wade zhang
701ac939c1
Merge pull request #22632 from taosdata/fix/ttl
fix/TD-26032: ttl cache entry record the original information for deletion
2023-08-29 18:27:50 +08:00
Haojun Liao
8dadae5d3a fix(stream): fix invalid read. 2023-08-29 18:24:32 +08:00
Haojun Liao
d6368798da fix(query): disable the set quit flag for tsdbreader. 2023-08-29 18:11:17 +08:00
Haojun Liao
b740ce8ad6 fix(tsdb): add more check for block rows to detect the duplicate rows in data block. 2023-08-29 17:49:43 +08:00
Haojun Liao
204fb97f68 fix(stream): fix error in snode, fix memory leaks, fix repeatly create bug in sink task. 2023-08-29 17:34:31 +08:00
Shungang Li
df2b2484df fix: ttl cache entry record the original information for deletion 2023-08-29 17:18:22 +08:00
Minglei Jin
82fc1f41fb fix(tsdb/cache read): remove all null row 2023-08-29 16:15:00 +08:00
wade zhang
53a24e04f2
Merge pull request #22604 from taosdata/feat/audit-db-detail
audit db detail
2023-08-29 15:46:10 +08:00
Haojun Liao
6e4acddd96
Merge pull request #22603 from taosdata/fix/TD-25962
fix:unregister push manager if delete subscribe to avoid tqProcessPol…
2023-08-29 10:58:49 +08:00
Minglei Jin
432eaf5110 Merge branch '3.0' into fix/TD-25963 2023-08-29 10:51:53 +08:00
Minglei Jin
b88d79d2ef fix(tsdb/open fs): fix retval 2023-08-29 10:33:20 +08:00
wade zhang
ff6de8ebe2
Merge pull request #22600 from taosdata/fix/TD-25010-10
enh(s3/evict): evict cache data files for other migrated head, stt, etc.
2023-08-28 18:12:40 +08:00
dmchen
1372cede48 init 2023-08-28 17:44:10 +08:00
dapan1121
778e50da79
Merge pull request #22592 from taosdata/fix/m23.0
Fix/m23.0
2023-08-28 17:35:35 +08:00
wangmm0220
2e04d513a4 fix:unregister push manager if delete subscribe to avoid tqProcessPollPush 2023-08-28 16:41:28 +08:00
Minglei Jin
9ef455a454 enh(s3/evict): evict cache data files for other migrated head, stt, etc. 2023-08-28 15:57:13 +08:00
Haojun Liao
05bb164675 fix(stream): release tasks. 2023-08-28 15:56:02 +08:00
Haojun Liao
391103bc5a fix(stream): do dome internal refactor. 2023-08-28 15:54:01 +08:00
Haojun Liao
33672f2f35 fix(stream): release stream tasks. 2023-08-28 15:52:54 +08:00
wade zhang
cd3702e91c
Merge pull request #22596 from taosdata/fix/TD-25963
fix(tsdb/s3): fix scan and fix with s3 files
2023-08-28 15:19:47 +08:00
Minglei Jin
f299ae9bc3 fix(tsdb/s3): fix scan and fix with s3 files 2023-08-28 12:22:53 +08:00
Haojun Liao
da0b4e1f92 Merge branch '3.0' into enh/triggerCheckPoint2 2023-08-28 10:55:38 +08:00
dapan1121
0cce255a02 Merge remote-tracking branch 'origin/3.0' into fix/m23.0 2023-08-28 10:06:53 +08:00
Haojun Liao
b75ae217f6 fix(stream): notify close for snode. 2023-08-28 09:53:08 +08:00
Haojun Liao
b11975f00c
Merge pull request #22534 from taosdata/mark/tmq
fix:[TD-258900]modify tmq trans conflict to db level
2023-08-28 09:45:47 +08:00
dapan1121
1f40da6fef Merge remote-tracking branch 'origin/main' into 3.0 2023-08-28 09:28:55 +08:00
dmchen
06083e1eb9 init 2023-08-28 09:15:51 +08:00
dapan1121
cda08087b9
Merge pull request #22586 from taosdata/fix/TD-25936.3
fix: no explain response issue
2023-08-28 08:52:13 +08:00
Haojun Liao
6b327886e9 fix(stream): add null ptr check. 2023-08-27 00:27:04 +08:00
Haojun Liao
d8a6ec024a Merge remote-tracking branch 'origin/enh/triggerCheckPoint2' into enh/triggerCheckPoint2 2023-08-25 19:21:00 +08:00
Haojun Liao
7ceb8dc627 fix(stream): disable the follower to execute the stop/resume instruction. 2023-08-25 19:20:49 +08:00
yihaoDeng
22b8d90fbc fix invalid read 2023-08-25 18:34:11 +08:00
wangmm0220
047707fa17 fix:error in privilege 2023-08-25 17:56:17 +08:00
Haojun Liao
43a3f9f535 Merge remote-tracking branch 'origin/enh/triggerCheckPoint2' into enh/triggerCheckPoint2 2023-08-25 17:32:50 +08:00
Haojun Liao
e013ba6795 fix(stream): fix the counter. 2023-08-25 17:32:34 +08:00
Minglei Jin
f0fdbaa3f3 fix(tsdb/close): fix uaf with file closing 2023-08-25 17:27:32 +08:00
Haojun Liao
60d1df0e25
Merge branch '3.0' into enh/triggerCheckPoint2 2023-08-25 15:49:28 +08:00
Haojun Liao
ab2aad2b2c fix(stream): update the task status even it is in stop status. 2023-08-25 15:32:55 +08:00
dapan1121
e24498b0fd enh: add task notify 2023-08-25 14:59:34 +08:00
Hongze Cheng
e1095e7d02 fix: improve upgrade speed 2023-08-25 14:40:29 +08:00
Haojun Liao
4c7ad54c9c fix(stream): not restart tasks when in restoring procedure. 2023-08-25 11:28:28 +08:00
Haojun Liao
38bb123701 fix(stream): release task. 2023-08-25 10:33:15 +08:00
Haojun Liao
ce1a3a379c fix(stream): handle the case when the task not in ready state, and the checkpoint msg arrived. 2023-08-25 10:12:12 +08:00
Haojun Liao
6f514b2bee
Merge pull request #22561 from taosdata/fix/TD-25651
fix:[TD-25651] reset epoch if consumer changed to avoid consumeing no…
2023-08-25 09:09:41 +08:00
wade zhang
abe97e1e21
Merge pull request #22566 from taosdata/fix/TD-25952
enh(tsdb/openfile): lazy file open to read/write
2023-08-25 07:57:52 +08:00
Haojun Liao
f7ea875af1 fix(stream): add some logs. 2023-08-24 23:27:48 +08:00
Haojun Liao
f7dada53b9
Merge pull request #22559 from taosdata/fix/liaohj
other: merge main.
2023-08-24 19:01:18 +08:00
Haojun Liao
82e356a9b5 Merge remote-tracking branch 'origin/enh/triggerCheckPoint2' into enh/triggerCheckPoint2
# Conflicts:
#	source/dnode/vnode/src/tq/tq.c
2023-08-24 18:04:44 +08:00
Haojun Liao
70396aa5a5 fix(stream): wakeup the hb timer every 200ms, to speeup the close db procedure. 2023-08-24 17:54:50 +08:00
yihaoDeng
286ac3e3fc fix merge error 2023-08-24 17:35:53 +08:00
yihaoDeng
9310d138a6 Merge branch 'enh/3.0' into enh/triggerCheckPoint2 2023-08-24 17:30:35 +08:00
yihaoDeng
8883d66227 Merge branch 'enh/triggerCheckPoint2' of https://github.com/taosdata/TDengine into enh/triggerCheckPoint2 2023-08-24 17:20:12 +08:00
yihaoDeng
090515c684 refactor code 2023-08-24 17:20:04 +08:00
Haojun Liao
caf20d7b5d fix(stream): release the stream task. 2023-08-24 17:16:34 +08:00
Minglei Jin
70871139bf enh(tsdb/openfile): lazy file open to read/write 2023-08-24 17:16:28 +08:00
wangmm0220
b83cc11043 fix:[TD-25651] reset epoch if consumer changed to avoid consumeing no data 2023-08-24 16:38:53 +08:00
wangmm0220
49216ffa54 fix:transaction in tmq 2023-08-24 16:32:15 +08:00
Haojun Liao
7d9d4ae850 fix(stream): fix the ptr ref. 2023-08-24 16:26:30 +08:00
Haojun Liao
5344efe181 fix(stream): adjust the ptr. 2023-08-24 16:22:24 +08:00
Haojun Liao
4c6bc4d2c3 fix(stream): continue check wal when meeting empty delete block msg. 2023-08-24 16:10:03 +08:00
Haojun Liao
523bad3951 fix(stream): continue check wal when meeting empty delete block msg. 2023-08-24 15:45:43 +08:00
Haojun Liao
2c9f28d97f Merge branch 'main' into fix/liaohj 2023-08-24 14:12:15 +08:00
wangjiaming0909
7fad1de841 Revert "feat: show create table, desc table with col comments"
This reverts commit 16754b5ac1.
2023-08-24 13:38:41 +08:00
wade zhang
0860670d37
Merge pull request #22526 from taosdata/enh/TD-25445-new
enh/TD-25445: ttl split flush and drop table
2023-08-24 13:57:07 +08:00
shenglian-zhou
e76e6bbb84
Update tsdbReadUtil.c 2023-08-24 11:34:56 +08:00
shenglian-zhou
1965703d53
Merge branch '3.0' into szhou/tsdb-neigh-block 2023-08-24 10:58:23 +08:00
yihaoDeng
3afea998ac refactor log 2023-08-24 10:15:15 +08:00
Haojun Liao
230d8c2d80 fix(stream): fix error in stream computing after vnode is transferred to other dnodes. 2023-08-23 17:50:44 +08:00
Haojun Liao
b0c5fa8290 fix(stream): reset the task status. 2023-08-23 14:10:41 +08:00
Hongze Cheng
a601cd21bd fix: alter stt_trigger bug 2023-08-23 13:31:19 +08:00
Shungang Li
458fd07976 enh: ttl split flush and drop table
config:
1. ttlFlushThreshold: maximum number of dirty items in memory
2. tsTtlBatchDropNum: number of tables dropped per batch
3. tsTrimVDbIntervalSec: interval of trimming db in all vgroups
2023-08-23 11:13:34 +08:00
wade zhang
8c4062c1b0
Merge pull request #22515 from taosdata/fix/TD-25819-2
fix(tsdb/file): save errno for shell reporting
2023-08-23 10:49:48 +08:00
wade zhang
e069d72229
Merge pull request #22513 from taosdata/FIX/TD-25864-3.0
enh: adjust the order of pre-close actions in vnodePreClose
2023-08-22 17:27:17 +08:00
Haojun Liao
af80d94a88 refactor: do some internal refactor. 2023-08-22 15:43:50 +08:00
Minglei Jin
345743aba5 fix(tsdb/file): save errno for shell reporting 2023-08-22 14:44:32 +08:00
Haojun Liao
cc8b2263cb fix(stream): fix error in heartbeat from vnode. 2023-08-22 13:37:38 +08:00
Benguang Zhao
8be670aee8 enh: place vnodeSyncPreStop ahead of vnodeQueryPreStop in vnodePreClose 2023-08-22 11:30:40 +08:00
Haojun Liao
64556c9f1d Merge remote-tracking branch 'origin/enh/triggerCheckPoint2' into enh/triggerCheckPoint2 2023-08-22 10:47:02 +08:00
Haojun Liao
def0877980 fix(stream): add timer to check stream task status. 2023-08-22 10:46:49 +08:00
dm chen
b899a8b7df
Merge branch '3.0' into feat/TD-22970 2023-08-22 10:12:30 +08:00
yihaoDeng
1531e2d7d3 Merge branch 'enh/3.0' into enh/triggerCheckPoint2 2023-08-22 09:51:38 +08:00
Haojun Liao
873c22abb1 fix(stream): reset update stream tasks. 2023-08-21 19:32:27 +08:00
Haojun Liao
8b27f98ee4 refactor: do some internal refactor. 2023-08-21 19:08:27 +08:00
yihaoDeng
387b4d365e refactor checkpoint 2023-08-21 19:07:15 +08:00
Hongze Cheng
b8d62a86ed feat: clean useless file 2023-08-21 16:29:48 +08:00
Haojun Liao
29bb5854cc fix(stream): fix error in check stream state. 2023-08-21 11:48:55 +08:00
dm chen
dda51d8019
Merge branch '3.0' into feat/TD-22970 2023-08-21 10:50:45 +08:00
wade zhang
54e1f69db4
Merge pull request #22481 from taosdata/fix/TD-25822
enh(tsdb/write): timestamp out of range with s3's last tier storage
2023-08-21 09:33:06 +08:00
yihaoDeng
6fe649c458 Merge branch 'enh/3.0' into enh/triggerCheckPoint2 2023-08-21 09:12:40 +08:00
Haojun Liao
b8101afbe2 fix(stream): refactor the tasks restart mechanism. 2023-08-18 22:36:38 +08:00
Minglei Jin
c4a3a5da35 vnode: fix write to s3 last tier 2023-08-18 18:02:53 +08:00
wade zhang
a7e11c2520
Merge pull request #22482 from taosdata/fix/TD-25818
fix(vnode/destroy): delete objects
2023-08-18 17:49:22 +08:00
wade zhang
87a8fe34ef
Merge pull request #22480 from taosdata/fix/TD-25819
fix(tsdb/reader): use ENOENT for not found objects
2023-08-18 16:49:54 +08:00
Minglei Jin
7b4bb72392 fix(vnode/destroy): delete objects 2023-08-18 16:38:26 +08:00
Minglei Jin
0242d4ce33 enh(tsdb/write): timestamp out of range with s3's last tier storage 2023-08-18 16:07:15 +08:00
Haojun Liao
c26d625152
Merge branch '3.0' into enh/triggerCheckPoint2 2023-08-18 15:51:07 +08:00
Minglei Jin
73ae493fa7 tsdb/reader: use ENOENT for not found objects 2023-08-18 15:43:08 +08:00
liuyao
9afd5155ed Merge branch 'enh/triggerCheckPoint2' of https://github.com/taosdata/TDengine into enh/triggerCheckPoint2 2023-08-18 15:33:37 +08:00
liuyao
87c65f3b68 Merge remote-tracking branch 'origin/3.0' into enh/triggerCheckPoint2 2023-08-18 15:27:02 +08:00
dapan1121
9b953707b8 Merge remote-tracking branch 'origin/main' into fix/mainto3.0 2023-08-18 14:42:43 +08:00
Haojun Liao
edbad45772 other: add logs. 2023-08-18 13:45:08 +08:00
dm chen
e393a98ee9
Merge branch '3.0' into feat/TD-22970 2023-08-18 13:37:18 +08:00
Haojun Liao
a8b53c2e69 other: add logs. 2023-08-18 10:51:59 +08:00
Haojun Liao
633ecf66e5 other: add logs. 2023-08-18 10:39:32 +08:00
Haojun Liao
9ad8cf48dc other: add logs. 2023-08-18 10:38:24 +08:00
Haojun Liao
87785b1d9c refactor: add some logs. 2023-08-18 10:30:00 +08:00
slzhou
4e2a26e8be Merge branch '3.0' of github.com:taosdata/TDengine into szhou/select-tags-scan 2023-08-18 08:46:17 +08:00
Hongze Cheng
8582e0c71a fix: important fix for compact file corruption 2023-08-17 18:13:37 +08:00
dmchen
5f138941f8 tsdb snap core 2023-08-17 17:58:35 +08:00
slzhou
85d1dbe00a Merge branch '3.0' of github.com:taosdata/TDengine into szhou/select-tags-scan 2023-08-17 16:27:37 +08:00
Benguang Zhao
7735e6f388 Merge branch 3.0 to FEAT/TD-25699-3.0 2023-08-17 16:04:23 +08:00