Commit graph

7411 commits

Author SHA1 Message Date
stephenkgu
8d6f129154 fix: skip empty data in delete file 2022-11-10 08:14:21 +08:00
Shengliang Guan
b067079d7a
Merge pull request #18007 from taosdata/feature/3_liaohj
refactor: do some internal refactor.
2022-11-10 08:12:36 +08:00
Shengliang Guan
b882e45505
Merge pull request #17988 from taosdata/perf/default_buffer_size
perf: optimize write by changing default buffer size
2022-11-09 21:45:26 +08:00
Shengliang Guan
b595c07149 fix: wait until db created while process use db req 2022-11-09 19:47:22 +08:00
Cary Xu
14dc1d3f84 Merge branch '3.0' into feat/TS-2007 2022-11-09 19:42:41 +08:00
Shengliang Guan
808ee3775d fix: wait until db created while process use db req 2022-11-09 19:42:30 +08:00
Haojun Liao
015f1a1df8 other: merge 3.0 2022-11-09 19:17:45 +08:00
Hongze Cheng
0c92bd6e37 make pass ci 2022-11-09 18:58:35 +08:00
dapan1121
2f807a85d0 Merge remote-tracking branch 'origin/3.0' into enh/TD-20288 2022-11-09 18:52:00 +08:00
dapan1121
9f7e06721c enh: support vgroup epset update 2022-11-09 18:48:52 +08:00
Cary Xu
25148268e6 enh: support wider range for future timestamp 2022-11-09 18:10:30 +08:00
Shengliang Guan
35bacc2f96
Merge pull request #17995 from taosdata/feature/3.0_mhli
refactor(sync): add FpApplyQueueItems in fsm
2022-11-09 17:40:28 +08:00
Minghao Li
d672e7a05c refactor(sync): add FpApplyQueueItems in fsm 2022-11-09 16:18:28 +08:00
Hongze Cheng
86ed5e5674 more optimize 2022-11-09 16:02:04 +08:00
Shengliang Guan
7f03194299
Merge pull request #17991 from taosdata/feature/stream
enh(tmq): auto clear lost consumer
2022-11-09 15:57:08 +08:00
Liu Jicong
4000176dc7 enh(tmq): auto clear lost consumer 2022-11-09 14:44:09 +08:00
Haojun Liao
411371ba48 refactor: do some internal refactor. 2022-11-09 14:16:21 +08:00
Haojun Liao
b48752686e refactor: do some internal refactor. 2022-11-09 13:45:46 +08:00
Haojun Liao
0c427b5f5b refactor: do some internal refactor. 2022-11-09 10:39:32 +08:00
Haojun Liao
68f2f92116 fix(query): fix the syntax errors. 2022-11-09 09:07:21 +08:00
Haojun Liao
d238f387ce other: merge 3.0 2022-11-09 00:00:14 +08:00
Shengliang Guan
9a5e82267e Merge branch '3.0' into fix/TD-20286 2022-11-08 19:37:01 +08:00
Shengliang Guan
ecd75c0d2c
Merge pull request #17967 from taosdata/fix/TD-20264
enh: check vgroup status changes at regular intervals
2022-11-08 19:34:35 +08:00
dapan1121
718ccf5dfa Merge remote-tracking branch 'origin/fix/TD-20264' into enh/TD-20288 2022-11-08 19:11:01 +08:00
Shengliang Guan
0183fe6e73 fix: crash if db not exist while process status msg 2022-11-08 18:36:24 +08:00
Haojun Liao
663138ba1e enh(query): optimize the query perf. 2022-11-08 18:33:57 +08:00
Shengliang Guan
6c0f46ff03 fix: the vnode was created to the wrong dnode when network was misconfigured 2022-11-08 18:10:26 +08:00
dapan1121
d5874419ec
Merge pull request #17935 from taosdata/fix/TD-20246
fix: fix tsdbRead crash caused by reuse after free
2022-11-08 17:48:25 +08:00
Haojun Liao
cc3e97da36 fix(query): fix an typo. 2022-11-08 17:41:29 +08:00
Haojun Liao
6e1a2b1e54 enh(query): improve the perf. 2022-11-08 17:38:09 +08:00
Shengliang Guan
5a5d2c448b enh: process some msg in mnode read queue 2022-11-08 17:26:31 +08:00
Shengliang Guan
1dbb086045 enh: check vgroup status changes at regular intervals 2022-11-08 17:17:24 +08:00
Haojun Liao
f85d66f5ee enh(query): opt query perf. 2022-11-08 16:59:00 +08:00
Shengliang Guan
8f0ecd46cd
Merge pull request #17961 from taosdata/feature/stream
enh(tmq): offset commit only when needed
2022-11-08 16:08:10 +08:00
Shengliang Guan
920fc400a0
Merge pull request #17964 from taosdata/fix/TD-20264
fix: add log for use db rsp
2022-11-08 16:07:55 +08:00
Haojun Liao
b0a54d3fab enh(query): improve the perf. 2022-11-08 14:54:42 +08:00
Liu Jicong
118e5bf5d3 enh(tmq): offset commit only when needed 2022-11-08 14:46:34 +08:00
Hongze Cheng
7bba9996cf perf: optimize insert 2022-11-08 14:36:18 +08:00
Shengliang Guan
7a5be43d14
Merge pull request #17954 from taosdata/feature/stream
fix(stream): delete tb should be checked in write thread
2022-11-08 13:50:15 +08:00
Shengliang Guan
fb7d45303b fix: add log for use db rsp 2022-11-08 13:46:18 +08:00
Shengliang Guan
75d7ae5a9d
Merge pull request #17953 from taosdata/enh/TD-20263
enh: optimize the error code when create and drop a db in dropping status
2022-11-08 13:24:48 +08:00
Shengliang Guan
abd95562a0 enh: optimize the error code when create and delete a database in a dropping state 2022-11-08 12:36:28 +08:00
Liu Jicong
bc469f7f7b fix(stream): delete tb should be checked in write thread 2022-11-08 11:30:29 +08:00
Shengliang Guan
30058f7619 enh: optimize the error code when create and delete a database in a dropping state 2022-11-08 10:46:32 +08:00
dapan1121
ba570b2bb7
Merge pull request #17938 from taosdata/fix/preprocessQuery
fix: fix error handling for query msg preprocessing
2022-11-08 10:13:14 +08:00
Shengliang Guan
dfbb958128
Merge pull request #17929 from taosdata/perf/sma_optimize
perf: optimize sma calculation
2022-11-08 10:03:12 +08:00
Shengliang Guan
1f91eef038
Merge pull request #17932 from taosdata/fix/tdb-prep-async-commit
enh(tdb): new interface of meta prep async commit
2022-11-08 09:43:36 +08:00
Minglei Jin
1a4cf6964b fix: add post commit to tq & stream 2022-11-07 20:34:23 +08:00
Shengliang Guan
e04c821292
Merge pull request #17933 from taosdata/perf/block_data_optimize
perf: optimize block data operation
2022-11-07 19:32:23 +08:00
dapan1121
f7fa81ee2c fix: fix error handling for query msg preprocessing 2022-11-07 18:51:13 +08:00
Shengliang Guan
b0f469adb1
Merge pull request #17931 from taosdata/feature/3.0_mhli
fix(sync): when apply queue not empty, can not read
2022-11-07 18:29:53 +08:00
dapan1121
3e3354782c
Merge pull request #17930 from taosdata/enh/showWrite
enh: support grant check in vnode query
2022-11-07 17:42:15 +08:00
Ganlin Zhao
d590a38d62 fix: fix tsdbRead crash caused by reuse after free
TD-20246
2022-11-07 17:36:58 +08:00
Hongze Cheng
d52b8966fc perf: optimize block data operation 2022-11-07 16:56:35 +08:00
Minglei Jin
c17f8244be enh(tdb): new interface of meta prep async commit 2022-11-07 16:45:29 +08:00
Minghao Li
b6aea6ae17 fix(sync): when apply queue not empty, can not read 2022-11-07 16:40:53 +08:00
dapan1121
5ab6166efe enh: support grant check in vnode query 2022-11-07 16:23:40 +08:00
Hongze Cheng
70f7e3e379 perf: optimize sma calculation 2022-11-07 15:42:53 +08:00
Haojun Liao
be5b72b25d enh(query): add cache for table meta entry in table scan. 2022-11-07 10:17:42 +08:00
Haojun Liao
e3d0bc6bf7 other: merge 3.0. 2022-11-06 11:37:21 +08:00
Haojun Liao
d200fe5145 refactor: do some internal refactor. 2022-11-06 01:34:04 +08:00
Shengliang Guan
adb335467c fix: coverity issues 2022-11-05 19:59:18 +08:00
dapan1121
7081ebe9e6
Merge pull request #17877 from taosdata/enh/TD-19920
enh: add show rewrite mask in query msg
2022-11-05 15:35:51 +08:00
Shengliang Guan
1c892e3e3d
Merge pull request #17894 from taosdata/fix/TD-20161
enh: adjust tqueue and tworker log
2022-11-04 19:11:13 +08:00
Haojun Liao
03f4918b3b refactor: do some internal refactor. 2022-11-04 18:46:48 +08:00
Shengliang Guan
8ce2e12423
Merge pull request #17886 from taosdata/feature/3_liaohj
fix(query): set the last key of each tablescan info
2022-11-04 18:46:17 +08:00
Shengliang Guan
2e6f75107d enh: adjust the number of vnode threads so that one vnode has one write thread 2022-11-04 17:38:35 +08:00
Shengliang Guan
4fc0e3bd55 enh: adjust the number of vnode threads so that one vnode has one write thread 2022-11-04 17:21:00 +08:00
Shengliang Guan
fc41f43c9d fix: deadlock while drop db 2022-11-04 16:15:37 +08:00
Shengliang Guan
8886785be1
Merge pull request #17853 from taosdata/fix/liao_cov
refactor(query): do some internal  refactor.
2022-11-04 15:58:11 +08:00
Shengliang Guan
a0983cf8c1 enh: adjust tqueue and tworker log 2022-11-04 10:58:23 +08:00
dapan1121
65636154ea fix: fix msg head issue 2022-11-04 10:47:49 +08:00
dapan1121
9ce099a2cf Merge remote-tracking branch 'origin/3.0' into enh/TD-19920 2022-11-04 08:51:43 +08:00
Haojun Liao
0315414623 other: merge 3.0 2022-11-03 19:03:09 +08:00
Haojun Liao
4959529127 fix(query): set the last key of each tablescan info 2022-11-03 19:00:13 +08:00
Shengliang Guan
0a45e80613
Merge pull request #17875 from taosdata/fix/TD-20125-2
fix: reset last load info when close dataf reader for stt last files
2022-11-03 17:05:17 +08:00
dapan1121
ae5f6ef175 enh: add show rewrite mask in query msg 2022-11-03 16:28:48 +08:00
Shengliang Guan
33d64b67ff
Merge pull request #17865 from taosdata/feature/stream
refactor(stream): stream deploy and state transfer
2022-11-03 15:24:39 +08:00
Shengliang Guan
c392abb0ad
Merge pull request #17869 from taosdata/feature/3.0_mhli
refactor(sync): add syncIsReadyForRead
2022-11-03 15:23:34 +08:00
Minglei Jin
93190003a3 fix: reset last load info when close dataf reader for stt last files 2022-11-03 15:19:21 +08:00
Liu Jicong
31f574359d fix:log empty msg 2022-11-03 14:29:51 +08:00
Minghao Li
a4b54c4f0e refactor(sync): add syncIsReadyForRead 2022-11-03 13:57:57 +08:00
Shengliang Guan
6cef3fa4fa
Merge pull request #17856 from taosdata/feature/3.0_mhli
enh(sync): add sync pre stop, local-cmd:follower-commit
2022-11-03 13:40:35 +08:00
Liu Jicong
15ad147c40 Merge branch '3.0' into feature/stream 2022-11-03 13:37:59 +08:00
Liu Jicong
2fbc8e6306 refactor(stream): stream deploy and state transfer 2022-11-03 13:22:23 +08:00
Shengliang Guan
df53e7591d
Merge pull request #17857 from taosdata/fix/TD-20095
refact: add sequence for status msg
2022-11-03 12:23:27 +08:00
Shengliang Guan
e61829856a
Merge pull request #17852 from taosdata/fix/TD-20125
fix: set fs and last dataf reader to null when resetting
2022-11-03 11:31:34 +08:00
Shengliang Guan
843e17744e refact: add sequence for status msg 2022-11-03 10:57:20 +08:00
Haojun Liao
7a14bc05ea fix(query): fix some issues about the initialization of SColumnInfoData. 2022-11-03 10:57:05 +08:00
Shengliang Guan
0b69a23220 Merge remote-tracking branch 'origin/3.0' into fix/TD-20095 2022-11-03 10:42:57 +08:00
Shengliang Guan
108b9ffed6 refact: add sequence for status msg 2022-11-03 10:42:33 +08:00
dapan1121
2bf5b0405d
Merge pull request #17851 from taosdata/enh/TD-19956
enh: optimize submit response message
2022-11-03 10:19:39 +08:00
Haojun Liao
9f5a0f01ea Merge branch '3.0' into fix/liao_cov 2022-11-03 10:02:23 +08:00
Haojun Liao
11c4878ca1 refactor(query): do some internal refactor. 2022-11-03 10:01:11 +08:00
Minghao Li
1a4b7622e4 enh(sync): add sync pre stop 2022-11-03 09:39:20 +08:00
Shengliang Guan
9de4bfb230
Merge pull request #17834 from taosdata/fix/liao_cov
fix(query): update the lastkey for each table during table scanning.
2022-11-03 09:29:55 +08:00
Shengliang Guan
3a4101f724
Merge pull request #17849 from taosdata/fix/TD-20095
fix: status access time error
2022-11-03 09:29:22 +08:00
Minglei Jin
0bbe049e58 fix: set fs and last dataf reader to null when resetting 2022-11-03 09:14:51 +08:00
Haojun Liao
fc20be8699 fix(query): get if the correct rows in last block exists, and set the correct flag to denote if the last blocks index have been loaded or not. 2022-11-02 22:27:23 +08:00
Shengliang Guan
0d60817268
Merge pull request #17837 from taosdata/feature/3.0_mhli
refactor(sync): add pre snapshot
2022-11-02 21:11:22 +08:00
Shengliang Guan
b98e3f343d fix: status access time error 2022-11-02 18:57:38 +08:00
Shengliang Guan
1bae78464c
Merge pull request #17842 from taosdata/fix/TD-20128
fix: failed to read sdb while udf larger then 10M
2022-11-02 18:37:59 +08:00
dapan1121
83fc35c8ee
Merge pull request #17841 from taosdata/fix/TS-1965-new
fix: fix reverse table scan when checking if block from file overlap with skyline deletion result error
2022-11-02 17:56:09 +08:00
Shengliang Guan
5625ea9f2a fix: failed to read sdb while udf larger then 10M 2022-11-02 17:09:53 +08:00
Shengliang Guan
f2e1bd55d3 fix: failed to read sdb while udf larger then 10M 2022-11-02 17:08:15 +08:00
Ganlin Zhao
215257f7f1 fix: fix reverse table scan when checking if block from file overlap
with skyline deletion result error
2022-11-02 16:58:36 +08:00
Minghao Li
70aff8451f refactor(sync): add pre snapshot 2022-11-02 15:38:30 +08:00
Shengliang Guan
b5f0041af1 refact: adjust error code in vnode 2022-11-02 15:28:32 +08:00
dapan1121
ecfcf7054c enh: optimize submit response msg 2022-11-02 15:18:32 +08:00
Haojun Liao
ecad94c419 fix(query): add one more check. 2022-11-02 14:59:34 +08:00
Haojun Liao
f83b863cf9 fix(query): update the lastkey for each table during table scanning. 2022-11-02 14:36:00 +08:00
Shengliang Guan
21748defea Merge remote-tracking branch 'origin/3.0' into fix/TD-20095 2022-11-02 11:49:34 +08:00
Shengliang Guan
b2e3c7800f
Merge pull request #17813 from taosdata/feature/3_liaohj
fix(query): optimize the performance of tsdbread.
2022-11-02 11:23:29 +08:00
Shengliang Guan
17e3c59aeb fix: add dnode status log 2022-11-02 11:21:13 +08:00
Shengliang Guan
1c77d5381a Merge remote-tracking branch 'origin/3.0' into fix/TD-20052 2022-11-02 10:26:19 +08:00
Shengliang Guan
7a537bba09 refact: adjust sync.h and syncState 2022-11-02 10:24:55 +08:00
Haojun Liao
c3ece576f2 Merge branch '3.0' into feature/3_liaohj 2022-11-02 09:13:13 +08:00
Shengliang Guan
15308ad140
Merge pull request #17820 from taosdata/fix/msg_compat
fix(vnode): ignore unhandled msg
2022-11-02 08:37:08 +08:00
Haojun Liao
698ae90e5b fix(query): set correct value in hash map reset. 2022-11-01 23:37:51 +08:00
Haojun Liao
b610b23981 fix(query): check for null value. 2022-11-01 21:47:12 +08:00
Liu Jicong
a0910c87f7 fix(vnode): ignore unhandled msg 2022-11-01 21:23:50 +08:00
Shengliang Guan
a2abb3a411
Merge pull request #17817 from taosdata/fix/msg_compat
fix: msg compatibility
2022-11-01 21:10:46 +08:00
Shengliang Guan
cec6642700
Merge pull request #17811 from taosdata/feature/stream
fix(tmq): handle adding subtable for subscribing stb
2022-11-01 21:09:43 +08:00
Shengliang Guan
fee43064f7
Merge pull request #17812 from taosdata/fix/TD-20052
enh: refact sync callback func
2022-11-01 21:09:00 +08:00
Liu Jicong
411373a7c5 fix: msg compatibility 2022-11-01 20:02:17 +08:00
Shengliang Guan
0578d034bc
Merge pull request #17810 from taosdata/fix/TD-20028
fix: remove last stt load info resetting from fs iterator
2022-11-01 19:36:12 +08:00
Liu Jicong
f0a66602da fix(taosx): set version 2022-11-01 19:28:42 +08:00
Shengliang Guan
871757240a
Merge pull request #17777 from taosdata/FIX/TS-1984-3.0
fix: fsync wal files and meta if data size not synced yet beyond a threshold
2022-11-01 18:53:10 +08:00
Haojun Liao
026b82145e Merge branch '3.0' into feature/3_liaohj 2022-11-01 18:49:51 +08:00
Haojun Liao
68db9b1cc9 fix(query): optimize the performance of tsdbread. 2022-11-01 18:48:57 +08:00
Shengliang Guan
df2c443a80
Merge pull request #17807 from taosdata/fix/TD-20097
fix: wrong cpu usage
2022-11-01 18:35:28 +08:00
Shengliang Guan
64633e3dbb enh: refact syncMsg code 2022-11-01 18:34:09 +08:00
Liu Jicong
60bb2c18dc Merge branch '3.0' into feature/stream 2022-11-01 18:19:15 +08:00
Liu Jicong
94e7c79ca0 fix(tmq): handle adding subtable for subscribing stb 2022-11-01 18:18:26 +08:00
Minglei Jin
a5052eb2d5 fix: remove last stt load info resetting from fs iterator 2022-11-01 17:29:18 +08:00
Shengliang Guan
ee63b49800 Merge remote-tracking branch 'origin/3.0' into fix/TD-20052 2022-11-01 17:11:27 +08:00
Shengliang Guan
d53ab62f00
Merge pull request #17797 from taosdata/fix/td_19618
fix: invalid read problem of tsdb path
2022-11-01 16:46:58 +08:00
Shengliang Guan
18f7d658d9
Merge pull request #17804 from taosdata/feature/3_liaohj
refactor: add some logs.
2022-11-01 16:46:24 +08:00
Shengliang Guan
a71dbd83d2
Merge pull request #17803 from taosdata/fix/TD-20052
enh: refact syncMsg code
2022-11-01 16:45:56 +08:00
Shengliang Guan
92b1c12330 fix: wrong cpu usage 2022-11-01 16:38:52 +08:00
Shengliang Guan
68f1641991
Merge pull request #17800 from taosdata/feature/stream
refactor(stream): fill history
2022-11-01 16:15:07 +08:00
Shengliang Guan
d35028ae4a enh: refact sync callback func 2022-11-01 16:10:07 +08:00
Liu Jicong
92804d317e Merge branch '3.0' into feature/stream 2022-11-01 15:47:37 +08:00
Shengliang Guan
f4dc7d0766 enh: refact sync callback func 2022-11-01 15:35:10 +08:00
Cary Xu
872085bef4 fix: add TD_PATH_MAX to support multi-platform 2022-11-01 15:18:26 +08:00
Haojun Liao
cff5d9295e refactor: add some logs. 2022-11-01 15:00:02 +08:00
Cary Xu
65943b3712 chore: revert the code change 2022-11-01 14:51:37 +08:00
Shengliang Guan
e79e50ae36 enh: refact syncMsg code 2022-11-01 13:58:25 +08:00
Haojun Liao
b78249c548 other: merge 3.0 2022-11-01 13:52:08 +08:00
Haojun Liao
83495ed5d3 fix(query): set the correct buffer to accommodate the tag values. 2022-11-01 13:49:02 +08:00