Commit graph

4457 commits

Author SHA1 Message Date
Minglei Jin
c17f8244be enh(tdb): new interface of meta prep async commit 2022-11-07 16:45:29 +08:00
Minghao Li
b6aea6ae17 fix(sync): when apply queue not empty, can not read 2022-11-07 16:40:53 +08:00
dapan1121
5ab6166efe enh: support grant check in vnode query 2022-11-07 16:23:40 +08:00
Hongze Cheng
70f7e3e379 perf: optimize sma calculation 2022-11-07 15:42:53 +08:00
Haojun Liao
be5b72b25d enh(query): add cache for table meta entry in table scan. 2022-11-07 10:17:42 +08:00
Haojun Liao
e3d0bc6bf7 other: merge 3.0. 2022-11-06 11:37:21 +08:00
Haojun Liao
d200fe5145 refactor: do some internal refactor. 2022-11-06 01:34:04 +08:00
Shengliang Guan
1c892e3e3d
Merge pull request #17894 from taosdata/fix/TD-20161
enh: adjust tqueue and tworker log
2022-11-04 19:11:13 +08:00
Haojun Liao
03f4918b3b refactor: do some internal refactor. 2022-11-04 18:46:48 +08:00
Shengliang Guan
8ce2e12423
Merge pull request #17886 from taosdata/feature/3_liaohj
fix(query): set the last key of each tablescan info
2022-11-04 18:46:17 +08:00
Shengliang Guan
2e6f75107d enh: adjust the number of vnode threads so that one vnode has one write thread 2022-11-04 17:38:35 +08:00
Shengliang Guan
fc41f43c9d fix: deadlock while drop db 2022-11-04 16:15:37 +08:00
Shengliang Guan
8886785be1
Merge pull request #17853 from taosdata/fix/liao_cov
refactor(query): do some internal  refactor.
2022-11-04 15:58:11 +08:00
Shengliang Guan
a0983cf8c1 enh: adjust tqueue and tworker log 2022-11-04 10:58:23 +08:00
Haojun Liao
0315414623 other: merge 3.0 2022-11-03 19:03:09 +08:00
Haojun Liao
4959529127 fix(query): set the last key of each tablescan info 2022-11-03 19:00:13 +08:00
Shengliang Guan
0a45e80613
Merge pull request #17875 from taosdata/fix/TD-20125-2
fix: reset last load info when close dataf reader for stt last files
2022-11-03 17:05:17 +08:00
Shengliang Guan
33d64b67ff
Merge pull request #17865 from taosdata/feature/stream
refactor(stream): stream deploy and state transfer
2022-11-03 15:24:39 +08:00
Shengliang Guan
c392abb0ad
Merge pull request #17869 from taosdata/feature/3.0_mhli
refactor(sync): add syncIsReadyForRead
2022-11-03 15:23:34 +08:00
Minglei Jin
93190003a3 fix: reset last load info when close dataf reader for stt last files 2022-11-03 15:19:21 +08:00
Minghao Li
a4b54c4f0e refactor(sync): add syncIsReadyForRead 2022-11-03 13:57:57 +08:00
Shengliang Guan
6cef3fa4fa
Merge pull request #17856 from taosdata/feature/3.0_mhli
enh(sync): add sync pre stop, local-cmd:follower-commit
2022-11-03 13:40:35 +08:00
Liu Jicong
15ad147c40 Merge branch '3.0' into feature/stream 2022-11-03 13:37:59 +08:00
Liu Jicong
2fbc8e6306 refactor(stream): stream deploy and state transfer 2022-11-03 13:22:23 +08:00
Shengliang Guan
e61829856a
Merge pull request #17852 from taosdata/fix/TD-20125
fix: set fs and last dataf reader to null when resetting
2022-11-03 11:31:34 +08:00
Haojun Liao
7a14bc05ea fix(query): fix some issues about the initialization of SColumnInfoData. 2022-11-03 10:57:05 +08:00
dapan1121
2bf5b0405d
Merge pull request #17851 from taosdata/enh/TD-19956
enh: optimize submit response message
2022-11-03 10:19:39 +08:00
Haojun Liao
9f5a0f01ea Merge branch '3.0' into fix/liao_cov 2022-11-03 10:02:23 +08:00
Haojun Liao
11c4878ca1 refactor(query): do some internal refactor. 2022-11-03 10:01:11 +08:00
Minghao Li
1a4b7622e4 enh(sync): add sync pre stop 2022-11-03 09:39:20 +08:00
Shengliang Guan
9de4bfb230
Merge pull request #17834 from taosdata/fix/liao_cov
fix(query): update the lastkey for each table during table scanning.
2022-11-03 09:29:55 +08:00
Minglei Jin
0bbe049e58 fix: set fs and last dataf reader to null when resetting 2022-11-03 09:14:51 +08:00
Haojun Liao
fc20be8699 fix(query): get if the correct rows in last block exists, and set the correct flag to denote if the last blocks index have been loaded or not. 2022-11-02 22:27:23 +08:00
dapan1121
83fc35c8ee
Merge pull request #17841 from taosdata/fix/TS-1965-new
fix: fix reverse table scan when checking if block from file overlap with skyline deletion result error
2022-11-02 17:56:09 +08:00
Ganlin Zhao
215257f7f1 fix: fix reverse table scan when checking if block from file overlap
with skyline deletion result error
2022-11-02 16:58:36 +08:00
Shengliang Guan
b5f0041af1 refact: adjust error code in vnode 2022-11-02 15:28:32 +08:00
dapan1121
ecfcf7054c enh: optimize submit response msg 2022-11-02 15:18:32 +08:00
Haojun Liao
ecad94c419 fix(query): add one more check. 2022-11-02 14:59:34 +08:00
Haojun Liao
f83b863cf9 fix(query): update the lastkey for each table during table scanning. 2022-11-02 14:36:00 +08:00
Shengliang Guan
b2e3c7800f
Merge pull request #17813 from taosdata/feature/3_liaohj
fix(query): optimize the performance of tsdbread.
2022-11-02 11:23:29 +08:00
Shengliang Guan
1c77d5381a Merge remote-tracking branch 'origin/3.0' into fix/TD-20052 2022-11-02 10:26:19 +08:00
Shengliang Guan
7a537bba09 refact: adjust sync.h and syncState 2022-11-02 10:24:55 +08:00
Haojun Liao
c3ece576f2 Merge branch '3.0' into feature/3_liaohj 2022-11-02 09:13:13 +08:00
Shengliang Guan
15308ad140
Merge pull request #17820 from taosdata/fix/msg_compat
fix(vnode): ignore unhandled msg
2022-11-02 08:37:08 +08:00
Haojun Liao
698ae90e5b fix(query): set correct value in hash map reset. 2022-11-01 23:37:51 +08:00
Haojun Liao
b610b23981 fix(query): check for null value. 2022-11-01 21:47:12 +08:00
Liu Jicong
a0910c87f7 fix(vnode): ignore unhandled msg 2022-11-01 21:23:50 +08:00
Shengliang Guan
cec6642700
Merge pull request #17811 from taosdata/feature/stream
fix(tmq): handle adding subtable for subscribing stb
2022-11-01 21:09:43 +08:00
Shengliang Guan
fee43064f7
Merge pull request #17812 from taosdata/fix/TD-20052
enh: refact sync callback func
2022-11-01 21:09:00 +08:00
Shengliang Guan
0578d034bc
Merge pull request #17810 from taosdata/fix/TD-20028
fix: remove last stt load info resetting from fs iterator
2022-11-01 19:36:12 +08:00
Liu Jicong
f0a66602da fix(taosx): set version 2022-11-01 19:28:42 +08:00
Shengliang Guan
871757240a
Merge pull request #17777 from taosdata/FIX/TS-1984-3.0
fix: fsync wal files and meta if data size not synced yet beyond a threshold
2022-11-01 18:53:10 +08:00
Haojun Liao
026b82145e Merge branch '3.0' into feature/3_liaohj 2022-11-01 18:49:51 +08:00
Haojun Liao
68db9b1cc9 fix(query): optimize the performance of tsdbread. 2022-11-01 18:48:57 +08:00
Shengliang Guan
64633e3dbb enh: refact syncMsg code 2022-11-01 18:34:09 +08:00
Liu Jicong
60bb2c18dc Merge branch '3.0' into feature/stream 2022-11-01 18:19:15 +08:00
Liu Jicong
94e7c79ca0 fix(tmq): handle adding subtable for subscribing stb 2022-11-01 18:18:26 +08:00
Minglei Jin
a5052eb2d5 fix: remove last stt load info resetting from fs iterator 2022-11-01 17:29:18 +08:00
Shengliang Guan
ee63b49800 Merge remote-tracking branch 'origin/3.0' into fix/TD-20052 2022-11-01 17:11:27 +08:00
Shengliang Guan
d53ab62f00
Merge pull request #17797 from taosdata/fix/td_19618
fix: invalid read problem of tsdb path
2022-11-01 16:46:58 +08:00
Shengliang Guan
18f7d658d9
Merge pull request #17804 from taosdata/feature/3_liaohj
refactor: add some logs.
2022-11-01 16:46:24 +08:00
Shengliang Guan
a71dbd83d2
Merge pull request #17803 from taosdata/fix/TD-20052
enh: refact syncMsg code
2022-11-01 16:45:56 +08:00
Shengliang Guan
68f1641991
Merge pull request #17800 from taosdata/feature/stream
refactor(stream): fill history
2022-11-01 16:15:07 +08:00
Shengliang Guan
d35028ae4a enh: refact sync callback func 2022-11-01 16:10:07 +08:00
Liu Jicong
92804d317e Merge branch '3.0' into feature/stream 2022-11-01 15:47:37 +08:00
Shengliang Guan
f4dc7d0766 enh: refact sync callback func 2022-11-01 15:35:10 +08:00
Cary Xu
872085bef4 fix: add TD_PATH_MAX to support multi-platform 2022-11-01 15:18:26 +08:00
Haojun Liao
cff5d9295e refactor: add some logs. 2022-11-01 15:00:02 +08:00
Cary Xu
65943b3712 chore: revert the code change 2022-11-01 14:51:37 +08:00
Shengliang Guan
e79e50ae36 enh: refact syncMsg code 2022-11-01 13:58:25 +08:00
Haojun Liao
b78249c548 other: merge 3.0 2022-11-01 13:52:08 +08:00
Haojun Liao
83495ed5d3 fix(query): set the correct buffer to accommodate the tag values. 2022-11-01 13:49:02 +08:00
dapan1121
801d65cb9a
Merge pull request #17789 from taosdata/fix/TD-20027-D
fix(query): set search key according to order
2022-11-01 13:48:19 +08:00
Liu Jicong
8ca5f0bd9c Merge branch '3.0' into feature/stream 2022-11-01 13:20:20 +08:00
Liu Jicong
5e253cc8ea refactor(stream): fill history 2022-11-01 11:56:14 +08:00
Cary Xu
0b6d117121 fix: add TD_PATH_MAX to support multi-platform 2022-11-01 11:44:17 +08:00
Cary Xu
9f14c599f2 fix: add TD_PATH_MAX to support multi-platform 2022-11-01 10:49:27 +08:00
Shengliang Guan
1724aa5638
Merge pull request #17779 from taosdata/fix/sync
refactor(sync): check msgcb, putToQueueFp NULL
2022-11-01 10:43:56 +08:00
Cary Xu
64f07523e2 Merge branch '3.0' into fix/td_19618 2022-11-01 10:02:45 +08:00
Haojun Liao
e533794cb7 Merge branch '3.0' into fix/liao_cov 2022-11-01 09:08:46 +08:00
benguang.zhao
07db554fab enh: persist wal at the beginning of vnodeCommit 2022-11-01 00:00:18 +08:00
Shengliang Guan
9bc48e2658 Merge remote-tracking branch 'origin/3.0' into fix/TD-20052 2022-10-31 23:42:03 +08:00
Shengliang Guan
392564cc7d enh: refact syncEnv code 2022-10-31 23:40:43 +08:00
Shengliang Guan
1c446b1bf4 Merge branch '3.0' into fix/sync 2022-10-31 23:30:24 +08:00
Shengliang Guan
e817e649c6
Merge pull request #17788 from taosdata/enh/TD-19090
enh: force drop dnode
2022-10-31 23:27:54 +08:00
Shengliang Guan
26f1e64f2a enh: force drop dnode 2022-10-31 21:33:06 +08:00
Shengliang Guan
f32d347bb8
Merge pull request #17778 from taosdata/feature/3.0_mhli
refactor(sync): add sync local cmd, step down
2022-10-31 20:05:01 +08:00
Cary Xu
8a5f94d35c fix: set search key according to order 2022-10-31 20:03:21 +08:00
Shengliang Guan
1ad604a5f9 enh: force drop dnode 2022-10-31 19:57:27 +08:00
Liu Jicong
2e640c38a2 refactor: stream and tmq message 2022-10-31 18:37:28 +08:00
Minghao Li
d246514da2 refactor(sync): check msgcb, putToQueueFp NULL 2022-10-31 18:16:58 +08:00
Haojun Liao
f29127a15e ehn(query): do some internal refactor. 2022-10-31 18:04:58 +08:00
Haojun Liao
1fa5eafa8a ehn(query): keep the time window for each block to avoid repeatly parsing the compressed data block meta info. 2022-10-31 17:49:50 +08:00
Haojun Liao
fac516a196 refactor: do some internal refactor. 2022-10-31 17:20:27 +08:00
Haojun Liao
9703ab510b enh(query): avoid parse the mapdata to improve the performance by keep the offset of each block. 2022-10-31 17:19:15 +08:00
Haojun Liao
ba37a58192 enh(query): reduce the number of malloc. 2022-10-31 15:39:16 +08:00
Haojun Liao
f38e9664f9 other:merge 3.0 2022-10-31 15:32:48 +08:00
Haojun Liao
adf085ab46 refactor: do some internal refactor. 2022-10-31 14:17:57 +08:00
Minghao Li
0b20b914bb refactor(sync): add sync local cmd 2022-10-31 14:17:26 +08:00
Haojun Liao
c11ed48937 Merge branch 'fix/liao_cov' of github.com:taosdata/tdengine into fix/liao_cov 2022-10-31 14:00:31 +08:00
Haojun Liao
f790d91b45 fix(query): check null before free resource, and some internal refactor. 2022-10-31 10:41:25 +08:00
Haojun Liao
ab2c7c575b Merge branch '3.0' into feature/3_liaohj 2022-10-30 11:18:36 +08:00
Haojun Liao
52c3777823 refactor: do some internal refactor. 2022-10-30 03:01:40 +08:00
Haojun Liao
afdb1467b7 refactor: do some internal refactor. 2022-10-29 18:27:22 +08:00
Shengliang Guan
24f31b0f72
Merge pull request #17746 from taosdata/feature/3.0_mhli
refactor(sync): adjust elect timer, add some tests
2022-10-29 17:02:19 +08:00
Haojun Liao
d583f1fab6 other: merge 3.0 2022-10-29 13:05:35 +08:00
Haojun Liao
bae6d824c2 fix(query): init blockscanInfo correctly. 2022-10-29 12:54:39 +08:00
Haojun Liao
f9aa286dc1 fix(query): fix syntax error on aarch64 2022-10-29 10:22:02 +08:00
Haojun Liao
61634070ec other: merge 3.0 2022-10-28 23:23:53 +08:00
Haojun Liao
307b7e0ac8 other: merge 3.0 2022-10-28 22:27:57 +08:00
Haojun Liao
029a4a85b3 fix(query): support partition by + limit push down. 2022-10-28 22:21:02 +08:00
Haojun Liao
9b99c69785 fix(query): support partition by + limit push down. 2022-10-28 19:56:32 +08:00
Haojun Liao
0035bb1966
Merge branch '3.0' into feature/3_liaohj 2022-10-28 19:43:47 +08:00
Haojun Liao
6ff3668ecc refactor: update the log. 2022-10-28 19:38:37 +08:00
Haojun Liao
335cb6513e other:merge 3.0 2022-10-28 19:32:57 +08:00
Haojun Liao
9ccf7addb3 other:merge 3.0 2022-10-28 18:46:08 +08:00
Liu Jicong
d058475fac
Merge pull request #17750 from taosdata/feature/stream
feat(stream): snode
2022-10-28 18:12:48 +08:00
Haojun Liao
4b2fa43dae
Revert "fix(query): handle partition by in table scan operator" 2022-10-28 18:05:19 +08:00
Haojun Liao
d0a7a43e6f other:merge 3.0 2022-10-28 18:02:50 +08:00
Haojun Liao
d103581eb8 Merge remote-tracking branch 'origin/fix/liao_cov' into fix/liao_cov 2022-10-28 17:59:20 +08:00
Haojun Liao
6c19e886df fix(query): move the init function for different reader. 2022-10-28 17:58:59 +08:00
Shengliang Guan
8a8aa3efea
Merge pull request #17723 from taosdata/enh/optQueryMem
enh: opt query mem
2022-10-28 17:09:17 +08:00
Liu Jicong
b26cc9c318 feat(snode) 2022-10-28 17:03:17 +08:00
54liuyao
7d8a282258 fix(stream):tqBuildDeleteReq error 2022-10-28 11:19:26 +08:00
Haojun Liao
2c896012df fix(query): check status before add table uid in hash map. and do some other refactor. 2022-10-28 10:43:25 +08:00
Haojun Liao
f25387018d
Merge pull request #17724 from taosdata/fix/liao_cov
fix(query): handle the case of the single block in last files.
2022-10-28 10:25:38 +08:00
Liu Jicong
9b1ba10fbc Merge branch '3.0' into feature/stream 2022-10-28 09:18:59 +08:00
Haojun Liao
70c3825ee5 fix(query): remove invalid check. 2022-10-28 09:12:40 +08:00
Haojun Liao
4774baa635 fix(query): handle the case of the single block in last files. 2022-10-27 22:43:42 +08:00
Haojun Liao
e7b59ae56d fix(query): handle the case of the single block in last files. 2022-10-27 21:51:54 +08:00
yihaoDeng
9de0afc8ea opt mem 2022-10-27 21:22:36 +08:00
yihaoDeng
6a978be730 opt mem 2022-10-27 20:38:57 +08:00
Haojun Liao
d8102fc8d8 fix(query): fix invalid number of tables check. 2022-10-27 18:58:10 +08:00
Haojun Liao
a7ca29c627 other: merge 3.0 2022-10-27 18:04:28 +08:00
dapan1121
b6326c7f14
Merge pull request #17710 from taosdata/fix/TD-19815
fix: add query heartbeat configuration
2022-10-27 17:33:27 +08:00
Haojun Liao
5a971dd03a fix(query): handle partition by in table scan operator with limit/offset value. 2022-10-27 16:58:32 +08:00
Liu Jicong
70311d95dd refactor(vnode): add log for failed deletion 2022-10-27 16:45:40 +08:00
Liu Jicong
5abeacc299
Merge pull request #17711 from taosdata/feature/stream
refactor(mnode): remove unused code
2022-10-27 16:41:09 +08:00
Minghao Li
bbb624b63e Merge branch '3.0' of https://github.com/taosdata/TDengine into feature/3.0_mhli 2022-10-27 16:22:08 +08:00
Minghao Li
7cb3112d77 refactor(sync): modify function name 2022-10-27 15:52:48 +08:00
Liu Jicong
dcf2c980c5 refactor(mnode): remove unused code 2022-10-27 15:32:20 +08:00
Shengliang Guan
8b31cd87d1
Merge pull request #17706 from taosdata/feature/stream_ly
fix(stream): read preversion data with delete many times
2022-10-27 15:05:23 +08:00
54liuyao
012dd30baa fix(stream): read preversion data with delete many times 2022-10-27 14:18:48 +08:00
Shengliang Guan
a0c5cece12
Merge pull request #17697 from taosdata/feature/stream_ly
fix(stream): read preversion data for delete
2022-10-27 14:01:08 +08:00
Liu Jicong
320202e60e Merge branch '3.0' into feature/stream 2022-10-27 13:49:56 +08:00
Liu Jicong
52d8f7bd35
Merge pull request #17701 from taosdata/feature/stream
fix(query): set block capacity
2022-10-27 13:38:46 +08:00
Liu Jicong
6cd8361a0a refactor(sma) 2022-10-27 11:06:16 +08:00
54liuyao
442f89783f fix(stream): read preversion data for delete 2022-10-27 10:55:32 +08:00
Shengliang Guan
26aa0f1d37
Merge pull request #17684 from taosdata/fix/TD-19894
fix: continue to fetch next row if deleted
2022-10-27 09:50:41 +08:00
Liu Jicong
465982fd8d
Merge pull request #17681 from taosdata/feature/stream
test(wal): rollback multiple file
2022-10-27 09:05:05 +08:00