Commit graph

847 commits

Author SHA1 Message Date
wangjiaming
f50fcb85b8
feat(decimal): support decimal data type (#30060)
* decimal: create table

* decimal: add test case decimal.py

* decimal: add decimal.c

* support input decimal

* decimal test

* refactor svalue

* fix test cases

* add decimal unit test

* add decimal test cmake

* support insert and query decimal type

* define wide integer, support decimal128

* support decimal128 divide

* set decimal type expr res types

* scalar decimal

* convert to decimal

* fix decimal64/128 from str and to str

* fix decimal from str and decimal to str

* decimal simple conversion

* unit test for decimal

* decimal conversion and unit tests

* decimal + - * /

* decimal scalar ops and comparision

* start to refactor GET_TYPED_DATA

* support decimal max func, cast func

* refactor GET_TYPED_DATA interface

* decimal scalar comparision

* start to implement sum for decimal

* support sum and avg for decimal type

* decimal tests

* add decimal test

* decimal add test cases

* decimal use int256/int128

* decimal testing

* fix decimal table meta and add tests for decimal col streams

* fix create stream and create tsma

* test insert decimal values

* decimal from str

* test decimal input

* test parse decimal from string

* add taos_fetch_field_e api

* decimal insert tests

* test decimal operators

* decimal operator test

* feat:support decimal in raw block

* decimal operator tests

* decimal test

* feat:support decimal in raw block

* feat:support decimal in raw block

* feat:add schemaExt to SMqDataRsp

* feat:remove add schemaExt to SMqDataRsp

* feat:remove add schemaExt to SMqDataRsp

* feat:remove add schemaExt to SMqDataRsp

* decimal test operators

* decimal operator test

* test decimal operators

* test decimal compare operators

* decimal unary operator test

* decimal col with decimal col oper test

* test decimal col filtering

* fix decimal float operator test

* decimal test where filtering

* fix decimal filtering

* fix decimal order by

* fix decimal op test

* test decimal agg funcs

* test decimal functions

* remove assert

* fix ci build for ret check

* fix decimal windows build

* fix ci ret check

* skip decimal ret check

* skip decimal ret check

* fix decimal tests

* fix decimal ci test

* decimal test

* fix(tmq): heap user after free

* fix(tmq): double free

* fix(tmq): double free

* fix decimal tests

* fix(decimal): decimal test ci build

* fix(decimal): windows build

* fix(decimal): decimal test build

* fix(decimal): fix decimal build and tests

* fix(decimal): fix decimal tests

* fix(decimal): fix taos_fetch_fields_e api

* fix(decimal): fix decimal taos_fetch_fields_e api

* fix(decimal): rebase 3.0

* fix(decimal): fix decimal functions

* fix(decimal): fix decimal test case memory leak

* fix(decimal): fix decimal tests

* fix(decimal): fix decimal test case

* fix(decimal): fix decimal tests

* feat(decimal): fix unit tests

* feat(decimal): fix deicmal unit test

---------

Co-authored-by: wangmm0220 <wangmm0220@gmail.com>
Co-authored-by: yihaoDeng <yhdeng@taosdata.com>
2025-03-14 18:08:07 +08:00
WANG MINGMING
3d053e2c9b
feat(stream): optimize client logic of creating stream if with history (#30059)
* feat:[TS-5617]use last_row cache mode if creating stream in fill_history

* feat(stream): optimize client logic of creating stream if with history

* feat(stream): optimize client logic of creating stream if with history

* feat(stream): optimize client logic of creating stream if with history

* feat(stream): optimize client logic of creating stream if with history

* feat(stream): optimize client logic of creating stream if with history

* feat(stream): optimize client logic of creating stream if with history

* feat(stream): optimize client logic of creating stream if with history

* feat(stream): optimize client logic of creating stream if with history

* feat(stream): optimize client logic of creating stream if with history

* feat(stream): optimize client logic of creating stream if with history

* feat(stream): optimize client logic of creating stream if with history

* fix: heap use after free

* feat: add log

* fix: ci case error

* fix: compile error in windows

* fix: ci case error

* fix: heap user after free

* fix: memory leak

* fix: ci case error

* fix: ci case error

* fix: ci case error

---------

Co-authored-by: yihaoDeng <yhdeng@taosdata.com>
2025-03-14 13:55:52 +08:00
Kaili Xu
13f9bddf3d
feat: support customized taos/taosd (#29736)
* feat: support TDAcoreOS

* chore: cmake options for TD_ACORE

* chore: disable lemon for TD_ACORE

* chore: add lzma2 and msvcregex

* chore: cmake for lzma2

* chore: adapt for TD_ACORE

* chore: adapt strcasecmp for TD_ACORE

* chore: adapt for geos/threadName

* chore: build adapt for TD_ACORE

* chore: build adapt for TD_ACORE

* chore: build adapt for TD_ACORE

* chore: build adapt for TD_ACORE

* chore: build adapt for TD_ACORE termio

* chore: refact transComm.h for TD_ACORE

* chore: refact transportInt.h for TD_ACORE

* chore: refact trans.c for TD_ACORE

* chore: refact trpc.h for TD_ACORE

* chore: refact transCli.c/transComm.c/transSvr.c for TD_ACORE

* chore: refact uv.h for TD_ACORE

* chore: refact geosWrapper.h for TD_ACORE

* chore: refact token/builtins/udf for TD_ACORE

* chore: refact rocks for TD_ACORE

* chore: refact tsdbCache.c for TD_ACORE, use LRU cache for last/last_row, not use rocksdb

* chore: refact FAIL to _ERR to solve conflicts for TD_ACORE

* chore: restore lemon.c/lempar.c

* chore: support build lemon for TD_ACORE

* chore: refact trpc and siginfo_t for TD_ACORE

* chore: refact timezone for TD_ACORE

* chore: refact lz4 for TD_ACORE

* chore: refact TD_ACORE to make compile pass

* chore: code optimization for TD_ASTRA

* feat: support run taos with taosd integrated

* feat: support invoke taos shell

* feat: support invoke taos shell

* feat: support invoke taos shell

* chore: code optimization

* chore: fix undefined reference problem os TD_ASTRA

* chore: resolve compile problem for TD_ASTRA

* chore: fix undefined reference problem os TD_ASTRA

* chore: fix undefined reference problem os TD_ASTRA

* chore: fix undefined reference problem os TD_ASTRA

* chore: fix undefined reference problem os TD_ASTRA

* chore: fix undefined reference problem os TD_ASTRA

* chore: fix undefined reference problem os TD_ASTRA

* chore: fix undefined reference problem os TD_ASTRA

* chore: fix undefined reference problem os TD_ASTRA

* chore: fix undefined reference problem os TD_ASTRA

* chore: fix undefined reference problem os TD_ASTRA

* chore: fix undefined reference problem os TD_ASTRA

* chore: fix undefined reference problem os TD_ASTRA

* chore: fix undefined reference problem os TD_ASTRA

* chore: fix undefined reference problem os TD_ASTRA

* chore: fix undefined reference problem os TD_ASTRA

* chore: fix undefined reference problem os TD_ASTRA

* chore: fix undefined reference problem os TD_ASTRA

* chore: fix undefined reference problem os TD_ASTRA

* chore: fix getpid

* chore: fix typo

* chore: set stack size and ajust min pack size for TD_ASTRA

* chore: fix pthread create parameters

* chore: chmod adapt for TD_ASTRA

* chore: fix trans compile problem

* chore: adapt chmod for TD_ASTRA

* chore: byte alignment for TD_ASTRA

* chore: more code for adaption of TD_ASTRA

* chore: more code for adaption of TD_ASTRA

* chore: more code for adaption of TD_ASTRA

* chore: byte alignment for TD_ASTRA

* chore: conditional compile option

* chore: adapt for TD_ASTRA

* chore: adjust taosPId and msvcregex for TD_ASTRA

* chore: log dir separator for wal build name

* chore: fix type of pointer parameter

* chore: fix compile problem of tsdbGetS3Size

* enh: get last ver from wal log for TD_ASTRA

* enh: refact wal meta ver

* enh: refact wal meta ver

* fix: typo of taosUcs4Compare

* enh: process return value of CI

* chore: more code for TD_ASTRA adaption

* chore: return value of taosCloseFile in walMeta.c

* chore: fix compile problem

* chore: fix compile problem of TD_ASTRA

* fix: update macro for tq and stream task

* chore: code optimization for TD_ASTRA

* chore: restore create log and init cfg interface

* chore: restore strncasecmp and strcasecmp

* fix: adjust the field position of SDataBlockInfo

* fix: pragma pack min size

* fix: pragma pack min size

* chore: more code for TD_ASTRA adaption

* fix: type of parameters

* chore: adapt strncasecmp and strcasecmp for TD_ASTRA

* chore: restore interface of init log

* enh: pack push optimization

* fix: taos init cfg

* add astra support

* fix: fetch the value of suid

* chore: switch of  build with udf

* add temp code

* chore: more code for TD_ASTRA adaption

* chore: add macro ERRNO to replace errno

* chore: bytes align for TD_ASTRA

* fix: remove obsolete codes

* enh: support USE_UDF macro

* fix compile error

* fix: resolve redefinition problem

* fix: compile problem of log.cpp

* fix: compile problem of osTimezone

* fix: resolve compile problem of udf

* fix: pragma definition on windows

* fix: ucs4 and stpncpy for TD_ASTRA

* fix: memory align problem for TD_ASTRA

* enh: solve memory leak for TD_ASTRA_RPC

* fix: compile problem of taosSetInt64Aligned

* fix: restore mndSubscribe.c

* fix: scalar for udf

* chore: code adaption for TD_ASTRA

* chore: code optimization for TD_ASTRA

* fix: typo of add definition

* fix: typo of macro in tudf.h

* chore: remove void to make CI pass

* enh: move macro from cmake.platform to cmake.options

* enh: byte align for hash node and error code

* chore: restore the size for lru cache

* enh: restore some code about pack push

* chore: restore the pack push in tmsg.h

* fix: add macro of pack pop for windows

---------

Co-authored-by: yihaoDeng <luomoxyz@126.com>
2025-03-14 13:32:13 +08:00
Simon Guan
1a9f75e663 Merge branch '3.0' into fix/internal 2025-03-04 13:47:03 +08:00
Simon Guan
b74b038f4f Merge branch '3.0' into merge/mainto3.0 2025-03-04 11:32:29 +08:00
Simon Guan
6826edf5e0 Merge branch 'main' into merge/mainto3.0 2025-03-03 14:01:43 +08:00
Simon Guan
d7207861bb Merge remote-tracking branch 'origin/3.0' into fix/internal 2025-03-03 10:43:07 +08:00
Haojun Liao
a71f414d97 refactor(stream): stop all tasks before drop vnodes. 2025-03-02 02:06:14 +08:00
Simon Guan
24fae6adbb
Merge pull request #29900 from taosdata/fix/checkfh
refactor(stream): scan wal is driven by time, instead of insert events.
2025-02-28 13:42:43 +08:00
Simon Guan
f5429637fa Merge branch 'main' into merge/mainto3.0 2025-02-28 09:11:10 +08:00
Haojun Liao
7b56df0adc refactor(stream): create task epset update trans out of lock, to avoid blocking mnode-read thread. 2025-02-27 14:51:26 +08:00
Simon Guan
76ad364b93 enh: adjust some logs 2025-02-27 09:55:11 +08:00
Shengliang Guan
e429264432
Merge pull request #29884 from taosdata/enh/killtrans
enh(stream): auto kill checkpoint trans that last long time
2025-02-25 11:29:49 +08:00
Haojun Liao
91271f3fa4 refactor: do some internal refactor. 2025-02-24 13:53:28 +08:00
Haojun Liao
4d21d5e055 fix(stream): fix memory leak. 2025-02-24 13:50:38 +08:00
Haojun Liao
97071f1d29 other: add some logs. 2025-02-24 09:42:15 +08:00
Haojun Liao
ea549364d4 fix(stream): free array and check the return values. 2025-02-21 16:38:26 +08:00
Haojun Liao
99d6086c5a enh(stream): kill too long checkpoint trans. 2025-02-20 15:07:24 +08:00
Shengliang Guan
d0eb99809f Merge branch 'main' into merge/mainto3.03 2025-02-20 11:38:41 +08:00
Haojun Liao
3048654484 refactor(stream): kill too long checkpoint trans. 2025-02-20 10:01:30 +08:00
Haojun Liao
7d617a7cde
fix(stream): fix the invalid start task condition
1. Get the basic checkpointId, and wait for 10 s, it's allowed to go.  2. All tasks have the same checkpointId, go immedidately.
2025-02-19 00:39:01 +08:00
Shengliang Guan
b43aaa5f8a Merge branch '3.0' into merge/mainto3.02 2025-02-18 16:48:55 +08:00
Shengliang Guan
63fc5dea34 Merge branch '3.0' into merge/mainto3.0 2025-02-18 10:29:16 +08:00
Jinqing Kuang
fc2d2672b9 feat(stream)[TS-5469]: add more stream event notifications
- Add event notifications for all window types
- Implement window invalidation notifications
- Add statistics for event notification delivery
2025-02-16 11:07:16 +08:00
Shengliang Guan
cd52401303 Merge branch 'main' into merge/mainto3.02 2025-02-11 15:15:35 +08:00
Haojun Liao
267f7d3b08 refactor(stream): limit the maximum number of consensus checkpoint trans. 2025-02-11 10:31:11 +08:00
Haojun Liao
d08a81cc98 fix(stream): fix bug in limiting concurrent checkpoints 2025-02-10 19:24:37 +08:00
Haojun Liao
6c7a8e9774 refactor(stream): allow ready stream to start checkpoint procedure. 2025-02-10 19:24:36 +08:00
Haojun Liao
ec954bf408 refactor(stream): record the dropped stream. 2025-02-09 21:13:10 +08:00
Shengliang Guan
a004d95996 Merge branch '3.0' into merge/mainto3.02 2025-02-06 13:46:36 +08:00
Haojun Liao
2f065062c3 fix(stream): adjust position of checking trans. 2025-01-21 19:04:14 +08:00
Haojun Liao
88a94919c4 fix(stream): check for nodeupdate trans before create streams. 2025-01-21 18:41:52 +08:00
Jinqing Kuang
eb5d463490 feat(stream)[TS-5469]. add support for window event notifications in stream processing
- Introduce new syntax to specify notification type and destination address
- Collect relevant event information during window computations
- Implement websocket-based notification delivery to the specified address
2025-01-21 17:39:46 +08:00
Haojun Liao
efd33aa4d7 fix(stream): check the checkpoint-report transId and checkpointId, and identify the expired checkpoint-report info. 2025-01-21 01:25:53 +08:00
Shengliang Guan
c62e189496 Merge branch 'main' into merge/mainto3.0 2025-01-09 13:38:37 +08:00
Haojun Liao
130ad28d2c fix(stream): check stream task status before start checkpoint. 2025-01-08 15:31:15 +08:00
Shengliang Guan
fcca5e7c2e
Merge pull request #29381 from taosdata/fix/liaohj
test: add test cases and fix related bugs.
2024-12-31 11:10:34 +08:00
Haojun Liao
f03e2305e0 fix(stream): add logs and set the reset-stream rsp. 2024-12-30 14:26:20 +08:00
54liuyao
0bf9fb1e31 add reset stream 2024-12-27 15:11:19 +08:00
54liuyao
e6ac454303 Merge branch 'main' of https://github.com/taosdata/TDengine 2024-12-17 18:02:36 +08:00
54liuyao
63a6428e34 fix(stream):adjust status of ins_streams 2024-12-17 17:59:53 +08:00
xiao-77
c4ed102c11 Merge branch '3.0' into enh/3.0/TD-31709 2024-12-16 10:00:02 +08:00
xiao-77
da8dd39bd8 Fix ci problems. 2024-12-13 17:21:30 +08:00
xiao-77
c66a834d1e Replace unsafe memory functions with safe versions in wal&sync. 2024-12-13 15:19:29 +08:00
Haojun Liao
fa591f4667 other: merge 3.0 2024-12-11 19:07:30 +08:00
Haojun Liao
1d700079d4 refactor: 1) add update task nodeep trans for every involved streams. 2) do some internal refactor for conflict check. 2024-12-06 18:56:49 +08:00
Haojun Liao
6bbfc1aead fix(stream): do rsp for checkpoint reqs. 2024-12-04 19:19:57 +08:00
Haojun Liao
9a9a1828c1 refactor: display the time window for force_window_close. 2024-11-29 13:47:20 +08:00
Haojun Liao
9cc2aec9d6 fix(stream): set correct value and open inputQ for stream if reset checkpoint. 2024-11-15 17:12:01 +08:00
Haojun Liao
969db55e9f fix(stream): update the checkpoint report info in SCheckpointReport 2024-11-14 17:42:55 +08:00