The following commits could not be applied individually due to context
differences between the monorepo and the public repo's build files.
They have been applied as a cumulative diff to ensure the final state
matches the monorepo exactly:
- chore: sync CI files with 3.0 branch to eliminate merge conflicts (rd-public/tsdb!271)
- revert(refactor): dynamically link taosd taosudf taosmqtt against libtaosnative.so to reduce binary size (revert #183) (rd-public/tsdb!282)
- fix(docs): autofix formatting issues across all doc files (rd-public/tsdb!296)
- feat: support -DBUILD_SANITIZER=true on windows for debug build (rd-public/tsdb!291)
- feat(build): build cache, mirror, and sccache optimizations (rd-public/tsdb!326)
- docs: update image for three replica (rd-public/tsdb!324)
- enh: shared storage on windows (rd-public/tsdb!333)
- fix(cmake): convert ext_libs3 from git clone to URL tarball download (rd-public/tsdb!360)
- feat: dual-source deps and comprehensive docs/packaging (cherry-pick to main) (rd-public/tsdb!352)
- fix(cmake): guard DOWNLOAD_EXTRACT_TIMESTAMP for CMake < 3.24 and fix duplicate Cargo.lock entry (rd-public/tsdb!369)
- fix: test case execution failure in pytest.sh (rd-public/tsdb!338)
- enh: built-in compilation support for Python UDF plugins use abi3 (rd-public/tsdb!325)
* fix(scalar): prevent crash when IN expression has invalid type from unhandled node
Root cause: sclGetNodeType() silently set *type = -1 and returned
TSDB_CODE_SUCCESS for unhandled node types. This propagated -1 as
ctx->type.selfType, which was then cast to (uint32_t)-1 = 4294967295
and used as an index into tDataTypes[], causing an out-of-bounds
crash in scalarGenerateSetFromList().
The bug was triggered by semantically odd SQL such as:
ifnull(b not between vb and a, n in (...)) in (today(), today(), now())
where a boolean result is compared against a timestamp list.
Fixes:
1. sclGetNodeType(): return TSDB_CODE_QRY_INVALID_INPUT for unhandled
node types instead of silently setting type = -1.
2. sclInitParam(): validate selfType before using it as list element
type to prevent OOB access in vectorGetConvertType(-1, ...).
3. scalarGenerateSetFromList(): add IS_INVALID_TYPE() guard at entry
as a defensive check against any future invalid-type paths.
Add regression test: test_scalar_invalid_type.py
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* fix: case when and decimal query crash issues
* fix: CAST to JSON returns Unknown error 65535; mavg/csum/diff with constant arg returns invalid input
- builtins.c: reject CAST(expr AS JSON) in translateCast() with
TSDB_CODE_CAST_TO_JSON_NOT_ALLOWED instead of silently passing
validation and failing at execution with TSDB_CODE_FAILED (-1)
- scalar.c: add QUERY_NODE_LEFT_VALUE case in sclGetNodeType() so
rewritten constant operands (ASSIGN operator left side) resolve
their type from ctx->type.opResType instead of hitting the
'unsupported node type' fallthrough (error 0x070F invalid input)
- executorInt.c: create dummy column for the first arg of indefinite-
rows functions when it is a constant, preventing NULL pData[0]
- test_fun_sca_cast.py: add do_cast_to_json_invalid() covering all
19 sql-fuzzing repro queries for the CAST-to-JSON bug
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* fix: sclGetNodeType missing cases cause invalid input in query/subquery
Add missing node type cases to sclGetNodeType() that were hit after
ed9de01 changed the fallthrough from silent type=-1 to hard error:
1. QUERY_NODE_LEFT_VALUE (27): return ctx->type.opResType, used when
the planner rewrites constants into ASSIGN(LEFT_VALUE, VALUE)
operators for scalarSup pre-processing (e.g. mavg(2.0, 3))
2. QUERY_NODE_REMOTE_VALUE_LIST (61): read resType from SExprNode,
used when IN subquery RHS is a remote value list node
3. QUERY_NODE_REMOTE_VALUE / QUERY_NODE_REMOTE_ZERO_ROWS /
QUERY_NODE_REMOTE_ROW: read resType from embedded SValueNode,
defensive coverage for other remote node types
Also extend doSetInputDataBlock() dummy-column creation to cover
the first constant argument of indefinite-rows functions (mavg, csum,
diff) so pInput->pData[0] is not NULL at function execution time.
Fixes: select distinct t1,'abc',tbname from st1 -> invalid input
Fixes: 1 in (select 1 from st1) -> invalid input
Fixes: mavg(cast(2 as float), 3) -> invalid input
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Remove QUERY_NODE_LEFT_VALUE case from switch statement
Removed handling for QUERY_NODE_LEFT_VALUE case in scalar.c
---------
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* feat: support alter key expiration time.
* test: add test for storage encrypt.
* test: add cases to ci.
* fix: ci test.
* feat: support udf to load encrypt key.
* fix: remove debug log
* fix: remove assert
* fix: delete unused code
* enh: [TD-37251] Support expr in state window.
* feat(stream): support expr in state window trigger
* enh: [TD-37251] Fix SCL_IS_CONST_CALC condition.
* fix(stream): set ver in wal
* fix: print code
* fix: increase runner replica num
* fix: trigger mem error
* fix: sliding _tnext_ts value
* fix(stream): disable tagFilterCache in stream reader trigger
* fix: crash
* Revert "fix(stream): fix history calc finish check"
This reverts commit f93d17f1d2.
* Revert "fix(stream): fix calc request allocation in trigger"
This reverts commit c5410f6da0.
* fix(stream): fix calc request allocation in trigger
* enh: [TD-37251] External window support more placeholder.
* fix(stream): modify size of return from 1000000->4096
* enh: [TD-37251] Modify error msg when stream query do not have from clause.
* fix(stream): add log for group not found
* fix(stream): do not return gid=0 in walMetaData interface
* enh: [TD-37251] Fix missing ts column in vtable query.
* fix: test case build failed
* fix: invalid read issue
* fix(stream): add vtable logic
* fix(stream): encode error in wal
* fix(stream): add vtable logic
* fix(stream): add log
* fix: diff funcition crash
* Revert "Merge branch 'enh/TD-37251-3.0-dropoutput' into enh/TD-37251-3.0"
This reverts commit e93cbd6fd4, reversing
changes made to dc3230591d.
* Revert "Merge branch 'enh/TD-37251-3.0-vtable' into enh/TD-37251-3.0"
This reverts commit dc3230591d, reversing
changes made to 085e086782.
* fix(stream): fix block data len is too large if data type is vchar
* fix: drop output table
* feat: runner delete output table
* process pDropBlock in trigger task.
* fix(stream): opti log level
* fix(stream): build block for drop table
* fix(stream): set gid for normal table
* fix(stream): set gid for normal table
* feat: Support delete output.
* fix(stream): rows error
* fix(stream): memory leak
* enh: [TD-37251] Fix external window wrong ts column.
* fix(stream): fix calc time check in batch mode
* fix: merge aligned external window issue
* Revert "fix(stream): fix calc time check in batch mode"
This reverts commit d895b7f577.
* fix(stream): add test case
* fix(stream): add insert drop table logic
* fix: external window end issue
* fix(stream): add test case
* fix(stream): fix trigger pull data
* fix(stream): fix history calc request
* enh: drop table on snode
* fix(stream): adjust hash index if data is filtered in wal
* fix(stream): rollback
* enh: add merge aligned extwin window row idx
* fix: drop output table
* fix: compile issue
* enh: [TD-37251] Add flag to identify interval window is overlapped
* fix: overlap
* fix(stream): set gid=-1 for initialized
* fix(stream): modify log level
* fix: trigger slow issue
* fix(stream): add basic test for obj pool
* fix(stream): fix metadata clear in trigger
* fix(stream): fix idle runner allocation in trigger
* fix: handle agg output on externalWin
* fix: test case
* fix(stream): adjust log
* fix: reset pCtx pOutput
* fix: memory leak
* fix: search first win for tsCol
* fix(stream): add test case for schema change
* fix: mem leak
* fix: mem leak
* Reapply "Merge branch 'enh/TD-37251-3.0-vtable' into enh/TD-37251-3.0"
This reverts commit b508e66958.
* fix(stream): fix virtual table data pull
* fix(stream): fix set table request
* fix(stream): process empty uidlist
* fix(stream): fix set table request
* fix(stream): fix data new request in trigger
* fix(stream): tablelist error for vtable
* fix(stream): block ver is null
* fix(stream): remove version limition for wal
* fix(stream): block rows error
* fix(stream): fix pending calc param in batch mode
* fix(stream): auto create table
* fix(stream): fix stream vtable data merge
* fix: _tcurrentts
* fix(stream): destroy hash
* fix(stream): fix trigger status
* fix(stream): colId error in vtable
* fix(stream): update nrows of vtable data block
* fix(stream): fix trigger status
* fix(stream): enable low latency calc for period trigger
* fix: test case file path
* fix: test case file path
* fix: string to node in reader
* enh: add test log
* fix(stream): increase wait time of non-low-latency mode
* fix(stream): fix column capacity in scalar calculation
* fix(stream): fix column capacity in trigger expr calculation
* fix: get origTableInfos
* fix(stream): fix calc data pull in trigger
* fix(stream): fix calc data cache write in trigger
* enh: [TD-37251] Add flag to identify interval wind
* fix: external window memory usage issue
* fix(stream): fix epxr result column in trigger
* fix(stream): add metaCache for calc plan
* fix(stream): fix stream obj list clear
* fix(stream): rollback
* enh: [TD-37251] Add flag to identify interval wind
* fix: mem free
* fix(stream): add metaCache for calc plan
* fix(stream): fix calc data cache write in trigger
* fix(stream): fix calc data cache write in trigger
* enh: optimize external result block memory
* fix(stream): modify logic of judge table for create table
* fix(stream): fix event window check in trigger
* fix(stream): fix count window check in trigger
* fix(stream): colSize=0 while encoding block because pDataCol->hasNull is false in secode time & reload table list if create table
* enh: optimize stream memory
* fix(stream): trigger tag error
* fix: add log
* fix(stream): fix calc data write in trigger
* fix: drop output table
* fix(stream): fix max delay in trigger
* fix: drop output table
* fix(stream): fix max delay in trigger
* fix(build): handle return value of function
* fix(stream): read gid error if it is child table in stream
* fix(stream): fix calc param of history calculation
* fix(stream): fix recalc of delete data
* fix(stream): fix calc data of period trigger
* fix(stream): fix cache read check
* fix(stream): filter error in calc plan
* fix: reset externla window expr
* fix(stream): fix calc parm of max delay
* fix: time range and case issues
* fix(stream): fix crash in trigger
* fix: case issues
* fix: fix window node mem leak.
* fix(stream): fix meta data clear in trigger
* fix(stream): table schema is old in TsdbDataRequest for vtable
* fix: fix slingding window place holder check condition.
* Revert "fix: fix slingding window place holder check condition."
This reverts commit ad864a1dc1.
* fix(stream): null pointer error
* fix: case issue
* fix(stream): calc data error for vtable
* fix(stream): fix data sorter in trigger
* fix(stream): add log for delete data
* fix(stream): fix start version of realtime calculation
* fix(stream): fix cache data merger of vtable
* fix: case issues
* fix(ci): upgrade stream cases in test_cols_function
* fix(stream): gid not found if change tag value
* fix: fix slingding window place holder check condition.
* fix(stream): fix virt table info request in trigger
* fix(stream): set gid = uid if stream table type != SUPER table
* fix: clean cache data by group
* fix: add block info and case issues
* fix: fix heap-buffer-overflow.
* fix(stream): fix state window with extend param
* fix: fix access null pointer.
* fix: case issues
* fix: case issue
* fix(stream): fix ignore_nodata_trigger option for period trigger
* fix(stream): fix pseudo col fetch for calc data
* fix: Extend checking time to avoid timeout.
* fix: case issues
* fix(stream): fix group col fetch for virtual tables
* fix(stream): tag is NULL for non vtable
* fix(stream): fix sliding check of virtual table
* test(stream): check stream status after create all streams
* fix: add log
* fix(stream): fix wal meta truncate when ignore disorder
* fix(stream): gid not found for child table
* fix: fix place holder condition pushdown error.
* fix(stream): suid not equal when delete data for child table
* fix: id issue
* fix(stream): disable recalc for count trigger
* fix: cast result rowSize error in project
* fix(stream): add log for tsdb meta
* fix(stream): fix gid in tsdb meta request
* fix(stream): fix wend of unclosed windows
* fix(stream): fix ignore_nodata_trigger option for period trigger
* fix: case issues
* fix(stream): fix calc data pull for empty interval window
* fix: fix ext window condition.
* fix(ci): smaBasic performance check affectd by debug level log
* fix(stream): add suid when set table list for vtable
* fix: forbid using prefilter when using %%trows an trigger table is virtual table.
* fix: forbid prefilter %%trows cases.
* fix(stream): sort cid in tsdbVirtalDataReq
* fix: forbid prefilter %%trows cases.
* fix(stream): add log for virtual table tsdb data
* fix(stream): get delete msg for vtable
* fix: winRowIndex
* Revert "fix: winRowIndex"
This reverts commit e08b41cf96.
* fix(stream): fix data merge in trigger
* test(stream): fix case ans
* fix(stream): fix empty calc data pull for period trigger
* fix(stream): col index error for tsdbVirtalDataReq
* fix(stream): pTableList is NULL for vtable
* test(stream): fix case ans
* fix(stream): fix notification in trigger
* fix(stream): memory leak
* fix(stream): fix virtual data pull in trigger
* test(stream): fix case ans
* fix(stream): col index error for tsdbVirtalDataReq
* fix(stream): pTableList is NULL for vtable
* test(stream): fix case ans
* fix(stream): fix notification in trigger
* fix(stream): memory leak
* fix(stream): fix virtual data pull in trigger
* test(stream): fix case ans
* fix: case issue
* fix: crash issue
* fix: forbid prefilter %%trows cases.
* fix(stream): session case
* fix(stream): fix data pull for virtual tables
* fix(stream): add log
* fix(stream): fix calc req send in batch mode
* fix: fix stream UT
* fix(stream): tablelist is null for non vtable
* fix: mem leak
* fix(stream): fix compile error in trigger
* fix: return code issue
---------
Co-authored-by: dapan1121 <wpan@taosdata.com>
Co-authored-by: facetosea <285808407@qq.com>
Co-authored-by: Jing Sima <simondominic9997@outlook.com>
Co-authored-by: Jinqing Kuang <kuangjinqingcn@gmail.com>
Co-authored-by: Tony Zhang <tonyzhang@taosdata.com>
