3 commits

Author	SHA1	Message	Date
sonika-shah	f43d762f72	Fixes #27158: ingestion slowdown from tag_usage seq-scan on Postgres (#27745) ... * Fixes #27158: restore tag_usage prefix-LIKE index on Postgres The 1.11.0 perf migration (#23054) added four `WHERE state = 1` partial indexes on tag_usage; #24063 dropped the matching `state = 1` predicate from getTagsInternalByPrefix (Suggested-state rows are valid for both classification and glossary derivation), leaving every partial index inapplicable. Postgres fell back to a parallel seq scan; MySQL was unaffected because its 1.11.0 indexes were never partial. Adds a non-partial single-col btree on targetfqnhash_lower (mirrors MySQL's idx_targetfqnhash_lower) and rebuilds the four partials as non-partial -- same shape, same INCLUDE columns, predicate coupling removed so future query changes can't silently invalidate them. Verified end-to-end against a local Postgres with 50k rows: seq scan reproduced before the fix (matches reporter's EXPLAIN), bitmap index scan after, both for inline and prepared-statement paths. * Move #27158 tag_usage fix to 1.12.7 + add tagfqn_lower index Ship the fix in the 1.12.7 release line so customers on 1.12.x get it without waiting for 2.0.x. Also closes the second structural gap with MySQL: a non-partial single-col btree on tagfqn_lower mirroring MySQL's idx_tagfqn_lower (1.11.0). 1.12.8 directory removed; 1.12.7's existing schemaChanges.sql now carries the tag_usage indexes alongside the entity_extension migration. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>	2026-05-05 10:30:11 +05:30
Ram Narayan Balaji	368fae160b	Revert "Feature #18173: Version API Improvements" (#26307) (#27837) ... * Revert "Feature #18173: Version API Improvements, Last x versions order by desc, versions from specific timeline, versions for specific metadata changes, sdk support and UI integration (#26307)" This reverts commit e4d3e423e1. * fix: apply ruff formatting after conflict resolution in Python files	2026-04-30 11:23:42 +00:00
Sriharsha Chintalapani	e4d3e423e1	Feature #18173: Version API Improvements, Last x versions order by desc, versions from specific timeline, versions for specific metadata changes, sdk support and UI integration (#26307) ... * Feature #18173: Improve Version API, through paginatio, get x latest versions, specifict time, specific metadata changes * Feature #18173: Version API Improvements, Last x versions order by desc, versions from specific timeline, versions for specific metadata changes, sdk support and UI integration * Update generated TypeScript types * address comments * fix py check * Address comments * Address comments * Fix tests * Fix tests * Fix tests * Better way to lookup versions * Fix pytests * Fix tests * Address comments * chore(migrations): move version API schema additions from 1.13.0 to 1.12.7 Moves the PR's new entity_extension columns (versionNum, changedFieldKeys), indexes, and backfill scripts from the 1.13.0 migration directory into a new 1.12.7 directory. Keeps 1.13.0 identical to upstream main; only this PR's additions land in 1.12.7. Also updates MigrationSqlStatementHashTest to exercise the relocated files. * fix(versions): address CI failures and review feedback - testAPI.test.ts: update getTestCaseVersionList mock expectation to include the new params argument (APIClient.get is called with { params } since the function now supports limit/offset/fieldChanged). - PaginatedVersionHistory.spec.ts: replace banned networkidle waits and waitForSelector with web-first assertion on version-button visibility (satisfies playwright/no-networkidle and playwright/no-wait-for-selector). - EntityVersionTimeLine.tsx: implement infinite scroll via IntersectionObserver on a sentinel element at the bottom of the version list. Hooks up the onLoadMore/hasMore/isLoadingMore props that were in the interface but previously unused. - EntityVersionPage.component.tsx: fix stale-closure bugs in fetchMoreVersions (gitar-bot review). Use versionListRef for currentOffset and isLoadingMoreRef to gate concurrent invocations so IntersectionObserver double-firing does not cause duplicate appends. - EntityResource.java: accept offset > 0 with default limit when no fieldChanged is provided, so pagination params are no longer silently ignored (Copilot review). - datamodel_generation.py: raise explicit errors if generated files or expected replacement targets are missing, instead of silently succeeding when the generator output drifts (Copilot review). * fix(checkstyle): format Java, ESLint/Prettier on UI, relax datamodel_generation strict check - Java: spotless:apply on EntityResource.java (line-break formatting). - Python: relax datamodel_generation.py DIRECT_IMPORT_FIXES check — replacement targets are alternative forms the generator may or may not emit. Only require the final marker ('from .paging import Paging') is present after replacements; the prior strict per-target check broke 'make generate'. - UI lint: organize-imports, ESLint --fix, Prettier on all version-related files touched by the PR (resolves lint-src + lint-playwright CI checks). - EntityVersionTimeLine: guard IntersectionObserver effect with isLoadingMore so the observer is torn down while a fetch is in flight (Copilot review). - EntityVersionTimeline.test.tsx: add unit tests covering sentinel rendering conditions (hasMore, onLoadMore) and the isLoadingMore observer-guard (Copilot review). * fix(ui-checkstyle): prettier+eslint on EntityVersionTimeline.test.tsx Collapse import line and reorder JSX props (callbacks last) per repo lint rules. Reruns ui-checkstyle-changed caught these in the new test file from the previous commit. * test(playwright): address @aniketkatkar97 review on PaginatedVersionHistory spec - Add waitUntil: 'domcontentloaded' to every page.goto() call. - Wait for loaders (waitForAllLoadersToDisappear) before asserting the version-button to avoid racing the initial entity render. - Replace the manual { timeout: 15_000 } on versionSelectors.nth(1) with an explicit waitForResponse on the second paginated /versions call (offset > 0). This deterministically synchronises on the infinite-scroll fetch instead of a wall-clock timeout. * fix: address Copilot review — one-shot observer + local SQL splitter 1. EntityVersionTimeLine.tsx: call observer.unobserve(entry.target) as soon as the sentinel first intersects so onLoadMore fires only once per attached observer. The effect reattaches a fresh observer after isLoadingMore flips back to false, so subsequent pages still load — we just no longer rely on the parent's in-flight ref as the sole stopgap against repeated fires for the same page. 2. MigrationSqlStatementHashTest.java: replace Flyway's non-public org.flywaydb.core.internal.* parser classes with a small, local SQL statement splitter. Handles line (--) and block comments, single-, double-, and backtick-quoted strings, backslash escapes, and doubled- quote escapes. Removes a brittle dependency on Flyway internals that could break on upgrades. Tested: - mvn test -pl openmetadata-service -Dtest=MigrationSqlStatementHashTest → 2 tests pass. - yarn test EntityVersionTimeline.test.tsx → 8/8 tests pass. --------- Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> Co-authored-by: sonika-shah <sonika-shah@users.noreply.github.com> Co-authored-by: mohitdeuex <mohit.y@deuexsolutions.com> Co-authored-by: sonika-shah <sonikashah94@gmail.com> Co-authored-by: Mohit Yadav <105265192+mohityadav766@users.noreply.github.com>	2026-04-23 12:17:40 +02:00