mirror of
https://github.com/open-metadata/OpenMetadata
synced 2026-05-24 09:39:11 +00:00
* Feature #18173: Improve Version API, through paginatio, get x latest versions, specifict time, specific metadata changes * Feature #18173: Version API Improvements, Last x versions order by desc, versions from specific timeline, versions for specific metadata changes, sdk support and UI integration * Update generated TypeScript types * address comments * fix py check * Address comments * Address comments * Fix tests * Fix tests * Fix tests * Better way to lookup versions * Fix pytests * Fix tests * Address comments * chore(migrations): move version API schema additions from 1.13.0 to 1.12.7 Moves the PR's new entity_extension columns (versionNum, changedFieldKeys), indexes, and backfill scripts from the 1.13.0 migration directory into a new 1.12.7 directory. Keeps 1.13.0 identical to upstream main; only this PR's additions land in 1.12.7. Also updates MigrationSqlStatementHashTest to exercise the relocated files. * fix(versions): address CI failures and review feedback - testAPI.test.ts: update getTestCaseVersionList mock expectation to include the new params argument (APIClient.get is called with { params } since the function now supports limit/offset/fieldChanged). - PaginatedVersionHistory.spec.ts: replace banned networkidle waits and waitForSelector with web-first assertion on version-button visibility (satisfies playwright/no-networkidle and playwright/no-wait-for-selector). - EntityVersionTimeLine.tsx: implement infinite scroll via IntersectionObserver on a sentinel element at the bottom of the version list. Hooks up the onLoadMore/hasMore/isLoadingMore props that were in the interface but previously unused. - EntityVersionPage.component.tsx: fix stale-closure bugs in fetchMoreVersions (gitar-bot review). Use versionListRef for currentOffset and isLoadingMoreRef to gate concurrent invocations so IntersectionObserver double-firing does not cause duplicate appends. - EntityResource.java: accept offset > 0 with default limit when no fieldChanged is provided, so pagination params are no longer silently ignored (Copilot review). - datamodel_generation.py: raise explicit errors if generated files or expected replacement targets are missing, instead of silently succeeding when the generator output drifts (Copilot review). * fix(checkstyle): format Java, ESLint/Prettier on UI, relax datamodel_generation strict check - Java: spotless:apply on EntityResource.java (line-break formatting). - Python: relax datamodel_generation.py DIRECT_IMPORT_FIXES check — replacement targets are alternative forms the generator may or may not emit. Only require the final marker ('from .paging import Paging') is present after replacements; the prior strict per-target check broke 'make generate'. - UI lint: organize-imports, ESLint --fix, Prettier on all version-related files touched by the PR (resolves lint-src + lint-playwright CI checks). - EntityVersionTimeLine: guard IntersectionObserver effect with isLoadingMore so the observer is torn down while a fetch is in flight (Copilot review). - EntityVersionTimeline.test.tsx: add unit tests covering sentinel rendering conditions (hasMore, onLoadMore) and the isLoadingMore observer-guard (Copilot review). * fix(ui-checkstyle): prettier+eslint on EntityVersionTimeline.test.tsx Collapse import line and reorder JSX props (callbacks last) per repo lint rules. Reruns ui-checkstyle-changed caught these in the new test file from the previous commit. * test(playwright): address @aniketkatkar97 review on PaginatedVersionHistory spec - Add waitUntil: 'domcontentloaded' to every page.goto() call. - Wait for loaders (waitForAllLoadersToDisappear) before asserting the version-button to avoid racing the initial entity render. - Replace the manual { timeout: 15_000 } on versionSelectors.nth(1) with an explicit waitForResponse on the second paginated /versions call (offset > 0). This deterministically synchronises on the infinite-scroll fetch instead of a wall-clock timeout. * fix: address Copilot review — one-shot observer + local SQL splitter 1. EntityVersionTimeLine.tsx: call observer.unobserve(entry.target) as soon as the sentinel first intersects so onLoadMore fires only once per attached observer. The effect reattaches a fresh observer after isLoadingMore flips back to false, so subsequent pages still load — we just no longer rely on the parent's in-flight ref as the sole stopgap against repeated fires for the same page. 2. MigrationSqlStatementHashTest.java: replace Flyway's non-public org.flywaydb.core.internal.* parser classes with a small, local SQL statement splitter. Handles line (--) and block comments, single-, double-, and backtick-quoted strings, backslash escapes, and doubled- quote escapes. Removes a brittle dependency on Flyway internals that could break on upgrades. Tested: - mvn test -pl openmetadata-service -Dtest=MigrationSqlStatementHashTest → 2 tests pass. - yarn test EntityVersionTimeline.test.tsx → 8/8 tests pass. --------- Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> Co-authored-by: sonika-shah <sonika-shah@users.noreply.github.com> Co-authored-by: mohitdeuex <mohit.y@deuexsolutions.com> Co-authored-by: sonika-shah <sonikashah94@gmail.com> Co-authored-by: Mohit Yadav <105265192+mohityadav766@users.noreply.github.com>
49 lines
1.9 KiB
SQL
49 lines
1.9 KiB
SQL
WITH version_metadata AS (
|
|
SELECT
|
|
e.id,
|
|
e.extension,
|
|
split_part(e.extension, '.version.', 2)::DOUBLE PRECISION AS version_num,
|
|
COALESCE(
|
|
(
|
|
SELECT jsonb_agg(field_name ORDER BY field_name)
|
|
FROM (
|
|
SELECT DISTINCT
|
|
field_change ->> 'name' AS field_name
|
|
FROM jsonb_array_elements(
|
|
COALESCE(e.json -> 'changeDescription' -> 'fieldsAdded', '[]'::jsonb)
|
|
) AS field_change
|
|
WHERE field_change ->> 'name' IS NOT NULL
|
|
AND field_change ->> 'name' <> ''
|
|
|
|
UNION
|
|
|
|
SELECT DISTINCT
|
|
field_change ->> 'name' AS field_name
|
|
FROM jsonb_array_elements(
|
|
COALESCE(e.json -> 'changeDescription' -> 'fieldsUpdated', '[]'::jsonb)
|
|
) AS field_change
|
|
WHERE field_change ->> 'name' IS NOT NULL
|
|
AND field_change ->> 'name' <> ''
|
|
|
|
UNION
|
|
|
|
SELECT DISTINCT
|
|
field_change ->> 'name' AS field_name
|
|
FROM jsonb_array_elements(
|
|
COALESCE(e.json -> 'changeDescription' -> 'fieldsDeleted', '[]'::jsonb)
|
|
) AS field_change
|
|
WHERE field_change ->> 'name' IS NOT NULL
|
|
AND field_change ->> 'name' <> ''
|
|
) AS exact_field_names
|
|
),
|
|
'[]'::jsonb
|
|
) AS changed_field_keys
|
|
FROM entity_extension AS e
|
|
WHERE e.extension LIKE '%.version.%'
|
|
)
|
|
UPDATE entity_extension AS e
|
|
SET versionNum = version_metadata.version_num,
|
|
changedFieldKeys = version_metadata.changed_field_keys
|
|
FROM version_metadata
|
|
WHERE e.id = version_metadata.id
|
|
AND e.extension = version_metadata.extension;
|