mirror of
https://github.com/open-metadata/OpenMetadata
synced 2026-05-24 09:39:11 +00:00
69b4ab57ab
Some checks failed
Java Checkstyle / java-checkstyle (push) Waiting to run
Maven Collate Tests / maven-collate-ci (push) Waiting to run
OpenMetadata Service Unit Tests / Detect Changes (push) Waiting to run
OpenMetadata Service Unit Tests / openmetadata-service-unit-tests (push) Blocked by required conditions
OpenMetadata Service Unit Tests / k8s_operator-unit-tests (push) Blocked by required conditions
OpenMetadata Service Unit Tests / openmetadata-service-unit-tests-status (push) Blocked by required conditions
Integration Tests - MySQL + Elasticsearch / Detect Changes (push) Has been cancelled
Integration Tests - PostgreSQL + Elasticsearch + Redis / Detect Changes (push) Has been cancelled
Integration Tests - PostgreSQL + OpenSearch / Detect Changes (push) Has been cancelled
Publish Package to Maven Central Repository / publish-maven-packages (push) Has been cancelled
Integration Tests - MySQL + Elasticsearch / integration-tests-mysql-elasticsearch (push) Has been cancelled
Integration Tests - PostgreSQL + Elasticsearch + Redis / integration-tests-postgres-elasticsearch-redis (push) Has been cancelled
Integration Tests - PostgreSQL + OpenSearch / integration-tests-postgres-opensearch (push) Has been cancelled
* fix(search): index usageSummary so reindex preserves Explore weekly-usage sort
TableIndex.getRequiredReindexFields() declared "columns" but not
"usageSummary". usageSummary is fields-gated in TableRepository
(clearFields nulls it unless requested), so the reindex path — which
fetches only the declared required fields — dropped it from the table
search document's _source. Explore's "Sort by Weekly Usage" reads
_source.usageSummary.weeklyStats.count, so it silently broke after any
full reindex even though live-served docs looked fine.
Add "usageSummary" to the required reindex field set so the reindexed
document carries it, matching the live entity payload.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(search): update search index on usage report (live path)
Usage is recorded via direct DAO writes in UsageRepository, bypassing
EntityRepository.update — so the entity-lifecycle SearchIndexHandler
never fires, and a reported usage never reached the search document.
The search doc kept a stale/absent usageSummary until the next full
reindex, so Explore "Sort by Weekly Usage" didn't reflect freshly
reported usage live.
After recording usage, push the refreshed entity into the search index
(updateEntity re-fetches with all fields, so usageSummary is included).
Table usage rolls up to its schema + database, so refresh those docs
too. Search failures are logged, not propagated — the usage write is
already committed.
Together with the reindex-fields change this keeps usageSummary present
in _source on both the live and reindex paths.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(search): use index required fields for usage reindex, not "*"
The live-path usage search update called SearchRepository.updateEntity,
which re-fetches the entity with getFields("*"). For the rolled-up
database/schema that hydrates every child table — tens of thousands on
large catalogs — risking OOM, the exact over-fetch the reindex-fields
work exists to avoid.
Fetch each affected entity (table, schema, database) with only its
index's required reindex fields via ReindexingUtil.getSearchIndexFields,
then updateEntityIndex(entity) directly. Mirrors the reindex pipeline's
field selection and keeps the rollup cheap and bounded.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(search): bound updateEntity field fetch to index required fields
SearchRepository.updateEntity(EntityReference) re-fetched the entity with
getFields("*") before re-indexing. This runs on every live entity update
(SearchIndexHandler.onEntityUpdated) and tag-propagation refresh, for all
entity types. For container entities (database/schema) "*" hydrates every
child — tens of thousands of tables on large catalogs — so a single live
update could OOM the server. That's a far bigger blast radius than the
usage path alone.
Fetch only the fields the entity's search index declares as required
(searchIndexFactory.getReindexFieldsFor) — the same set the reindex
pipeline uses — so live updates and reindex stay consistent and bounded.
Reverts the per-call workaround in UsageRepository.updateUsageInSearch
back to plain updateEntity calls, now that updateEntity itself is bounded.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* refactor(usage): switch expression + drop schema/database live refresh
Address PR #28350 review:
- addUsage: convert the break/mutable-variable switch to a Java 21
switch expression (no mutable response, no break boilerplate).
- updateUsageInSearch: only refresh the reported entity's search doc.
Dropped the cascade to the rolled-up schema + database — usage
reporting can be high-volume and the table doc is the surface that
matters ("Sort by Weekly Usage"); schema/database usageSummary
reconciles on the next reindex. This also removes the redundant
second table fetch and the unguarded schema/database refs the bots
flagged. The single updateEntity call is bounded (required fields).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* Use safe list
---------
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|---|---|---|
| .. | ||
| src | ||
| .swp | ||
| LEARNING_RESOURCES_DESIGN.md | ||
| lombok.config | ||
| pom.xml | ||