OpenMetadata/bootstrap/sql/migrations/native/1.12.7/postgres/schemaChanges.sql

Ignoring revisions in .git-blame-ignore-revs. Click here to bypass and see the normal blame view.

32 lines
1.6 KiB
MySQL
Raw Normal View History

Fixes #27158: ingestion slowdown from tag_usage seq-scan on Postgres (#27745) * Fixes #27158: restore tag_usage prefix-LIKE index on Postgres The 1.11.0 perf migration (#23054) added four `WHERE state = 1` partial indexes on tag_usage; #24063 dropped the matching `state = 1` predicate from getTagsInternalByPrefix (Suggested-state rows are valid for both classification and glossary derivation), leaving every partial index inapplicable. Postgres fell back to a parallel seq scan; MySQL was unaffected because its 1.11.0 indexes were never partial. Adds a non-partial single-col btree on targetfqnhash_lower (mirrors MySQL's idx_targetfqnhash_lower) and rebuilds the four partials as non-partial -- same shape, same INCLUDE columns, predicate coupling removed so future query changes can't silently invalidate them. Verified end-to-end against a local Postgres with 50k rows: seq scan reproduced before the fix (matches reporter's EXPLAIN), bitmap index scan after, both for inline and prepared-statement paths. * Move #27158 tag_usage fix to 1.12.7 + add tagfqn_lower index Ship the fix in the 1.12.7 release line so customers on 1.12.x get it without waiting for 2.0.x. Also closes the second structural gap with MySQL: a non-partial single-col btree on tagfqn_lower mirroring MySQL's idx_tagfqn_lower (1.11.0). 1.12.8 directory removed; 1.12.7's existing schemaChanges.sql now carries the tag_usage indexes alongside the entity_extension migration. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-05 05:00:11 +00:00
-- Issue #27158: tag_usage seq-scan on Postgres. #24063 dropped the
-- `state = 1` predicate that 1.11.0's partial indexes required.
-- Fix: add single-col indexes on the `_lower` columns, and drop the
-- `WHERE state = 1` filter from the partials so changes can't invalidate them.
DROP INDEX CONCURRENTLY IF EXISTS idx_tag_usage_targetfqnhash_lower_pattern;
CREATE INDEX CONCURRENTLY IF NOT EXISTS idx_tag_usage_targetfqnhash_lower_pattern
ON tag_usage (targetfqnhash_lower text_pattern_ops);
DROP INDEX CONCURRENTLY IF EXISTS idx_tag_usage_tagfqn_lower_pattern;
CREATE INDEX CONCURRENTLY IF NOT EXISTS idx_tag_usage_tagfqn_lower_pattern
ON tag_usage (tagfqn_lower text_pattern_ops);
DROP INDEX CONCURRENTLY IF EXISTS idx_tag_usage_target_prefix_covering;
CREATE INDEX CONCURRENTLY IF NOT EXISTS idx_tag_usage_target_prefix_covering
ON tag_usage (source, targetfqnhash_lower text_pattern_ops)
INCLUDE (tagFQN, labelType, state);
DROP INDEX CONCURRENTLY IF EXISTS idx_tag_usage_tagfqn_prefix_covering;
CREATE INDEX CONCURRENTLY IF NOT EXISTS idx_tag_usage_tagfqn_prefix_covering
ON tag_usage (source, tagfqn_lower text_pattern_ops)
INCLUDE (targetFQNHash, labelType, state);
DROP INDEX CONCURRENTLY IF EXISTS idx_tag_usage_join_source;
CREATE INDEX CONCURRENTLY IF NOT EXISTS idx_tag_usage_join_source
ON tag_usage (tagFQNHash, source)
INCLUDE (targetFQNHash, tagFQN, labelType, state);
CREATE EXTENSION IF NOT EXISTS pg_trgm;
DROP INDEX CONCURRENTLY IF EXISTS gin_tag_usage_targetfqn_trgm;
CREATE INDEX CONCURRENTLY IF NOT EXISTS gin_tag_usage_targetfqn_trgm
ON tag_usage USING GIN (targetFQNHash gin_trgm_ops);