OpenMetadata/bootstrap/sql/migrations/native/2.0.0/postgres/postDataMigrationSQLScript.sql
Sriharsha Chintalapani 51ecf4502f
Task redesign (#25894)
* Task Redesign: Add Task entity & tests

* Task Redesign: Add Task entity & tests

* Task Redesign: Add Permissions checks for Task APIs

* Task UI changed to the new APIs

* Migrate UI and APIs to new tasks system inlcuding suggestions

* Add Suggestions integration

* Activity Feed Refactor

* ActivityFeed -> ActivityStream publisher

* Activity Feed redesign

* Activity Feed redesign, adding tests

* Incident Manager update

* Migrate Incidents to new tasks

* Migrate Incidents to new tasks

* Update generated TypeScript types

* Update generated TypeScript types

* feat(tasks): add domain-aware task cutover and workflow v2 migration

* test(tasks): cover domain filters and task feed visibility flows

* Address comments

* Fix workflow tests to use new Task entity API and fix UserApprovalTaskV2 candidate transformation

Migrated 9 WorkflowDefinitionResourceIT tests from legacy Feed/Thread API to the new
Task entity API (UserApprovalTaskV2 creates Task entities, not Thread entities). Fixed
a bug in UserApprovalTaskV2 where candidates were passed as raw EntityReferences instead
of being transformed into users/teams FQN arrays for SetApprovalAssigneesImpl.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Fix tests

* refactor: stabilize task entity workflows

* refactor: finish task entity cutover and activity migration

* refactor: migrate legacy thread feed during cutover

* refactor: split legacy thread rename and archive migrations

* Merge main; fix tests

* Update generated TypeScript types

* feat: advance task redesign through phase 2

* Merge main; fix tests

* Update generated TypeScript types

* Fix failing tests

* Update generated TypeScript types

* fininsh phase 6 of the design, configurable task forms

* Update generated TypeScript types

* Update generated TypeScript types

* Fix linting

* Address gitar comments

* Address gitar comments

* Fix build

* Address giar comments

* fix build

* Add task custom forms

* Fix tests

* Address tests

* Apply UI lint autofixes

* Fix tess

* Fix linter

* Fix task patching

* Fix tests

* Fix playwright tests

* fix java checkstyle

* Add python sdk support for tasks, annoucements

* Fix playwright tests

* Fix playwright tests

* Fix playwright tests

* Fix python tests

* Fix python tests

* Fix linting workflows

* fix pycheck

* fix pycheck

* Fix tests

* Fix build

* Address deviations from main and fix tests

* Fix integration tests

* Fix integration tests

* Fix integration tests

* Update generated TypeScript types

* Fix Playwright tests

* Fix Playwright tests

* feat(incident): wire incident manager to task-first architecture (#27369)

* feat(incident): wire incident manager to task-first architecture

Connect the incident manager to the task redesign so it works
end-to-end: resolve data persistence, backward transitions,
reopen from resolved, and incident discovery via TCRS.

* Update generated TypeScript types

* refactor: single-query incident task lookup with parameterized statuses

Replace two sequential queries (Open, InProgress) in
getOrCreateIncident with one findByAboutAndTypeAndStatuses
query using @BindList for status IN (...).

---------

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>

* Fix Playwright tests

* Update generated TypeScript types

* Fix linter

* Fix tests

* Fix tests

* Fix checkstyle

* Fix tests

* Fix checkstyle

* Update FeedResourceIT.java

* Update TableRepository.java

* fix tests

* Update ActivityFeedProvider.tsx

* fix tests

* fix tests

* Address Task comments

* Fix unit test

* Fix the feed summary panel showing on landing page

* Fix comment functionality

* Fix pytests

* Fix failing playwright tests

* Fix test flakiness

* Fix ui-checkstyle

* Fix advanced search spec failure

* Fix playwright tests

Co-authored-by: Copilot <copilot@github.com>

* Fix checkstyle

* Fix the flaky tests

Co-authored-by: Copilot <copilot@github.com>

* fix checkstyle

* Reduce the workflow polling

* Update generated TypeScript types

* skip failing tests

Co-authored-by: Copilot <copilot@github.com>

* Fix ui-checkstyle

---------

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Aniket Katkar <aniketkatkar97@gmail.com>
Co-authored-by: IceS2 <pablo.takara@getcollate.io>
Co-authored-by: karanh37 <karanh37@gmail.com>
Co-authored-by: Karan Hotchandani <33024356+karanh37@users.noreply.github.com>
Co-authored-by: Copilot <copilot@github.com>
2026-04-23 15:52:30 +02:00

84 lines
3.9 KiB
SQL

-- Post data migration script for Task System Redesign - OpenMetadata 2.0.0
-- This script runs after the data migration completes
-- =====================================================
-- NOTE: Suggestion migration (suggestions → task_entity),
-- thread-based task migration (thread_entity → task_entity),
-- and legacy system activity migration
-- (thread_entity generated feed rows → activity_stream)
-- are handled in Java MigrationUtil because they require
-- entity-link aware transformation logic.
-- =====================================================
-- =====================================================
-- PHASE 2D: Migrate announcements from thread_entity → announcement_entity
-- =====================================================
INSERT INTO announcement_entity (id, json, fqnhash)
SELECT
json->>'id' AS id,
jsonb_build_object(
'id', json->>'id',
'name', 'announcement-' || (json->>'id'),
'fullyQualifiedName', 'announcement-' || (json->>'id'),
'displayName', NULLIF(json->>'message', ''),
'description', COALESCE(
json->'announcement'->>'description',
json->>'message',
''
),
'entityLink', json->>'about',
'startTime', (json->'announcement'->>'startTime')::bigint,
'endTime', (json->'announcement'->>'endTime')::bigint,
'status', CASE
WHEN (json->'announcement'->>'endTime')::bigint < (extract(epoch from now()) * 1000)::bigint
THEN 'Expired'
WHEN (json->'announcement'->>'startTime')::bigint > (extract(epoch from now()) * 1000)::bigint
THEN 'Scheduled'
ELSE 'Active'
END,
'createdBy', json->>'createdBy',
'updatedBy', COALESCE(json->>'updatedBy', json->>'createdBy'),
'createdAt', (json->>'threadTs')::bigint,
'updatedAt', COALESCE((json->>'updatedAt')::bigint, (json->>'threadTs')::bigint),
'deleted', false,
'version', 0.1,
'reactions', COALESCE(json->'reactions', '[]'::jsonb)
) AS json,
md5('announcement-' || (json->>'id')) AS fqnhash
FROM thread_entity t
WHERE json->>'type' = 'Announcement'
AND NOT EXISTS (
SELECT 1 FROM announcement_entity a WHERE a.id = t.json->>'id'
)
ON CONFLICT (id) DO NOTHING;
-- =====================================================
-- PHASE 2E: Rename legacy thread storage to fail stale references
-- =====================================================
ALTER TABLE IF EXISTS thread_entity RENAME TO thread_entity_legacy;
-- =====================================================
-- PHASE 2F: Lower workflow trigger polling intervals
-- =====================================================
-- Reduce WorkflowEventConsumer poll interval from 10s to 1s.
-- The legacy 10s default added up to a 10s wait between an entity change and the
-- workflow-triggered approval task being created. On CI under resource pressure this
-- often drifted to >2 minutes when combined with Flowable's 60s async job poll. The
-- new value keeps the trigger pipeline near-real-time.
UPDATE event_subscription_entity
SET json = jsonb_set(json, '{pollInterval}', '1'::jsonb)
WHERE name = 'WorkflowEventConsumer'
AND (json->>'pollInterval')::int > 1;
-- Lower Flowable async/timer job acquisition intervals to keep workflow-driven
-- task creation responsive. The previous 60s default was a Flowable production setting
-- carried over verbatim; for OpenMetadata's interactive task UX we want sub-second pickup.
UPDATE openmetadata_settings
SET json = jsonb_set(
jsonb_set(json, '{executorConfiguration,asyncJobAcquisitionInterval}', '1000'::jsonb),
'{executorConfiguration,timerJobAcquisitionInterval}', '5000'::jsonb)
WHERE configtype = 'workflowSettings'
AND json->'executorConfiguration' IS NOT NULL
AND ((json->'executorConfiguration'->>'asyncJobAcquisitionInterval')::int > 1000
OR (json->'executorConfiguration'->>'timerJobAcquisitionInterval')::int > 5000);