Commit graph

4 commits

Author SHA1 Message Date
Teddy
8986523ea6
chore: realign main migration with 1.11.1 branch (#24938) 2025-12-22 09:03:28 +01:00
Eugenio
c89bc2adac
Allow multiple classifications in TagProcessor (#24545)
* Oh boy, factory-boy

Created a bunch of `factory-boy` factories that help creating mock test data easily

* Update `try_bind` docker utility to ease debugging

* Resolve conflicts between `Classification` tags

* Refactor `TagClassifier` into another entity

This is so:
1. We're not tied to the `ColumnClassifier` interface that forced returning `Mapping[T, float]` (unnecessary since we're returning `List[ScoredTag]`
2. The tag analyzer uses the same `recognizer_factories` registry we used for `PIIProcessor`
3. Create a separate service that abstracts using `TagScorer` and `TagAnalyzer` to return `TagScore`s (makes testing upstream code easier)

* Interface to retrieve available `Tag`s and `Classification`s

* Refactor `TagProcessor` to support multi-classification

- Depends `ClassificationManagerInterface` to retrieve `Tag`s and `Classification`s
- Uses a callable dependency to score tags for a column
- Accepts a classification filter parameter
- Leverages `ConflictResolver` to resolve conflicts between tags of the same `Classification`

* Add an integration test for the `TagProcessor`

* Ensure `PII` classification is configured with migrations

# Conflicts:
#	bootstrap/sql/migrations/native/1.11.1/mysql/postDataMigrationSQLScript.sql
#	bootstrap/sql/migrations/native/1.11.1/postgres/postDataMigrationSQLScript.sql

* Move `FakeClassificationManager` to `_openmetadata_testutils`

This is because importing from `tests` breaks in the CI when running pytests from the root of the repo

* Fix broken mutually exclusive classifications

This is because the implementation did not take into account previous tags when resolving conflicts.

This caused that running the classifier twice for a classification, with a mutually exclusive configuration, would end up breaking the exclusivity
2025-12-10 07:26:12 -08:00
Teddy
d5c6e5b19a
ISSUE #24020: add supportedServcices for relavnt service DQ display (#24706)
* fix: add supportedServcices for relavnt service DQ display

data diff is not supported by all services. We need to only
display it on supported services

* fix: added query param and create filed
2025-12-05 15:40:33 +01:00
Ram Narayan Balaji
1bec15ea39
Feat: #24424 Flowable History Timed Cleanup and Ops Command for cleanup (#24367)
* Flowable History Timed Cleanup and Ops Command for cleanup

* Remove Commented Code

* Update generated TypeScript types

* Fix Migration column name

* Test Cases

* Remove cleanupAll parameter

* cron for default every sunday 12 am

* cron for default every sunday 12 am - 0 0 0 ? * 1

* Move Migrations to 1.11.1

---------

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2025-12-05 16:35:23 +05:30