* Fix MySQL timestamp precision for tag_usage.appliedAt
MySQL's TIMESTAMP type defaults to second precision, while PostgreSQL
returns microsecond precision. This causes _normalize_datetime_strings
in the Python ingestion client to produce spurious appliedAt diffs in
JSON patches, which then fail with "Failed to convert JsonValue to
target class" during deserialization in JsonUtils.applyPatch().
Upgrade appliedAt to TIMESTAMP(6) to match PostgreSQL behavior and
eliminate the spurious patch diffs.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* Add 1.11.8 migration for MySQL appliedAt timestamp precision
Backport the TIMESTAMP(6) fix to the 1.11.x release line so existing
deployments on 1.11.x pick up the fix without requiring a 1.12.0 upgrade.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
* Add design doc for search indexing stats redesign
Covers:
- Simplified 4-stage pipeline model (Reader, Process, Sink, Vector)
- Per-entity index promotion instead of batch promotion
- Alias management from indexMapping.json
- Payload-aware vector bulk processor
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* Add Support for Per Entity Index Promotion
* Add UI Bit
* Add Lang
* Add AppLog View Test coverage
* Add Bathced Vector index querying
* Add Improvements for Vector to be async and also stats to be better handled
* Use Virtual Thread
* Use Virtual Thread
* Fix Tests
* Make reading stats easier
* Fixed Stats to be accurate
* Fix Stats getting null
* Fix partition worker stats
* Fix Reader Stats - final
* Update generated TypeScript types
* Make updates in 1.12.0
* Revert "Use Virtual Thread"
This reverts commit 4eb23374d1.
* Revert "Use Virtual Thread"
This reverts commit efe8d03b5d.
* Reapply "Use Virtual Thread"
This reverts commit d59cde18b2.
* Reapply "Use Virtual Thread"
This reverts commit 769e5710c3.
* Fix Final Update on stat
* - Add atomic alias swap
- remove unnecessary migration
* Fix Sonar test jest
* Fix Final Update on stat
---------
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
* Ensure columns are retrieved in the right order
This is because since introducing ordering for `getTableColumnsByFQN`, the patches created in `removeTagFromEntity` were open to pointing to different columns if the default order didn't match how they were persisted in db
* Allow exception list to be updated on all feedback
* Apply gitar comments
* Add `metadata` to `tag_usage` table
* Update JSON schema object to include `TagLabel.metadata`
* Apply feedback to selected recognizer
* Add backend integration tests
* Update `ingestion` to return `TagLabel.metadata.recognizer`
* Update generated TypeScript types
* Update generated TypeScript types
* Send recognizer result metadata in feedback approval task (#25485)
* Send `TagLabelRecognizerMetadata` in `TaskDetails`
This is so we can show an explanation behind the classification in the feedback approval card
* Update typescript types
* Run Spotless
* Ensure `applyTagsBatchInternal` works equally for pg and mysql
* Tag metadata fixes
* Fix CI test
---------
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Rohit Jain <60229265+Rohit0301@users.noreply.github.com>
Co-authored-by: Pere Miquel Brull <peremiquelbrull@gmail.com>
* Fix Stats
* Add Warning logs and reindex failure analysis
* Add Search Insights in Preferences
* Add Label
* Fix Full Error not available
* Add check for reindex run
* FIX#24374 - Data Contract at Data Product level
* Update generated TypeScript types
* FIX#24374 - Data Contract at Data Product level
* fix DP page
* fix: preserve termsOfUse object format in filtered contract
The termsOfUse field was being converted to a string during filtering,
but the form components expect it to be an object with {content: string}.
This was causing test failures where form elements were not visible.
- Keep termsOfUse as object format when not inherited
- Convert old string format to new object format for consistency
- Fixes 21 test failures in DataContracts.spec.ts and DataContractInheritance.spec.ts
* fix: address code review findings - state sync and immutability
Frontend changes:
- Add useEffect to sync formValues with filteredContract changes
- Ensures edit form updates when contract prop changes
Backend changes:
- Create deep copy at start of mergeContracts() to avoid mutating input
- Prevents side effects if contract object is reused elsewhere
Co-authored-by: pmbrull <pmbrull@users.noreply.github.com>
* Addressing feedback
Co-authored-by: pmbrull <pmbrull@users.noreply.github.com>
* fix tests
* fix inherited contract delete and status
* fix inherited contract delete and status
* fix inherited contract execution in app
* fix test
* fix: resolve playwright postgresql ci test failure
Co-authored-by: pmbrull <pmbrull@users.noreply.github.com>
* ci: fix yaml validation and checkstyle failures
Co-authored-by: pmbrull <pmbrull@users.noreply.github.com>
* fix: correct JSON/YAML validation errors
Co-authored-by: pmbrull <pmbrull@users.noreply.github.com>
* fix: resolve maven-collate and ui-coverage test failures
Co-authored-by: pmbrull <pmbrull@users.noreply.github.com>
* gitar feedback
* fix ci
* fix ci
* fix ci
* fix ci
* include .claude
* validate
* fix playwright
* playwright
* fix playwright
---------
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Gitar <gitar@collate.io>
Co-authored-by: Gitar <noreply@gitar.ai>
Co-authored-by: pmbrull <pmbrull@users.noreply.github.com>
Co-authored-by: Karan Hotchandani <33024356+karanh37@users.noreply.github.com>
Co-authored-by: karanh37 <karanh37@gmail.com>
* feat: added repository logic to list all versions (including latest) for a specific entity type
* feat: added list all versions for all the entity resources
* feat: moved endpoint to EntityResource
* feat: renamed endpoint to /history and methods to EntityHistory
* feat: ran java linting
* feat: remove v1 implementation left over code
* feat: fix failing tests
* feat: ran klinting
* feat: fix psql query
* feat: address PR comments
* feat: ran klinting
* feat: increase cache duration
* feat: address query edge cases
* fix(dq): psql migration for row insert test parameters
* fix(dq): use name and add trailing new line
* Fix description formatting in postDataMigrationSQLScript.sql
* Apply suggestion from @Copilot
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
---------
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* Add DQ Rules Library
* Add DQ Rules Library - Add Tests and enable testDefinitions through APIs to list
* Add DQ Rules Library - Add Tests and enable testDefinitions through APIs to list
* Add DQ Rules Library - Add Tests and enable testDefinitions through APIs to list
* Update generated TypeScript types
* Add DQ Rules Library - Add Tests and enable testDefinitions through APIs to list
* Add DQ Rules Library - Add Tests and enable testDefinitions through APIs to list
* Update generated TypeScript types
* Refactor tests to use toStrictEqual for string comparisons and improve consistency
- Updated various test files to replace `toBe` with `toStrictEqual` for string assertions in ImportStatus, SummaryCard, TabsLabel, and others.
- Enhanced regex tests to ensure accurate validation of entity names and tags.
- Added new translations for test platform warnings in en-us.json.
- Improved utility tests for alerts, authentication, CSV handling, and task messages to use `toEqual` for better clarity.
* Refactor TestDefinitionForm and TestDefinitionList components to use updated API methods and improve SQL expression handling
* Enhance TestDefinitionList component with permission checks for edit and delete actions, and update tests to reflect changes in permission handling
* Remove debug log from handleSubmit in TestDefinitionForm component
* Add permission loading state and enhance permission handling in TestDefinitionList component
* Update generated TypeScript types
* Update generated TypeScript types
* Update generated TypeScript types
* fix build failure
* Revert "Update generated TypeScript types"
This reverts commit 67b062216f.
* Enhance TestDefinitionForm and TestDefinitionList components with improved UI and pagination handling
* fix: update RulesLibrary tests and enhance TestDefinitionForm styling
* fix: Enhance TestDefinitionForm with error handling and improved UX
* fix: Update test definition handling and improve rendering in TestDefinitionList
* fix: Refactor TestDefinitionPermissions tests for improved permission checks and API context handling
* fix: Update system test definition retrieval to use findLast for improved accuracy
* feat: Add end-to-end tests for Rules Library and Test Definition Permissions
* fix: Update edit button visibility check to use beDisabled for better clarity
* fix: Refactor response handling in TestDefinitionPermissions tests for improved reliability
* move migrations execution order
* fix: remove existing columns
* style: remove migration extra line break
* chore: fix migration
---------
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Shailesh Parmar <shailesh.parmar.webdev@gmail.com>
Co-authored-by: TeddyCr <teddy.crepineau@gmail.com>
* Added `appliedAt` field to `TagLabel`s
This is to track insertions to `tag_usage` with timestamps
* Capture and format recognition explanations in `TagAnalyzer`
This creates a function to build an explanation to why something was scored as it was.
# Conflicts:
# ingestion/src/metadata/pii/algorithms/presidio_utils.py
* Refactor `TagProcessor`
* Capture results for the old-style `PIIProcessor`
* Move strings to constants
* Add `TagLabel.appliedBy` field
This change also patches the user's name into the tags declared in `JsonPatch` objects to fill it up
* Update typescript types
* Fix python tests
* Fix java tests
* Simplify setting tag's `appliedBy` using `EntityUpdater.updatingUser`
* Remove unnecessary f-string
* Moar fixes
* Move migrations to 1.11.5
* Fix#23853: AI Governance and Compliance Framework for AI Applications
* Update generated TypeScript types
* Update generated TypeScript types
* trigger ci
* Fix#23853: AI Governance and Compliance Framework for AI Applications
* Fix test failures
* Merge origin/main into ai_agents - added pipeline execution features and resolved conflicts
* Update generated TypeScript types
---------
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Pere Miquel Brull <peremiquelbrull@gmail.com>
* Oh boy, factory-boy
Created a bunch of `factory-boy` factories that help creating mock test data easily
* Update `try_bind` docker utility to ease debugging
* Resolve conflicts between `Classification` tags
* Refactor `TagClassifier` into another entity
This is so:
1. We're not tied to the `ColumnClassifier` interface that forced returning `Mapping[T, float]` (unnecessary since we're returning `List[ScoredTag]`
2. The tag analyzer uses the same `recognizer_factories` registry we used for `PIIProcessor`
3. Create a separate service that abstracts using `TagScorer` and `TagAnalyzer` to return `TagScore`s (makes testing upstream code easier)
* Interface to retrieve available `Tag`s and `Classification`s
* Refactor `TagProcessor` to support multi-classification
- Depends `ClassificationManagerInterface` to retrieve `Tag`s and `Classification`s
- Uses a callable dependency to score tags for a column
- Accepts a classification filter parameter
- Leverages `ConflictResolver` to resolve conflicts between tags of the same `Classification`
* Add an integration test for the `TagProcessor`
* Ensure `PII` classification is configured with migrations
# Conflicts:
# bootstrap/sql/migrations/native/1.11.1/mysql/postDataMigrationSQLScript.sql
# bootstrap/sql/migrations/native/1.11.1/postgres/postDataMigrationSQLScript.sql
* Move `FakeClassificationManager` to `_openmetadata_testutils`
This is because importing from `tests` breaks in the CI when running pytests from the root of the repo
* Fix broken mutually exclusive classifications
This is because the implementation did not take into account previous tags when resolving conflicts.
This caused that running the classifier twice for a classification, with a mutually exclusive configuration, would end up breaking the exclusivity
* fix: add supportedServcices for relavnt service DQ display
data diff is not supported by all services. We need to only
display it on supported services
* fix: added query param and create filed
* Revert deleting Old Deployments for Periodic batch Workflows
* Revert "Revert deleting Old Deployments for Periodic batch Workflows"
This reverts commit 7bd1be5a81.
* TRUNCATE FLOWABLE history tables in both 1.10.5 and 1.10.7 migrations
* TRUNCATE FLOWABLE history tables in both 1.10.5 and 1.10.7 migrations
* fix: migration
* fix: playwright test DBT -> dbt
* feat: added rentention for profile and dq data
* feat: fix failing tests
* feat: address error in postgres delete sql
* feat: fixed missing parameter in psql query
* fix: added the deletion step in test case
* feat: fixed postgres query for deletion before cutoffs
* Fix: Search Slowness when painless scripts aggregates for terms and classifications
* Fix Sql
* Add fields to security service index
---------
Co-authored-by: mohitdeuex <mohit.y@deuexsolutions.com>
* Remove data placed in the wrong directory
* Update `MigrationUtil` to use data from `piiTagsWithRecognizers`
That way we can also remove duplicate json and have a single source of data
* Update migration queries to use prepared statements
* Minor fix in the `piiTagsWithRecognizers` definitions
* Initial implementation for Dimensionality on Data Quality Tests
* Fix ColumnValuesToBeUnique and create TestCaseResult API
* Refactor dimension result
* Initial E2E Implementation without Impact Score
* Dimensionality Thin Slice
* Update generated TypeScript types
* Update generated TypeScript types
* Removed useless method to use the one we already had
* Fix Pandas Dimensionality checks
* Remove useless comments
* Implement PR comments, fix Tests
* Improve the code a bit
* Fix imports
---------
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
* Draft Implementation of Custom Workflows
* Multiple Entities in the Same Trigger for Workflow, along with draft implementation of signal id
* Improved User Approval Task and Impl
* Custom Workflows - Draft Implementation 2, improved periodic batch entity filter and other improvements
* feat(governance): Implement transactional custom workflows - improved
This commit introduces a robust, transactional, and extensible framework for custom governance workflows in OpenMetadata.
Key features and improvements include:
Transactional Workflow Management: A new WorkflowTransactionManager ensures atomic operations for creating, updating, and deleting workflow definitions, maintaining consistency between the OpenMetadata database and the Flowable engine.
Safe ID Encoding: Implemented a WorkflowIdEncoder to generate safe, Base64-encoded, and collision-resistant IDs for Flowable processes, preventing errors from ID truncation.
Rollback and Deprecation Tasks:
Added RollbackEntityTask to revert entities to their last approved state.
Introduced DeprecateStaleEntityTask for automated lifecycle management of stale assets.
Enhanced Workflow Engine:
Improved WorkflowHandler to validate workflow definitions before deployment.
Added new custom functions to the rule engine for checking entity update timestamps and calculating field completeness scores.
CI/CD and Build Improvements:
Updated the CI Dockerfile with a multi-stage build and refined dependency installation.
Modified POM files to include necessary dependencies for new features.
* Adding DataCompleteness Task Node, Flowable Debug logs
* Transaction handling for Custom Workflow - Initial Draft
* add new tasks to node definition interface
* Update generated TypeScript types
* Draft Implementation of Multi Reviewer Approval Task with Migration
* Update generated TypeScript types
* Transaction handling fix, id truncation fix by migration, feed repo fix for multi reviewer pattern, copilot comments
* Update generated TypeScript types
* Fixed Multi Reviewer approval to take consideration of the namespaced variables, Fixed RollBackEntity task to follow subprocess like other automated tasks, copilot nitpicks
* Remove conditionalSetEntityAttributes as it is not needed anymore
* Update generated TypeScript types
* Completely remove the setConditionalAttributes to fix compilation errors
* Removed the comments in the schemaChanges
* Created a new Task called CreateDetailedApprovalTaskImpl, Fixed RollBackEntityTask to roll back to either Approved or Rejected state, use namespaced variables, Updated the workflow handler to resolve the tasks and remove them from the user who has approved the task in the feed during multi reviewer approval, TransactionManager updated to keep Transactions in place, Improve the Validation Node input in the Mainworkflow java to handle proper graph traversal, Find Proper Termination Message for the two user events that are conflicting, Include Message in the request approval thread task for proper messages
* Update generated TypeScript types
* Rendering of messages in task thread
* Fix PeriodicBatchEntityTriggerTask to separate workflows based on entities Draft, Fix: SetEntityAttributes Impl to handle multiple fields, Fix: DataCompletenessTask Draft
* Fix DataCompletenessTask BoundaryEvent in Flowable
* Introduced Wf Deployment Strategy for transaction Handling, Improved user tasks for better termination event naming, fix periodic batch entity trigger to fetch the entity instead of all the entities in the trigger, Migrated GlossaryApprovalWorkflow.json with new nodes and edges, Fixed test cases and some edge cases
* Update generated TypeScript types
* Added performTasks for TagRepository and DataProductRepository, Removed test api, removed unnecessary change from workflow json, improved DataCompletenessImpl, Improved CreateDetailedApprovalTaskImpl to show what updated and not updated in the thread response
* Remove Entity Certification and Entity Status tasks and make use of generic set entity attribute task
* Update generated TypeScript types
* Fix the compilation issues!
* Remove setCertification and setEntityAttributes from createWorkflowDefinition
* Test cases for custom workflows related to glossaryTermApprovalWorkflow
* Test cases for custom workflows
* Changed info to debug logs
* Update generated TypeScript types
* DetailedUserApprovalTask changed to reviewChange task, Have validations for workflows where user approval tasks can only be available for entities that support reviewers
* Fix compilation issues and mvn spotless apply
* Update generated TypeScript types
* Remove Extra assignees from user tasks
* Update generated TypeScript types
* Replace Tags/GlossaryTerms during Mutual Exclusivity and Append when there is no Mutual Exclusivity
* Workflow Transaction Manager to handle authorization as part of its functionality, Added Validation for workflows using /validate endpoint
* Increase the Flowable's polling time to 60 seconds to poll timer and async jobs table
* Update generated TypeScript types
* Enum for UserTask Termination Messages, Removed WorkflowUtils to use WorkflowHandler's terminateDuplicate Instances, Approval and rejecters list in the variables, using namespaced variables for updatedBy
* Reverted the enum for userTaskType for now
* Added new tests for dataContract, dataProduct and Tag for user approval tasks
* Glossary Approval Workflow changed to handle jsonLogic according to UI expectations
* Added a new Task type for change description tasks to review and suggest the changes, Added checkUpdatedByReviewer incase of perfom tasks, increase wf test timeout
* Update generated TypeScript types
* TaskWorkflow constructor public to be used by Collate Repo
* AutoApproveServiceTaskImpl incase the assignees are not there for the userTask with ExlusiveGateway builder with hasAssignees
* Fix Compilation Issues - Upgrade of deps to lang3
* ExclusiveGatewayBuilder set exclusive as true, and other minor changes for test and approval tasks
* Added a different wait mechanism for user tasks in WorkflowDefinitionResourceTest.java
* Combined UserApprovalTask and ChangeReviewTask into one to handle suggestions in the same task!
* Update generated TypeScript types
* Test Case Approval Workflow, Removing orphaned changeReviewTask, Test case fix and new test cases
* Update generated TypeScript types
* Treat empty strings, arrays as missing by default
* Update generated TypeScript types
* fix compilation issues by changing the schema properly
* Remove Stale ChangeReview Task
* Update generated TypeScript types
* Enhanced User Approval task to show changed fields along with what changed and the task resolve function as well
* Update generated TypeScript types
* Add Knowledge center page to workflows event consumer, remove legacy backward compatibility for triggers, lenient validations for workflows when there are no nodes
* EntitySpecific Filters for EventBased Trigger, Removing Backward Compatibility logic for filters, Removed unnecessary comments and execution variables, Handle Structured task resolver for dataProduct, dataContract, tag and testCase, Modified GlossaryApprovalWorkflow.json and improved the MigrationUtil
* Update generated TypeScript types
* Bring back entitycertification and glosarystatus task for backward compatibility
* Update generated TypeScript types
* Filter is a map, entity specific filters are stringified, have certification and glossary status tasks in nodeinterface
* Merge Main
* Remove Suggestions for RequestApproval
* Update generated TypeScript types
* Remove Old Deployments of Periodic Batch Entity Trigger and use hiphen to trigger to avoid accidental triggering
* Resolve Merge conflicts, Java Checkstyle
* Update generated TypeScript types
* Fix Migrations
* Added alter table queries in 1.6.0 to avoid cached plan queries in flowable
* Increase timeout in workflow definition resource test
* Increase polling in workflow definition resource test
* Fix java Checkstyle
* comment the flaky test
* COmmented out Flaky Test Cases, fixed a bug on team reviewers
* Resolve java checkstyle after resolving conflicts
* add updated at field in json logic for glossary
* update fields
* add version field
* Update the rule config of new term node with "and" and update the migration as well
* add equal not equal op
* Delete the trigger workflows using like condition
* Defensive Fallback for multiple task instances created by race condition, Terminate duplicate instances of main workflows
* reverting operators
* Approval Capabilities for Metrics
* Update generated TypeScript types
* Move Migrations to 1.10.1 from 1.10.0
* Removed the extra spaces in Migration 1.10
---------
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: karanh37 <karanh37@gmail.com>
* Update `TableDiffParamsSetter` to move data at table level
This means that `key_columns` and `extra_columns` will be defined per table instead of "globally", just like `data_diff` expects
* Update `TableDiffValidator` to use table's `key_columns`
Call `data_diff` and run validations using each table's `key_columns`
* Create migration to update `tableDiff` test definition
* Fix Playwright test
* Add the migration classes and data for recognizers
This is so that we can run a migration that sets `json->recognizers` of `PII.Sensitive` and `PII.NonSensitive` tags from json values.
The issue with normal migrations was that the value of recognizers was too long to be persisted in the server migrations log.
Created a common `migration.utils.v1110.MigrationProcessBase`
* Ensure building automatically with the right parameters
* Update typescript types
* Add `reason` property to `TagLabel`
This is to understand what score was used for selecting the entity
* Build `TagLabel`s with `reason`
* Increase `PIIProcessor._tolerance`
This is so we correctly filter out low scores from classifiers while still maintaining the normalization that filters out confusing outcomes.
e.g: an output with scores 0.3, 0.7 and 0.75, would initially filter the 0.3 and then discard the other two because they're both relatively high results.
* Make database and DAO changes needed to persist `TagLabel.reason`
* Update generated TypeScript types
---------
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
* Add support for translations in multi lang
* Add Tag Feedback System
* Update generated TypeScript types
* Fix typing issues and add tests to reocgnizer factory
* Updated `TagResourceTest.assertFieldChange` to fix broken test
This is because change description values had been serialized into strings and for some reason the keys ended up in a different order. So instead of performing String comparison, we do Json comparisons
---------
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Eugenio Doñaque <eugenio.donaque@getcollate.io>
* Domain Only Access Role to be evaluated as part of PolicyEvaluator
* Domain Only Access Role to be evaluated as part of PolicyEvaluator
* Cleanup unnecessary comments
* Add migration fix for domain only policy
* fix playwright domain rbac
* fix hasDomain role issues for TestCaseResourceContext
* allow user to create only with their domain
* move DomainOnlyAccessPolicy migrations to 1.9.10 from 1.9.9
* For resources that don't support domains (like DataInsights), always returns true
* fix adding test case to bundleSuite
* revert supportsDomains changes in RuleEvaluator and handle it in postFiltering steps
* Inherit domains when creating task from test case incident, and skip few entities for domain check + other remaining DQ related domain fixes
* for CreateResourceContext consider assigned domains + inherited domains
* fix permission for table metrics page
* remove SKIP_DOMAIN_CHECK_ENTITY_LIST
* fix test
---------
Co-authored-by: sonika-shah <58761340+sonika-shah@users.noreply.github.com>
Co-authored-by: sonikashah <sonikashah94@gmail.com>
Co-authored-by: Ayush Shah <ayush@getcollate.io>
* feat: databricks oauth and azure ad auth setup
* refactor: add auth type changes in databricks.md
* fix: test after oauth changes
* refactor: unity catalog connection to databricks connection code
* Fix post profiler time series migration
* add prefix index
* Update prefix index length for entityFQNHash
* Update prefix index length for LIKE queries
* add prefix index
* fix mysql syntax
* Initial Implementation of Custom Metrics Measurement Units
* Update generated TypeScript types
* Removed Regex patterns and length validations as they are not needed
* Add a new column with index for custom units
* Remove comments in the sql
* update ui and add playwright
* fix metric selector
* fix tests
* address feedbacks
* remove unused field
---------
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Karan Hotchandani <33024356+karanh37@users.noreply.github.com>
Co-authored-by: karanh37 <karanh37@gmail.com>
* MINOR WIP - tag_usage performance
* Update EntityRepository.java
remove unnecessary call
* update query and indexes
* Merge 1.9.3 migration queries from main branch
- Add performance indexes for tag_usage and entity_relationship tables
- Add entity relationship update for dataContract relations
- Merge queries from both main and tag-usage-perf branches
Co-authored-by: Pere Miquel Brull <pmbrull@users.noreply.github.com>
---------
Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>
Co-authored-by: Pere Miquel Brull <pmbrull@users.noreply.github.com>
* fix change event handling for alerts
* contract is deleted when asset is deleted
* add support for custom properties
* Update generated TypeScript types
* handle suite index deletion
* validate owner is not coming back if not requested
---------
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
* Add migrations to update widget descriptions
* move to postDataMigrationSQLScript.sql
* move to postDataMigrationSQLScript.sql
* move to postDataMigrationSQLScript.sql
* feat: security service initial code
* support secrity type for service from ui
* refactor: added missing java files for security service and require json files
* refactor: added doc file, oneOf java changes for ranger
* Merge branch 'main' of https://github.com/open-metadata/OpenMetadata into issue-20145
* update the security services schema to have domains instead of domain
* fix: added security service metadata pipeline
* Make changes for security service UI
* Update missing translation
* update type
* Fix the java checkstyle
* fix: added ingestionRunner in security service json
* Hide security service for OSS
* fix: reverse metadata documentation
* refactor: added securityConnection in test connection
* fix: documentation and ts file changes
* fix: deleted __init__ file and added SSISConnection again in the workflowclassconverter.java
* fix: convert domain to domains
* fix: added new file for basicAuth for ranger and change in rangerConnectionClassConverter.java file
* fix: java checkstyle
* fix: mysql security service sql
* fix: security service index mapping
* fix: moved migrations in new version and fix classConverterFactory
---------
Co-authored-by: Chirag Madlani <12962843+chirag-madlani@users.noreply.github.com>
Co-authored-by: Aniket Katkar <aniketkatkar97@gmail.com>
* fix(dq): enable ''Column values to be in set'' test case for boolean columns
Add BOOLEAN to supportedDataTypes array in columnValuesToBeInSet.json
to allow boolean column validation with predefined allowed values.
This enables users to enforce strict true/false validation on boolean
columns directly at the column level, resolving issue #22099.
Co-authored-by: IceS2 <IceS2@users.noreply.github.com>
* Add tests to the new feature
* Add migrations and columnValuesToBeNotInSet
---------
Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>
Co-authored-by: IceS2 <IceS2@users.noreply.github.com>
* Update databaseServiceMetadataPipeline.json
markDeletedSchemas and markDeletedDatabases should be false by default
* add migrations
---------
Co-authored-by: Pere Miquel Brull <peremiquelbrull@gmail.com>
Co-authored-by: sonika-shah <58761340+sonika-shah@users.noreply.github.com>
* - Fix Entity Relationship Cleanup to not delted entities without repositories
* - Add Migration from Test Case to Test Case Resolution
* - Typo should be resolution id
* Add Tests and Fix Condition
* Update BatchSize
