* feat: security service initial code
* support secrity type for service from ui
* refactor: added missing java files for security service and require json files
* refactor: added doc file, oneOf java changes for ranger
* Merge branch 'main' of https://github.com/open-metadata/OpenMetadata into issue-20145
* update the security services schema to have domains instead of domain
* fix: added security service metadata pipeline
* Make changes for security service UI
* Update missing translation
* update type
* Fix the java checkstyle
* fix: added ingestionRunner in security service json
* Hide security service for OSS
* fix: reverse metadata documentation
* refactor: added securityConnection in test connection
* fix: documentation and ts file changes
* fix: deleted __init__ file and added SSISConnection again in the workflowclassconverter.java
* fix: convert domain to domains
* fix: added new file for basicAuth for ranger and change in rangerConnectionClassConverter.java file
* fix: java checkstyle
* fix: mysql security service sql
* fix: security service index mapping
* fix: moved migrations in new version and fix classConverterFactory
---------
Co-authored-by: Chirag Madlani <12962843+chirag-madlani@users.noreply.github.com>
Co-authored-by: Aniket Katkar <aniketkatkar97@gmail.com>
* fix(dq): enable ''Column values to be in set'' test case for boolean columns
Add BOOLEAN to supportedDataTypes array in columnValuesToBeInSet.json
to allow boolean column validation with predefined allowed values.
This enables users to enforce strict true/false validation on boolean
columns directly at the column level, resolving issue #22099.
Co-authored-by: IceS2 <IceS2@users.noreply.github.com>
* Add tests to the new feature
* Add migrations and columnValuesToBeNotInSet
---------
Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>
Co-authored-by: IceS2 <IceS2@users.noreply.github.com>
* Update databaseServiceMetadataPipeline.json
markDeletedSchemas and markDeletedDatabases should be false by default
* add migrations
---------
Co-authored-by: Pere Miquel Brull <peremiquelbrull@gmail.com>
Co-authored-by: sonika-shah <58761340+sonika-shah@users.noreply.github.com>
* - Fix Entity Relationship Cleanup to not delted entities without repositories
* - Add Migration from Test Case to Test Case Resolution
* - Typo should be resolution id
* Add Tests and Fix Condition
* Update BatchSize
* feat: implemented microstrategy lineage & dbServicePrefix
* feat: added dbServicePrefixes support in other dashboards
* fix: test_metabase and powerbi extra code remove
* fix: python checkstyle
* refactor: added prefix support for other connectors - superset, tableau, etc
* refactor: added migration for prefix change and fix dbServicePrefixes field description
* refactor: added prefix changes in superset db source
* doc: add prefix in tableau doc
* fix: typescript files and postgres migration for prefix
* fix: moved migration in 1.8.2
---------
Co-authored-by: Pere Miquel Brull <peremiquelbrull@gmail.com>
* Fix#20621: User Status Tracking in the System
* Fix#20621: User Status Tracking in the System
* Fix#20621: Add Playwright tests
* Update generated TypeScript types
* fix import
* remove unused imports and fix broken ui for settings user page
* revert path change for PersonaDetailsPage
* fix unit tests
* fix tests
* Add separate api for last login time
* update last login time api
* remove last login time patch
* remove updateLoginTime
---------
Co-authored-by: shrushti2000 <shrushtipolekar@gmail.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: sonikashah <sonikashah94@gmail.com>
Co-authored-by: karanh37 <karanh37@gmail.com>
Co-authored-by: mohitdeuex <mohit.y@deuexsolutions.com>
* Improve indexing auto tune based on heap mem, parse custer metrics for new versions of OpenSearch
* Use native bulk processors
* lower the batch size, add composite index for app_runs table
* while updated secrets, if there is entityNotFoundException ignore and move on
* Fix deadlock
* Fix deadlock
* Fix gzip headers
* remove unnecessary code to check compression, batch sizes etc..
* Improved performance
* Improved performance
* fix merge error
* fix styling
* fix tests
* Fix build issues
* improve queue size recommendations
* put max limits on threads
* fix checkstyle
* feat: prep for collate support
- Added support for delayed background jobs. This is to delete the support user after a while.
- added ScheduleType.OnlyManual for applications to support only manual triggers
- Added a four hour token expiration
- Allow a null trigger type in job data (not sure what it's for)
* added test for delayed job
* added type checking for enumCleanupArgs in navbar
* - added a support toten type
- added job handler for delete token
* fixed whitelist links
* patch OnlyManual apps
* Update openmetadata-spec/src/main/resources/json/schema/entity/applications/app.json
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* added indexes for runAt
* use NO_JOB_SLEEP_SECONDS in background job test
* Fixed NavBar test
* removed backticls
* fix indexes
* change for support app
* fix tests
---------
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Karan Hotchandani <33024356+karanh37@users.noreply.github.com>
Co-authored-by: karanh37 <karanh37@gmail.com>
Co-authored-by: Pere Miquel Brull <peremiquelbrull@gmail.com>
* Add missing workflow config status migrations
* Moved to Java based migrations to redeploy pipeline
* Moved migrations to Java. Updated Listener based on config
* Improved update on the migrations
* fix: made databricks httpPath required and added a migration file for the same
* fix: added sql migration in postDataMigration file and fix databricks tests
* fix: added httpPath in test_source_connection.py and test_source_parsing.py files
* fix: added httpPath in test_databricks_lineage.py
* fix: table name in postgres migration
* chore: added index for apps_extension_time_series.timestamp
* - added timestamp index for workflow_instance_time_series
- added timestamp index for workflow_instance_state_time_series
* Refactor: Introduce openMetadataBaseUrlConfiguration to set up the server endpoint and remove OpenMetadataUrl from smtpSettings.json
* migrations.
* refactor(cli): add setOpenMetadataUrl command to store OpenMetadata URL
* refactor(cli): add configureEmailSettings command to manage SMTP/Email configuration
* load initialData when config is not present in the db
* add om url settings page
* update locales
* add e2e
* load omBaseUrl with default host and port: localhost:8585 when not present
---------
Co-authored-by: Karan Hotchandani <33024356+karanh37@users.noreply.github.com>
Co-authored-by: karanh37 <karanh37@gmail.com>
* feat(apps): support event subscriptions
- added support for apps with event subscriptions.
- added support for custom consumers on event subscriptions.
- use native application methods instead of reflection in ApplicationHandler
* removed runMethodFromApplication reflection and use concrete methods from AbstractNativeApplication
* format
* fix(governance-workflows): defined interface in workflow schema
- defined types for workflow nodes
- fixed updateEdge method in WorkflowDefinitionRepository
* ref(governance-workflows): use explicit types in schema
use explicit interface type in json schemas and use annotations to infer the concrete classes when deserializing
* - implemented CustomSignal trigger
- use JsonTypeInfo and JsonSubTypes to infer types in jsonschema for nodes and triggers
- Implemented JsonLogicFilter task
- implemented placeholder noop task
* - simplified trigger type names
- removed governance workflow root "type"
- added "deployed" field to governance workflow
- applied changes to existing gov-workflows
- migrations
* fixed migrations
* backend
* format & tests
* rename backend
* migrations and ingestion
* format & tests
* format & tests
* tests
* format & tests
* tests
* updated ui side of changes
* addressing comment
* fixed failing unit test
* fix test list
* added e2e test, and fixed existing test
---------
Co-authored-by: Shailesh Parmar <shailesh.parmar.webdev@gmail.com>
* fix: combined index on offset and eventTime in changeEvent table
* fix: correct syntax error in Postgres query.
* fix: correct syntax error in Postgres query.
* fix: correct syntax error in Postgres query.
* fix: Update configuration for activity feed.
* fix: Migrations
* remove the customProperty and testSuite feed beautification case from the feed
* fix: Remove custom properties from the source
* fix: add metric in the matchAnySource.
---------
Co-authored-by: Ashish Gupta <ashish@getcollate.io>
* Move the alert components to the components folder from pages folder
* auto generated localization changes and linting changes
* Fix the imports
* Add AlertConfigDetails component
* Add tabs in the alert details page
* auto generated localization changes
* Apply permissions for alert and add owner update functionality
* Change the alert update method from PUT to PATCH
* Add the recent events tab
* auto generated localization changes
* Improve the recent events tab for alert details
* Implement the logic to show alert events count on the alert details page
* auto generated localization and linting changes
* Add the pagination logic for the recent events list on the alert details page
* Fix the logic for previous click in NextPreviousWithOffset
* Localization change for other languages
* Hide pagination when not required
* Add pagination and permissions to API; create eventsRecord API; fix timestamp issue
* fix: remove publisherName from test destination message.
* Fix the recent events tab error
* fix: Resolve PATCH API issue with destination ID creation.
* fix: Delete associated alert metrics upon alert deletion
* feat: Detailed HTTP Response for Destination Test
* Add the test external destination functionality
* localization changes for other languages
* fix: Set status for testing destination in EmailPublisher
* Add advanced configurations for webhook destinations
* Localization changes for other languages
* Fix the destination status not showing
Add the POST and PUT radio button in advanced configs
* Localization changes for other languages
* Update the API to fetch the alert event diagnostic counts
* Improve the AlertConfigDetails component to show the alert configs properly
* fix: Display displayName instead of fullyQualifiedName in templates
* Fix the test destination result not showing the first time
* Fix the test destination status not showing for already existing destinations
* Fix the unit tests
* Fix playwright tests
* refactor: Separate sendMessage and sendTestMessage methods; add statusContext
* fix: Use name or fqn as fallback if displayName is null or empty
* fix: upsert successfulChangeEvent
* eliminate the GovernanceWorkflowChangeEvent option from external destination
* Add code comments
* Fix the unit test
---------
Co-authored-by: Siddhant <86899184+Siddhanttimeline@users.noreply.github.com>
Co-authored-by: Siddhanttimeline <siddhanttimeline9@gmail.com>
* Initial implementation for Governance Workflows
* Remove file that shouldn't be there
* Initial implementation for the governance workflows
* Refactor JSONSchema for Governance Workflows
* Refactored WorkflowInstanceStatus to keep the stage properly
* Fix Migration for MySQL for the WorkflowInstanceStates change
* Update WorkflowInstanceState to track all the states propertly
* Initial code tro migrate Request type of Tasks
* copy governance schemas into json directory
* update schema names
* Extract WorkflowInstance into its own resource
* Update WorkflowInstance to have the workflow starting variables
* Changed relatedEntity to be an EntityLink
* First iteration on checkEntityAttributes JSONLogic
* Refactored Workflow into TriggerWorkfow and MainWorkflow
* Fixed WorkflowInstance and WorkflowInstanceState
* Fixed WorkflowInstance and WorkflowInstanceState when process is terminated
* Fix UserTask
* json logic changes to workflow
* Fix Patch request. Add exclude filter to EventBasedEntityTrigger
* update query builder
* Fix Migrations
* Fix Migrations
* modify query builder widget
* fix search props
* update widget and use default fields
* Update edges to always have a true/false condition
* made changes to provider for fields
* push progress
* fix grouping of array field
* Remove duplicated migrations
* Fix migrations
* Fix migrations
* Fix Tests and commit some improvements suggestion
* fix unit tests
* Add Certification Workflow
* Fix Tests
* Fix Tests
* fix tests
* add delay in pagination tests
* fix minor config changes for json logic
* change the glossary page
* Remove Table Certification Workflow seed Data
* add json logic and elastic search parsers
* minor schedule interval changes for workflows
* minor fixes to the parsing logic
* certification ui
* fix tests
* fix tests
* add other search fields
* Add Certification to search index
* Update index mapping for certification
* add quick filter for certification
* update locales
* Fix index mapping
* Add missing index updates
* Fix checkstyle
* increase timeout
* skipping user test
* skipping team test
---------
Co-authored-by: karanh37 <karanh37@gmail.com>
Co-authored-by: Karan Hotchandani <33024356+karanh37@users.noreply.github.com>
Co-authored-by: Sriharsha Chintalapani <harshach@users.noreply.github.com>
Co-authored-by: sonikashah <sonikashah94@gmail.com>
Co-authored-by: sonika-shah <58761340+sonika-shah@users.noreply.github.com>
* fix getSuccessfullySentChangeEventsForAlert
* migrations.
* offset-based pagination
* add startingOffset in EventSubscriptionDiagnosticInfo
* fix missing paginationOffset
* Record successfully sent change events in a separate table, clean up associated successful and failed records on alert deletion.
* Remove oldest 30 records if record count reaches 100 for successful_sent_change_events.
* list unprocessed changeEvents based on alert subscription rules, ensuring only relevant unprocessed events are listed.
* refactor: enhance eventSubscriptionDiagnosticInfo
Added new fields to the eventSubscriptionDiagnosticInfo schema to support detailed diagnostics:
- `relevantUnprocessedEventsCount`: Count of relevant unprocessed events based on alert filtering rules.
- `relevantUnprocessedEventsList`: List of relevant unprocessed events based on alert filtering rules.
- `totalUnprocessedEventsCount`: Count of all unprocessed events.
- `totalUnprocessedEventsList`: List of all unprocessed events.
* Refactor: delete 30 oldest records per alert when the count reaches 100 records
* fix: table creation issue in PostgreSQL
* fix tests.
* scheduleCleanupJob
* remove in-line deletion of records.
* fix postgres table structure issue.
* create EventSubscriptionDiagnosticInfo
* API for failedEvents
* migrations
* add index on source
* list all failedEvents optionally filtered by source
* refactor
* refactor
* getSuccessfullySentChangeEventsForAlert API
* increase defaultValue of limit to 100
* resolve conflicts
* listEvents API with 'failed', 'successful,' and 'unprocessed' query parameters
* fix description
* eventSource as enums
* refactor name.
---------
Co-authored-by: Sriharsha Chintalapani <harshach@users.noreply.github.com>
* MINOR - Renaming MetaPilot to CollateAI
* MINOR - Renaming MetaPilot to CollateAI
* MINOR - Renaming MetaPilot to CollateAI
* MINOR - Renaming MetaPilot to CollateAI
* fix
* moving limits migrations for 1.5.9
* fix(ui): pagination issue with search in services list page
* Add Apps Data store
* Revert Services Changes
---------
Co-authored-by: Chirag Madlani <12962843+chirag-madlani@users.noreply.github.com>
* Fix OpenSearch Content Size Issue
entity content is too long [204857600] for the configured buffer limit [104857600]
* Change Type to Long
* Add Payload Size to take dynamic entry
* Migrations for 1.5.6
* Mark Stale Entries Stopped
* Format checkstyle
* Fix failure
---------
Co-authored-by: Sriharsha Chintalapani <harshach@users.noreply.github.com>
* feat: indexed test case results
* feat: added indexation logic for test case results
* style: ran java linting
* fix: IDE warnigns
* chore: added test case results migration
* style: ran java linting
* fix: postgres migration column json ref
* empty commit to trigger queued
* chore: extracted test case results to its own resource
* chore: fix failing tests
* chore: move testCaseResult state from testSuite and testCase to dynamic field fetched from test case results search index
* chore: clean up test case repository
* style: ran java linting
* chore: removed testCaseResultSummary and testCaseResult state from db
* fix: test failures
* chore: fix index mapping type for result value
* chore: fix test failure
* feat: indexed test case results
* feat: added indexation logic for test case results
* style: ran java linting
* fix: IDE warnigns
* chore: added test case results migration
* style: ran java linting
* fix: postgres migration column json ref
* empty commit to trigger queued
* fix: added index on id field
* Remove owner field from json in various entities
* Add Psql migrations for owner
* Modify operator type for query
* chore: Update table_entity JSON structure to include dataModel owners
* Add domain field to entity jsons
* add subdomains ui
* Add Domain To create Request
* Typo fix
* commit progress
* allow multiple domains
* - add userDomains and teamDomains
* - Restrict Api Access
* allow multiple domain support
* - Add Listing Restricition
- Add Search Restrictions
* - Field Fix
* - Inherit Team Domains
* remove glossary filter from ui
* - Fix Tag and User Filter
* - Id -> FQN in create
* - Missing domain on indexes
* add domain in creation of team, user and glossary
* fix merge conflicts
* fix feedbacks
* fix tests
* add domain playwright
* minor changes
* fix domain fetching calls
* Add Thread Listing with EntityDomain
* Fixes for Team Join
* Mkae list mutable
* Add missing import
* Fix Team Test
* User Test case fix
* Fix User Domain
* Minor bug fixes related to sub domains
* Fix data retaining in add sub domain modal after closing
* fix NavBar unit test
* Make Domain Access Work Via Policies
* Fix errors introduced after conflict resolutions
* update render function in SubDomainsTable.component.tsx to accept an array of owners instead of a single owner
* fix java checkstyle
* fix py-test
* format
* Fix the domain e2e owner field type
* Fix Failing Tests
* fixed pytest list index
* update userDomains and teamDomains to domains
* Entity Domain to Domain for thread and changeEvent
* FIx fields
* @Ovveride
* FIx
* Remove Duplicate Field
* Remove Duplicate Field
* Fix Failing Test
* fix domain related changes
* fix subdomains loader
* Fix Owners Order
* Enforce Domain Permissions
* fix domain cleanup
* Fix Test
* rever owner change
---------
Co-authored-by: karanh37 <karanh37@gmail.com>
Co-authored-by: Aniket Katkar <aniketkatkar97@gmail.com>
Co-authored-by: Pere Miquel Brull <peremiquelbrull@gmail.com>
Co-authored-by: sonikashah <sonikashah94@gmail.com>
Co-authored-by: Sriharsha Chintalapani <harshach@users.noreply.github.com>
Co-authored-by: Onkar Ravgan <onkar.10r@gmail.com>
Co-authored-by: Karan Hotchandani <33024356+karanh37@users.noreply.github.com>
* Add multiple owners
* Multi Ownership
* Issue #17012: Multi User/Team Ownership
* Issue #17012: Multi User/Team Ownership
* Issue #17012: Multi User/Team Ownership - Fix Tests - Part 1
* Issue #17012: Multi User/Team Ownership - Fix Tests - Part 2
* Issue #17012: Multi User/Team Ownership - Fix Tests - Part 3
* Issue #17012: Multi User/Team Ownership - Fix Tests - Part 4
* Issue #17012: Multi User/Team Ownership - Fix Tests - Part 5
* Issue #17012: Multi User/Team Ownership - Fix Tests - Part 6
* Issue #17012: Multi User/Team Ownership - Fix Tests - Part 7
* Issue #17012: Multi User/Team Ownership - Fix Tests - Part 8
* Add Migrations for Owner Thread
* update ingestion for multi owner
* fix pytests
* fixed checkstyle
* Add Alert Name to Publishers (#17108)
* Add Alert Name to Publishers
* Fix Test
* Add Bound to Setuptools (#17105)
* Minor: fixed testSummaryGraph issue (#17115)
* feat: updated multi pipeline ui as per new mock (#17106)
* feat: updated multi pipeline ui as per new mock
* translation sync
* fixed failing unit test
* fixed playwright test
* fixed viewService click issue
* sorted pipeline based on test case length
* Added domo federated dataset support (#17061)
* fix usernames (#17122)
* Doc: Updated Doris & Redshift Docs (#17123)
Co-authored-by: Prajwal Pandit <prajwalpandit@Prajwals-MacBook-Air.local>
* Fix#12677: Added Synapse Connector - docs and side docs (#17041)
* Fix#17098: Fixed case sensitive partition column name in Bigquery (#17104)
* Fixed case sensitive partiion col name bigquery
* update test
* #13876: change placement of comment and close button in task approval workflow (#17044)
* change placment of comment and close button in task approval workflow
* minor change
* playwright test for the close and comment function
* supported ref in activityFeedEditor
* fix playwright test
* added playwright test for data steward
* fix the test for the data streward user
* fix the close button not showing if task has no suggestions and icon fixes
* fix sonar issue
* change glossary and add suggestion button to dropdown button
* fix the glossary failure due to button change
* icon change for add tag and description
* fix glossary cypress failure due to button chnages
* changes as per comments
* MINOR: docs links fix (#17125)
* alation link fix
* dbt yaml config source link fix
* bigquery doc fix
* Explore tree feedbacks (#17078)
* fix explore design
* update switcher icon
* show menu when search query exists
* fix selection of active service
* fix type error
* fix tests
* fix tests
* fix tests
* MINOR: Databricks view TableType fix (#17124)
* Minor: fixed AUT test (#17128)
* Fix#16692: Override Lineage Support for View & Dashboard Lineage (#17064)
* #17065: fix the tags not rendering in selector after selection in edit tags task (#17107)
* fix the tags not rendering in selector after selection in edit tags taks
* added playwright test
* minor changes
* minor fix
* fix the tags not updating in edit and accept tag
* fix explore type changes for collate (#17131)
* MINOR: changed log level to debug (#17126)
* changed log level to debug
* fixed type
* changed type to optional
* Get feed and count data of soft deleted user (#17135)
* Doc: Adding OIDC Docs (#17139)
Co-authored-by: Prajwal Pandit <prajwalpandit@Prajwals-MacBook-Air.local>
* Doc: Updating Profiler Workflow Docs URL (#17140)
Co-authored-by: Prajwal Pandit <prajwalpandit@Prajwals-MacBook-Air.local>
* fix playwright and cypress (#17138)
* Minor: fixed edit modal issue for sql test case (#17132)
* Minor: fixed edit modal issue for sql test case
* fixed test
* Minor: Added whats new content for 1.4.6 release (#17148)
* MINOR [GEN-799]: add option to disable manual trigger using scheduleType (#17031)
* fix: raise for triggering system app
* added scheduleType ScheduledOrManual
* minor: remove "service" field from required properties in createAPIEndpoint schema (#17147)
* initial commit multi ownership
* update glossary and other entities
* update owners
* fix version pages
* fix tests
* Update entity_extension to move owner to array (#17200)
* fix tests
* fix api page errors
* fix owner label design
* locales
* fix owners in elastic search source
* fix types
* fix tests
* fix tests
* Updated CustomMetric owner to entityReferenceList. (#17211)
* Fix owners field in search mappings
* fix search aggregates
* fix inherited label
* Issue #17012: Multi User/Team Ownership - Fix Tests - Part 9
* Fix QUeries
* Fix Mysql Queries
* Typo
* fix tests
* fix tests
* fix tests
* fix advanced search constants
* fix service ingestion tests
* fix tests
---------
Co-authored-by: mohitdeuex <mohit.y@deuexsolutions.com>
Co-authored-by: Onkar Ravgan <onkar.10r@gmail.com>
Co-authored-by: Mohit Yadav <105265192+mohityadav766@users.noreply.github.com>
Co-authored-by: Ayush Shah <ayush@getcollate.io>
Co-authored-by: Shailesh Parmar <shailesh.parmar.webdev@gmail.com>
Co-authored-by: k.nakagaki <141020064+nakaken-churadata@users.noreply.github.com>
Co-authored-by: Prajwal214 <167504578+Prajwal214@users.noreply.github.com>
Co-authored-by: Prajwal Pandit <prajwalpandit@Prajwals-MacBook-Air.local>
Co-authored-by: Suman Maharana <sumanmaharana786@gmail.com>
Co-authored-by: Ashish Gupta <ashish@getcollate.io>
Co-authored-by: harshsoni2024 <64592571+harshsoni2024@users.noreply.github.com>
Co-authored-by: Karan Hotchandani <33024356+karanh37@users.noreply.github.com>
Co-authored-by: Mayur Singal <39544459+ulixius9@users.noreply.github.com>
Co-authored-by: Imri Paran <imri.paran@gmail.com>
Co-authored-by: sonika-shah <58761340+sonika-shah@users.noreply.github.com>
Co-authored-by: Sachin Chaurasiya <sachinchaurasiyachotey87@gmail.com>
Co-authored-by: karanh37 <karanh37@gmail.com>
Co-authored-by: Siddhant <86899184+Siddhanttimeline@users.noreply.github.com>
* Fix Users Issue due to capital names
* Fixed Tests
* No lowercasing name
* Revert
* toLowerCase
* fix email casing from UI side on basic auth
* compare lowercase email only from the loggedInUser
* address comments
* Updated LowerCase UserName on updates
* revert ui changes
* Migrate Name and Email
* cypress failure
* fix glossary test
* fix activity feed test
* Compare by Lowercasing Email and username
---------
Co-authored-by: Chirag Madlani <12962843+chirag-madlani@users.noreply.github.com>
Co-authored-by: Chira Madlani <chirag@getcollate.io>
Co-authored-by: karanh37 <karanh37@gmail.com>
* fix(#15683): backend validation error for the following property types
* feat: Update supported date and datetime formats
* test: add e2e playwright test for custom properties
* chore: remove cypress test for custom properties
* Add date, dateTime, time backend support for custom property
* fix: update custom property display names in AddCustomProperty component
* feat: Update supported date and time formats
* chore: Update supported date and time formats
* feat: Update date and time formats in CustomProperty
* test: add playwright test
* Update supported date and time formats
* feat: Add tests for custom properties in Playwright in 2 parts
* Update supported date and time formats
* Update supported date and time formats
* chore: Update date format to uppercase in PropertyValue component
* feat: Add support for custom date format in PropertyValue component
* Update supported date and time formats
* test: add playwright test for time, dateTime, timeInterval and date properties
* Update supported time formats
* chore: Add focus to time and date pickers before filling values
* chore: Add focus to time and date pickers before filling values
* Remove date, dateTime, time from type_entity
---------
Co-authored-by: sonikashah <sonikashah94@gmail.com>
* User Email Based lookup for loggedInUser
* Add Unique on name
* support email only login
* fix cypress
* fix playwright
* reset user for 404
inject enableSelfSignup for auth config
* Add Self Signup Flag
* Add error messages
* Better check
* Change to Custom Exception Message
* Update message as suggested
* fix claim issue for confidential type
* fix sonar
---------
Co-authored-by: Chira Madlani <chirag@getcollate.io>
* Added Test Case for Matching Enum
1. Implemented the test case using the `matchEnum` parameter.
2. Added integration tests.
3. Added migrations.
* fix tests
* fixed tests
* format
* fixed tests
* clear search cache before running ingestion
* format
* changed scopt of aws fixture
* moved migrations to 1.5.0
* Activity feed card ui fixes
* Fix feed card for test suite
* fix testCase and testSuite feed card title and links
* Fix the profile picture not loading in the feeds
* fix the custom property and bot icon
* fix the deleted feed card by adding opacity on it
* Fix feed card hover effect
Add edit and delete button for posts
* - Fix Activityt feed event
* - Ignore workflow
* - Ignore pipelienStatus as well
* fix test case card feed
* localization keys
* sonar fix
* changes made as per comments and fix sonar issue
* Update schemaChanges.sql
---------
Co-authored-by: Mohit Yadav <105265192+mohityadav766@users.noreply.github.com>
Co-authored-by: Aniket Katkar <aniketkatkar97@gmail.com>
Co-authored-by: mohitdeuex <mohit.y@deuexsolutions.com>
* Fixes Test Suite Reference in Table Schema
* fix: fix test suite to interact with entity reference
---------
Co-authored-by: Teddy Crepineau <teddy.crepineau@gmail.com>
* Add predefined color palette for UI styling assets - backend
* Update UI components to use customLogoConfig in applicationConfig
* Update UI components to use customLogoConfig and customTheme in applicationConfig
* Update UI components to use customLogoConfig and customTheme in applicationConfig
* Update UI components to use customLogoConfig and customTheme in AppearanceConfigSettingsPage.tsx
* Update UI components to use customLogoConfig and customTheme in AppearanceConfigSettingsPage.test.tsx
* Remove existing customLogoConfiguration
* Refactor App.tsx and ThemeUtils.ts files
* Update AppearanceConfigSettingsPage.tsx with new imports and default theme values
* Update AppearanceConfigSettingsPage.tsx with new imports and reset functionality
* add cypress test
* Fix color update issue in CustomThemeConfig.spec.ts
* Remove existing customLogoConfiguration -migration changes
* Fix theme configuration initialization in useApplicationStore.ts
* Update CustomThemeConfig.spec.ts to check if the color is updated
* update existing customLogoConfiguration with customUiThemePreference -migration changes
* Update CustomThemeConfig.spec.ts to fix color update check
* Add unit tests for ThemeUtils
* move validateJsonSchema method to JsonUtils
---------
Co-authored-by: Sachin Chaurasiya <sachinchaurasiyachotey87@gmail.com>
* feat(nosql-profiler): row count
1. Implemented the NoSQLProfilerInterface as an entrypoint for the nosql profiler.
2. Added the NoSQLMetric as an abstract class.
3. Implemented the interface for the MongoDB database source.
4. Implemented an e2e test using testcontainers.
* added profiler support for mongodb connection
* doc
* use int_admin_ometa in test setup
* - fixed linting issue in gx
- removed unused inheritance
* moved the nosql function into the metric class
* feat(profiler): add dynamodb row count
* feat(profiler): add dynamodb row count
* formatting
* fixed import
* format
* dded dynamodb row count
* format
* removed unused factory file
* removed "validate"
* migrations
* removed validations
* format
* linting
* fixed: test_amundsen.py
* Update schemaChanges.sql
* feat: add validation field to test case definition parameters
* feat: added parameter validation logic
* fix: set ES testcontainer startup attemp to 3
* style: ran java linting
* fix: added waitForEsAsyncOp in post_trigger_app_200 test
* Application cannot be installed with None as schedule type
* MINOR: Add / Fix GCS and ADLS - docs, bugs (#15502)
Add GCS and ADLS docs
* Add stack trace while throwing an error to debug (#15522)
* Fix#15533: Fix name & display name for kafka json schema parser (#15534)
* MINOR: postgres add ssl options in yaml (#15538)
* Small change to fix the Glossary TErm Tasks not fetching Owner automatically (#15535)
* remove the DataInsightAlert spec as the alerts test are covered in ObservabilityAlerts spec (#15531)
* MINOR: fix help dropdown item font and icon sizes (#15511)
* fix help dropdown item font and icon sizes
* added unit test for navbarUtils
* fix sonar failure
* changes as per comments
* minor fixes
* Minor: fixed DQ edit test case issue and searchIndexDetails typescript issue (#15528)
* Minor: fixed DQ edit test case issue and searchIndexDetails typescript issue
* fixed failing unit test
* fixed unit test for Data quality test
* reverting e2eLabeler changes
* chore(ui): separate routes as per categories (#15512)
* chore(ui): seprate routes as per categories
* add tests
* fix test
* domain path fix
* [MINOR] GX logging hierarchy (#15542)
* fix: GX module logging hierarchy
* style: ran python linting
* - Use Entity Type (#15546)
* removed docs 1.4.x (#15550)
* Suggestions Alert new design (#15532)
* update ux of suggestions alert
* locales
* minor changes
* fix descriptions
* minor css fixes
* Fix#11868: Duplicated queries cannot be created (#15519)
* Fix#11868: Duplicate query should throw an error of entityExists
* Fix#11868: Duplicate query should throw an error of entityExists
* fix test
* fix test
* Fix uniquee constraint for checksum in Postgres
---------
Co-authored-by: Pere Miquel Brull <peremiquelbrull@gmail.com>
* added OpenMetadataApplication.getDao (#15549)
* Add the entityUrl to the header for ThreadMessages (#15552)
* feat: AppResource (#15555)
forbid modification of system app
* MINOR: Skip source hash generation for service (#15516)
* sync the documentation roadmap page with the getcollate roadmap (#15551)
* Application cannot be installed with None as schedule type
* None type should not register in scheduler
* scheduleTimeline fix
* fix payload of schedule type none and change it to scheduleTimeline
* fix unit test
---------
Co-authored-by: Ashish Gupta <ashish@getcollate.io>
Co-authored-by: Ayush Shah <ayush@getcollate.io>
Co-authored-by: Mayur Singal <39544459+ulixius9@users.noreply.github.com>
Co-authored-by: IceS2 <pjt1991@gmail.com>
Co-authored-by: Aniket Katkar <aniketkatkar97@gmail.com>
Co-authored-by: Shailesh Parmar <shailesh.parmar.webdev@gmail.com>
Co-authored-by: Chirag Madlani <12962843+chirag-madlani@users.noreply.github.com>
Co-authored-by: Teddy <teddy.crepineau@gmail.com>
Co-authored-by: Mohit Yadav <105265192+mohityadav766@users.noreply.github.com>
Co-authored-by: Imri Paran <imri.paran@gmail.com>
Co-authored-by: Karan Hotchandani <33024356+karanh37@users.noreply.github.com>
Co-authored-by: Pere Miquel Brull <peremiquelbrull@gmail.com>
Co-authored-by: mohitdeuex <mohit.y@deuexsolutions.com>
* Fix#11868: Duplicate query should throw an error of entityExists
* Fix#11868: Duplicate query should throw an error of entityExists
* fix test
* fix test
* Fix uniquee constraint for checksum in Postgres
---------
Co-authored-by: Pere Miquel Brull <peremiquelbrull@gmail.com>
* chore: clean up TestCaseResource tests
* fix: added status and type query params for test cases
* style: ran java linting
* fix: refactor addCondition method in ListFilter
* style: ran java linting
* - Added Jdbc Store for Quartz Job
- Added tables schema for Mysql and Postgresql
* Catch Object Already Exists
* Drop tables extra statements
* Use Proper Dialects
* format cron from unix to quartz for internal apps , for others use unix
* Add Migration Files for 131
* Add Migrations
* Fix Json Issue and Add Exception Handling for force
---------
Co-authored-by: Ashish Gupta <ashish@getcollate.io>
* Fix#14786: Suggestions API
* Handle suggestions in ometa
* Minor: Optimise Databricks Client (#14776)
* MINOR - Fix SP topology context & Looker usage context (#14816)
* MINOR - Fix SP topology context & Looker usage context
* MINOR - Fix SP topology context & Looker usage context
* Fix tests
* Fixes#14598: Fix Tags / Labels ingestion on includeTags as False (#14782)
* fix(ui): password error message for char limits (#14808)
* fix(ui): password error message for char limits
* fix java side code
* Fixes#13556: Support for Salesforce table description ingestion (#14733)
* ISSUE-13556: Add suport for Salesforce table description ingestion
* ISSUE-13556: Remove unnecessary blank line
* ISSUE-13556: Fix to get description for each table
---------
Co-authored-by: Teddy <teddy.crepineau@gmail.com>
* MINOR - Better handling of Ingestion Pipeline Status (#14792)
* MINOR - Better handling of Ingestion Pipeline Status
* format
* format
* MINOR: Added table validation for cost analysis data (#14793)
* Added validation for cost analysis source
* centralized life cycle logic
* CYPRESS: simplify side navigation click in cypress (#14818)
* simplify side navigation click in cypress
* make sidbar item uses common enum
* fix cypress failure of outside import
* fix(#14326): tier dropdown is not working in advance search (#14780)
* improvement in advance search based on custom property
* fix a reading undefined property issue
* wip: advance search based on tier
* some code cleanup and improvement
* some fixes
* fix: ui flicker when advanceSearched is apply and refresh the page
* some cleanup
* no need to call customproperty api call, if entity not suppport customProperties
* minor change
* fix: autocomplete not working in tier search option in advance search modal
* added unit test for advance search provider component
* some cleanup
* added testcase for open modal
* added testcase for resetAllFilters method
* removed unwanted code
* added e2e test for testing tier advance search
* fix: e2e search flow for single field
* fix: string field not working after giving listValues in TierSearch
* fix: group query e2e test fix
* used asyncFetch way to get the tierOptions synchronously
* some cleanup
* remove unwanted lines
* some cleanup
* fix: selected option show option value instead of option title
* fix(minor): update skip icon for executions (#14809)
* Fixes#14803: ignore capitalization when confirming deletes (#14804)
* ignore case when confirming deletes
* Test confirmation of deletes works when case differs
Added test case for 'delete' as the confirmation text.
* minor(config): update openmetadata-ui code reviewers (#14823)
* Add Tests
* Add list/accept/reject apis
* initial ui changes
* localisation
* show suggestion for empty description
* ui feedbacks
* Fix permission check for entities without owner
* Fix entityLink and add tests
* Add update suggestion WIP
* Fix test
* Fix PUT and Pagination
* Fix styling
* update test
* Update status
* add OM server connection in apps
* add permissions check
* Fix CI
* Remove TODO
* Fix feedResourceTest
* fix unit tests
* add private configs for apps
* add private configs for apps
* fix update application icons
* minor center align icon
* add private configs for apps
* Format
* Fix pydantic gen
* Remove token
* Update name
* Rework private conf
* Fix apps
* Fix apps
* Format
* Format
* show metapilot only if its installed
---------
Co-authored-by: Pere Miquel Brull <peremiquelbrull@gmail.com>
Co-authored-by: Mayur Singal <39544459+ulixius9@users.noreply.github.com>
Co-authored-by: Ayush Shah <ayush@getcollate.io>
Co-authored-by: Chirag Madlani <12962843+chirag-madlani@users.noreply.github.com>
Co-authored-by: kwgdaig <18678754+kwgdaig@users.noreply.github.com>
Co-authored-by: Teddy <teddy.crepineau@gmail.com>
Co-authored-by: Onkar Ravgan <onkar.10r@gmail.com>
Co-authored-by: Ashish Gupta <ashish@getcollate.io>
Co-authored-by: Abhishek Porwal <80886271+Abhishek332@users.noreply.github.com>
Co-authored-by: Carlo Q <carlo@machina.bio>
Co-authored-by: karanh37 <karanh37@gmail.com>
* Centralize EventType
* Format better for readability
* Add Handling for Thread as Change Event
* Centralize logic for Entity Message Creation
* Add Thread Parent Message in Conversation
* Add Task and Announcement
* Support for multiple destinations
* Delete existing alerts for schema changes
* Delete all tables data for event subscription
* Centralize logic for Notifications and Observability
* Change Field Name and Remove redundant code
* Test Fixes
* Add more fields
* Add Optional
* Logical Test Case Addition Chnage Event
* Add Filter By Owner
* Fix Email Entity Url
* added data observability settings page
* localization changes for other languages
* Handler closure and cleanup
* complete create observibility flow
* fix searchIndex method error
* Update Entity Observability Filter
* working observability creation
* complete creation flow
* fix action with arguments
* fix form validations and form data formulation for 'input.actions' field
* localization changes for other languages
* Fixes for Data Observability
* Add mention notification
* Only apply mention to conversation
* fIx test case
* bug fixes for add and edit observability alert form functionality
* fIx test case
* add feature to config internal along with external destinations in alert form
* Fix Formatting
* Fix Url Redirection
* add observability alert details page
* localization changes for other languages
* remove filter and action required rules
* fix styling for observability form and details page
* add delete functionality on observability alert details page
* update view page layout and add a warning in configure destination step
* localization changes for other languages
* feat(ui): support notification with new flow (#14848)
* feat(ui): support notification with new flow
* fix notifications
* add missing filters for notification alert creation flow
* update the filter hiding logic to show at the start always
* add check to show Actions section in observability alert creation form
* code cleanup and improvements
* add breadcrumb to add notification alert page
* Add General Metadata Filters
* remove code smells and code cleanup for add notification page
* add new filter
* show activity feed alert on notifications list page
* add notification alert details page changes
* localization changes for other languages
* show icons for all the entities in the alert trigger dropdown
* redirect to notification alert listing page after deleting the notification alert
* change add alert page title
* add logic to disable all destinations except email for users
* fixed unit tests
* code improvements and cleanups
---------
Co-authored-by: mohitdeuex <mohit.y@deuexsolutions.com>
Co-authored-by: Mohit Yadav <105265192+mohityadav766@users.noreply.github.com>
Co-authored-by: Aniket Katkar <aniketkatkar97@gmail.com>
* Centralize EventType
* Format better for readability
* Add Handling for Thread as Change Event
* Centralize logic for Entity Message Creation
* Add Thread Parent Message in Conversation
* Add Task and Announcement
* Support for multiple destinations
* Delete existing alerts for schema changes
* Delete all tables data for event subscription
* Centralize logic for Notifications and Observability
* Change Field Name and Remove redundant code
* Test Fixes
* Add more fields
* Add Optional
* Logical Test Case Addition Chnage Event
* Add Filter By Owner
* Fix Email Entity Url
* Handler closure and cleanup
* Remove unnecessary field
* Remove unnecessary field
* Support query comments in MSSQL
* Remove unnecessary field
* Format
* Add external type
* Add MSSQL SP support
* Remove LMAX Disruptor , use queue and Quartz
* Spotless Fix
* Spotless formatting
* Review Comments
* Improvements
* Improvements to add Retry and DLQ
* Spotless fix
* Update Stats
* Fix stats
* Fix delete Failures
* Fix Failing Tests
* Fix Failing Tests
* Fix Failing Tests
* Postgres Fix
* Fix Schema Change for Change Event Table
* Failing test
* Fix Error for non ChangeDescriptions
* Adds Observability On Top of Event Subscription
* Make List Separate
* Add more filters for Test Suite
* Populate Default Schema Updates to Look for
* typo
* Add matchFieldChange and other in paramAdditionalContext
* Review Comments
* Spotless fix
* Merge issue
* Update Filter as per new Syntax
* Fix Failing Test
* Add field and index
* MINOR - Move test case incident to the top of the data_quality_data_time_series table
* Fix test
* Fix compile
* Format
* Update incidentId
* Rename field
* Fix patch
* Add new OpenMetadataSetup command line application to migrate/deploy and re-index
* Add new OpenMetadataSetup command line application to migrate/deploy and re-index
* Add deployPipelines option
* Add reIndex option
* add subcommands
* add provision to store upgrade metrics
* rename bootstrap script
* fix styling checks
* Add changelog and store metrics into SERVER_CHANGE_LOG
* Cast jsonb
* Cast jsonb
---------
Co-authored-by: Pere Miquel Brull <peremiquelbrull@gmail.com>
* feat: add test case resolution task workflow
* chore: add migration for test case resolution feature
* fix: removed required field for object compatibiity in older migrations
* fix: minor testCaseResolution status logic
* chore: revert migration for test case incident
* chore: update migration file
* style: renamed variables
* feat: added logic to compute failed/passed rows
* feat: add support for row level computation in schema
* chore: add test definition migration
* feat: add logic to explicitly compute row level failure
* chore: clean up code
* style: fix java
* style: fix pyton format
* fix: unhidde API for incident manager
* style: fix java styling
* feat: add test case resolution task workflow
* chore: add migration for test case resolution feature
* fix: removed required field for object compatibiity in older migrations
* fix: minor testCaseResolution status logic
* chore: revert migration for test case incident
* chore: update migration file
* chore: clean up code
* style: fix java stylecheck
* Fix supported characters in SM
* Update SM
* Fixes
* Fixes
* Improve class conversion exceptions
* Comments
* Rename noop to db secrets manager providee
* Update sm
* Fix
* db SM
* db SM
* Fix test
* UI
* Update openmetadata-ui/src/main/resources/ui/src/mocks/IngestionListTable.mock.ts
* update default
* refactor: entityFQN as ListFilter condition
* feat: implement resolution entity timeseries
* fix: rename to testCaseResolutionStatus
* ref: extracted ES query builder into private method
* ref: extract OS query builder in its own method
* ref: remove ingestion logic for test case resolution
* fix: reorganize json schemas to fix circular import in Python
* ref: object names in typescript code
* feat: added indexing of test case resolution
* feat: added test case resolution sample data
* fix: test case resolution api logic
* fix: audit logger for entityTimeSeriesInterface
* fix: DDL generation
* style: python linting
* fix: skip UI test case resolution tests
* fix: remove extension field
* fix: renamed testCaseFailureStatus to testCaseResolutionStatus
* fix: remove reviewer
* fix: rename sequenceId to stateId
* fix: re adjust search weights
* fix: removed InReview status
* style: ran python linting
* Clean Airflow Lineage Backend and migrate status to millis
* Format
* chore(ui): update executions startTs and endTs to millis
* Remove lineage providers
---------
Co-authored-by: Sachin Chaurasiya <sachinchaurasiyachotey87@gmail.com>
* Remove duplicates from entity_extension_time_series and add the constraing if missing
* Add sort buffer and work mem
* Revert "Add sort buffer and work mem"
This reverts commit fcfff5feb6.
* Fix#12655: Customizable Landing Page - Add Personas
* Add Knowledge Panels
* Add Doc Store to store KnowledgePanels and Pages
* Add Doc Store to store KnowledgePanels and Pages schemas
* Fix tests
* Add Multiple Persona support for a user
* add more elaborate tests for docstore
* Fix Page and KnowledgePanel required fields
* Fix Page and KnowledgePanel required fields
* Fix Page and KnowledgePanel required fields
* Fix build
