Commit graph

617 commits

Author SHA1 Message Date
Pere Miquel Brull
abcadb8a1d
FIX - K8s Operator Release CI (#25785) 2026-02-10 10:48:41 +01:00
Sriharsha Chintalapani
6f577656c1
Fix integration tests (#25753)
* Fix - disk space in github workflows

* Fix - disk space in github workflows

* Fix - disk space in github workflows

* Fix running tests with bulk apis

* Fix running tests with bulk apis

* Address comments; make awaitability for tests

* Address comments
2026-02-08 21:16:28 -08:00
Sriharsha Chintalapani
c89b7ee757
Fix disk space (#25750)
* Fix - disk space in github workflows

* Fix - disk space in github workflows

* Fix - disk space in github workflows
2026-02-08 12:18:16 -08:00
Sriharsha Chintalapani
dbddaa9f07
Fix disk space (#25749)
* Fix - disk space in github workflows

* Fix - disk space in github workflows
2026-02-08 12:10:31 -08:00
Sriharsha Chintalapani
1caa3e8e88
Fix - disk space in github workflows (#25748) 2026-02-08 11:43:33 -08:00
Chirag Madlani
d84ff8ebe2
WIP - chore(ci): update playwright workflow to run ingestion with dataAsset… (#24753)
* chore(ci): update playwright workflow to run ingestion with dataAssetRules

* limit setup to required steps

* rearrnage spec files

* update condition

* update workflow

* update runners

* update console

* update config

* update arrangements

* fix typo

* rearrange specs

* remove unwanted import

* fix tests

* fix tests issues

* fix comment

* fix failing tests

* fix advance search

* fix test arrangements

* fix failing tests
2026-02-06 17:57:35 +05:30
Eugenio
ac56fb7690
Fix: Replace maven-surefire-plugin with maven-failsafe-plugin for integration tests (#25663)
This change addresses two critical issues with TagRecognizerFeedbackIT and the integration test suite:

1. **Test Execution Flow**: Replaced maven-surefire-plugin with maven-failsafe-plugin to ensure all integration tests run even when individual tests fail. Failsafe runs tests in the `integration-test` phase and validates results in the `verify` phase, allowing complete test coverage while still failing the build on test failures.

2. **Test Flakiness**: Fixed race conditions and timing issues in TagRecognizerFeedbackIT:
   - Replaced Thread.sleep() with Awaitility-based workflow readiness check
   - Increased timeout from 2 to 3 minutes to accommodate CI server load
   - Decreased poll interval from 5 to 3 seconds for faster failure detection
   - Enhanced error handling with detailed exception messages
   - Fixed compilation error by using WorkflowHandler.isDeployed() instead of non-existent getWorkflow()

3. **GitHub Workflows**: Updated CI workflows to use `mvn verify` instead of `mvn test` and corrected report paths from `surefire-reports` to `failsafe-reports`.

Changes applied to all Maven profiles: mysql-elasticsearch, postgres-opensearch, postgres-elasticsearch, mysql-opensearch, and postgres-rdf-tests.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Claude <noreply@anthropic.com>
2026-02-03 18:31:04 +01:00
Pere Miquel Brull
5002f82669
CI - Slack link monitor w/ playwright (#25641)
* CI - Improve Slack Monitor

* CI - Check links with playwright
2026-01-30 10:23:52 +01:00
Pere Miquel Brull
62f1d5d168
CI - Improve Slack Monitor (#25639) 2026-01-30 09:56:48 +01:00
Ayush Shah
3db4b1b75b
Fix environment variable reference for PLAYWRIGHT_BQ_PROJECT_ID in workflow files (#25602) 2026-01-29 14:20:15 +05:30
Shailesh Parmar
d4f8d11c99
chore: Exclude playwright documentation and doc-generator paths from various CI workflow triggers. (#25568) 2026-01-28 05:12:03 +00:00
Shailesh Parmar
0422260729
feat: Introduce GitHub Actions workflow for automated Playwright E2E documentation updates. (#25545)
* feat: Introduce GitHub Actions workflow for automated Playwright E2E documentation updates and update README.

* feat: refine Playwright E2E docs workflow and documentation

* addressing review comment
2026-01-27 18:46:07 +05:30
Shailesh Parmar
6d68360439
docs: Update Playwright test documentation to reflect new scenarios and add a deflake prompt. (#25544) 2026-01-27 12:44:19 +05:30
IceS2
f562bf5fca
Guarantee sonar report is sent even if the pytests fail (#25417) 2026-01-21 08:29:38 +01:00
IceS2
5702d5b069
MINOR: Fix sonar coverage (#25276)
* Fix Sonar Coverage

* Tests

* Fix Sonar Cloud Coverage Upload

* test

* Fix Sonar Cloud Coverage Upload

* Add Sonar Cloud Nightly Run for Ingestion

* Fix Sonar Cloud Coverage Upload

* Fix Sonar Cloud Coverage Upload

* Test

* Fix test branch name

* Undo Makefile testing changes

* Fix Sonar Coverage
2026-01-16 08:35:00 +01:00
Pere Miquel Brull
fa4373054e
Finish K8sPipelineClient Implementation (#25172)
* config cleanup

* add missing configs

* fix auto pilot

* fix lifecycle

* fix logs and tests

* fix test

* move integration tests

* fix

* fix

* Address code review feedback

- Fix UsageWorkflowConfig to set stageFileLocation instead of queryLogFilePath
- Add error handling for parseInt in IngestionLogHandler to catch NumberFormatException

* fix

* fix lifecycle

* prepare cronOMJob

* remove PR target

* fix

* fix

* fix

* fix

* fix

* fix tests

* fix review

* fix review

* fix review

* fix

---------

Co-authored-by: Gitar <gitar@gitar.ai>
Co-authored-by: Gitar <noreply@gitar.ai>
Co-authored-by: pmbrull <pmbrull@users.noreply.github.com>
2026-01-15 08:17:55 +01:00
IceS2
e9a5bd219e
Add Sonar Cloud Nightly Run for Ingestion (#25293) 2026-01-14 12:05:33 +01:00
miriann-uu
c998a31490
fix: Add cleanup for runner (#25277) 2026-01-14 10:29:46 +05:30
Adrià Manero
4d615245a2
fix: add disk space cleanup to maven postgres tests workflow (#25264) 2026-01-13 21:31:21 +05:30
Sriharsha Chintalapani
2c8a45d2a8
Upgrade to Dropwizard 5x and Jetty 12.1 (#24776)
* Add support for Dropwizard 5.0 and Jetty 12.1.x

* Dropwizard 5x and Jetty 12.1 upgrade

* Fix test behavior

* Fix rdf tests

* revert enableVirtualThreads

* fix tests

* Fix Tests

* Fix tests

* Switch to jersey-jetty-connector for Jetty 12 compatibility

- Replace jersey-apache-connector with jersey-jetty-connector
- Jersey 3.1.4+ jersey-jetty-connector supports Jetty 12.0.x+
- Use JettyConnectorProvider and JettyHttpClientSupplier for HTTP client
- Keep reasonable timeouts (30s connect, 2min read) to prevent CI hangs
- Set SYNC_LISTENER_RESPONSE_MAX_SIZE for large responses

This fixes the 1,093 InterruptedException test failures caused by
using the default Jersey client (HttpURLConnection-based) which doesn't
handle concurrent test execution properly.

* Fix: Start Jetty HttpClient before use

Jetty 12 HttpClient implements LifeCycle and must be explicitly
started with httpClient.start() before use. This fixes the 163
InterruptedException test failures.

* Fix: Force jetty-client to 12.1.1 for jersey-jetty-connector

jersey-jetty-connector brings transitive jetty-client:12.0.22 but
Dropwizard 5.0 uses Jetty 12.1.1. The ClientConnector.newTransport()
API changed between 12.0.x and 12.1.x, causing NoSuchMethodError.

Fix: Exclude transitive jetty-client and add explicit 12.1.x dependency.

* Use Java 11+ HttpClient connector for tests (jersey-jnh-connector)

Switch from the broken jersey-jetty-connector (incompatible with Jetty 12.1.x)
to jersey-jnh-connector which uses Java's built-in java.net.http.HttpClient.
This connector:
- Natively supports all HTTP methods including PATCH
- Works with Java 21
- No external dependencies required
- Avoids compatibility issues with Jetty versions

* Use Apache HttpClient 5.x connector for tests (jersey-apache5-connector)

Switch from jersey-jetty-connector (incompatible with Jetty 12.1.x)
to jersey-apache5-connector which uses Apache HttpClient 5.x.
This connector:
- Supports all HTTP methods including PATCH
- Lenient with empty PUT request bodies
- Has proper timeout support to prevent indefinite hangs
- Works with Jetty 12.1.x

* Fix  tests

* Fix  docker compose

* Fix tests

* Fix tests - make url compatible

* Add URL parsing

* Fix URL decode

* fix tests

* fix test

* fix tests

* Fix integration with new dropwizard-5x changes

---------

Co-authored-by: Karan Hotchandani <33024356+karanh37@users.noreply.github.com>
Co-authored-by: karanh37 <karanh37@gmail.com>
Co-authored-by: Mohit Yadav <105265192+mohityadav766@users.noreply.github.com>
2026-01-12 12:18:29 -08:00
Shailesh Parmar
a283702a4f
Upgrade Playwright from 1.51.1 to 1.57.0 (#24330)
* Upgrade Playwright from 1.51.1 to 1.54.2

- Updated @playwright/test package to version 1.54.2 in package.json
- Updated Playwright browser installation to 1.54.2 in all GitHub workflow files:
  - mysql-nightly-e2e.yml
  - playwright-mysql-e2e.yml
  - playwright-postgresql-e2e.yml
  - postgresql-nightly-e2e.yml
- Updated yarn.lock to reflect the new Playwright version and its dependencies

Using 1.54.2 instead of 1.56.x to avoid regression bug with "Internal error: step id not found"
that affects fixtures in Playwright 1.55+ (see microsoft/playwright#37147).

This upgrade brings stable Playwright features, improvements, and bug fixes.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* fix: update Playwright tests and utility functions for improved query handling and response validation

* fix: update URL wait conditions in Data Insight tests for accurate navigation

* fix: refine search response handling in verifyDeletedEntityNotVisible function

* fix: correct Playwright worker configuration in workflow files

- Fixed missing backslash continuation in playwright-postgresql-e2e.yml causing --workers=50% to be ignored
- Added missing --workers=50% flag in else block of playwright-postgresql-e2e.yml
- Fixed missing backslash continuation in postgresql-nightly-e2e.yml
- Fixed missing backslash continuation in mysql-nightly-e2e.yml

These syntax errors caused Playwright tests to run with unlimited workers (default CPU cores × 2),
leading to resource exhaustion and GitHub Actions runner communication failures. With this fix,
tests will properly run with 50% workers (1 worker on 2-core runners) preventing memory/CPU starvation.

* fix: increase Playwright worker count to 75% in workflow files

* fix: update Playwright tests to use correct role selectors and encode query parameters

* fix: reduce Playwright worker count from 75% to 50% in CI workflows

* fixed failing test

* fix: remove worker count specification from Playwright test commands in CI workflows

* fix: update Playwright test commands to specify worker count and adjust ESLint ignore settings

* fix: reduce worker count in Playwright configurations to prevent resource exhaustion

* fix: change waitForLoadState from 'networkidle' to 'domcontentloaded' in redirectToHomePage function

* fix: increase worker count to 3 in Playwright config and add navigation timeout to prevent hangs

* fix: reduce worker count to 2 in Playwright config to optimize resource usage

* Upgrade Playwright from 1.51.1 to 1.56.1

- Updated @playwright/test package to version 1.56.1 in package.json
- Updated Playwright browser installation to 1.56.1 in all GitHub workflow files:
  - mysql-nightly-e2e.yml
  - playwright-mysql-e2e.yml
  - playwright-postgresql-e2e.yml
  - postgresql-nightly-e2e.yml
- Updated yarn.lock to reflect the new Playwright version and its dependencies
- Reduced workers from 4 to 2 in playwright.config.ts to prevent resource exhaustion
- Added navigationTimeout: 60000 to prevent infinite hangs on networkidle waits
- Changed networkidle to domcontentloaded in redirectToHomePage function

This upgrade brings the latest stable Playwright features, improvements, and bug fixes while addressing resource exhaustion issues.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>

* fix: downgrade Playwright to version 1.55.1 in workflow files and package.json

* chore: upgrade Playwright to version 1.57.0 in workflow files and package.json

* trying firfox

* chore: upgrade Playwright to version 1.57.0 in OpenMetadata

- Updated @playwright/test package to version 1.57.0 in package.json
- Updated Playwright browser installation to 1.57.0 in all GitHub workflow files:
  - mysql-nightly-e2e.yml
  - playwright-postgresql-e2e.yml
  - postgresql-nightly-e2e.yml
- Updated yarn.lock to reflect the new Playwright version and its dependencies
- Maintained Chromium browser configuration for all test projects
- Kept worker optimization at 2 workers and --workers=50% flag for resource management

This upgrade brings the latest stable Playwright features, improvements, and bug fixes, including resolution of the "step id not found" regression bug.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>

* chore: update Playwright trace configuration to 'on-first-retry'

* revert worker setting

* fix failing test

---------

Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Sriharsha Chintalapani <harshach@users.noreply.github.com>
2025-12-31 17:06:09 +05:30
Sriharsha Chintalapani
dfd82688e8
fix integration tests (#25001) 2025-12-27 01:40:23 -08:00
Sriharsha Chintalapani
d09416058f
fix integration tests wf build - try #4 (#25000)
* fix integration tests wf build - try #4

* fix integration tests wf build - try #4
2025-12-27 00:45:23 -08:00
Sriharsha Chintalapani
498b44adf3
Add Free Disk Space step to prevent disk exhaustion (#24999)
* Add Free Disk Space step to prevent disk exhaustion

* Simplify integration test workflows
   with single maven command

   - Replace two-step build (mvn -DskipTests install + mvn test) with single
     mvn clean install -pl :openmetadata-integration-tests -P<profile> -am
   - This matches maven-build.yml pattern and builds only required modules
2025-12-27 00:35:45 -08:00
Sriharsha Chintalapani
4dab801243
Exclude openmetadata-ui and openmetadata-dist from integration test build (#24998) 2025-12-27 00:03:58 -08:00
Sriharsha Chintalapani
fcb1f7ffff
Fix maven build command - remove non-existent module exclusions (#24997) 2025-12-26 23:53:49 -08:00
Sriharsha Chintalapani
ab535900da
Faster tests (#24948)
* Add Parallel tests using the new SDK

* Make tests faster and use new SDK

* Add SDK based parallel tests

* Add SDK based parallel tests

* Fix from main

* Add Fluen APIs for Tests

* Add Fluen APIs for Tests

* Add missing Fluent APIs for SDK

* Add missing Fluent APIs for SDK - Data Contracts

* Migrate all the integration tests to new module

* Migrate all the integration tests to new module

* Improve pagination test performance

* Fix tests

* Migration Complete

* Fix the code styling; add github workflows, fix tags parallel issues

* Update migration tracker, address flaky tests

* Address comments

* rename env -> bootstrap for java package

* Fix YAML syntax in playwright-sso-tests.yml and update integration test workflows
2025-12-26 23:47:49 -08:00
Copilot
cbea0c0355
Add GitHub Actions workflow for SSO Playwright tests (#24980)
* Initial plan

* Add Playwright SSO tests workflow with @sso tag filtering

Co-authored-by: chirag-madlani <12962843+chirag-madlani@users.noreply.github.com>

* Add comprehensive documentation to SSO tests workflow

Co-authored-by: chirag-madlani <12962843+chirag-madlani@users.noreply.github.com>

* update workflow

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: chirag-madlani <12962843+chirag-madlani@users.noreply.github.com>
2025-12-24 18:16:36 +05:30
Pere Miquel Brull
2220cb41f0
MINOR - Fix RDF CI & repository cleanup (#24918)
* MINOR - Fix RDF CI & repository cleanup

* MINOR - Fix RDF CI & repository cleanup

* fix

* MINOR - Fix RDF CI & repository cleanup

* MINOR - Fix RDF CI & repository cleanup

* MINOR - Fix RDF CI & repository cleanup

* fix
2025-12-21 12:54:13 +01:00
Sriharsha Chintalapani
e71715ad6c
Single RDF knowledge graph for all entities (#24839)
* Single RDF knowledge graph for all entities

* Fix RDF Resource Test

* fix test

* fix test

* Add support for TagLabel objects

---------

Co-authored-by: Pere Miquel Brull <peremiquelbrull@gmail.com>
Co-authored-by: lautel <laura92cp2@gmail.com>
2025-12-18 16:33:15 +01:00
Shailesh Parmar
6f7c21ce8d
feat: Add e2e test documentation site with generation scripts and VitePress configuration. (#24837)
* feat: Add e2e test documentation site with generation scripts and VitePress configuration.

* feat: migrate e2e documentation generation to Playwright and update service entity permission tests.

* docs: Improve Playwright E2E test documentation with JSDoc comments and update doc generation.

* docs: Add link to E2E Test Documentation in README.

* fix: Correctly escape backslashes in Playwright generated markdown tables and refine the documentation CI workflow.

* minor fix

* fix node version issue

* minor change
2025-12-18 11:16:04 +05:30
Mayur Singal
48c3f4e98c
MINOR: Python E2E Test Fixes (#24821) 2025-12-15 08:07:05 +05:30
Mayur Singal
3051382b8b
MINOR: Fix Trivy Scan For OpenMetadata Ingestion (#24766) 2025-12-10 18:44:39 +05:30
Suman Maharana
cdc5595c52
Temporarily remove py 3.12 CI (#24690) 2025-12-04 10:23:55 +01:00
sonika-shah
e1d1fbc0c3
Complete Migration to Central Publishing Portal (Replace Nexus Staging Plugin) (#24614)
* Complete Maven Central Publishing Migration to Central Portal

* switch to latest 0.9.0 release of central-publishing-maven-plugin

* Remove distributionManagement section when using central-publishing-maven-plugin

---------

Co-authored-by: Pere Miquel Brull <peremiquelbrull@gmail.com>
2025-12-01 12:41:29 +05:30
Chirag Madlani
fa241865f7
fix postgres workflow error 2025-11-24 20:22:14 +05:30
Chirag Madlani
16e9692fc4
chore(ci): limit worker to avoid OOM killed for CI (#24518) 2025-11-24 17:32:29 +05:30
Chirag Madlani
d2ff014fa2
chore(CI): adding placeholder to run tests for core-components (#24512) 2025-11-24 12:05:52 +05:30
Mohit Tilala
ca0e673547
HOTFIX: Docker Ingestion CI add free disk space step (#24491)
Co-authored-by: Akash Jain <15995028+akash-jain-10@users.noreply.github.com>
2025-11-21 18:55:17 +05:30
miriann-uu
5db13f8937
GEN-4784 Add validation action for OMD dockerfile (#24335)
* Add validation pipeline for OMD dockerfile

* Add validation pipeline for OMD dockerfile

* Add validation pipeline for OMD dockerfile

* Add validation pipeline for OMD dockerfile

* Add validation pipeline for OMD dockerfile

* Add validation pipeline for OMD dockerfile

* Update diskspace step
2025-11-19 14:55:30 -05:00
Ashish Gupta
6b3a95f252
allocate 1 worflow to dataAssetRule in nightly (#24368) 2025-11-17 14:14:05 +05:30
Ashish Gupta
e18c848851
Data Asset rule enforcement on ui (#24049)
* data asset rule enforcement on ui

* added multi user and multi team support at same time

* fix the multi user case which wasn't working correctly and clean the ui as well

* fix the rule condition for glossary

* fix the glossary dropdown getting closed on select select after selection where we should wait for footer dropdown button action

* added playwright test for data asset rules table

* fix unit test and supported playwright test for table entity

* change the method and variable name for better readability

* naming convention fix around the glossary validation rule

* supported playwright test for all the entities

* supported in service entities

* spearate the test of rules enable and disable state and added supported in glossaryand teams page of rules owner

* Update generated TypeScript types

* fix the unit test and supported rule in tags page

* fix the generated type issue

* fix he domain single select where de-selecting throwing error and fix type

* modify the playwrigth config so dataAssetRules should be run after the chromium is finish executing so the global run changes won't interrupt other test

* fix all unit test

* fix the test failing in dataAssetHeader

* trying the chromium parallel test running

* modifyed yaml files to run the data asset rules after all machine runs is completed

* change the test running order in playwright config

* reverted the playwright config

* run the data asset rules in only one runner

* added support for multiple user and team combination in ownerLabel compoennt

* fix the placemnet issue, reduce the team font to 12 and minor cleanup

* modify many owner component cases as per ui

* clear ruleEnforcementProvider and useEntityRules hooks

* change the dropdown on owner user to mui component

* fix unit test and make chnages in playwright postgress around test running

* fix unit test failing

* fix some playwright test due to rule enforcement and made changes in CI playeright config to only run dataasset on container 6

* reverted the baseurl

* fix the version view for owner, fix some failing playwright test and re-modify the postgress script to run test

* supported the placement option for owner in case of user and team both and postgress e2e change

* remove the other e2e file changes and fix playwright around rules changes

* fix the e2e command

* fix the failing test around disabled rule and based on comments made fix

---------

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2025-11-14 16:19:45 +01:00
Mohit Tilala
79f04bda65
Fix workflow python version picking with quotes (#24263) 2025-11-11 13:24:22 +05:30
Mohit Tilala
9f6c07cb1c
Upgrade github workflows to use python 3.10 (#24203) 2025-11-07 16:43:04 +05:30
Mohit Tilala
fda5480918
Fix playwright pytest (#24197)
* Bump python version to 3.10 in playwright workflow

* Minor change to trigger playwright tests
2025-11-06 14:41:48 +05:30
Aniket Katkar
048e6ebc9b
Change ingestion_dependency to all (#24174) 2025-11-05 12:10:55 +05:30
Chirag Madlani
ef1459aaf0
Fix customize test aut & update nightly workflows (#24077)
* fix(test): separate count for OSS & AUTs

* update report to json

* fix tests
2025-10-30 13:33:06 +05:30
Chirag Madlani
0127ad6db1
fix(ci): support playwright trace for nightly workflow (#24088)
* fix(ci): support playwright trace for nightly workflow

* update compression level
2025-10-30 13:20:58 +05:30
Chirag Madlani
288f5d0ce7
chore(ci): increase machine for nightly playwright (#24070) 2025-10-29 17:37:27 +05:30
Aniket Katkar
489aeb5bc8
Update the reporter to html for better trace viewing (#23958) 2025-10-20 14:05:56 +02:00