Commit graph

54 commits

Author SHA1 Message Date
Sriharsha Chintalapani
2c8a45d2a8
Upgrade to Dropwizard 5x and Jetty 12.1 (#24776)
* Add support for Dropwizard 5.0 and Jetty 12.1.x

* Dropwizard 5x and Jetty 12.1 upgrade

* Fix test behavior

* Fix rdf tests

* revert enableVirtualThreads

* fix tests

* Fix Tests

* Fix tests

* Switch to jersey-jetty-connector for Jetty 12 compatibility

- Replace jersey-apache-connector with jersey-jetty-connector
- Jersey 3.1.4+ jersey-jetty-connector supports Jetty 12.0.x+
- Use JettyConnectorProvider and JettyHttpClientSupplier for HTTP client
- Keep reasonable timeouts (30s connect, 2min read) to prevent CI hangs
- Set SYNC_LISTENER_RESPONSE_MAX_SIZE for large responses

This fixes the 1,093 InterruptedException test failures caused by
using the default Jersey client (HttpURLConnection-based) which doesn't
handle concurrent test execution properly.

* Fix: Start Jetty HttpClient before use

Jetty 12 HttpClient implements LifeCycle and must be explicitly
started with httpClient.start() before use. This fixes the 163
InterruptedException test failures.

* Fix: Force jetty-client to 12.1.1 for jersey-jetty-connector

jersey-jetty-connector brings transitive jetty-client:12.0.22 but
Dropwizard 5.0 uses Jetty 12.1.1. The ClientConnector.newTransport()
API changed between 12.0.x and 12.1.x, causing NoSuchMethodError.

Fix: Exclude transitive jetty-client and add explicit 12.1.x dependency.

* Use Java 11+ HttpClient connector for tests (jersey-jnh-connector)

Switch from the broken jersey-jetty-connector (incompatible with Jetty 12.1.x)
to jersey-jnh-connector which uses Java's built-in java.net.http.HttpClient.
This connector:
- Natively supports all HTTP methods including PATCH
- Works with Java 21
- No external dependencies required
- Avoids compatibility issues with Jetty versions

* Use Apache HttpClient 5.x connector for tests (jersey-apache5-connector)

Switch from jersey-jetty-connector (incompatible with Jetty 12.1.x)
to jersey-apache5-connector which uses Apache HttpClient 5.x.
This connector:
- Supports all HTTP methods including PATCH
- Lenient with empty PUT request bodies
- Has proper timeout support to prevent indefinite hangs
- Works with Jetty 12.1.x

* Fix  tests

* Fix  docker compose

* Fix tests

* Fix tests - make url compatible

* Add URL parsing

* Fix URL decode

* fix tests

* fix test

* fix tests

* Fix integration with new dropwizard-5x changes

---------

Co-authored-by: Karan Hotchandani <33024356+karanh37@users.noreply.github.com>
Co-authored-by: karanh37 <karanh37@gmail.com>
Co-authored-by: Mohit Yadav <105265192+mohityadav766@users.noreply.github.com>
2026-01-12 12:18:29 -08:00
Teddy
11c2d2f6a9
MINOR - Airflow serialized limit (#24617)
* chore: set lower DAG serialization defaults

* chore: increase timeout for testSuite

* chore: update DAG processor interval postgres

* chore: lower DAG parse interval and delay

* fix: remove internal parsing and trigger dag parsing automatically on deploy
2025-12-01 11:09:19 +01:00
Mayur Singal
acb1be97f4
Fix #23096: Add Airflow 3.x support (#24338)
* Fix #23096: Add Airflow 3.x support

* airflow auth fixes

* fix airflow tests

* fix airflow 3 ingestion

* pyformat

* fix pytest

* pyformat

* bump version

* fix version

* fix mlflow

* custom pydoris

* fix airflow tests

* fix spotless

* final test fixs

* playwrite debug

* fix pytests

* checkstyle fix

* fit get status api and revert playwrite debug

* fix airflow version

---------

Co-authored-by: Ashish Gupta <ashish@getcollate.io>
2025-11-21 12:28:28 +01:00
Mohit Yadav
0b2321e976
Added Session Age for Cookies (#22166)
* - Added Session Age for Cookies

* Make OIDC Session Expiry Configurable

* Update generated TypeScript types

* Updated Docker Files

* Update Session to 7 days

---------

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2025-07-08 15:07:52 +05:30
Akash Jain
799e3ca900
chore(docker-compose): Bump indices.query.bool.max_clause_count=4096 (#21301) 2025-05-27 14:29:05 +02:00
Mohit Yadav
20f17a3367
Fixes #16062: Added prompt config to allow config (#20959)
* Fixes #16062
Make prompt=login as optional

* update null or empty
2025-04-25 08:37:25 +05:30
Mohit Yadav
3a01ad7da5
[Fix-20125] OIDC: Allow max_age to be optional (#20721)
* Make Max Age Optional

* spotless fix
2025-04-09 15:09:57 +05:30
Mohit Yadav
c28f3274d1
Adds new param to docker files (#20338) 2025-03-19 18:13:22 +05:30
Chirag Madlani
a43835df32
Revert "fixes #18820: updated docker compose files (#18821)" (#19297)
This reverts commit 69dd8b99f9.
2025-01-09 15:34:07 +05:30
tarunpandey23
69dd8b99f9
fixes #18820: updated docker compose files (#18821) 2025-01-09 10:50:00 +05:30
Prajwal214
1c90eaaf3d
Docs: Updated Domain in Docker Compose & Docs (#17603)
* Minor: Updating Domain in Docker Compose & Docs

* replace openmetadata to open-metadata

---------

Co-authored-by: Prajwal Pandit <prajwalpandit@Prajwals-MacBook-Air.local>
Co-authored-by: Chirag Madlani <12962843+chirag-madlani@users.noreply.github.com>
Co-authored-by: Shailesh Parmar <shailesh.parmar.webdev@gmail.com>
2024-09-09 11:47:09 +05:30
Mohit Yadav
05baba9dc1
Change Default Value of Batch Size and Payload Size (#17391) 2024-08-12 11:18:18 +05:30
Mohit Yadav
aa5b9b2985
Increase Payload Size to 100MIB for ELastic Search (#17366)
* Increase Payload Size to 100MIB

* Increase batchSize Size to 500

* Add Payload Size as option in openmetadataoperations , and searchIndexJob

* //Comment

* Add Logs
2024-08-11 21:58:55 +05:30
sonika-shah
8d5661a7bf
Support Elasticsearch cluster alias for E2E tests  (#17117)
* Fix backend tests and have index alias with clusterAlias appended

* Fix backend tests and have index alias with clusterAlias appended

* Fix failing tests

* alias setup

* fix suggestion not working due to alias

* fix getIndexOrAliasName method for multiple indexes

* update openmetadata.yaml

* update childAliases with clusterAlias

---------

Co-authored-by: Ashish Gupta <ashish@getcollate.io>
Co-authored-by: Sriharsha Chintalapani <harshach@users.noreply.github.com>
2024-08-10 11:26:49 -07:00
Mohit Yadav
37e1472fa3
Add Jwt Claim Mapping Env (#16780) 2024-06-24 23:15:19 +05:30
Mohit Yadav
a5295396bd
Add WebConf Env to Compose (#16713) 2024-06-19 12:34:33 +05:30
Shailesh Parmar
1d4e3e0825
Revert alias setup (#16610)
* Revert "fix(CI): update `run_local_docker.sh` script with cluster alias ES endpoint (#16604)"

This reverts commit baab52076f.

* Revert "chore(CI): Enable Elasticsearch cluster alias by default (#16599)"

This reverts commit a3524e14d3.
2024-06-11 19:14:25 +05:30
Akash Jain
a3524e14d3
chore(CI): Enable Elasticsearch cluster alias by default (#16599) 2024-06-11 11:34:24 +05:30
sonika-shah
c010939072
Added payload size check logic for Elasticsearch (#15540)
* Added payload size check logic for Elasticsearch

* Added payload size check logic for Elasticsearch - made review changes

* Added payload size check logic for Elasticsearch - added env in docker files

* Optimize payload logic, fix search application error stats issue
2024-04-15 16:25:06 +05:30
Shin-ichi Hashiba
ba7b5d248a
delete duplicate env (#15709) 2024-04-01 18:21:51 +05:30
Mohit Yadav
e8f8271b77
- Add docker vars (#15619)
- Modified Azure refresh token logic
2024-03-20 11:47:25 +05:30
Imri Paran
aade838020
Fixes #15388: Use native backup tools (#15393)
* feat: use native backup tools

1. added mysqldump 8.3 to the ingestion container.
2. documented how to use native tools to back up and restore.
3. added deprecated message on the cli backup and restore.

* added deprecation notice for 1.3 backup

* removed 1.3.x deprecation notice

* added another backup page in 1.3 introducing SQL dump tools

* added --set-gtid-purged=OFF to the mysql dump process
2024-03-12 06:23:05 +01:00
Pere Miquel Brull
62c0cc7563
#13985 - Azure KV Secrets Manager (#15192)
* #13985 - Azure KV Secrets Manager

* Format

* #13985 - Azure KV Secrets Manager

* #13985 - Azure KV Secrets Manager

* Simplify credentials loading

* Simplify credentials loading

* Simplify credentials loading
2024-02-20 07:18:35 +01:00
Akash Jain
065a8e30da
fix: docker-compose failed randomly without healthcheck of elasticsearch. #14854 (#15034) 2024-02-05 18:28:00 +05:30
Akash Jain
8574e9a742
feat(docker-compose): Use openmetadata-ops.sh script for migrations (#14990) 2024-02-05 11:22:53 +05:30
Akash Jain
69e6f50769
chore(docker-compose): fix DB_PARAMS for Docker Compose files (#14931) 2024-01-29 19:28:07 +05:30
Akash Jain
07eb1ccdf5
fix: Add SAML and LDAP env configs; disabled by default (#14453) 2023-12-25 18:08:38 -08:00
Pere Miquel Brull
d8984d267e
#14340 & #13849 - Clean secret ID and improve encrypt/decrypt exception management (#14356)
* Fix supported characters in SM

* Update SM

* Fixes

* Fixes

* Improve class conversion exceptions

* Comments

* Rename noop to db secrets manager providee

* Update sm

* Fix

* db SM

* db SM

* Fix test

* UI

* Update openmetadata-ui/src/main/resources/ui/src/mocks/IngestionListTable.mock.ts

* update default
2023-12-18 06:43:20 +01:00
Akash Jain
6eefd51180
fix: AIRFLOW_DB_PROPERTIES default value in docker compose files (#14274) 2023-12-06 19:56:19 +05:30
Akash Jain
a5a3347fd3
fix: Include DB_PROPERTIES env for docker-compose files (#14264) 2023-12-06 13:13:49 +05:30
Mohit Yadav
19dbc217b0
Add code flow Configuration (#14026) 2023-11-20 08:44:13 +05:30
Mohit Yadav
2ae0333551
Fix Email Issues (#13962)
* - Added Unauthenticated Server Emails
- Removed redundant OM_URI
- Add Test Email in SystemResources

* remove default port

* remove fernet
2023-11-14 09:04:53 +05:30
Mohit Yadav
2577490700
- Removed Logo Configuration from the openmetadata.yaml (#13873)
* - Removed Logi Configuration from the openmetadata.yaml
- Migrated Login Configuration to the UI

* Removed ApplicationConfig from tests

* Added Custom Header Logo Url

* Update field to favicon

* support login conifguration

* chore(ui): update logo and login config md files

* fix validation issue

* add cypress

* restructure folders and set default fallback for favicon

* autofocus to first field of edit form

---------

Co-authored-by: Chirag Madlani <12962843+chirag-madlani@users.noreply.github.com>
Co-authored-by: Sachin Chaurasiya <sachinchaurasiyachotey87@gmail.com>
2023-11-08 20:05:00 +05:30
Teddy
e57849b732
Fixes #12298 - Update report data type to camel case (#13505)
* fix: updated DI to camelCase

* fix: ran linting

* fix: added migration

* fix: remove extra parenthesis in migration file

* fix: psql migration query

* fix: OS compose host

* fix: removed commented code block
2023-10-11 08:14:21 +02:00
Akash Jain
8b028a737b
fix(CI): postgresql ci with opensearch (#13500)
* fix(CI): Postgresql Docker Compose Services Dependency

* fix(CI): update `SEARCH_TYPE` to `opensearch`
2023-10-09 20:18:13 +05:30
Akash Jain
9549df48b4
fix(CI): Postgresql Docker Compose Services Dependency (#13492) 2023-10-09 07:42:20 -07:00
Sriharsha Chintalapani
35c4e18663
Upgrade search versions (#13479)
* working on maven shading

* update es in docker

* add maven shade for elasticsearch/opensearch dependencies

* add submodule for es/os dependencies

* upgrade docker elasticsearch to latest 8.10.2

* upgrade docker elasticsearch to latest 8.10.2

* Update Postgres dev to use OpenSearch

* Fix container service dependency

* revert conf change

---------

Co-authored-by: 07Himank <himank07mehta@gmail.com>
Co-authored-by: 07Himank <112613760+07Himank@users.noreply.github.com>
2023-10-08 11:55:34 -07:00
Akash Jain
7d4957537c
fix(CI): enable force migrations for CI (#13026) 2023-08-29 13:39:08 +05:30
Akash Jain
0a7f30977a
fix(chore): Remove OPENMETADATA_DEBUG Param (#12997) 2023-08-24 20:17:50 +05:30
Mohit Yadav
9712337b01
[SecurityHeaders] Fixed issue with Security Headers (#12921) 2023-08-18 17:48:17 +05:30
Dhruvin Maniar
403e407478
[FEAT] Implement Docker InitContainer for migration (#12714)
* [FEAT] Implement Docker InitContainer for migration

* [FEAT] Implement Docker InitContainer for migration

* [FEAT] Implement Docker InitContainer for migration

* [FEAT] Implement Docker InitContainer for migration

---------

Co-authored-by: Dhruvin Maniar <dhruvinmaniar@192.168.1.27>
Co-authored-by: Dhruvin Maniar <dhruvinmaniar@Dhruvins-MacBook-Pro.local>
Co-authored-by: Akash Jain <15995028+akash-jain-10@users.noreply.github.com>
2023-08-03 16:01:20 +05:30
Akash Jain
1b54a9d6bf
fix: Docker Compose Postgres Command Args (#12693) 2023-08-01 18:56:51 +05:30
Anuj359
9ed9fb172e
Updating docker env and symbolic link for openmetadata.yaml (#12684) 2023-08-01 11:07:29 +05:30
Anuj359
948ddf36c6
Fix/issue 12532 (#12550)
* Updating Dockerfile with multistage

* Updating reviewed changes

* Docker Development changes

* Docker workflow changes

* Arguments update

* Script path update

* Arguments update

* Resolving the reviewed suggestions

* Issue-12532 and security-scan fix

* Removing security-scan fix

* Update Dockerfile

* updating wget from curl
2023-07-21 22:34:47 +05:30
Pere Miquel Brull
a73f5821ea
Fix postgres container init (#12407) 2023-07-13 08:12:04 +02:00
Anuj359
7f55b808e5
fix(chore): Prepare Main Branch with next release (1.2.0) (#12364)
* Updating Dockerfile with multistage

* Updating reviewed changes

* Docker Development changes

* Docker workflow changes

* Arguments update

* Script path update

* Arguments update

* Resolving the reviewed suggestions

* Preparing for next release and updating docker sort value changes

* Adding .dev0 in the python related release version

* Adding .dev0 in the python related release version
2023-07-12 16:55:18 +05:30
Anuj359
8431f57ef2
Docker env variables (#12262)
* Updating Dockerfile with multistage

* Updating reviewed changes

* Docker Development changes

* Docker workflow changes

* Arguments update

* Script path update

* Arguments update

* Resolving the reviewed suggestions

* Adding the missing Env variables

* Updating SAML env variable removal

* Updating searchtype
2023-07-03 19:13:28 +05:30
07Himank
405896e009
updated es to 7.16.3 (#12218) 2023-06-29 16:03:51 +05:30
07Himank
569539f280
updated elastic search version from 7.16.0 to 7.16.1 (#12203)
* updated elastic search version from 7.16.0 to 7.16.1

* dummy commit

---------

Co-authored-by: Pere Miquel Brull <peremiquelbrull@gmail.com>
2023-06-28 07:05:00 -07:00
Anuj359
73f5859ed9
Docker script update (#12115)
* Create manual.yml

* Fixing Typo

* Removing manual test github action file

* Testing IP runner

* Updating docker script for sandbox db task

* removing the ip test action

* Updating docker script for sandbox db task postgres

* Volume cleaning argument addition

* Volume cleaning argument addition
2023-06-26 15:44:53 +05:30