Nahuel
235117c28c
Add missing healthcheck to docker-compose-postgres ( #7080 )
2022-08-31 08:40:17 +02:00
Nahuel
db41be4c4a
Fix#6929: docker-compose services should wait until OpenMetadata DB is created ( #7025 )
...
* Added the docker compose for postgres
* Added changes as per the comments on PR
* Airflow Connectivity to Postgres
* Changes are done as per the comments
* Replocate changes to local dockerfile
* Fixed Airflow Dockerfile
* docker-compose services should wait until OpenMetadata DB is created
* remove unnecesary wait for db service
* Add dependencies to airflow image
* Add missing dependencies to airflow Dockerfile
Co-authored-by: “Vijay” <“vijay.l@deuexsolutions.com”>
Co-authored-by: ulixius9 <mayursingal9@gmail.com>
2022-08-30 23:55:46 +02:00
NiharDoshi99
4c7fd03b5a
Configured ODBC drivers ( #6924 )
2022-08-25 08:19:20 -07:00
Vj-L
8bf34fcdc1
Fix/airflow dag issue ( #6759 )
...
* Added the correct path for dags volume
* Updated the dags volume path
* Removed the library from the Dockerfile of ingestion
* Removed the libmxl2 library from Docker file
Co-authored-by: “Vijay” <“vijay.l@deuexsolutions.com”>
2022-08-25 18:49:46 +05:30
Akash Jain
e987cb878e
feat: Prepare metadata docker cli with 0.11.5 release ( #6900 )
2022-08-24 16:43:14 +05:30
Vj-L
ffdb32193f
Added libxml2 in the ingestion Dockerfile ( #6778 )
...
* Added libxml2 in the ingestion Dockerfile
* Added libxml2 in the airflow Dockerfile
Co-authored-by: “Vijay” <“vijay.l@deuexsolutions.com”>
2022-08-18 15:54:00 +05:30
Vj-L
3728fa635b
Added docker-compose file for postgressql ( #6695 )
...
Co-authored-by: “Vijay” <“vijay.l@deuexsolutions.com”>
2022-08-11 12:12:14 +05:30
Ayush Shah
a6db2e8a84
Fix for profiler: modified filter patterns and added error handling ( #6608 )
2022-08-08 10:43:17 +05:30
Akash Jain
cac59328d0
feat: prepare metadata docker cli with 0.11.4 ( #6490 )
2022-08-02 00:57:09 +05:30
Vj-L
c4e88b69fa
Updated the image tag to 0.11.3 in docker compose file ( #6388 )
...
Co-authored-by: “Vijay” <“vijay.l@deuexsolutions.com”>
2022-07-30 09:48:07 -07:00
Kirill Popov
b32b11982e
Replaced wget with nc in entrypoint to support PostgreSQL ( #6403 )
...
Co-authored-by: Kirill Popov <kirill.popov@sbermarket.ru>
2022-07-30 09:42:01 -07:00
Pere Miquel Brull
ea918495ff
Fix #6382 - Snyk workflows ( #6413 )
...
Fix #6382 - Snyk workflows (#6413 )
2022-07-28 23:06:33 +02:00
Pere Miquel Brull
b7eb41a560
Fix #5840 - Upgrade Airflow container to 2.3.3 ( #6362 )
...
Fix #5840 - Upgrade Airflow container to 2.3.3 (#6362 )
2022-07-28 14:46:25 +02:00
Nahuel
30ff470922
Reduce the CI times for py-tests ( #6352 )
...
* Testing if we can reduce the CI times
* Update POM and run_local_docker script
2022-07-26 17:47:03 +02:00
Pere Miquel Brull
bbfb0dd7d2
Give more verbosity for CI ( #6048 )
...
Give more verbosity for CI (#6048 )
2022-07-18 14:44:50 +02:00
Akash Jain
fa5105191b
Fix/issue-5883 ( #5958 )
2022-07-12 19:55:38 +05:30
Vj-L
011651f078
"Updated the tags with latest tag" ( #5962 )
...
Co-authored-by: “Vijay” <“vijay.l@deuexsolutions.com”>
2022-07-08 16:43:39 +05:30
Vivek Ratnavel Subramanian
da93e7c925
Fix Authentication Public Keys env variable in docker compose ( #5948 )
2022-07-07 12:19:16 -07:00
Sriharsha Chintalapani
7e14dfa0d0
Update docker-compose.yml
...
Add 0.11.0 as image tag
2022-07-04 09:58:07 -07:00
Pere Miquel Brull
97761b97a6
Prepare postgres migrations ( #5814 )
2022-07-01 10:18:24 +02:00
mohitdeuex
7f25b82408
[Backend][TaskNotification] Update the Group Id for to org.open-metadata ( #5663 )
2022-06-28 11:10:07 +05:30
mohitdeuex
be18feb928
[Backend] Websocket implementation added ( #5440 )
...
* [Backend] Websocket implementation added
* [Backend] Websocket implementation added [google formatter fix]
* [backend] Updated Reiew Comments
* [backend] Updated CheckStyle
2022-06-15 10:14:47 +05:30
Akash Jain
a9167cdcf9
feat: Bump Docker Images to Release 0.10.2 ( #5276 )
2022-06-03 15:29:35 +05:30
Vivek Ratnavel Subramanian
059acfc529
Backend: Enforce principal domain in the JWT filter ( #5155 )
2022-06-01 10:07:12 -07:00
Sriharsha Chintalapani
cf3d4314dd
Fix #1607 Operational metrics for OpenMetadata system observability ( #4386 )
...
Co-authored-by: Akash-Jain <Akash.J@deuexsolutions.com>
2022-05-27 15:39:13 +05:30
Pere Miquel Brull
8e121fb306
Fix #4096 - PipelineService JSON refactor & Airflow metadata source connector ( #5125 )
...
Fix #4096 - PipelineService JSON refactor & Airflow metadata source connector (#5125 )
2022-05-25 08:35:16 +02:00
Sriharsha Chintalapani
2e9774dfe0
Update docker-compose.yml ( #5023 )
...
Update to latest release version
2022-05-18 08:32:57 -07:00
Akash Jain
18194e374e
Docker image and CI for Airflow Image ( #4791 )
2022-05-09 14:09:46 +02:00
Vj-L
2c7a327c6b
Issue #3094 ( #4500 )
...
* Added the named volumes for improvement
* Mapped the named volumes
* Updated the Named Volumes
* Updated the Volumes on top level
* Added the named volumes
2022-05-03 14:19:31 +05:30
Akash Jain
0f63a4063e
Enable custom oidc provider name env override. ( #4642 )
2022-05-02 22:38:11 -07:00
Akash Jain
792c67a8f1
Arguments to provide ingestion depenedency. defaults to 'all' ( #4613 )
2022-05-02 16:09:44 +05:30
Ayush Shah
9cb9792b7e
Fix Docker and revert openmetadata-start.sh changes ( #4518 )
...
* Fix Docker and revert openmetadata-start.sh changes
* Fix Docker - added error handling
* Fix Docker - added error handling
* Added time.sleep
* Resolve Code smell
2022-04-27 14:20:09 +05:30
Vivek Ratnavel Subramanian
12a4936671
Fix ingestion bot principals in docker-compose ( #4507 )
2022-04-26 15:33:16 -07:00
Akash Jain
55e1be9ca4
feat: Prepare 0.11.0-snapshot version changes ( #4505 )
...
* updated version to 0.11.0-SNAPSHOT
* updated version to 0.11.0.dev* for ingestion
* docker version updates for metadata docker cli
* OM Server Env Variables from #4197 #4261 #4473
2022-04-27 00:41:17 +05:30
Akash Jain
0beeef3f97
pluralize list input environment varialbles ( #4473 )
2022-04-25 17:11:02 +05:30
Ayush Shah
777e9e8097
Update docker-compose.yml ( #4370 )
2022-04-22 21:21:56 +05:30
Ayush Shah
c863fd9d90
Modified snowflake, utils ( #4364 )
2022-04-22 16:43:11 +05:30
Akash Jain
54162b7c6d
fix: List Configuration from Environment Variable for Authorizer and Authentication Configuration ( #4305 )
...
* ability to provide list from env in yaml config
* remove security config file as we do not maintain
2022-04-21 11:29:26 +05:30
Akash Jain
018cd0aec6
fix: Google SSO Audience default value ( #4285 )
...
* Update openmetadata.yaml
* updated audience default value
2022-04-20 15:48:13 +02:00
Akash Jain
7958e81918
server host api url env for airflow configuration ( #4261 )
2022-04-19 23:24:48 -07:00
Akash Jain
ff26b6d93e
fix: OpenMetadata Airflow config with SSO ( #4197 )
2022-04-19 10:39:50 -07:00
Akash Jain
540f5ca3b5
Fix: Prepare 0.10.0 snapshot ( #4190 )
...
* update pom versions to 0.10.0-SNAPSHOT
* fix version to latest release 0.9.1
* fix ingestion core version to 0.10.0
2022-04-19 17:40:42 +05:30
Pere Miquel Brull
6a6507e754
Fix #3962 - Profiler uses DatabaseSchema & Sample Data fix ( #4056 )
2022-04-12 13:40:59 +05:30
Akash Jain
22bbc407ad
Update docker-compose.yml ( #3966 )
2022-04-08 12:08:00 -07:00
Pere Miquel Brull
93525aea0d
Fix #3823 & Fix generate default Enum field ( #3909 )
...
* Fix sample data DAG
* Fix callback imports
* Use --set-default-enum-member in generate
* Format
* Add faulty merge hard_delete
* Fix airflow lineage, improve naming and fix lineage tests
* Add mysql url test
* Add mysql url test
* Update CI name
* Fix test ometa endpoint
* Format
* Fix metadata config
2022-04-07 18:22:50 +05:30
Sriharsha Chintalapani
b14c8dc2c4
Issue-3685: Variable based separator used for fullyQualifiedName instead of hardcoded . for Python and make the separator to : ( #3778 )
...
* Issue-3685: Variable based separator used for fullyQualifiedName instead of hardcoded . for Python and make the sepearator to :
* Fix failing test
* Use colon for run_local_docker validation
* Update tests FQDN
* Update tests FQDN
Co-authored-by: Sachin-chaurasiya <sachinchaurasiyachotey87@gmail.com>
Co-authored-by: pmbrull <peremiquelbrull@gmail.com>
2022-03-31 19:20:27 +05:30
Ayush Shah
351db56687
Added fix for Sample_Usage not being ingested in Local docker ( #3650 )
2022-03-27 22:38:13 -07:00
Ayush Shah
9a620c380d
Fixing Metadata indexing issue ( #3580 )
2022-03-22 14:37:41 +05:30
Ayush Shah
b865d85d85
Docker Calculation fix + Reset DB command added ( #3459 )
2022-03-21 10:28:58 -07:00
Akash Jain
59867aaad7
making docker compose compatible with metadata CLI ( #3369 )
2022-03-11 10:19:24 +05:30
Vivek Ratnavel Subramanian
f0927ecebb
Fix #3236 : Add support for secure docker compose with an env file ( #3241 )
...
* Fix #3236 : Add support for secure docker compose with an env file
2022-03-08 07:05:31 -08:00
Snyk bot
01e17d0d66
fix: docker/images/minimal-ubuntu/Dockerfile to reduce vulnerabilities ( #3126 )
...
The following vulnerabilities are fixed with an upgrade:
- https://snyk.io/vuln/SNYK-UBUNTU1604-LIBGCRYPT20-1585790
- https://snyk.io/vuln/SNYK-UBUNTU1604-SYSTEMD-1320131
- https://snyk.io/vuln/SNYK-UBUNTU1604-SYSTEMD-1320131
- https://snyk.io/vuln/SNYK-UBUNTU1604-SYSTEMD-1320131
- https://snyk.io/vuln/SNYK-UBUNTU1604-SYSTEMD-1320131
2022-03-04 12:03:11 -08:00
Akash Jain
123ef9cab7
Bump up release version ( #2968 )
2022-02-24 12:48:10 +05:30
Akash Jain
3744cd250b
fix: cherry pick for 0.8.3 ( #2893 )
...
* update dockerfile with new release download url
* update tags to 0.8.3
2022-02-21 13:55:29 +05:30
Pere Miquel Brull
76f4ccd590
Fix #2735 - Simplify Airflow properties extraction ( #2749 )
...
* Simplify lineage properties extraction
* Add network name
* Format
2022-02-14 16:53:42 +01:00
Pere Miquel Brull
edea57b4ad
Add env vars for host and port + verbosity ( #2490 )
2022-01-29 16:02:36 -08:00
Akash Jain
48c1ad63c4
fix: ElasticSearch and Ingestion connectivity from openmetadata post soft link ( #2470 )
...
* soft linked openmetadata.yaml
* quick fix for ingestion and elasticsearch connect from server
2022-01-27 13:55:54 +05:30
Akash Jain
59aa59e284
soft linked openmetadata.yaml ( #2469 )
2022-01-27 13:07:10 +05:30
Akash Jain
20309cde64
fix:cherry pick from 0.8.0 and miscellaneous fixes ( #2445 )
...
* updating ingestion-core version to 0.8.0
* updating ingestion version to 0.8.0
* fixing file copy path
* update docker images to point to 0.8.0
* set up new release version
* reset ingestion version to 0.9.0.dev0
* reset ingestion-core version to 0.9.0
2022-01-26 14:00:27 +05:30
Pere Miquel Brull
1eb3f1ad41
[issue-2322] - Backup CLI ( #2349 )
...
* Prepare cli module
* Check call
* Prepare backup
* Run pytest on schema changes
* Remove traces
* Add backup plugin
* run generate during CI
* Install after generate
* Address logger hotspot
2022-01-22 12:08:14 -08:00
Pere Miquel Brull
6693b4ae40
[issue-2262] - Simplify sample data DAG ( #2337 )
...
* Simplify sample data DAG
* Remove mkdir
* Generate sources before running compose
* Generate sources to install models to ingestion image
* Add python-on-whales for docker --start
* Remove python-on-whales from base
* Install venv
* Setup python
2022-01-22 16:37:56 +05:30
Pere Miquel Brull
94ecb1e030
Map ports to localhost ( #2078 )
2022-01-06 14:32:31 +01:00
Akash Jain
0bbb283e70
fix: update base image to alpine linux and refactor dockerflie instructions ( #2050 )
...
* +alpine img and refactor dockerflie instructions
* local-metadata server image with alpine linux
* correcting rm openmetadata-*.tar.gz
* fix COPY destination folder ref
2022-01-05 10:11:24 -08:00
Pere Miquel Brull
2ec32017bb
[issue-1698] - Airflow DockerOperator setup ( #1730 )
...
* Fix typo
* Clean setup
* Update ingestion local image to be barebone on connector dependencies
* Prepare ingestion connectors base image
* Add system dependencies
* Prepare docker CLI
* Add docker provider
* Prepare entrypoint for the image
* Remove DBT pipeline as per Issue 1658
* Add TODO for ingestion build
* Bind docker socket
* Update comment
* Update README
* Use DockerOperator in sample data
* Build images with latest tag
* Prepare symlink to pass the volume to the DockerOperator
* Update README
* Prepare Base image for CI
* COPY multiple files into dir
* COPY multiple files into dir
* Remove DBT source as is now part of table ingestion
* Build docker base in run_local_docker
2021-12-18 16:41:38 +01:00
Sriharsha Chintalapani
39e4163d6f
Fix #1814 : Add migrate-all option to update ElasticSearch index mappings ( #1815 )
...
* Fix #1814 : Add migrate-all option to update ElasticSearch index mappings
* Fix #1814 : Add migrate-all option to update ElasticSearch index mappings
2021-12-16 20:56:58 -08:00
Akash Jain
b79784b921
updated url to match with 0.7.0 release ( #1798 )
2021-12-16 16:04:53 +05:30
Ayush Shah
f288cef6d1
Update docker-compose.yml ( #1792 )
...
* Update docker-compose.yml
* Update docker-compose.yml
2021-12-16 14:39:45 +05:30
Ayush Shah
07924f80c3
Added Metabase Connector ( #1726 )
...
* Added Metabase Connector
* Metabase Added support for lineage
* Modified Metabase as per comments
* Metabase filter patterns added
* Update databaseService.json
2021-12-15 09:53:55 -08:00
Akash Jain
a9969dec99
update docker image tags to 0.6.0 ( #1766 )
2021-12-15 14:55:51 +05:30
Akash Jain
d0d3fce332
fix: add container_name to docker-compose ( #1639 )
...
* adding container_name
* fixed typo --run to --start
* updated container_name prefixing openmetadata_
* updated container_name to prefix openmetadata_
2021-12-10 08:01:08 -08:00
Ayush Shah
777afd1d3f
Openmetadata.yaml for docker updated ( #1606 )
...
* Dockerfiles changed - updated elasticsearch, openmetadata.yaml
* Added blue color to docker info
2021-12-07 11:27:16 -08:00
parthp2107
a9e70e6a89
License header update ( #1498 )
...
* updated liscense header
* updated license header
* updated license header
* updated license header
* addressing stylecheck findings
2021-12-01 12:46:28 +05:30
Ayush Shah
47a97e7be0
FQN changed from bigquery to bigquery_gcp
2021-11-29 16:57:39 +05:30
Ayush Shah
267a3d7aa2
Removed Platform from docker-compose.yml files, emulation fix
2021-11-26 18:10:27 +01:00
Akash Jain
74a10382a2
Fix: Docker Image to use new Dist Location while running locally ( #1381 )
...
* fix path for dist folder
* updated docs wrt new dist folder
2021-11-25 12:33:20 +05:30
Ayush Shah
11e2229972
Added OMeta API, removed wget ( #1376 )
...
* Added OMeta API, removed wget
* Added curl instead of Wget in run_local_docker.sh
2021-11-24 11:57:33 -08:00
Akash Jain
e7072a3177
prepare for 0.6.0 Docker Release ( #1213 )
2021-11-17 14:57:12 +05:30
Ayush Shah
2111f6c938
Docker Commands Modified ( #1199 )
...
* Docker ingestion modified
* Command line commands updated
2021-11-16 00:25:40 -08:00
Akash Jain
58cf1b012f
feat: Docker Image Push Workflow ( #1191 )
...
* copy command use path from root folder
* Add workflows for docker release to github
2021-11-15 09:36:57 -08:00
Ayush Shah
5dc3bb9297
Docker Support from Python added ( #1158 )
...
* Script modified - supports running from different locations
* Docker support from Python CLI
* Docker plugin setup.py
* Paths for latest and local dockers updated
* Resolved Comments - Docker CLI optimized, timestamp added
* help attribute added to options
* Docker clean code refactored
2021-11-12 10:30:28 -08:00
Akash Jain
b2d1d0ca34
fix:Airflow configurations to match docker-compose ( #1157 )
...
Airflow configurations to match docker-compose yaml values
2021-11-11 19:23:24 +05:30
Ayush Shah
dd7bb5a79f
airflow config added to openmetadata conf file ( #1155 )
2021-11-11 18:31:53 +05:30
Ayush Shah
c95f1542eb
Docker fix ( #1101 )
...
* Docker Fix, User Ingestion Fix
* Docker file optimized
* ES docker version rolledback
* Healthcheck removed - incompatible with 3.9
* Docker & Airflow Config files updated with - resolved comments
* Openmetadata managed Airflow apis support added
* Dockerfile updated
* Docker file Updated with release package
* Elasticsearch allocated memory increased
2021-11-10 21:22:32 -08:00
Akash Jain
06b2f70e3e
kubernetes docker image compatibility for #1026 ( #1042 )
2021-11-03 08:18:52 -07:00
Ayush Shah
93921814af
Docker fix - latest release changes ( #983 )
...
* Docker fix - Architecture, MySQL
* Docker Airflow API Dagrun support
* Docker latest changes modified
2021-10-30 09:05:30 -07:00
Ayush Shah
759574a8de
Ingestion Optimization - Sample Users, Dockerfiles, Removal of Pandas ( #935 )
...
* Sample Profile Data for Sample Tables (#815 )
* Sample Profile Data for Sample Tables
* Disabling Profile as Default
* Added Sample Profile Data to 3 additional sample tables
* Sample Tables fixed (#850 )
* Pydantic fix, Docker update (#860 )
* Setup.py Modified with openmetadata-airflow package, docker update
* Setup.py Modified
* Update setup.py
* Removed Pandas from Sample Data
* Sample Users added under sample data
* Sample User Standalone pipelines and modules removed
* Docker release package updated
* Dockerfile updated, removed redundant files
* Setup.py removed from ingestion src directory
* User Resource failing check resolved
* Modifying Usage Columns Datatype
2021-10-26 09:14:24 -07:00
Pere Miquel Brull
1291bf6ec0
[Issue-892] - GET operations return service ( #916 )
...
* Force image build in local docker
* Use service as default field query param for entities with required service
* Return service in List
* Naming convention
* Update Repository setFields to include service
* Revert resource changes
* Revert resource changes
* Update tests to check we always return the service, even if not specified
* Return service for database
2021-10-24 10:26:28 -07:00
vijaypm
b8863648d6
Issue 898 ( #905 )
...
* ISSUE-898: additional information in the prerequisities for building and running code
* ISSUE-898: removed unreachable old doc
* ISSUE-898: added new docker compose to expose MySQL and ES ports to host machines
* ISSUE-898: changed jdbc connect url to allow Public Key Retrieval
* ISSUE-898: fixed log name to openmetadata.log
Co-authored-by: Vijay Mariadassou <vijay@mariadassou.com>
2021-10-22 19:35:12 -07:00
Ayush Shah
85b6b72848
Airflow docker ( #762 )
...
* Airflow Docker implementation - Ingestion
* Dockerfiles modified
2021-10-14 07:46:24 -07:00
Ayush Shah
319f7a795c
Docker local ( #723 )
...
* added docker file for local build and ingestion
* Docker Modified for local setup
* Ingestion script optimized
* Added script to directly run Local Docker
Co-authored-by: Parth Panchal <parthpanchal@Parths-MacBook-Air.local>
2021-10-09 10:58:33 -07:00
Ayush Shah
0ae4ef21cd
Folder not found fixed
2021-08-16 23:55:28 +05:30
Ayush Shah
ca5ca88f37
change release path
2021-08-16 23:55:28 +05:30
Ayush Shah
71444a815a
Docker Platform and Commands Modified
2021-08-16 23:55:28 +05:30
Ayush Shah
d4d66ea21d
Docker fix ( #200 )
...
* Modifications in Docker
* Docker config and script Modifications
* Config changes ElasticSearch
2021-08-16 18:37:04 +05:30
Ayush Shah
03ad583744
Registry dependency removed
2021-08-14 00:09:51 +05:30
Suresh Srinivas
1619b6f2e2
Fix Docker: update the binary names
2021-08-12 11:10:30 -07:00
parthp2107
8a92d253db
replaced catalog to openmetadata #2 ( #47 )
...
Co-authored-by: parthp2107 <parth.panchal@deuexsoultions.com>
2021-08-05 10:15:52 +05:30
Ayush Shah
982ca639b5
Docker Scripts and Commands modified
2021-08-03 01:26:20 +05:30
parthp2107
b5baa9f2db
applied license to the backend files
2021-08-02 15:08:30 +05:30
sureshms
8ce7c879e8
OpenMetadata snapshot release 0.3
2021-08-01 14:27:44 -07:00
