mirror of
https://github.com/open-metadata/OpenMetadata
synced 2026-05-24 09:39:11 +00:00
9921dc1389
* Fixes #28245: ingest valueless Databricks/Unity Catalog tags Databricks/Unity Catalog exposes system-generated (and some user-defined) tags as (tag_name, tag_value=null). The connectors mapped tag_name -> Classification and tag_value -> Tag, so an empty tag_value was either skipped (Unity Catalog) or coerced to a "NONE" sentinel (Databricks). When tag_value is empty, fall back to a dedicated per-connector classification (DATABRICKS_TAGS / UNITY_CATALOG_TAGS) and use tag_name verbatim as the tag under it (no dot-splitting). Valued tags are unchanged: classification = tag_name, tag = tag_value. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * Address review: harden valueless-tag mapping - Treat whitespace-only tag_value as valueless (strip-based check) so it falls back to the *_TAGS classification instead of being silently dropped downstream by get_ometa_tag_and_classification. - Skip rows with empty/None tag_name in the Databricks connector, for parity with Unity Catalog, so an empty classification name is never sent to the API. - Add tests for whitespace-only tag_value (both connectors) and the empty tag_name skip (Databricks). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|---|---|---|
| .. | ||
| .basedpyright | ||
| docs/design | ||
| examples | ||
| operators | ||
| pipelines | ||
| src | ||
| stubs | ||
| tests | ||
| __init__.py | ||
| airflow-constraints-3.1.7.txt | ||
| airflow-constraints-3.2.1.txt | ||
| Dockerfile | ||
| Dockerfile.ci | ||
| ingestion_dependency.sh | ||
| LICENSE | ||
| Makefile | ||
| noxfile.py | ||
| pyproject.toml | ||
| README.md | ||
| setup.py | ||
| sonar-project.properties | ||
This guide will help you setup the Ingestion framework and connectors
This guide will help you setup the Ingestion framework and connectors
OpenMetadata Ingestion is a simple framework to build connectors and ingest metadata of various systems through OpenMetadata APIs. It could be used in an orchestration framework(e.g. Apache Airflow) to ingest metadata. Prerequisites
- Python >= 3.9.x
Docs
Please refer to the documentation here https://docs.open-metadata.org/connectors
TopologyRunner
All the Ingestion Workflows run through the TopologyRunner.
The flow is depicted in the images below.
TopologyRunner Standard Flow
TopologyRunner Multithread Flow
