Elgato_dark/OpenMetadata
1
0
Fork 0
mirror of https://github.com/open-metadata/OpenMetadata synced 2026-05-24 09:39:11 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 5
OpenMetadata/docker
History
Mohit Yadav 1fe8a02fde Improve indexing (#26154) 
* Add Prometheus metrics for reindexing pipeline via Micrometer                                                       Bridge the existing reindexing atomic counters to Prometheus so operators     can alert on failures, latency spikes, and backpressure without relying      solely on database-flushed stats.

  - Add ReindexingMetrics singleton (initialize/getInstance pattern matching
    CacheMetrics) with job lifecycle counters, stage success/failed/warnings
    counters, bulk request timers with SLA buckets, payload size distribution,
    backpressure and promotion counters, and active/pending gauges
  - Register in MicrometerBundle after StreamableLogsMetrics
  - Instrument ReindexingOrchestrator.run() with job started/completed/failed/stopped
  - Bridge StageStatsTracker.flush() deltas to Prometheus per stage and entity type
  - Add bulk request latency timer and payload size recording in OpenSearchBulkSink
  - Record backpressure events in SearchIndexExecutor.handleBackpressure()
  - Record promotion success/failure in DefaultRecreateHandler
  - Add ReindexingMetricsTest with 24 tests covering all metric types

* Add Improvements

* Auto Gene

* Use Auto Config in distributed

* Fix Partition Claim Spread

* Make partition use config

* Correct total count

* Fix Wait time to 5 mins

* Revert om yaml

* Fix Sink sync

* Add Failure Handling at different stages

* Update script to create entities

* Move to scripts

* Add usage and fix script

* Fix Script

* Update generated TypeScript types

* Fix Staging miss

* Fix Stats reconcilation issue

* Revert workflow handler

* Fix Partition worker early sync

* Update Logs

* Update logs EntityRepository

* Error failure test

* Review Comments fix

* Fix Non Distributed live feed

* Fix Non Distributed stats feed

* Fix Review comments

* Fix Time Series cutt off

* Update generated TypeScript types

* Md

* Benchmark addition

* Fix date time warning

* Update load test to do benchmark analysis

* Disagnostic and update perf test

* Move load test to bin

* Fix Review Comments

* Add numeric values

* Move to localhost by default

* Fix Perf test issues

* Review Comments

* Add Preflight Fixes

* Add Preflight fixes for stale entry

* Remove stale entry on ApplicationHandler

---------

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
(cherry picked from commit b59aa7fc44)
2026-03-03 16:41:16 +05:30
..
development Improve indexing (#26154) 2026-03-03 16:41:16 +05:30
docker-compose-ingestion chore(release): Prepare Branch for 1.12.2 2026-02-24 23:08:12 +00:00
docker-compose-openmetadata chore(release): Prepare Branch for 1.12.2 2026-02-24 23:08:12 +00:00
docker-compose-quickstart chore(release): Prepare Branch for 1.12.2 2026-02-24 23:08:12 +00:00
images/minimal-ubuntu Fix java 21 in docker (#21746) 2025-06-12 18:24:25 -07:00
mysql Pluggable API/Features Limits (#16782) 2024-07-01 14:59:25 +05:30
postgresql Fix #8208: OpenMetadata server supports Postgres 15 (#12370) 2023-08-02 11:13:28 +05:30
rdf-store fix #21555: Automator - Separating terms and tags in action config (#22970) 2025-08-28 10:50:48 +02:00
openmetadata-start.sh MINOR - Pydantic V2 warnings and better exception msg (#16916) 2024-07-04 14:54:41 +02:00
openmetadata.yaml Updating docker env and symbolic link for openmetadata.yaml (#12684) 2023-08-01 11:07:29 +05:30
run_local_docker.sh MINOR: Add option to disable ingestion in run_local_docker (#24676) 2025-12-03 12:35:29 +00:00
run_local_docker_rdf.sh RDF Ontology, Json LD, DCAT vocabulary support by mapping OM Schemas to RDF (#22852) 2025-08-17 18:36:26 -07:00
validate_compose.py Fix #23096: Add Airflow 3.x support (#24338) 2025-11-21 12:28:28 +01:00