PostgreSQL JDBC 42.7.7 uses synchronized blocks around network I/O (sending queries, reading
responses). With virtual threads, a thread that blocks inside synchronized gets pinned to
its carrier thread — it cannot unmount even when waiting for I/O.
With -XX:ActiveProcessorCount=2, there are exactly 2 ForkJoinPool carrier threads. The
moment 2 concurrent SQL queries are executing on virtual threads, both carrier threads are
pinned. The health probe's virtual thread becomes runnable but can't be scheduled — no
carrier thread is free. Probe times out. Repeat indefinitely.
Disabling virtual threads switches Jetty back to a 150-thread platform thread pool. Even if
100 threads are blocked waiting for DB connections, 50 remain available for the health probe
and other requests. The complete deadlock is impossible with platform threads
* Update Perf
* Add multi asset scale count
* Update perf and Usage
* Fix recommendation
* Add Benchmarking script and doc
* Fix Perf
* Add --no break to benchmark
* add more metrics and validation for indexes miss
* Update generated TypeScript types
* Bound Doc Virtual Threads
* Remove Additional Properties from the UI
* Update doc
* Fix Job Getting Marked Stopped
* Server killed logs fixes
* Add Server stat to Quartz Progress
* Fix CPU spiking
* Make Auto Tune Consider JVm configs
* Fix Partition Calculator and Recovery Job Stats
* Update Auto Tune to show up in logs and stored in config
* Fix Auto Tune Config not store in app run record
* Fix OnDemand Job type
* Indexing Failures not flushed fixed
* Fix Stat counting at job level with process job failures
* Add Reindex Job Identifier
* Add Thread Identifiers
* Wait for sink
* Wait for sink
* Fix Stopping to let partitions finish the job
* CPU Budgeting
* More Conservative settings
* Address Review Comment
* fix Open Search Index Manager
* Reapply OpenSearch BulkSink
---------
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
* Add Prometheus metrics for reindexing pipeline via Micrometer Bridge the existing reindexing atomic counters to Prometheus so operators can alert on failures, latency spikes, and backpressure without relying solely on database-flushed stats.
- Add ReindexingMetrics singleton (initialize/getInstance pattern matching
CacheMetrics) with job lifecycle counters, stage success/failed/warnings
counters, bulk request timers with SLA buckets, payload size distribution,
backpressure and promotion counters, and active/pending gauges
- Register in MicrometerBundle after StreamableLogsMetrics
- Instrument ReindexingOrchestrator.run() with job started/completed/failed/stopped
- Bridge StageStatsTracker.flush() deltas to Prometheus per stage and entity type
- Add bulk request latency timer and payload size recording in OpenSearchBulkSink
- Record backpressure events in SearchIndexExecutor.handleBackpressure()
- Record promotion success/failure in DefaultRecreateHandler
- Add ReindexingMetricsTest with 24 tests covering all metric types
* Add Improvements
* Auto Gene
* Use Auto Config in distributed
* Fix Partition Claim Spread
* Make partition use config
* Correct total count
* Fix Wait time to 5 mins
* Revert om yaml
* Fix Sink sync
* Add Failure Handling at different stages
* Update script to create entities
* Move to scripts
* Add usage and fix script
* Fix Script
* Update generated TypeScript types
* Fix Staging miss
* Fix Stats reconcilation issue
* Revert workflow handler
* Fix Partition worker early sync
* Update Logs
* Update logs EntityRepository
* Error failure test
* Review Comments fix
* Fix Non Distributed live feed
* Fix Non Distributed stats feed
* Fix Review comments
* Fix Time Series cutt off
* Update generated TypeScript types
* Md
* Benchmark addition
* Fix date time warning
* Update load test to do benchmark analysis
* Disagnostic and update perf test
* Move load test to bin
* Fix Review Comments
* Add numeric values
* Move to localhost by default
* Fix Perf test issues
* Review Comments
* Add Preflight Fixes
* Add Preflight fixes for stale entry
* Remove stale entry on ApplicationHandler
---------
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
