Closes HDX-3361
# Summary
This PR prevents various query errors caused by references to `bodyExpression` on trace sources. The `bodyExpression` should not exist on trace sources, and cannot be edited in the source form. Despite that, the `bodyExpression` would be set on trace sources during source inference.
- The `getEventBody` helper function will now correctly use the `spanNameExpression` field instead for trace sources. A few direct references to `bodyExpression` have been updated to `getEventBody` calls.
- Source configuration inference will no longer populate the `bodyExpression` for trace sources, and the default trace source will not be created with a `bodyExpression`.
- Introduce a new flag `HYPERDX_OTEL_EXPORTER_CREATE_LEGACY_SCHEMA` (default to false) to otel collector
- Custom ClickStack schemas should be enforced by default
- ClickHouse tables migration logs should be stored in `clickstack_db_version_xxx` tables
- The collector will run the migration at startup and retry if it fails to connect to the database (using exponential backoff).
- Fully backward compatible
Ref: HDX-3301
- Upgrades ClickHouse to 25.6, fixes breaking config change, needed for latest JSON type
- Upgrades OTel Collector to 0.129.1, fixes breaking config change, needed for latest JSON support in exporter
- Upgrades OTel OpAMP Supervisor to 0.128.0
- Fixes features to support JSON type columns in OTel in HyperDX (filtering, searching, graphing, opening rows, etc.)
Requires users to set `BETA_CH_OTEL_JSON_SCHEMA_ENABLED=true` in `ch-server` and `OTEL_AGENT_FEATURE_GATE_ARG='--feature-gates=clickhouse.json'` in `otel-collector` to enable JSON schema. Users must start a new ClickHouse DB or migrate their own table manually to enable as it is not schema compatible and migration is not automatic.
Closes HDX-1849, HDX-1969, HDX-1849, HDX-1966, HDX-1964
Co-authored-by: Tom Alexander <3245235+teeohhem@users.noreply.github.com>
like v1, users should be able to specify `HYPERDX_API_KEY` to export in-app telemetry to hyperdx itself.
running command like
```
HYPERDX_API_KEY=XXX docker compose up -d
```
- Support `CLICKHOUSE_ENDPOINT` to switch aio clickhouse endpoint (Ref: HDX-1758)
- Support `HYPERDX_OTEL_EXPORTER_CLICKHOUSE_DATABASE ` (Ref: HDX-1786)
Support command like
```
docker run -e CLICKHOUSE_ENDPOINT=<CH-CLOUD-ENDPOINT> -e CLICKHOUSE_USER=default -e CLICKHOUSE_PASSWORD='BLABLA' -e HYPERDX_OTEL_EXPORTER_CLICKHOUSE_DATABASE =hyperdx -p 8080:8080 -p 4317:4317 -p 4318:4318 hyperdx/hyperdx-local:2-nightly
```
So users can export data to other services like clickhouse cloud
For users connecting to ClickHouse Cloud or a TLS endpoint, add the `secure=true` query parameter or use the HTTPS protocol. Providing the full URL via the `CLICKHOUSE_SERVER_ENDPOINT` in the exporter's endpoint field should resolve this issue
Ref: HDX-1743
1. new env var RUN_SCHEDULED_TASKS_EXTERNALLY to opt out in-app task process
2. introduce new `prod-extended` build that includes mongodb process
3. GA k8s dashboard (only picking the connection)
4. bake check-alert task into fullstack app build
For better self-hosting experience, users should be able to run
```
docker run -e MONGO_URI=xxx -p 8080:8080 hyperdx/hyperdx:2-beta
```
to spin up the project that includes the server components
for docker-vompose v1x, it throws error like:
```
ERROR: The Compose file './docker-compose.yml' is invalid because:
services.ingestor.environment.VECTOR_OPENSSL_LEGACY_PROVIDER contains false, which is an invalid type, it should be a string, number, or a null
```
