## Summary
- Enable multiple agents/developers to run `make dev-int` simultaneously from different git worktrees without Docker port conflicts
- Compute a deterministic port offset (0-99) from the worktree directory name via `cksum`, giving each worktree its own isolated Docker Compose project and port range
- Switch `.env.test` files to use `${HDX_CI_*:-default}` variable expansion (powered by `dotenv-expand`) so test processes connect to the correct dynamic ports
## How it works
Each worktree gets a unique **slot** derived from its directory name. All service ports are offset by that slot:
| Service | Base port | Example (slot 68) |
|-----------------|-----------|-------------------|
| ClickHouse HTTP | 18123 | 18191 |
| MongoDB | 39999 | 40067 |
| API test server | 19000 | 19068 |
| OpAMP | 14320 | 14388 |
Docker Compose project names are also unique (`int-<slot>`), isolating containers and networks.
Backward compatible — when no `HDX_CI_*` env vars are set, all ports fall back to their original defaults.
## Changes
- **Makefile**: Added `HDX_CI_SLOT` computation and dynamic project names/ports for all `dev-int` targets
- **docker-compose.ci.yml**: Ports use `${HDX_CI_*:-default}` env vars; removed unused OTel collector published port; removed hardcoded network name (auto-generated from project name)
- **packages/api/.env.test** / **packages/common-utils/.env.test**: Ports use `${HDX_CI_*:-default}` expansion syntax
- **packages/api/jest.config.js** / **packages/common-utils/jest.int.config.js**: Switched from `dotenv/config` to `dotenv-expand/config` to enable variable expansion
- **packages/api/package.json** / **packages/common-utils/package.json**: Added `dotenv-expand` devDependency
- **agent_docs/development.md**: Documented multi-agent worktree support
## Testing
Ran full Alert integration test suite (`make dev-int FILE=alerts`) — **6 test suites, 150 tests passed** on slot 68 with dynamic ports.
## Summary
- Fix tile alerts to support `groupBy` for Gauge/Sum metrics — each group-by value appears as its own column in the response
- Add missing `whereLanguage` to tile alert config so Lucene WHERE conditions are parsed correctly
- Replace stale fixture-based ClickHouse schema with otel-collector's canonical schema in integration tests
Ref: HDX-3576
# Summary
This PR updates the CI (integration tests) docker compose and env file so that the integration tests run on a separate set of ports, so that they can be run locally without interfering with an instance of the app running locally.
## Testing
To test locally
Start the app
```
yarn dev
```
Run the integration tests
```
make ci-int
```
Note that both can run at the same time without docker port conflicts. `make ci-int` can also be run without running the app - it uses its own independent mongo + clickhouse + api.
- Introduce a new flag `HYPERDX_OTEL_EXPORTER_CREATE_LEGACY_SCHEMA` (default to false) to otel collector
- Custom ClickStack schemas should be enforced by default
- ClickHouse tables migration logs should be stored in `clickstack_db_version_xxx` tables
- The collector will run the migration at startup and retry if it fails to connect to the database (using exponential backoff).
- Fully backward compatible
Ref: HDX-3301
- Upgrades ClickHouse to 25.6, fixes breaking config change, needed for latest JSON type
- Upgrades OTel Collector to 0.129.1, fixes breaking config change, needed for latest JSON support in exporter
- Upgrades OTel OpAMP Supervisor to 0.128.0
- Fixes features to support JSON type columns in OTel in HyperDX (filtering, searching, graphing, opening rows, etc.)
Requires users to set `BETA_CH_OTEL_JSON_SCHEMA_ENABLED=true` in `ch-server` and `OTEL_AGENT_FEATURE_GATE_ARG='--feature-gates=clickhouse.json'` in `otel-collector` to enable JSON schema. Users must start a new ClickHouse DB or migrate their own table manually to enable as it is not schema compatible and migration is not automatic.
Closes HDX-1849, HDX-1969, HDX-1849, HDX-1966, HDX-1964
Co-authored-by: Tom Alexander <3245235+teeohhem@users.noreply.github.com>
- Support `CLICKHOUSE_ENDPOINT` to switch aio clickhouse endpoint (Ref: HDX-1758)
- Support `HYPERDX_OTEL_EXPORTER_CLICKHOUSE_DATABASE ` (Ref: HDX-1786)
Support command like
```
docker run -e CLICKHOUSE_ENDPOINT=<CH-CLOUD-ENDPOINT> -e CLICKHOUSE_USER=default -e CLICKHOUSE_PASSWORD='BLABLA' -e HYPERDX_OTEL_EXPORTER_CLICKHOUSE_DATABASE =hyperdx -p 8080:8080 -p 4317:4317 -p 4318:4318 hyperdx/hyperdx-local:2-nightly
```
So users can export data to other services like clickhouse cloud
For users connecting to ClickHouse Cloud or a TLS endpoint, add the `secure=true` query parameter or use the HTTPS protocol. Providing the full URL via the `CLICKHOUSE_SERVER_ENDPOINT` in the exporter's endpoint field should resolve this issue
Ref: HDX-1743
This PR was opened by the [Changesets release](https://github.com/changesets/action) GitHub action. When you're ready to do a release, you can merge this and the packages will be published to npm automatically. If you're not ready to do a release yet, that's fine, whenever you add more changesets to v2, this PR will be updated.
# Releases
## @hyperdx/common-utils@0.0.10
### Patch Changes
- fc4548f: feat: add alert schema + types
Co-authored-by: Warren <5959690+wrn14897@users.noreply.github.com>
This PR includes changes:
1. Rename `type` field to `thresholdType` (enum 'above' or 'below')
2. Introduce alert source enum ('saved_search' or 'tile')
3. Rename `dashboardId` field to `dashboard`
4. Deprecate unused `cron` field
5. Deprecate external alert translation layers
6. Deprecate `appType` flag
7. Copied over 'common' pkgs from app dir (mostly clickhouse query helpers + types) -> will become a sharable pkg between app and api
For better self-hosting experience, users should be able to run
```
docker run -e MONGO_URI=xxx -p 8080:8080 hyperdx/hyperdx:2-beta
```
to spin up the project that includes the server components
for docker-vompose v1x, it throws error like:
```
ERROR: The Compose file './docker-compose.yml' is invalid because:
services.ingestor.environment.VECTOR_OPENSSL_LEGACY_PROVIDER contains false, which is an invalid type, it should be a string, number, or a null
```
