## Summary
- Fix dev environment port isolation so OTel collector, MongoDB, and ClickHouse resolve to the correct worktree-specific ports
- Add slot-specific session cookies so multiple worktrees on localhost don't interfere with each other's login sessions
- Share NX build cache across worktrees for faster builds
- Add worktrunk project config for automated worktree lifecycle management
## Changes
### Port resolution fix
**Root `.env`**: Replace self-referential `HDX_DEV_*` vars (e.g. `HDX_DEV_OTEL_HTTP_PORT=${HDX_DEV_OTEL_HTTP_PORT:-4318}`) with plain default values. The `${VAR:-default}` syntax caused `dotenv-expand` infinite recursion, preventing `dev-env.sh` port overrides from taking effect.
**`packages/api/.env.development`**: Remove redundant self-referential port declarations and `:-default` fallbacks from `${HDX_DEV_*}` references. Ports now come exclusively from `dev-env.sh` exports or root `.env` defaults.
**`packages/app/.env.development`**: Set `HYPERDX_API_KEY` to `super-secure-ingestion-api-key` (matching the API/collector) instead of a placeholder. Remove self-referential port declarations.
**`packages/app/src/config.ts`**: Add `process.env.OTEL_EXPORTER_OTLP_ENDPOINT` as fallback for `HDX_COLLECTOR_URL` so the browser OTel SDK picks up the correct collector endpoint.
### Session cookie isolation
**`packages/api/src/api-app.ts`**: Use `connect.sid.<slot>` as the session cookie name in dev mode so multiple worktrees on `localhost` maintain independent sessions. Guarded behind `config.IS_DEV && process.env.HDX_DEV_SLOT` — production uses the default `connect.sid`.
### Shared NX build cache
**`scripts/dev-env.sh`**: Set `NX_CACHE_DIRECTORY=~/.config/hyperdx/nx-cache` so all worktrees share a single content-hash-based build cache. Unchanged packages get cache hits regardless of worktree; changed packages rebuild correctly.
### Worktrunk project config
**`.config/wt.toml`**: New project config for [worktrunk](https://worktrunk.dev) (`wt`) worktree lifecycle hooks:
- `pre-start`: Symlink `node_modules/` from primary worktree (instant, no copy)
- `post-start`: Copy `.env.local` from primary worktree
- `post-remove`: Tear down Docker stacks (`dev-down`, `dev-int-down`, `dev-e2e-down`) for the removed worktree's slot
## Summary
- Isolate dev, E2E, and integration test environments so multiple git worktrees can run all three simultaneously without port conflicts
- Each worktree gets a deterministic slot (0-99) with unique port ranges: dev (30100-31199), E2E (20320-21399), CI integration (14320-40098)
- Dev portal dashboard (http://localhost:9900) auto-discovers all running stacks, streams logs, and provides a History tab for past run logs
## Port Isolation
| Environment | Port Range | Project Name |
|---|---|---|
| Dev stack | 30100-31199 | `hdx-dev-<slot>` |
| E2E tests | 20320-21399 | `e2e-<slot>` |
| CI integration | 14320-40098 | `int-<slot>` |
All three can run simultaneously from the same worktree with zero port conflicts.
## Dev Portal Features
**Live tab:**
- Auto-discovers dev, E2E, and integration Docker containers + local services (API, App)
- Groups all environments for the same worktree into a single card
- SSE log streaming with ANSI color rendering, capped at 5000 lines
- Auto-starts in background from `make dev`, `make dev-e2e`, `make dev-int`
**History tab:**
- Logs archived to `~/.config/hyperdx/dev-slots/<slot>/history/` on exit (instead of deleted)
- Each archived run includes `meta.json` with worktree/branch metadata
- Grouped by worktree with collapsible cards, search by worktree/branch
- View any past log file in the same log panel, delete individual runs or clear all
- Custom dark-themed confirm modal (no native browser dialogs)
## What Changed
- **`scripts/dev-env.sh`** — Slot-based port assignments, portal auto-start, log archival on exit
- **`scripts/test-e2e.sh`** — E2E port range (20320-21399), log capture via `tee`, portal auto-start, log archival
- **`scripts/ensure-dev-portal.sh`** — Shared singleton portal launcher (works sourced or executed)
- **`scripts/dev-portal/server.js`** — Discovery for dev/E2E/CI containers, history API (list/read/delete), local service port probing
- **`scripts/dev-portal/index.html`** — Live/History tabs, worktree-grouped cards, search, collapse/expand, custom confirm modal, ANSI color log rendering
- **`docker-compose.dev.yml`** — Parameterized ports/volumes/project name with `hdx.dev.*` labels
- **`packages/app/tests/e2e/docker-compose.yml`** — Updated to new E2E port defaults
- **`Makefile`** — `dev-int`/`dev-e2e` targets with log capture + portal auto-start; `dev-portal-stop`; `dev-clean` stops everything + wipes slot data
- **`.env` files** — Ports use `${VAR:-default}` syntax across dev, E2E, and CI environments
- **`agent_docs/development.md`** — Full documentation for isolation, port tables, E2E/CI port ranges
## How to Use
```bash
# Start dev stack (auto-starts portal)
make dev
# Run E2E tests (auto-starts portal, separate ports)
make dev-e2e FILE=navigation
# Run integration tests (auto-starts portal, separate ports)
make dev-int FILE=alerts
# All three can run simultaneously from the same worktree
# Portal at http://localhost:9900 shows everything
# Stop portal
make dev-portal-stop
# Clean up everything (all stacks + portal + history)
make dev-clean
```
## Dev Portal
<img width="1692" height="944" alt="image" src="https://github.com/user-attachments/assets/6ed388a3-43bc-4552-aa8d-688077b79fb7" />
<img width="1689" height="935" alt="image" src="https://github.com/user-attachments/assets/8677a138-0a40-4746-93ed-3b355c8bd45e" />
## Test Plan
- [x] Run `make dev` — verify services start with slot-assigned ports
- [x] Run `make dev` in a second worktree — verify different ports, no conflicts
- [x] Run `make dev-e2e` and `make dev-int` simultaneously — no port conflicts
- [x] Open http://localhost:9900 — verify all stacks grouped by worktree
- [x] Click a service to view logs — verify ANSI colors render correctly
- [x] Stop a stack — verify logs archived to History tab with correct worktree
- [x] History tab — search, collapse/expand, view archived logs, delete
- [x] `make dev-clean` — stops everything, wipes slot data and history
## Summary
- Bump `@hyperdx/browser` to 0.22.0 and pass `service.version` as an OTel resource attribute to the browser SDK, so all frontend telemetry includes the app version
- Inject `OTEL_RESOURCE_ATTRIBUTES=service.version=$CODE_VERSION` in API and all-in-one Dockerfiles (prod) and `.env.development` (dev) so the Node OTel SDK attaches service version to all backend traces, metrics, and logs
- Remove the standalone Next.js health endpoint — `/api/health` now proxies through to the API server. The original `/api/health` page was redundant since `/api/config` already serves the same purpose
Closes HDX-3361
# Summary
This PR prevents various query errors caused by references to `bodyExpression` on trace sources. The `bodyExpression` should not exist on trace sources, and cannot be edited in the source form. Despite that, the `bodyExpression` would be set on trace sources during source inference.
- The `getEventBody` helper function will now correctly use the `spanNameExpression` field instead for trace sources. A few direct references to `bodyExpression` have been updated to `getEventBody` calls.
- Source configuration inference will no longer populate the `bodyExpression` for trace sources, and the default trace source will not be created with a `bodyExpression`.
Closes HDX-2283
This adds an environment variable `INGESTION_API_KEY` that can be set by the user. This apiKey will be valid and accepted by the Otel collector. It is in addition to the autogenerated apiKey and will not show in the team settings apiKey section.
* Adds support for auto-openapi doc generation
* Adds a swagger route for local development
* Adds a script to manually generate the openapi doc (to be used later if we publish to our site, etc...)
Ref: HDX-1661
