## Summary
- Isolate dev, E2E, and integration test environments so multiple git worktrees can run all three simultaneously without port conflicts
- Each worktree gets a deterministic slot (0-99) with unique port ranges: dev (30100-31199), E2E (20320-21399), CI integration (14320-40098)
- Dev portal dashboard (http://localhost:9900) auto-discovers all running stacks, streams logs, and provides a History tab for past run logs
## Port Isolation
| Environment | Port Range | Project Name |
|---|---|---|
| Dev stack | 30100-31199 | `hdx-dev-<slot>` |
| E2E tests | 20320-21399 | `e2e-<slot>` |
| CI integration | 14320-40098 | `int-<slot>` |
All three can run simultaneously from the same worktree with zero port conflicts.
## Dev Portal Features
**Live tab:**
- Auto-discovers dev, E2E, and integration Docker containers + local services (API, App)
- Groups all environments for the same worktree into a single card
- SSE log streaming with ANSI color rendering, capped at 5000 lines
- Auto-starts in background from `make dev`, `make dev-e2e`, `make dev-int`
**History tab:**
- Logs archived to `~/.config/hyperdx/dev-slots/<slot>/history/` on exit (instead of deleted)
- Each archived run includes `meta.json` with worktree/branch metadata
- Grouped by worktree with collapsible cards, search by worktree/branch
- View any past log file in the same log panel, delete individual runs or clear all
- Custom dark-themed confirm modal (no native browser dialogs)
## What Changed
- **`scripts/dev-env.sh`** — Slot-based port assignments, portal auto-start, log archival on exit
- **`scripts/test-e2e.sh`** — E2E port range (20320-21399), log capture via `tee`, portal auto-start, log archival
- **`scripts/ensure-dev-portal.sh`** — Shared singleton portal launcher (works sourced or executed)
- **`scripts/dev-portal/server.js`** — Discovery for dev/E2E/CI containers, history API (list/read/delete), local service port probing
- **`scripts/dev-portal/index.html`** — Live/History tabs, worktree-grouped cards, search, collapse/expand, custom confirm modal, ANSI color log rendering
- **`docker-compose.dev.yml`** — Parameterized ports/volumes/project name with `hdx.dev.*` labels
- **`packages/app/tests/e2e/docker-compose.yml`** — Updated to new E2E port defaults
- **`Makefile`** — `dev-int`/`dev-e2e` targets with log capture + portal auto-start; `dev-portal-stop`; `dev-clean` stops everything + wipes slot data
- **`.env` files** — Ports use `${VAR:-default}` syntax across dev, E2E, and CI environments
- **`agent_docs/development.md`** — Full documentation for isolation, port tables, E2E/CI port ranges
## How to Use
```bash
# Start dev stack (auto-starts portal)
make dev
# Run E2E tests (auto-starts portal, separate ports)
make dev-e2e FILE=navigation
# Run integration tests (auto-starts portal, separate ports)
make dev-int FILE=alerts
# All three can run simultaneously from the same worktree
# Portal at http://localhost:9900 shows everything
# Stop portal
make dev-portal-stop
# Clean up everything (all stacks + portal + history)
make dev-clean
```
## Dev Portal
<img width="1692" height="944" alt="image" src="https://github.com/user-attachments/assets/6ed388a3-43bc-4552-aa8d-688077b79fb7" />
<img width="1689" height="935" alt="image" src="https://github.com/user-attachments/assets/8677a138-0a40-4746-93ed-3b355c8bd45e" />
## Test Plan
- [x] Run `make dev` — verify services start with slot-assigned ports
- [x] Run `make dev` in a second worktree — verify different ports, no conflicts
- [x] Run `make dev-e2e` and `make dev-int` simultaneously — no port conflicts
- [x] Open http://localhost:9900 — verify all stacks grouped by worktree
- [x] Click a service to view logs — verify ANSI colors render correctly
- [x] Stop a stack — verify logs archived to History tab with correct worktree
- [x] History tab — search, collapse/expand, view archived logs, delete
- [x] `make dev-clean` — stops everything, wipes slot data and history
## Summary
- Adds worktree-aware port isolation for E2E tests, mirroring the existing `dev-int` slot mechanism so multiple agents/developers can run E2E tests in parallel without port conflicts
- Fixes the navigation E2E test that was broken by Live Tail URL updates swallowing client-side navigation
- Adds `dev-e2e` Makefile target for running specific tests with `FILE=` and `GREP=` filters, plus `REPORT=1` to open the HTML report after tests finish
## Port Isolation
Each worktree gets a deterministic slot (0–99) computed from its directory name. All E2E service ports are offset by that slot in the **44000–50100** range, avoiding collisions with `dev` (4317–27017) and `dev-int` (14320–40098).
| Service | Base + slot | Variable |
|---|---|---|
| ClickHouse HTTP | 48123 + slot | `HDX_E2E_CH_PORT` |
| ClickHouse Native | 49000 + slot | `HDX_E2E_CH_NATIVE_PORT` |
| MongoDB | 49998 + slot | `HDX_E2E_MONGO_PORT` |
| API server | 49100 + slot | `HDX_E2E_API_PORT` |
| App (fullstack) | 48081 + slot | `HDX_E2E_APP_PORT` |
| App (local) | 48001 + slot | `HDX_E2E_APP_LOCAL_PORT` |
| OpAMP | 44320 + slot | `HDX_E2E_OPAMP_PORT` |
## New Make Targets
```bash
make dev-e2e FILE=navigation # Run specific test file
make dev-e2e FILE=navigation GREP="help menu" # Filter by test name
make dev-e2e GREP="should navigate" # Grep across all files
make dev-e2e FILE=navigation REPORT=1 # Open HTML report after run
make dev-e2e-clean # Remove test artifacts
```
## Linear
https://linear.app/hyperdx/issue/HDX-3796
## Summary
- Enable multiple agents/developers to run `make dev-int` simultaneously from different git worktrees without Docker port conflicts
- Compute a deterministic port offset (0-99) from the worktree directory name via `cksum`, giving each worktree its own isolated Docker Compose project and port range
- Switch `.env.test` files to use `${HDX_CI_*:-default}` variable expansion (powered by `dotenv-expand`) so test processes connect to the correct dynamic ports
## How it works
Each worktree gets a unique **slot** derived from its directory name. All service ports are offset by that slot:
| Service | Base port | Example (slot 68) |
|-----------------|-----------|-------------------|
| ClickHouse HTTP | 18123 | 18191 |
| MongoDB | 39999 | 40067 |
| API test server | 19000 | 19068 |
| OpAMP | 14320 | 14388 |
Docker Compose project names are also unique (`int-<slot>`), isolating containers and networks.
Backward compatible — when no `HDX_CI_*` env vars are set, all ports fall back to their original defaults.
## Changes
- **Makefile**: Added `HDX_CI_SLOT` computation and dynamic project names/ports for all `dev-int` targets
- **docker-compose.ci.yml**: Ports use `${HDX_CI_*:-default}` env vars; removed unused OTel collector published port; removed hardcoded network name (auto-generated from project name)
- **packages/api/.env.test** / **packages/common-utils/.env.test**: Ports use `${HDX_CI_*:-default}` expansion syntax
- **packages/api/jest.config.js** / **packages/common-utils/jest.int.config.js**: Switched from `dotenv/config` to `dotenv-expand/config` to enable variable expansion
- **packages/api/package.json** / **packages/common-utils/package.json**: Added `dotenv-expand` devDependency
- **agent_docs/development.md**: Documented multi-agent worktree support
## Testing
Ran full Alert integration test suite (`make dev-int FILE=alerts`) — **6 test suites, 150 tests passed** on slot 68 with dynamic ports.
* Also adds a helpful test:e2e:ci script to test playwright tests inside of a docker image with a clean build to more closely resemble running in CI
* Upgrades playwright version
