6e8ddd3736
## Summary - Isolate dev, E2E, and integration test environments so multiple git worktrees can run all three simultaneously without port conflicts - Each worktree gets a deterministic slot (0-99) with unique port ranges: dev (30100-31199), E2E (20320-21399), CI integration (14320-40098) - Dev portal dashboard (http://localhost:9900) auto-discovers all running stacks, streams logs, and provides a History tab for past run logs ## Port Isolation | Environment | Port Range | Project Name | |---|---|---| | Dev stack | 30100-31199 | `hdx-dev-<slot>` | | E2E tests | 20320-21399 | `e2e-<slot>` | | CI integration | 14320-40098 | `int-<slot>` | All three can run simultaneously from the same worktree with zero port conflicts. ## Dev Portal Features **Live tab:** - Auto-discovers dev, E2E, and integration Docker containers + local services (API, App) - Groups all environments for the same worktree into a single card - SSE log streaming with ANSI color rendering, capped at 5000 lines - Auto-starts in background from `make dev`, `make dev-e2e`, `make dev-int` **History tab:** - Logs archived to `~/.config/hyperdx/dev-slots/<slot>/history/` on exit (instead of deleted) - Each archived run includes `meta.json` with worktree/branch metadata - Grouped by worktree with collapsible cards, search by worktree/branch - View any past log file in the same log panel, delete individual runs or clear all - Custom dark-themed confirm modal (no native browser dialogs) ## What Changed - **`scripts/dev-env.sh`** — Slot-based port assignments, portal auto-start, log archival on exit - **`scripts/test-e2e.sh`** — E2E port range (20320-21399), log capture via `tee`, portal auto-start, log archival - **`scripts/ensure-dev-portal.sh`** — Shared singleton portal launcher (works sourced or executed) - **`scripts/dev-portal/server.js`** — Discovery for dev/E2E/CI containers, history API (list/read/delete), local service port probing - **`scripts/dev-portal/index.html`** — Live/History tabs, worktree-grouped cards, search, collapse/expand, custom confirm modal, ANSI color log rendering - **`docker-compose.dev.yml`** — Parameterized ports/volumes/project name with `hdx.dev.*` labels - **`packages/app/tests/e2e/docker-compose.yml`** — Updated to new E2E port defaults - **`Makefile`** — `dev-int`/`dev-e2e` targets with log capture + portal auto-start; `dev-portal-stop`; `dev-clean` stops everything + wipes slot data - **`.env` files** — Ports use `${VAR:-default}` syntax across dev, E2E, and CI environments - **`agent_docs/development.md`** — Full documentation for isolation, port tables, E2E/CI port ranges ## How to Use ```bash # Start dev stack (auto-starts portal) make dev # Run E2E tests (auto-starts portal, separate ports) make dev-e2e FILE=navigation # Run integration tests (auto-starts portal, separate ports) make dev-int FILE=alerts # All three can run simultaneously from the same worktree # Portal at http://localhost:9900 shows everything # Stop portal make dev-portal-stop # Clean up everything (all stacks + portal + history) make dev-clean ``` ## Dev Portal <img width="1692" height="944" alt="image" src="https://github.com/user-attachments/assets/6ed388a3-43bc-4552-aa8d-688077b79fb7" /> <img width="1689" height="935" alt="image" src="https://github.com/user-attachments/assets/8677a138-0a40-4746-93ed-3b355c8bd45e" /> ## Test Plan - [x] Run `make dev` — verify services start with slot-assigned ports - [x] Run `make dev` in a second worktree — verify different ports, no conflicts - [x] Run `make dev-e2e` and `make dev-int` simultaneously — no port conflicts - [x] Open http://localhost:9900 — verify all stacks grouped by worktree - [x] Click a service to view logs — verify ANSI colors render correctly - [x] Stop a stack — verify logs archived to History tab with correct worktree - [x] History tab — search, collapse/expand, view archived logs, delete - [x] `make dev-clean` — stops everything, wipes slot data and history
4.4 KiB
Contributing
Architecture Overview
Service Descriptions:
- OpenTelemetry Collector (otel-collector): Receives OpenTelemetry data from instrumented applications and forwards it to ClickHouse for storage. Includes OpAMP supervisor that dynamically pulls configuration from HyperDX API.
- ClickHouse (ch-server): ClickHouse database, stores all telemetry.
- MongoDB (db): Stores user/saved search/alert/dashboard data.
- HyperDX API (api): Node.js API, executes ClickHouse queries on behalf of the frontend and serves the frontend. serves the frontend. Can also run alert checker.
- HyperDX UI (app): Next.js frontend, serves the UI.
Development
Pre-requisites:
- Docker
- Node.js (
>=22) - Yarn (v4)
You can get started by deploying a complete development stack in dev mode.
yarn dev
This will start the Node.js API, Next.js frontend locally and the OpenTelemetry collector and ClickHouse server in Docker.
Each worktree automatically gets unique ports so multiple developers (or agents)
can run yarn dev simultaneously without conflicts. A dev portal at
http://localhost:9900 auto-starts and shows all running stacks with their
assigned ports. Check the portal to find the URL for your instance.
To stop the stack:
yarn dev:down
To enable self-instrumentation and demo logs, you can set the HYPERDX_API_KEY
to your ingestion key (visit the Team settings page after creating your account).
To do this, create a .env.local file in the root of the project and add the
following:
HYPERDX_API_KEY=<YOUR_INGESTION_API_KEY_HERE>
Then restart the stack using yarn dev.
The core services are all hot-reloaded, so you can make changes to the code and see them reflected in real-time.
Volumes
The development stack mounts volumes locally for persisting storage under
.volumes. Each worktree gets its own volume directory (e.g.
.volumes/ch_data_dev_89). Clear the .volumes directory to reset ClickHouse
and MongoDB storage.
Windows
If you are running WSL 2, Hot module reload on Nextjs (Frontend) does not work out of the box on windows when run natively on docker. The fix here is to open project directory in WSL and run the above docker compose commands directly in WSL. Note that the project directory should not be under /mnt/c/ directory. You can clone the git repo in /home/{username} for example.
To develop from WSL, follow instructions here.
Testing
All test environments use slot-based port isolation, so they can run simultaneously with the dev stack and across multiple worktrees.
E2E Tests
E2E tests run against a full local stack (MongoDB + ClickHouse + API). Docker must be running.
# Run all E2E tests
make e2e
# Run a specific spec file (dev mode: hot reload, containers kept running)
make dev-e2e FILE=search
# Run with grep pattern
make dev-e2e FILE=search GREP="filter"
# Run via script directly for more control
./scripts/test-e2e.sh --ui --last-failed
Tests live in packages/app/tests/e2e/. Page objects are in page-objects/,
shared components in components/.
Integration Tests
# Build dependencies (run once before first test run)
make dev-int-build
# Run a specific test file
make dev-int FILE=checkAlerts
Unit Tests
To run unit tests or update snapshots, you can go to the package you want (ex. common-utils) to test and run:
yarn dev:unit
AI-Assisted Development
The repo ships with configuration for AI coding assistants that enables interactive browser-based E2E test generation and debugging via the Playwright MCP server.
Claude Code
The project includes agents and skills for test generation, healing, and planning under .claude/. These are loaded automatically when you open the project in Claude Code. No additional setup required.
Cursor
A Playwright MCP server config is included at .cursor/mcp.json. To activate it:
- Open Cursor Settings → Tools & MCP
- The
playwright-testserver should appear automatically from the project config - Enable it
This gives Cursor's AI access to a live browser for test exploration and debugging.
Additional support
If you need help getting started, join our Discord and we're more than happy to get you set up!