hyperdx/packages/app/scripts/run-e2e.js
Warren Lee 6e8ddd3736
feat: isolate dev environment for multi-agent worktree support (#1994)
## Summary
- Isolate dev, E2E, and integration test environments so multiple git worktrees can run all three simultaneously without port conflicts
- Each worktree gets a deterministic slot (0-99) with unique port ranges: dev (30100-31199), E2E (20320-21399), CI integration (14320-40098)
- Dev portal dashboard (http://localhost:9900) auto-discovers all running stacks, streams logs, and provides a History tab for past run logs

## Port Isolation

| Environment | Port Range | Project Name |
|---|---|---|
| Dev stack | 30100-31199 | `hdx-dev-<slot>` |
| E2E tests | 20320-21399 | `e2e-<slot>` |
| CI integration | 14320-40098 | `int-<slot>` |

All three can run simultaneously from the same worktree with zero port conflicts.

## Dev Portal Features

**Live tab:**
- Auto-discovers dev, E2E, and integration Docker containers + local services (API, App)
- Groups all environments for the same worktree into a single card
- SSE log streaming with ANSI color rendering, capped at 5000 lines
- Auto-starts in background from `make dev`, `make dev-e2e`, `make dev-int`

**History tab:**
- Logs archived to `~/.config/hyperdx/dev-slots/<slot>/history/` on exit (instead of deleted)
- Each archived run includes `meta.json` with worktree/branch metadata
- Grouped by worktree with collapsible cards, search by worktree/branch
- View any past log file in the same log panel, delete individual runs or clear all
- Custom dark-themed confirm modal (no native browser dialogs)

## What Changed

- **`scripts/dev-env.sh`** — Slot-based port assignments, portal auto-start, log archival on exit
- **`scripts/test-e2e.sh`** — E2E port range (20320-21399), log capture via `tee`, portal auto-start, log archival
- **`scripts/ensure-dev-portal.sh`** — Shared singleton portal launcher (works sourced or executed)
- **`scripts/dev-portal/server.js`** — Discovery for dev/E2E/CI containers, history API (list/read/delete), local service port probing
- **`scripts/dev-portal/index.html`** — Live/History tabs, worktree-grouped cards, search, collapse/expand, custom confirm modal, ANSI color log rendering
- **`docker-compose.dev.yml`** — Parameterized ports/volumes/project name with `hdx.dev.*` labels
- **`packages/app/tests/e2e/docker-compose.yml`** — Updated to new E2E port defaults
- **`Makefile`** — `dev-int`/`dev-e2e` targets with log capture + portal auto-start; `dev-portal-stop`; `dev-clean` stops everything + wipes slot data
- **`.env` files** — Ports use `${VAR:-default}` syntax across dev, E2E, and CI environments
- **`agent_docs/development.md`** — Full documentation for isolation, port tables, E2E/CI port ranges

## How to Use

```bash
# Start dev stack (auto-starts portal)
make dev

# Run E2E tests (auto-starts portal, separate ports)
make dev-e2e FILE=navigation

# Run integration tests (auto-starts portal, separate ports)
make dev-int FILE=alerts

# All three can run simultaneously from the same worktree
# Portal at http://localhost:9900 shows everything

# Stop portal
make dev-portal-stop

# Clean up everything (all stacks + portal + history)
make dev-clean
```

## Dev Portal

<img width="1692" height="944" alt="image" src="https://github.com/user-attachments/assets/6ed388a3-43bc-4552-aa8d-688077b79fb7" />

<img width="1689" height="935" alt="image" src="https://github.com/user-attachments/assets/8677a138-0a40-4746-93ed-3b355c8bd45e" />

## Test Plan
- [x] Run `make dev` — verify services start with slot-assigned ports
- [x] Run `make dev` in a second worktree — verify different ports, no conflicts
- [x] Run `make dev-e2e` and `make dev-int` simultaneously — no port conflicts
- [x] Open http://localhost:9900 — verify all stacks grouped by worktree
- [x] Click a service to view logs — verify ANSI colors render correctly
- [x] Stop a stack — verify logs archived to History tab with correct worktree
- [x] History tab — search, collapse/expand, view archived logs, delete
- [x] `make dev-clean` — stops everything, wipes slot data and history
2026-03-31 18:24:24 +00:00

104 lines
3 KiB
JavaScript
Executable file

#!/usr/bin/env node
/**
* E2E Test Runner
*
* Usage:
* yarn test:e2e # Run all tests (full-stack mode, default)
* yarn test:e2e --local # Run frontend-only tests
* yarn test:e2e --ui # Open Playwright UI (full-stack)
* yarn test:e2e --ui --local # Open UI (local mode)
* yarn test:e2e --debug # Debug mode (full-stack)
* yarn test:e2e --debug --local # Debug (local mode)
* yarn test:e2e --dev # Hot reload (next dev) instead of build+start
*/
import { spawn } from 'child_process';
import fs from 'fs';
import path from 'path';
import { fileURLToPath } from 'url';
// Get __dirname equivalent in ES modules
const __filename = fileURLToPath(import.meta.url);
const __dirname = path.dirname(__filename);
// Parse command line arguments
const args = process.argv.slice(2);
const useLocal = args.includes('--local');
const useUI = args.includes('--ui');
const useDebug = args.includes('--debug');
const useDev = args.includes('--dev');
// Remove our custom flags from args
const playwrightArgs = args.filter(
arg => !['--local', '--ui', '--debug', '--dev'].includes(arg),
);
// Build playwright command
const playwrightCmd = ['playwright', 'test'];
// Add mode flags
if (useUI) {
playwrightCmd.push('--ui');
}
if (useDebug) {
playwrightCmd.push('--debug');
}
// Add grep-invert for local mode (exclude @full-stack tests)
if (useLocal) {
playwrightCmd.push('--grep-invert', '@full-stack');
}
// Add any additional playwright arguments
playwrightCmd.push(...playwrightArgs);
// Set environment variables (Playwright and its webServer children inherit these)
const env = {
...process.env,
...(!useLocal && { E2E_FULLSTACK: 'true' }),
...(useDev && { E2E_USE_DEV: 'true' }),
};
// Port configuration from HDX_E2E_* env vars (set by scripts/test-e2e.sh)
const chPort = env.HDX_E2E_CH_PORT || '20500';
// Ensure CLICKHOUSE_HOST is set for seed-clickhouse.ts (used by both modes)
env.CLICKHOUSE_HOST = `http://localhost:${chPort}`;
// Full-stack: inject DEFAULT_CONNECTIONS/DEFAULT_SOURCES from fixture so the API gets them
if (!useLocal) {
const fixturePath = path.join(
__dirname,
'../tests/e2e/fixtures/e2e-fixtures.json',
);
const fixture = JSON.parse(fs.readFileSync(fixturePath, 'utf8'));
// Patch connection hosts with dynamic ClickHouse port
const connections = (fixture.connections ?? []).map(conn => ({
...conn,
host: conn.host?.replace(/localhost:\d+/, `localhost:${chPort}`),
}));
env.DEFAULT_CONNECTIONS = JSON.stringify(connections);
env.DEFAULT_SOURCES = JSON.stringify(fixture.sources ?? []);
}
// Run playwright
// eslint-disable-next-line no-console
console.info(`Running: ${playwrightCmd.join(' ')}`);
// eslint-disable-next-line no-console
console.info(
`Mode: ${useLocal ? 'Local (frontend only)' : 'Full-stack'}${useDev ? ' + dev (hot reload)' : ''}`,
);
const child = spawn('npx', playwrightCmd, {
stdio: 'inherit',
shell: true,
env,
cwd: path.join(__dirname, '..'),
});
child.on('exit', code => {
process.exit(code);
});