mirror of
https://github.com/hyperdxio/hyperdx
synced 2026-04-21 13:37:15 +00:00
## Summary - Isolate dev, E2E, and integration test environments so multiple git worktrees can run all three simultaneously without port conflicts - Each worktree gets a deterministic slot (0-99) with unique port ranges: dev (30100-31199), E2E (20320-21399), CI integration (14320-40098) - Dev portal dashboard (http://localhost:9900) auto-discovers all running stacks, streams logs, and provides a History tab for past run logs ## Port Isolation | Environment | Port Range | Project Name | |---|---|---| | Dev stack | 30100-31199 | `hdx-dev-<slot>` | | E2E tests | 20320-21399 | `e2e-<slot>` | | CI integration | 14320-40098 | `int-<slot>` | All three can run simultaneously from the same worktree with zero port conflicts. ## Dev Portal Features **Live tab:** - Auto-discovers dev, E2E, and integration Docker containers + local services (API, App) - Groups all environments for the same worktree into a single card - SSE log streaming with ANSI color rendering, capped at 5000 lines - Auto-starts in background from `make dev`, `make dev-e2e`, `make dev-int` **History tab:** - Logs archived to `~/.config/hyperdx/dev-slots/<slot>/history/` on exit (instead of deleted) - Each archived run includes `meta.json` with worktree/branch metadata - Grouped by worktree with collapsible cards, search by worktree/branch - View any past log file in the same log panel, delete individual runs or clear all - Custom dark-themed confirm modal (no native browser dialogs) ## What Changed - **`scripts/dev-env.sh`** — Slot-based port assignments, portal auto-start, log archival on exit - **`scripts/test-e2e.sh`** — E2E port range (20320-21399), log capture via `tee`, portal auto-start, log archival - **`scripts/ensure-dev-portal.sh`** — Shared singleton portal launcher (works sourced or executed) - **`scripts/dev-portal/server.js`** — Discovery for dev/E2E/CI containers, history API (list/read/delete), local service port probing - **`scripts/dev-portal/index.html`** — Live/History tabs, worktree-grouped cards, search, collapse/expand, custom confirm modal, ANSI color log rendering - **`docker-compose.dev.yml`** — Parameterized ports/volumes/project name with `hdx.dev.*` labels - **`packages/app/tests/e2e/docker-compose.yml`** — Updated to new E2E port defaults - **`Makefile`** — `dev-int`/`dev-e2e` targets with log capture + portal auto-start; `dev-portal-stop`; `dev-clean` stops everything + wipes slot data - **`.env` files** — Ports use `${VAR:-default}` syntax across dev, E2E, and CI environments - **`agent_docs/development.md`** — Full documentation for isolation, port tables, E2E/CI port ranges ## How to Use ```bash # Start dev stack (auto-starts portal) make dev # Run E2E tests (auto-starts portal, separate ports) make dev-e2e FILE=navigation # Run integration tests (auto-starts portal, separate ports) make dev-int FILE=alerts # All three can run simultaneously from the same worktree # Portal at http://localhost:9900 shows everything # Stop portal make dev-portal-stop # Clean up everything (all stacks + portal + history) make dev-clean ``` ## Dev Portal <img width="1692" height="944" alt="image" src="https://github.com/user-attachments/assets/6ed388a3-43bc-4552-aa8d-688077b79fb7" /> <img width="1689" height="935" alt="image" src="https://github.com/user-attachments/assets/8677a138-0a40-4746-93ed-3b355c8bd45e" /> ## Test Plan - [x] Run `make dev` — verify services start with slot-assigned ports - [x] Run `make dev` in a second worktree — verify different ports, no conflicts - [x] Run `make dev-e2e` and `make dev-int` simultaneously — no port conflicts - [x] Open http://localhost:9900 — verify all stacks grouped by worktree - [x] Click a service to view logs — verify ANSI colors render correctly - [x] Stop a stack — verify logs archived to History tab with correct worktree - [x] History tab — search, collapse/expand, view archived logs, delete - [x] `make dev-clean` — stops everything, wipes slot data and history
109 lines
3.2 KiB
YAML
109 lines
3.2 KiB
YAML
name: E2E Tests
|
|
on:
|
|
workflow_call:
|
|
|
|
jobs:
|
|
e2e-tests:
|
|
name: E2E Tests - Shard ${{ matrix.shard }}
|
|
runs-on: ubuntu-24.04
|
|
timeout-minutes: 15
|
|
permissions:
|
|
contents: read
|
|
strategy:
|
|
fail-fast: false
|
|
matrix:
|
|
shard: [1, 2, 3, 4]
|
|
|
|
# E2E port configuration (slot 0 defaults — must match scripts/test-e2e.sh)
|
|
env:
|
|
HDX_E2E_SLOT: '0'
|
|
HDX_E2E_OPAMP_PORT: '20320'
|
|
HDX_E2E_CH_PORT: '20500'
|
|
HDX_E2E_CH_NATIVE_PORT: '20600'
|
|
HDX_E2E_API_PORT: '21000'
|
|
HDX_E2E_MONGO_PORT: '21100'
|
|
HDX_E2E_APP_LOCAL_PORT: '21200'
|
|
HDX_E2E_APP_PORT: '21300'
|
|
|
|
steps:
|
|
- name: Checkout
|
|
uses: actions/checkout@v4
|
|
|
|
- name: Setup Node.js
|
|
uses: actions/setup-node@v4
|
|
with:
|
|
node-version-file: '.nvmrc'
|
|
cache-dependency-path: 'yarn.lock'
|
|
cache: 'yarn'
|
|
|
|
- name: Install dependencies
|
|
run: yarn install
|
|
|
|
- name: Build dependencies
|
|
run: npx nx run-many -t ci:build
|
|
|
|
- name: Install Playwright browsers
|
|
run: cd packages/app && npx playwright install --with-deps chromium
|
|
|
|
- name: Start E2E Docker Compose
|
|
run: |
|
|
docker compose -p e2e-0 -f packages/app/tests/e2e/docker-compose.yml up -d
|
|
echo "Waiting for MongoDB..."
|
|
for i in $(seq 1 30); do
|
|
if docker compose -p e2e-0 -f packages/app/tests/e2e/docker-compose.yml exec -T db mongosh --quiet --eval "db.adminCommand({ping:1})" >/dev/null 2>&1; then
|
|
echo "MongoDB is ready"
|
|
break
|
|
fi
|
|
if [ "$i" -eq 30 ]; then
|
|
echo "MongoDB failed to become ready after 30 seconds"
|
|
exit 1
|
|
fi
|
|
echo "Waiting for MongoDB... ($i/30)"
|
|
sleep 1
|
|
done
|
|
echo "Waiting for ClickHouse..."
|
|
for i in $(seq 1 60); do
|
|
if curl -sf http://localhost:${HDX_E2E_CH_PORT}/ping >/dev/null 2>&1; then
|
|
echo "ClickHouse is ready"
|
|
break
|
|
fi
|
|
if [ "$i" -eq 60 ]; then
|
|
echo "ClickHouse failed to become ready after 60 seconds"
|
|
exit 1
|
|
fi
|
|
echo "Waiting for ClickHouse... ($i/60)"
|
|
sleep 1
|
|
done
|
|
|
|
- name: Run Playwright tests (full-stack mode)
|
|
env:
|
|
E2E_FULLSTACK: 'true'
|
|
E2E_UNIQUE_USER: 'true'
|
|
E2E_API_HEALTH_CHECK_MAX_RETRIES: '60'
|
|
MONGO_URI:
|
|
mongodb://localhost:${{ env.HDX_E2E_MONGO_PORT }}/hyperdx-e2e
|
|
run: |
|
|
cd packages/app
|
|
yarn test:e2e --shard=${{ matrix.shard }}/4
|
|
|
|
- name: Upload Playwright report
|
|
uses: actions/upload-artifact@v4
|
|
if: always()
|
|
with:
|
|
name: playwright-report-${{ matrix.shard }}
|
|
path: packages/app/playwright-report/
|
|
retention-days: 30
|
|
|
|
- name: Upload test results
|
|
uses: actions/upload-artifact@v4
|
|
if: always()
|
|
with:
|
|
name: test-results-${{ matrix.shard }}
|
|
path: packages/app/test-results/
|
|
retention-days: 30
|
|
|
|
- name: Stop E2E containers
|
|
if: always()
|
|
run:
|
|
docker compose -p e2e-0 -f packages/app/tests/e2e/docker-compose.yml
|
|
down -v
|