## Summary
- Refactor for readability + add tests
- Allow small agent-branch PRs to reach Tier 2 (< 50 prod lines, ≤ 3 files)
- Add cross-layer detection to keep multi-package changes at Tier 3+
- Raise Tier 2 ceiling from 100 → 150 production lines
- Exclude test and changeset files from line counts used for tier decisions
- Extract pure classification logic into .github/scripts/ for testability
- Gate classify job on tests passing in pr-triage.yml
- Only CI changes to main.yaml and release.yaml are tier 4
- Test changes in critical paths don't count toward tiering decisions
## Summary
- Isolate dev, E2E, and integration test environments so multiple git worktrees can run all three simultaneously without port conflicts
- Each worktree gets a deterministic slot (0-99) with unique port ranges: dev (30100-31199), E2E (20320-21399), CI integration (14320-40098)
- Dev portal dashboard (http://localhost:9900) auto-discovers all running stacks, streams logs, and provides a History tab for past run logs
## Port Isolation
| Environment | Port Range | Project Name |
|---|---|---|
| Dev stack | 30100-31199 | `hdx-dev-<slot>` |
| E2E tests | 20320-21399 | `e2e-<slot>` |
| CI integration | 14320-40098 | `int-<slot>` |
All three can run simultaneously from the same worktree with zero port conflicts.
## Dev Portal Features
**Live tab:**
- Auto-discovers dev, E2E, and integration Docker containers + local services (API, App)
- Groups all environments for the same worktree into a single card
- SSE log streaming with ANSI color rendering, capped at 5000 lines
- Auto-starts in background from `make dev`, `make dev-e2e`, `make dev-int`
**History tab:**
- Logs archived to `~/.config/hyperdx/dev-slots/<slot>/history/` on exit (instead of deleted)
- Each archived run includes `meta.json` with worktree/branch metadata
- Grouped by worktree with collapsible cards, search by worktree/branch
- View any past log file in the same log panel, delete individual runs or clear all
- Custom dark-themed confirm modal (no native browser dialogs)
## What Changed
- **`scripts/dev-env.sh`** — Slot-based port assignments, portal auto-start, log archival on exit
- **`scripts/test-e2e.sh`** — E2E port range (20320-21399), log capture via `tee`, portal auto-start, log archival
- **`scripts/ensure-dev-portal.sh`** — Shared singleton portal launcher (works sourced or executed)
- **`scripts/dev-portal/server.js`** — Discovery for dev/E2E/CI containers, history API (list/read/delete), local service port probing
- **`scripts/dev-portal/index.html`** — Live/History tabs, worktree-grouped cards, search, collapse/expand, custom confirm modal, ANSI color log rendering
- **`docker-compose.dev.yml`** — Parameterized ports/volumes/project name with `hdx.dev.*` labels
- **`packages/app/tests/e2e/docker-compose.yml`** — Updated to new E2E port defaults
- **`Makefile`** — `dev-int`/`dev-e2e` targets with log capture + portal auto-start; `dev-portal-stop`; `dev-clean` stops everything + wipes slot data
- **`.env` files** — Ports use `${VAR:-default}` syntax across dev, E2E, and CI environments
- **`agent_docs/development.md`** — Full documentation for isolation, port tables, E2E/CI port ranges
## How to Use
```bash
# Start dev stack (auto-starts portal)
make dev
# Run E2E tests (auto-starts portal, separate ports)
make dev-e2e FILE=navigation
# Run integration tests (auto-starts portal, separate ports)
make dev-int FILE=alerts
# All three can run simultaneously from the same worktree
# Portal at http://localhost:9900 shows everything
# Stop portal
make dev-portal-stop
# Clean up everything (all stacks + portal + history)
make dev-clean
```
## Dev Portal
<img width="1692" height="944" alt="image" src="https://github.com/user-attachments/assets/6ed388a3-43bc-4552-aa8d-688077b79fb7" />
<img width="1689" height="935" alt="image" src="https://github.com/user-attachments/assets/8677a138-0a40-4746-93ed-3b355c8bd45e" />
## Test Plan
- [x] Run `make dev` — verify services start with slot-assigned ports
- [x] Run `make dev` in a second worktree — verify different ports, no conflicts
- [x] Run `make dev-e2e` and `make dev-int` simultaneously — no port conflicts
- [x] Open http://localhost:9900 — verify all stacks grouped by worktree
- [x] Click a service to view logs — verify ANSI colors render correctly
- [x] Stop a stack — verify logs archived to History tab with correct worktree
- [x] History tab — search, collapse/expand, view archived logs, delete
- [x] `make dev-clean` — stops everything, wipes slot data and history
## Summary
- Adds worktree-aware port isolation for E2E tests, mirroring the existing `dev-int` slot mechanism so multiple agents/developers can run E2E tests in parallel without port conflicts
- Fixes the navigation E2E test that was broken by Live Tail URL updates swallowing client-side navigation
- Adds `dev-e2e` Makefile target for running specific tests with `FILE=` and `GREP=` filters, plus `REPORT=1` to open the HTML report after tests finish
## Port Isolation
Each worktree gets a deterministic slot (0–99) computed from its directory name. All E2E service ports are offset by that slot in the **44000–50100** range, avoiding collisions with `dev` (4317–27017) and `dev-int` (14320–40098).
| Service | Base + slot | Variable |
|---|---|---|
| ClickHouse HTTP | 48123 + slot | `HDX_E2E_CH_PORT` |
| ClickHouse Native | 49000 + slot | `HDX_E2E_CH_NATIVE_PORT` |
| MongoDB | 49998 + slot | `HDX_E2E_MONGO_PORT` |
| API server | 49100 + slot | `HDX_E2E_API_PORT` |
| App (fullstack) | 48081 + slot | `HDX_E2E_APP_PORT` |
| App (local) | 48001 + slot | `HDX_E2E_APP_LOCAL_PORT` |
| OpAMP | 44320 + slot | `HDX_E2E_OPAMP_PORT` |
## New Make Targets
```bash
make dev-e2e FILE=navigation # Run specific test file
make dev-e2e FILE=navigation GREP="help menu" # Filter by test name
make dev-e2e GREP="should navigate" # Grep across all files
make dev-e2e FILE=navigation REPORT=1 # Open HTML report after run
make dev-e2e-clean # Remove test artifacts
```
## Linear
https://linear.app/hyperdx/issue/HDX-3796
## Summary
- **Replace QEMU-emulated multi-platform builds with native ARM64 runners** for both `release.yml` and `release-nightly.yml`, significantly speeding up CI build times
- Each architecture (amd64/arm64) now builds in parallel on native hardware, then a manifest-merge job combines them into a multi-arch Docker tag using `docker buildx imagetools create`
- Migrate from raw Makefile `docker buildx build` commands to `docker/build-push-action@v6` for better GHA integration
## Changes
### `.github/workflows/release.yml`
- Removed QEMU setup entirely
- Replaced single `release` matrix job with per-image build+publish job pairs:
- `build-otel-collector` / `publish-otel-collector` (runners: `ubuntu-latest` / `ubuntu-latest-arm64`)
- `build-app` / `publish-app` (runners: `Large-Runner-x64-32` / `Large-Runner-ARM64-32`)
- `build-local` / `publish-local` (runners: `Large-Runner-x64-32` / `Large-Runner-ARM64-32`)
- `build-all-in-one` / `publish-all-in-one` (runners: `Large-Runner-x64-32` / `Large-Runner-ARM64-32`)
- Added `check_version` job to centralize skip-if-exists logic (replaces per-image `docker manifest inspect` in Makefile)
- Removed `check_release_app_pushed` artifact upload/download — `publish-app` now outputs `app_was_pushed` directly
- Scoped GHA build cache per image+arch (e.g. `scope=app-amd64`) to avoid collisions
- All 4 images build in parallel (8 build jobs total), then 4 manifest-merge jobs, then downstream notifications
### `.github/workflows/release-nightly.yml`
- Same native runner pattern (no skip logic since nightly always rebuilds)
- 8 build + 4 publish jobs running in parallel
- Slack failure notification and OTel trace export now depend on publish jobs
### `Makefile`
- Removed `release-*` and `release-*-nightly` targets (lines 203-361) — build logic moved into workflow YAML
- Local `build-*` targets preserved for developer use
## Architecture
Follows the same pattern as `release-ee.yml` in the EE repo:
```
check_changesets → check_version
│
┌───────────────────┼───────────────────┬───────────────────┐
v v v v
build-app(x2) build-otel(x2) build-local(x2) build-aio(x2)
│ │ │ │
publish-app publish-otel publish-local publish-aio
│ │ │ │
└─────────┬─────────┴───────────────────┴───────────────────┘
v
notify_helm_charts / notify_clickhouse_clickstack
│
otel-cicd-action
```
## Notes
- `--squash` flag dropped — it's an experimental Docker feature incompatible with `build-push-action` in multi-platform mode. `sbom` and `provenance` are preserved via action params.
- Per-arch intermediate tags (e.g. `hyperdx/hyperdx:2.21.0-amd64`) remain visible on DockerHub — this is standard practice.
- Dual DockerHub namespace tagging (`hyperdx/*` + `clickhouse/clickstack-*`) preserved.
## Sample Run
https://github.com/hyperdxio/hyperdx/actions/runs/23362835749
## Summary
- Enable multiple agents/developers to run `make dev-int` simultaneously from different git worktrees without Docker port conflicts
- Compute a deterministic port offset (0-99) from the worktree directory name via `cksum`, giving each worktree its own isolated Docker Compose project and port range
- Switch `.env.test` files to use `${HDX_CI_*:-default}` variable expansion (powered by `dotenv-expand`) so test processes connect to the correct dynamic ports
## How it works
Each worktree gets a unique **slot** derived from its directory name. All service ports are offset by that slot:
| Service | Base port | Example (slot 68) |
|-----------------|-----------|-------------------|
| ClickHouse HTTP | 18123 | 18191 |
| MongoDB | 39999 | 40067 |
| API test server | 19000 | 19068 |
| OpAMP | 14320 | 14388 |
Docker Compose project names are also unique (`int-<slot>`), isolating containers and networks.
Backward compatible — when no `HDX_CI_*` env vars are set, all ports fall back to their original defaults.
## Changes
- **Makefile**: Added `HDX_CI_SLOT` computation and dynamic project names/ports for all `dev-int` targets
- **docker-compose.ci.yml**: Ports use `${HDX_CI_*:-default}` env vars; removed unused OTel collector published port; removed hardcoded network name (auto-generated from project name)
- **packages/api/.env.test** / **packages/common-utils/.env.test**: Ports use `${HDX_CI_*:-default}` expansion syntax
- **packages/api/jest.config.js** / **packages/common-utils/jest.int.config.js**: Switched from `dotenv/config` to `dotenv-expand/config` to enable variable expansion
- **packages/api/package.json** / **packages/common-utils/package.json**: Added `dotenv-expand` devDependency
- **agent_docs/development.md**: Documented multi-agent worktree support
## Testing
Ran full Alert integration test suite (`make dev-int FILE=alerts`) — **6 test suites, 150 tests passed** on slot 68 with dynamic ports.
## Summary
- Silence noisy `console.debug` and `console.info` logs in test output across `api` and `common-utils` packages
- Add `DOTENV_CONFIG_OVERRIDE=true` to API integration test scripts so `.env.test` values take precedence
- Add shared jest setup for `common-utils` to suppress verbose console output during tests
Now the CI stdout should be much cleaner and readable (int tests especially)
## Summary
- Simplified `CLAUDE.md` integration test instructions to use `make dev-int-build` and `make dev-int FILE=<TEST_FILE_NAME>` instead of manual docker compose steps
- Added `npx nx run-many -t ci:build` to `dev-int-build` Makefile target to ensure common-utils is built before running tests
You can now prompt
```
run int tests for renderChartConfig
```
## Summary
- Fix tile alerts to support `groupBy` for Gauge/Sum metrics — each group-by value appears as its own column in the response
- Add missing `whereLanguage` to tile alert config so Lucene WHERE conditions are parsed correctly
- Replace stale fixture-based ClickHouse schema with otel-collector's canonical schema in integration tests
Ref: HDX-3576
## Summary
Addresses npm security vulnerabilities in transitive dependencies. Prefer direct dependency upgrades over broad resolutions where possible.
## Changes
**Direct upgrade:**
- **`@slack/webhook`**: `^6.1.0` → `^7.0.0` — v7 natively uses axios v1, eliminating the axios@0.21.4 SSRF/redirect vulnerabilities. Only breaking change in v7 is dropping Node <18 (we're on Node 22).
**Resolutions for transitive deps with no direct upgrade path:**
- **`fast-xml-parser`**: `^4.4.0` — fixes prototype pollution (High)
- **`systeminformation`**: `^5.24.0` — fixes command injection (High)
## Removed/Not Done
- `axios` resolution removed — covered by the `@slack/webhook` upgrade instead
- `tar` resolution removed — was a v6→v7 major jump on build-only tools (`cacache`, `node-gyp`); not present in the production image
- `glob` resolution removed — was breaking test coverage tooling (`test-exclude@6` depends on glob@^7)
## Related
Follow-up to #1731 which addressed base image vulnerabilities (Node, Go, ClickHouse).
TLDR: This PR changes playwright full-stack tests to run against a local clickhouse instance (with seeded data) instead of relying on the clickhouse demo server, which can be unpredictable at times. This workflow allows us to fully control the data to make tests more predictable.
This PR:
* Adds local CH instance to the e2e dockerfile
* Adds a schema creation script
* Adds a data seeding script
* Updates playwright config
* Updates various tests to change hardcoded fields, metrics, or areas relying on play demo data
* Updates github workflow to use the dockerfile instead of separate services
* Runs against a local clickhouse instead of the demo server
Fixes: HDX-3193
- Upgrade OTel collector-contrib and opampsupervisor from 0.136.0 to 0.145.0 to resolve Go stdlib CVEs from outdated binaries
- Pin Alpine base to 3.21 with fresh digest replacing stale alpine:latest pin
- Add HEALTHCHECK to both dev and prod stages using the health_check extension on port 13133
- Fix Makefile otel-collector build targets to use repo-root context with -f flag, matching the repo-root relative COPY paths
Followup from #1697#1698
- fix race condition when switching saved searches after changing sources
- Add e2e tests for saved search bug
- add ability to run e2e tests with UI easier
No changeset needed.. can bump off last commit
Fixes HDX-3127
Enables broader testing
Fixes: HDX-3069
To test:
- By default `make e2e` runs playwright tests with a docker compose for mongo
- To test the local-only mode, run `make e2e local=true`
- Since we manage play.hyperdx.io, I envision us running both commands on release
Closes HDX-2623
# Summary
This change improves the performance of `getKeyValues` when getting values of a JSON key.
Generally, columns that are not referenced outside of a CTE will be pruned by the query planner. For JSON however, if the outer select references one field in a JSON column, then the inner select will read (it seems) the entire JSON object.
This PR also adds integration tests for `getKeyValues` to ensure that the function generates queries that work as expected in ClickHouse.
## Performance impact (on single JSON Dashboard Filter)
- Original: 15.03s
<img width="584" height="71" alt="Screenshot 2025-10-21 at 3 28 07 PM" src="https://github.com/user-attachments/assets/184de198-cee1-4b1d-beed-ec4465d3e248" />
- Optimized: 0.443s
<img width="590" height="61" alt="Screenshot 2025-10-21 at 3 25 47 PM" src="https://github.com/user-attachments/assets/690d0ef0-15b8-47c5-9a7e-8b8f6b8f5e92" />
Ref: HDX-1976
1. Updated release-xxx commands to prevent image tag overrides
2. Updated release workflow so that notify-xxx steps won't be triggered if no new app image was pushed
1. Merge 'fullstack' and 'local' (auth + noauth) builds into a single Dockerfile
2. Introduce 'all-in-one-auth' and 'all-in-one-noauth' build stages
3. Lock `IS_LOCAL_APP_MODE` env var
4. Fix bug in ctrl-c exit with docker run
5. Enable alerts in local mode (no-auth)
6. Build `common-utils` on the fly (no longer needing pulling pkg from npm)
Ref: HDX-1709
Ref: HDX-1713
Ref: HDX-1254
Ref: HDX-1729
To match v2 product definition, we are going to release three images:
- hyperdx/hyperdx (--target=prod): app only without any other deps (clickhouse, otelcol, mongodb), used in default compose + helm deployment
- hyperdx/hyperdx-all-in-one (--target=all-in-one-auth): all-in-one build + auth
- hyperdx/hyperdx-local (--target=all-in-one-noauth): all-in-one build + no-auth
Production impacts:
- hyperdx/hyperdx: none
- hyperdx/hyperdx-all-in-one: new
- hyperdx/hyperdx-local: add server components (alerts, saved searches, dashboards)
This PR was opened by the [Changesets release](https://github.com/changesets/action) GitHub action. When you're ready to do a release, you can merge this and the packages will be published to npm automatically. If you're not ready to do a release yet, that's fine, whenever you add more changesets to v2, this PR will be updated.
⚠️⚠️⚠️⚠️⚠️⚠️
`v2` is currently in **pre mode** so this branch has prereleases rather than normal releases. If you want to exit prereleases, run `changeset pre exit` on `v2`.
⚠️⚠️⚠️⚠️⚠️⚠️
# Releases
## @hyperdx/common-utils@0.2.0-beta.3
### Patch Changes
- 092a292: fix: autocomplete for key-values complete for v2 lucene
- 2f626e1: fix: metric name filtering for some metadata
- b16c8e1: feat: compute charts ratio
- 4865ce7: Fixes the histogram query to perform quantile calculation across all data points
## @hyperdx/api@2.0.0-beta.14
### Patch Changes
- e5dfefb: Added test cases for the webhook and source routes.
- f5e9a07: chore: bump node version to v22
- Updated dependencies [092a292]
- Updated dependencies [2f626e1]
- Updated dependencies [b16c8e1]
- Updated dependencies [4865ce7]
- @hyperdx/common-utils@0.2.0-beta.3
## @hyperdx/app@2.0.0-beta.14
### Patch Changes
- 56e39dc: 36c3edc fix: remove several source change forms throughout the log drawer
- 092a292: fix: autocomplete for key-values complete for v2 lucene
- 2f626e1: fix: metric name filtering for some metadata
- f5e9a07: chore: bump node version to v22
- b16c8e1: feat: compute charts ratio
- 08009ac: feat: add saved filters for searches
- db761ba: fix: remove originalWhere tag from view. not used anyways
- 8c95b9e: Add search history
- Updated dependencies [092a292]
- Updated dependencies [2f626e1]
- Updated dependencies [b16c8e1]
- Updated dependencies [4865ce7]
- @hyperdx/common-utils@0.2.0-beta.3
Co-authored-by: Warren <5959690+wrn14897@users.noreply.github.com>
This PR was opened by the [Changesets release](https://github.com/changesets/action) GitHub action. When you're ready to do a release, you can merge this and the packages will be published to npm automatically. If you're not ready to do a release yet, that's fine, whenever you add more changesets to v2, this PR will be updated.
⚠️⚠️⚠️⚠️⚠️⚠️
`v2` is currently in **pre mode** so this branch has prereleases rather than normal releases. If you want to exit prereleases, run `changeset pre exit` on `v2`.
⚠️⚠️⚠️⚠️⚠️⚠️
# Releases
## @hyperdx/common-utils@0.2.0-beta.2
### Minor Changes
- a9dfa14: Added support to CTE rendering where you can now specify a CTE using a full chart config object instance. This CTE capability is then used to avoid the URI too long error for delta event queries.
- e002c2f: Support querying a sum metric as a value instead of a rate
### Patch Changes
- 50ce38f: Histogram metric query test cases
- 2e350e2: feat: implement logs > metrics correlation flow + introduce convertV1ChartConfigToV2
- a6fd5e3: feat: introduce k8s preset dashboard
- b9f7d32: Refactored renderWith to simplify logic and ship more tests with the changes.
- eaa6bfa: fix: transform partition_key to be the same format as others
- bd9dc18: perf: reuse existing queries promises to avoid duplicate requests
- 5db2767: Fixed CI linting and UI release task.
- 414ff92: feat: export 'Connection' type
- e884d85: fix: metrics > logs correlation flow
- e5a210a: feat: support search on multi implicit fields (BETA)
## @hyperdx/app@2.0.0-beta.13
### Minor Changes
- 9579251: Stores the collapse vs expand status of the side navigation in local storage so it's carried across browser windows/sessions.
### Patch Changes
- 3be7f4d: fix: input does not overlap with language select button anymore
- 2e350e2: feat: implement logs > metrics correlation flow + introduce convertV1ChartConfigToV2
- a6fd5e3: feat: introduce k8s preset dashboard
- a9dfa14: Added support to CTE rendering where you can now specify a CTE using a full chart config object instance. This CTE capability is then used to avoid the URI too long error for delta event queries.
- 5a10ae1: fix: delete huge z-value for tooltip
- 6864836: fix: don't show ellipses on search when query is in-flight
- b99236d: fix: autocomplete options for dashboard page
- 5db2767: Fixed CI linting and UI release task.
- 2580ddd: chore: bump next to v13.5.10
- 5044083: Session Replay tab for traces is disabled unless the source is configured with a sessionId
- 6dafb87: fix: View Events not shown for multiple series; grabs where clause when single series
- decd622: fix: k8s dashboard uptime metrics + warning k8s event body
- e884d85: fix: metrics > logs correlation flow
- e5a210a: feat: support search on multi implicit fields (BETA)
- Updated dependencies [50ce38f]
- Updated dependencies [2e350e2]
- Updated dependencies [a6fd5e3]
- Updated dependencies [a9dfa14]
- Updated dependencies [e002c2f]
- Updated dependencies [b9f7d32]
- Updated dependencies [eaa6bfa]
- Updated dependencies [bd9dc18]
- Updated dependencies [5db2767]
- Updated dependencies [414ff92]
- Updated dependencies [e884d85]
- Updated dependencies [e5a210a]
- @hyperdx/common-utils@0.2.0-beta.2
## @hyperdx/api@2.0.0-beta.13
### Patch Changes
- 50ce38f: Histogram metric query test cases
- 2e350e2: feat: implement logs > metrics correlation flow + introduce convertV1ChartConfigToV2
- b9f7d32: Refactored renderWith to simplify logic and ship more tests with the changes.
- 5db2767: Fixed CI linting and UI release task.
- d326610: feat: introduce RUN_SCHEDULED_TASKS_EXTERNALLY + enable in-app task
- 414ff92: perf + fix: single clickhouse proxy middleware instance
- Updated dependencies [50ce38f]
- Updated dependencies [2e350e2]
- Updated dependencies [a6fd5e3]
- Updated dependencies [a9dfa14]
- Updated dependencies [e002c2f]
- Updated dependencies [b9f7d32]
- Updated dependencies [eaa6bfa]
- Updated dependencies [bd9dc18]
- Updated dependencies [5db2767]
- Updated dependencies [414ff92]
- Updated dependencies [e884d85]
- Updated dependencies [e5a210a]
- @hyperdx/common-utils@0.2.0-beta.2
Co-authored-by: Warren <5959690+wrn14897@users.noreply.github.com>
1. new env var RUN_SCHEDULED_TASKS_EXTERNALLY to opt out in-app task process
2. introduce new `prod-extended` build that includes mongodb process
3. GA k8s dashboard (only picking the connection)
4. bake check-alert task into fullstack app build
This PR was opened by the [Changesets release](https://github.com/changesets/action) GitHub action. When you're ready to do a release, you can merge this and the packages will be published to npm automatically. If you're not ready to do a release yet, that's fine, whenever you add more changesets to v2, this PR will be updated.
# Releases
## @hyperdx/common-utils@0.0.10
### Patch Changes
- fc4548f: feat: add alert schema + types
Co-authored-by: Warren <5959690+wrn14897@users.noreply.github.com>
For better self-hosting experience, users should be able to run
```
docker run -e MONGO_URI=xxx -p 8080:8080 hyperdx/hyperdx:2-beta
```
to spin up the project that includes the server components
The goal is to generate a page that shows what alerts have been created and what their current status is i.e. alarmed or OK, with a small historical view of alert failures/successes recently
- [x] Testing alertHistory changes
- [x] Testing alert API changes
- One of
- [ ] Making the disable button work
- [x] Omitting disable button for now
- [ ] ~Ensuring that alertHistory and alert queries are indexed~ - ticket filed in lieu
- [ ] ~converting dashboard/logview/alerthistory queries to single $in query for performance~ - ticket filed in lieu
- [x] Add explanatory text about how to create alerts (on another page!)
- [x] Comment out AppNav link for now
Co-authored-by: Warren <5959690+wrn14897@users.noreply.github.com>
fixes#120
Intended to handle the case where a nonsensical rate would be returned (e.g. massively negative) or the metric `_string_attributes` change mid-query, to either clamp to zero or filter out possible NaN being currently returned. This should create cleaner graphs for rate metrics, allowing the rate to interpolate over missing data instead of creating a gap in the graph
1. Implement an additional API route to enable users to access data from the HyperDX API using a standard bearer token authentication method
2. Setup Mongo DB migration tool
