## Summary
Fixes the release workflow bug where the `cli-v*` git tag and GitHub Release are created successfully, but `@hyperdx/cli` never reaches npm.
### Root cause
The `check_changesets` job in `.github/workflows/release.yml` runs only `yarn install` before calling `changesets/action@v1` with `publish: yarn release`. When changesets attempts to publish `@hyperdx/cli`, the package's `prepublishOnly` hook runs `yarn build` (tsup). tsup bundles all `@hyperdx/common-utils/dist/*` imports, but because `common-utils` was never built in this job, esbuild fails with 14 "Could not resolve" errors and aborts the CLI publish. The release logs for `cli-v0.3.0` confirm this:
```
🦋 info Publishing "@hyperdx/cli" at "0.3.0"
🦋 error an error occurred while publishing @hyperdx/cli: 1 command failed
🦋 error sh -c yarn build
🦋 error ✘ [ERROR] Could not resolve "@hyperdx/common-utils/dist/clickhouse"
(... 14 similar errors ...)
🦋 error packages failed to publish:
🦋 @hyperdx/cli@0.3.0
```
The rest of the release jobs (GitHub Release, Docker images, downstream notifications) proceeded because the changesets step has `continue-on-error: true`.
### Fix
Add a `Build common-utils` step (`make ci-build`, which runs `@hyperdx/common-utils:ci:build` via Nx) between `yarn install` and the changesets action. This populates `packages/common-utils/dist/`, so `@hyperdx/cli`'s `prepublishOnly` build can resolve its dist paths.
### Follow-up
`@hyperdx/cli@0.3.0` was never published to npm (`npm view @hyperdx/cli versions` shows only up to `0.2.0`). Because changesets calls `npm info` on each run and retries publishing any local version that isn't yet on npm, the next push to `main` with this fix will publish `0.3.0` automatically — no version bump needed.
### How to test locally
```bash
# From a clean workspace (no common-utils/dist yet)
rm -rf packages/common-utils/dist
cd packages/cli && yarn build
# => fails with "Could not resolve '@hyperdx/common-utils/dist/*'" (reproduces bug)
# With the fix applied:
cd ../..
make ci-build # builds common-utils
cd packages/cli
yarn build # succeeds, produces dist/cli.js
```
### References
- Failing release run: https://github.com/hyperdxio/hyperdx/actions/runs/24652970126
- Published release (CLI binaries only, npm skipped): https://github.com/hyperdxio/hyperdx/releases/tag/cli-v0.3.0
<div><a href="https://cursor.com/agents/bc-43427d62-7a6b-4d6a-b42e-74db8dc0d2b9"><picture><source media="(prefers-color-scheme: dark)" srcset="https://cursor.com/assets/images/open-in-web-dark.png"><source media="(prefers-color-scheme: light)" srcset="https://cursor.com/assets/images/open-in-web-light.png"><img alt="Open in Web" width="114" height="28" src="https://cursor.com/assets/images/open-in-web-dark.png"></picture></a> <a href="https://cursor.com/background-agent?bcId=bc-43427d62-7a6b-4d6a-b42e-74db8dc0d2b9"><picture><source media="(prefers-color-scheme: dark)" srcset="https://cursor.com/assets/images/open-in-cursor-dark.png"><source media="(prefers-color-scheme: light)" srcset="https://cursor.com/assets/images/open-in-cursor-light.png"><img alt="Open in Cursor" width="131" height="28" src="https://cursor.com/assets/images/open-in-cursor-dark.png"></picture></a> </div>
Co-authored-by: Cursor Agent <199161495+cursoragent@users.noreply.github.com>
The "Release CLI Binaries" job fails because `bun build --compile` can't resolve `@hyperdx/common-utils/dist/*` imports. In CI it's a fresh checkout — `yarn install` puts source code in place but `common-utils` needs to be compiled (tsup) to produce its `dist/` output.
Adds a `make ci-build` step between `yarn install` and the CLI compile step.
Fixes https://github.com/hyperdxio/hyperdx/actions/runs/24220292098/job/70710194631
## Summary
- **Replace QEMU-emulated multi-platform builds with native ARM64 runners** for both `release.yml` and `release-nightly.yml`, significantly speeding up CI build times
- Each architecture (amd64/arm64) now builds in parallel on native hardware, then a manifest-merge job combines them into a multi-arch Docker tag using `docker buildx imagetools create`
- Migrate from raw Makefile `docker buildx build` commands to `docker/build-push-action@v6` for better GHA integration
## Changes
### `.github/workflows/release.yml`
- Removed QEMU setup entirely
- Replaced single `release` matrix job with per-image build+publish job pairs:
- `build-otel-collector` / `publish-otel-collector` (runners: `ubuntu-latest` / `ubuntu-latest-arm64`)
- `build-app` / `publish-app` (runners: `Large-Runner-x64-32` / `Large-Runner-ARM64-32`)
- `build-local` / `publish-local` (runners: `Large-Runner-x64-32` / `Large-Runner-ARM64-32`)
- `build-all-in-one` / `publish-all-in-one` (runners: `Large-Runner-x64-32` / `Large-Runner-ARM64-32`)
- Added `check_version` job to centralize skip-if-exists logic (replaces per-image `docker manifest inspect` in Makefile)
- Removed `check_release_app_pushed` artifact upload/download — `publish-app` now outputs `app_was_pushed` directly
- Scoped GHA build cache per image+arch (e.g. `scope=app-amd64`) to avoid collisions
- All 4 images build in parallel (8 build jobs total), then 4 manifest-merge jobs, then downstream notifications
### `.github/workflows/release-nightly.yml`
- Same native runner pattern (no skip logic since nightly always rebuilds)
- 8 build + 4 publish jobs running in parallel
- Slack failure notification and OTel trace export now depend on publish jobs
### `Makefile`
- Removed `release-*` and `release-*-nightly` targets (lines 203-361) — build logic moved into workflow YAML
- Local `build-*` targets preserved for developer use
## Architecture
Follows the same pattern as `release-ee.yml` in the EE repo:
```
check_changesets → check_version
│
┌───────────────────┼───────────────────┬───────────────────┐
v v v v
build-app(x2) build-otel(x2) build-local(x2) build-aio(x2)
│ │ │ │
publish-app publish-otel publish-local publish-aio
│ │ │ │
└─────────┬─────────┴───────────────────┴───────────────────┘
v
notify_helm_charts / notify_clickhouse_clickstack
│
otel-cicd-action
```
## Notes
- `--squash` flag dropped — it's an experimental Docker feature incompatible with `build-push-action` in multi-platform mode. `sbom` and `provenance` are preserved via action params.
- Per-arch intermediate tags (e.g. `hyperdx/hyperdx:2.21.0-amd64`) remain visible on DockerHub — this is standard practice.
- Dual DockerHub namespace tagging (`hyperdx/*` + `clickhouse/clickstack-*`) preserved.
## Sample Run
https://github.com/hyperdxio/hyperdx/actions/runs/23362835749
## Summary
- Add `otel-cicd-action` to `release.yml` and `release-nightly.yml` workflows to export OpenTelemetry traces for CI/CD observability
- Add `actions: read` permission to `release.yml` (already present in `release-nightly.yml`)
- Service names: `release-hyperdx-oss-workflow` and `release-nightly-hyperdx-oss-workflow`
Ref: HDX-3670
## Test plan
- [x] Verify `release-nightly` workflow runs successfully with the new otel export job
- [x] Verify `release` workflow runs successfully with the new otel export job
- [x] Confirm traces appear in the configured OTLP endpoint
- Multiple workflow runs can now run in parallel for different commits
- The release job (Docker builds) won't be cancelled once it starts
- New commits will queue their release jobs to run after the current one finishes (due to the concurrency group per matrix item)
Ref: HDX-3008
Ref: HDX-1976
1. Updated release-xxx commands to prevent image tag overrides
2. Updated release workflow so that notify-xxx steps won't be triggered if no new app image was pushed
1. Merge 'fullstack' and 'local' (auth + noauth) builds into a single Dockerfile
2. Introduce 'all-in-one-auth' and 'all-in-one-noauth' build stages
3. Lock `IS_LOCAL_APP_MODE` env var
4. Fix bug in ctrl-c exit with docker run
5. Enable alerts in local mode (no-auth)
6. Build `common-utils` on the fly (no longer needing pulling pkg from npm)
Ref: HDX-1709
Ref: HDX-1713
Ref: HDX-1254
Ref: HDX-1729
To match v2 product definition, we are going to release three images:
- hyperdx/hyperdx (--target=prod): app only without any other deps (clickhouse, otelcol, mongodb), used in default compose + helm deployment
- hyperdx/hyperdx-all-in-one (--target=all-in-one-auth): all-in-one build + auth
- hyperdx/hyperdx-local (--target=all-in-one-noauth): all-in-one build + no-auth
Production impacts:
- hyperdx/hyperdx: none
- hyperdx/hyperdx-all-in-one: new
- hyperdx/hyperdx-local: add server components (alerts, saved searches, dashboards)
This PR was opened by the [Changesets release](https://github.com/changesets/action) GitHub action. When you're ready to do a release, you can merge this and the packages will be published to npm automatically. If you're not ready to do a release yet, that's fine, whenever you add more changesets to v2, this PR will be updated.
⚠️⚠️⚠️⚠️⚠️⚠️
`v2` is currently in **pre mode** so this branch has prereleases rather than normal releases. If you want to exit prereleases, run `changeset pre exit` on `v2`.
⚠️⚠️⚠️⚠️⚠️⚠️
# Releases
## @hyperdx/common-utils@0.2.0-beta.4
### Minor Changes
- 79fe30f: Queries depending on numeric aggregates now use the type's default value (e.g. 0) instead of null when dealing with non-numeric data.
### Patch Changes
- cfdd523: feat: clickhouse queries are by default conducted through the clickhouse library via POST request. localMode still uses GET for CORS purposes
- 92a4800: feat: move rrweb event fetching to the client instead of an api route
- 7f0b397: feat: queryChartConfig method + events chart ratio
## @hyperdx/api@2.0.0-beta.15
### Minor Changes
- 79fe30f: Queries depending on numeric aggregates now use the type's default value (e.g. 0) instead of null when dealing with non-numeric data.
### Patch Changes
- 9a9581b: Adds external API for alerts and dashboards
- 293a2af: Adds openapidoc annotations for spec generation and swagger route for development
- 92a4800: feat: move rrweb event fetching to the client instead of an api route
- 7f0b397: feat: queryChartConfig method + events chart ratio
- b4b5f6b: style: remove unused routes/components + clickhouse utils (api)
- Updated dependencies [79fe30f]
- Updated dependencies [cfdd523]
- Updated dependencies [92a4800]
- Updated dependencies [7f0b397]
- @hyperdx/common-utils@0.2.0-beta.4
## @hyperdx/app@2.0.0-beta.15
### Patch Changes
- 7de8916: Removes trailing slash for connection urls
- cfdd523: feat: clickhouse queries are by default conducted through the clickhouse library via POST request. localMode still uses GET for CORS purposes
- 6dc6989: feat: Automatically use last used source when loading search page
- 92a4800: feat: move rrweb event fetching to the client instead of an api route
- 7f0b397: feat: queryChartConfig method + events chart ratio
- b4b5f6b: style: remove unused routes/components + clickhouse utils (api)
- Updated dependencies [79fe30f]
- Updated dependencies [cfdd523]
- Updated dependencies [92a4800]
- Updated dependencies [7f0b397]
- @hyperdx/common-utils@0.2.0-beta.4
Co-authored-by: Warren <5959690+wrn14897@users.noreply.github.com>
This PR was opened by the [Changesets release](https://github.com/changesets/action) GitHub action. When you're ready to do a release, you can merge this and the packages will be published to npm automatically. If you're not ready to do a release yet, that's fine, whenever you add more changesets to v2, this PR will be updated.
⚠️⚠️⚠️⚠️⚠️⚠️
`v2` is currently in **pre mode** so this branch has prereleases rather than normal releases. If you want to exit prereleases, run `changeset pre exit` on `v2`.
⚠️⚠️⚠️⚠️⚠️⚠️
# Releases
## @hyperdx/common-utils@0.2.0-beta.3
### Patch Changes
- 092a292: fix: autocomplete for key-values complete for v2 lucene
- 2f626e1: fix: metric name filtering for some metadata
- b16c8e1: feat: compute charts ratio
- 4865ce7: Fixes the histogram query to perform quantile calculation across all data points
## @hyperdx/api@2.0.0-beta.14
### Patch Changes
- e5dfefb: Added test cases for the webhook and source routes.
- f5e9a07: chore: bump node version to v22
- Updated dependencies [092a292]
- Updated dependencies [2f626e1]
- Updated dependencies [b16c8e1]
- Updated dependencies [4865ce7]
- @hyperdx/common-utils@0.2.0-beta.3
## @hyperdx/app@2.0.0-beta.14
### Patch Changes
- 56e39dc: 36c3edc fix: remove several source change forms throughout the log drawer
- 092a292: fix: autocomplete for key-values complete for v2 lucene
- 2f626e1: fix: metric name filtering for some metadata
- f5e9a07: chore: bump node version to v22
- b16c8e1: feat: compute charts ratio
- 08009ac: feat: add saved filters for searches
- db761ba: fix: remove originalWhere tag from view. not used anyways
- 8c95b9e: Add search history
- Updated dependencies [092a292]
- Updated dependencies [2f626e1]
- Updated dependencies [b16c8e1]
- Updated dependencies [4865ce7]
- @hyperdx/common-utils@0.2.0-beta.3
Co-authored-by: Warren <5959690+wrn14897@users.noreply.github.com>
This PR was opened by the [Changesets release](https://github.com/changesets/action) GitHub action. When you're ready to do a release, you can merge this and the packages will be published to npm automatically. If you're not ready to do a release yet, that's fine, whenever you add more changesets to v2, this PR will be updated.
⚠️⚠️⚠️⚠️⚠️⚠️
`v2` is currently in **pre mode** so this branch has prereleases rather than normal releases. If you want to exit prereleases, run `changeset pre exit` on `v2`.
⚠️⚠️⚠️⚠️⚠️⚠️
# Releases
## @hyperdx/common-utils@0.2.0-beta.2
### Minor Changes
- a9dfa14: Added support to CTE rendering where you can now specify a CTE using a full chart config object instance. This CTE capability is then used to avoid the URI too long error for delta event queries.
- e002c2f: Support querying a sum metric as a value instead of a rate
### Patch Changes
- 50ce38f: Histogram metric query test cases
- 2e350e2: feat: implement logs > metrics correlation flow + introduce convertV1ChartConfigToV2
- a6fd5e3: feat: introduce k8s preset dashboard
- b9f7d32: Refactored renderWith to simplify logic and ship more tests with the changes.
- eaa6bfa: fix: transform partition_key to be the same format as others
- bd9dc18: perf: reuse existing queries promises to avoid duplicate requests
- 5db2767: Fixed CI linting and UI release task.
- 414ff92: feat: export 'Connection' type
- e884d85: fix: metrics > logs correlation flow
- e5a210a: feat: support search on multi implicit fields (BETA)
## @hyperdx/app@2.0.0-beta.13
### Minor Changes
- 9579251: Stores the collapse vs expand status of the side navigation in local storage so it's carried across browser windows/sessions.
### Patch Changes
- 3be7f4d: fix: input does not overlap with language select button anymore
- 2e350e2: feat: implement logs > metrics correlation flow + introduce convertV1ChartConfigToV2
- a6fd5e3: feat: introduce k8s preset dashboard
- a9dfa14: Added support to CTE rendering where you can now specify a CTE using a full chart config object instance. This CTE capability is then used to avoid the URI too long error for delta event queries.
- 5a10ae1: fix: delete huge z-value for tooltip
- 6864836: fix: don't show ellipses on search when query is in-flight
- b99236d: fix: autocomplete options for dashboard page
- 5db2767: Fixed CI linting and UI release task.
- 2580ddd: chore: bump next to v13.5.10
- 5044083: Session Replay tab for traces is disabled unless the source is configured with a sessionId
- 6dafb87: fix: View Events not shown for multiple series; grabs where clause when single series
- decd622: fix: k8s dashboard uptime metrics + warning k8s event body
- e884d85: fix: metrics > logs correlation flow
- e5a210a: feat: support search on multi implicit fields (BETA)
- Updated dependencies [50ce38f]
- Updated dependencies [2e350e2]
- Updated dependencies [a6fd5e3]
- Updated dependencies [a9dfa14]
- Updated dependencies [e002c2f]
- Updated dependencies [b9f7d32]
- Updated dependencies [eaa6bfa]
- Updated dependencies [bd9dc18]
- Updated dependencies [5db2767]
- Updated dependencies [414ff92]
- Updated dependencies [e884d85]
- Updated dependencies [e5a210a]
- @hyperdx/common-utils@0.2.0-beta.2
## @hyperdx/api@2.0.0-beta.13
### Patch Changes
- 50ce38f: Histogram metric query test cases
- 2e350e2: feat: implement logs > metrics correlation flow + introduce convertV1ChartConfigToV2
- b9f7d32: Refactored renderWith to simplify logic and ship more tests with the changes.
- 5db2767: Fixed CI linting and UI release task.
- d326610: feat: introduce RUN_SCHEDULED_TASKS_EXTERNALLY + enable in-app task
- 414ff92: perf + fix: single clickhouse proxy middleware instance
- Updated dependencies [50ce38f]
- Updated dependencies [2e350e2]
- Updated dependencies [a6fd5e3]
- Updated dependencies [a9dfa14]
- Updated dependencies [e002c2f]
- Updated dependencies [b9f7d32]
- Updated dependencies [eaa6bfa]
- Updated dependencies [bd9dc18]
- Updated dependencies [5db2767]
- Updated dependencies [414ff92]
- Updated dependencies [e884d85]
- Updated dependencies [e5a210a]
- @hyperdx/common-utils@0.2.0-beta.2
Co-authored-by: Warren <5959690+wrn14897@users.noreply.github.com>
