hyperdx/.github/workflows/release.yml
Warren Lee 470b2c2992
ci: Replace QEMU with native ARM64 runners for release builds (#1952)
## Summary

- **Replace QEMU-emulated multi-platform builds with native ARM64 runners** for both `release.yml` and `release-nightly.yml`, significantly speeding up CI build times
- Each architecture (amd64/arm64) now builds in parallel on native hardware, then a manifest-merge job combines them into a multi-arch Docker tag using `docker buildx imagetools create`
- Migrate from raw Makefile `docker buildx build` commands to `docker/build-push-action@v6` for better GHA integration

## Changes

### `.github/workflows/release.yml`
- Removed QEMU setup entirely
- Replaced single `release` matrix job with per-image build+publish job pairs:
  - `build-otel-collector` / `publish-otel-collector` (runners: `ubuntu-latest` / `ubuntu-latest-arm64`)
  - `build-app` / `publish-app` (runners: `Large-Runner-x64-32` / `Large-Runner-ARM64-32`)
  - `build-local` / `publish-local` (runners: `Large-Runner-x64-32` / `Large-Runner-ARM64-32`)
  - `build-all-in-one` / `publish-all-in-one` (runners: `Large-Runner-x64-32` / `Large-Runner-ARM64-32`)
- Added `check_version` job to centralize skip-if-exists logic (replaces per-image `docker manifest inspect` in Makefile)
- Removed `check_release_app_pushed` artifact upload/download — `publish-app` now outputs `app_was_pushed` directly
- Scoped GHA build cache per image+arch (e.g. `scope=app-amd64`) to avoid collisions
- All 4 images build in parallel (8 build jobs total), then 4 manifest-merge jobs, then downstream notifications

### `.github/workflows/release-nightly.yml`
- Same native runner pattern (no skip logic since nightly always rebuilds)
- 8 build + 4 publish jobs running in parallel
- Slack failure notification and OTel trace export now depend on publish jobs

### `Makefile`
- Removed `release-*` and `release-*-nightly` targets (lines 203-361) — build logic moved into workflow YAML
- Local `build-*` targets preserved for developer use

## Architecture

Follows the same pattern as `release-ee.yml` in the EE repo:

```
check_changesets → check_version
                        │
    ┌───────────────────┼───────────────────┬───────────────────┐
    v                   v                   v                   v
build-app(x2)   build-otel(x2)    build-local(x2)    build-aio(x2)
    │                   │                   │                   │
publish-app      publish-otel       publish-local      publish-aio
    │                   │                   │                   │
    └─────────┬─────────┴───────────────────┴───────────────────┘
              v
     notify_helm_charts / notify_clickhouse_clickstack
              │
     otel-cicd-action
```

## Notes

- `--squash` flag dropped — it's an experimental Docker feature incompatible with `build-push-action` in multi-platform mode. `sbom` and `provenance` are preserved via action params.
- Per-arch intermediate tags (e.g. `hyperdx/hyperdx:2.21.0-amd64`) remain visible on DockerHub — this is standard practice.
- Dual DockerHub namespace tagging (`hyperdx/*` + `clickhouse/clickstack-*`) preserved.


## Sample Run
https://github.com/hyperdxio/hyperdx/actions/runs/23362835749
2026-03-20 23:04:49 +00:00

567 lines
20 KiB
YAML
Raw Blame History

This file contains ambiguous Unicode characters

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

name: Release
on:
push:
branches: [main]
permissions:
contents: write
packages: write
pull-requests: write
actions: read
jobs:
check_changesets:
name: Check Changesets
runs-on: ubuntu-24.04
outputs:
changeset_outputs_hasChangesets:
${{ steps.changesets.outputs.hasChangesets }}
steps:
- name: Checkout
uses: actions/checkout@v4
- name: Setup node
uses: actions/setup-node@v4
with:
node-version-file: '.nvmrc'
cache-dependency-path: 'yarn.lock'
cache: 'yarn'
- name: Install root dependencies
run: yarn install
- name: Create Release Pull Request or Publish to npm
if: always()
continue-on-error: true
id: changesets
uses: changesets/action@v1
with:
commit: 'chore(release): bump HyperDX app/package versions'
title: 'Release HyperDX'
version: yarn run version
publish: yarn release
env:
YARN_ENABLE_IMMUTABLE_INSTALLS: false
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
NPM_TOKEN: ${{ secrets.NPM_TOKEN }}
# ---------------------------------------------------------------------------
# Check if version already published (skip-if-exists)
# ---------------------------------------------------------------------------
check_version:
name: Check if version exists
needs: check_changesets
runs-on: ubuntu-24.04
if:
needs.check_changesets.outputs.changeset_outputs_hasChangesets == 'false'
outputs:
should_release: ${{ steps.check.outputs.should_release }}
steps:
- name: Checkout
uses: actions/checkout@v4
- name: Load Environment Variables from .env
uses: xom9ikk/dotenv@v2
- name: Login to Docker Hub
uses: docker/login-action@v3
with:
username: ${{ secrets.DOCKER_HUB_USERNAME }}
password: ${{ secrets.DOCKER_HUB_ACCESS_TOKEN }}
- name: Check if app image tag already exists
id: check
run: |
TAG_EXISTS=$(docker manifest inspect ${{ env.IMAGE_NAME_DOCKERHUB }}:${{ env.IMAGE_VERSION }}${{ env.IMAGE_VERSION_SUB_TAG }} > /dev/null 2>&1 && echo "true" || echo "false")
if [ "$TAG_EXISTS" = "true" ]; then
echo "Tag ${{ env.IMAGE_NAME_DOCKERHUB }}:${{ env.IMAGE_VERSION }}${{ env.IMAGE_VERSION_SUB_TAG }} already exists. Skipping release."
echo "should_release=false" >> $GITHUB_OUTPUT
else
echo "Tag does not exist. Proceeding with release."
echo "should_release=true" >> $GITHUB_OUTPUT
fi
# ---------------------------------------------------------------------------
# OTel Collector build each arch natively, then merge into multi-arch tag
# ---------------------------------------------------------------------------
build-otel-collector:
name: Build OTel Collector (${{ matrix.arch }})
needs: [check_changesets, check_version]
if: needs.check_version.outputs.should_release == 'true'
strategy:
fail-fast: true
matrix:
include:
- arch: amd64
platform: linux/amd64
runner: ubuntu-latest
- arch: arm64
platform: linux/arm64
runner: ubuntu-latest-arm64
runs-on: ${{ matrix.runner }}
steps:
- name: Checkout
uses: actions/checkout@v4
- name: Setup Docker Buildx
uses: docker/setup-buildx-action@v3
- name: Login to Docker Hub
uses: docker/login-action@v3
with:
username: ${{ secrets.DOCKER_HUB_USERNAME }}
password: ${{ secrets.DOCKER_HUB_ACCESS_TOKEN }}
- name: Login to GitHub Container Registry
uses: docker/login-action@v3
with:
registry: ghcr.io
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
- name: Load Environment Variables from .env
uses: xom9ikk/dotenv@v2
- name: Build and Push
uses: docker/build-push-action@v6
with:
context: .
file: ./docker/otel-collector/Dockerfile
platforms: ${{ matrix.platform }}
target: prod
tags: |
${{ env.OTEL_COLLECTOR_IMAGE_NAME_DOCKERHUB }}:${{ env.IMAGE_VERSION }}${{ env.IMAGE_VERSION_SUB_TAG }}-${{ matrix.arch }}
${{ env.NEXT_OTEL_COLLECTOR_IMAGE_NAME_DOCKERHUB }}:${{ env.IMAGE_VERSION }}${{ env.IMAGE_VERSION_SUB_TAG }}-${{ matrix.arch }}
push: true
cache-from: type=gha,scope=otel-collector-${{ matrix.arch }}
cache-to: type=gha,mode=max,scope=otel-collector-${{ matrix.arch }}
publish-otel-collector:
name: Publish OTel Collector Manifest
needs: [check_version, build-otel-collector]
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v4
- name: Load Environment Variables from .env
uses: xom9ikk/dotenv@v2
- name: Login to Docker Hub
uses: docker/login-action@v3
with:
username: ${{ secrets.DOCKER_HUB_USERNAME }}
password: ${{ secrets.DOCKER_HUB_ACCESS_TOKEN }}
- name: Setup Docker Buildx
uses: docker/setup-buildx-action@v3
- name: Create multi-arch manifests
run: |
VERSION="${{ env.IMAGE_VERSION }}${{ env.IMAGE_VERSION_SUB_TAG }}"
MAJOR="${{ env.IMAGE_VERSION }}"
LATEST="${{ env.IMAGE_LATEST_TAG }}"
for IMAGE in "${{ env.OTEL_COLLECTOR_IMAGE_NAME_DOCKERHUB }}" "${{ env.NEXT_OTEL_COLLECTOR_IMAGE_NAME_DOCKERHUB }}"; do
docker buildx imagetools create \
-t "${IMAGE}:${VERSION}" \
-t "${IMAGE}:${MAJOR}" \
-t "${IMAGE}:${LATEST}" \
"${IMAGE}:${VERSION}-amd64" \
"${IMAGE}:${VERSION}-arm64"
done
# ---------------------------------------------------------------------------
# App (fullstack prod) build each arch natively, then merge
# ---------------------------------------------------------------------------
build-app:
name: Build App (${{ matrix.arch }})
needs: [check_changesets, check_version]
if: needs.check_version.outputs.should_release == 'true'
strategy:
fail-fast: true
matrix:
include:
- arch: amd64
platform: linux/amd64
runner: Large-Runner-x64-32
- arch: arm64
platform: linux/arm64
runner: Large-Runner-ARM64-32
runs-on: ${{ matrix.runner }}
steps:
- name: Checkout
uses: actions/checkout@v4
- name: Setup Docker Buildx
uses: docker/setup-buildx-action@v3
- name: Login to Docker Hub
uses: docker/login-action@v3
with:
username: ${{ secrets.DOCKER_HUB_USERNAME }}
password: ${{ secrets.DOCKER_HUB_ACCESS_TOKEN }}
- name: Login to GitHub Container Registry
uses: docker/login-action@v3
with:
registry: ghcr.io
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
- name: Load Environment Variables from .env
uses: xom9ikk/dotenv@v2
- name: Build and Push
uses: docker/build-push-action@v6
with:
file: ./docker/hyperdx/Dockerfile
platforms: ${{ matrix.platform }}
target: prod
build-contexts: |
hyperdx=./docker/hyperdx
api=./packages/api
app=./packages/app
build-args: |
CODE_VERSION=${{ env.CODE_VERSION }}
tags: |
${{ env.IMAGE_NAME_DOCKERHUB }}:${{ env.IMAGE_VERSION }}${{ env.IMAGE_VERSION_SUB_TAG }}-${{ matrix.arch }}
push: true
sbom: true
provenance: true
cache-from: type=gha,scope=app-${{ matrix.arch }}
cache-to: type=gha,mode=max,scope=app-${{ matrix.arch }}
publish-app:
name: Publish App Manifest
needs: [check_version, build-app]
runs-on: ubuntu-latest
outputs:
app_was_pushed: 'true'
steps:
- name: Checkout
uses: actions/checkout@v4
- name: Load Environment Variables from .env
uses: xom9ikk/dotenv@v2
- name: Login to Docker Hub
uses: docker/login-action@v3
with:
username: ${{ secrets.DOCKER_HUB_USERNAME }}
password: ${{ secrets.DOCKER_HUB_ACCESS_TOKEN }}
- name: Setup Docker Buildx
uses: docker/setup-buildx-action@v3
- name: Create multi-arch manifest
run: |
VERSION="${{ env.IMAGE_VERSION }}${{ env.IMAGE_VERSION_SUB_TAG }}"
MAJOR="${{ env.IMAGE_VERSION }}"
LATEST="${{ env.IMAGE_LATEST_TAG }}"
IMAGE="${{ env.IMAGE_NAME_DOCKERHUB }}"
docker buildx imagetools create \
-t "${IMAGE}:${VERSION}" \
-t "${IMAGE}:${MAJOR}" \
-t "${IMAGE}:${LATEST}" \
"${IMAGE}:${VERSION}-amd64" \
"${IMAGE}:${VERSION}-arm64"
# ---------------------------------------------------------------------------
# Local (all-in-one-noauth) build each arch natively, then merge
# ---------------------------------------------------------------------------
build-local:
name: Build Local (${{ matrix.arch }})
needs: [check_changesets, check_version]
if: needs.check_version.outputs.should_release == 'true'
strategy:
fail-fast: true
matrix:
include:
- arch: amd64
platform: linux/amd64
runner: Large-Runner-x64-32
- arch: arm64
platform: linux/arm64
runner: Large-Runner-ARM64-32
runs-on: ${{ matrix.runner }}
steps:
- name: Checkout
uses: actions/checkout@v4
- name: Setup Docker Buildx
uses: docker/setup-buildx-action@v3
- name: Login to Docker Hub
uses: docker/login-action@v3
with:
username: ${{ secrets.DOCKER_HUB_USERNAME }}
password: ${{ secrets.DOCKER_HUB_ACCESS_TOKEN }}
- name: Login to GitHub Container Registry
uses: docker/login-action@v3
with:
registry: ghcr.io
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
- name: Load Environment Variables from .env
uses: xom9ikk/dotenv@v2
- name: Build and Push
uses: docker/build-push-action@v6
with:
file: ./docker/hyperdx/Dockerfile
platforms: ${{ matrix.platform }}
target: all-in-one-noauth
build-contexts: |
clickhouse=./docker/clickhouse
otel-collector=./docker/otel-collector
hyperdx=./docker/hyperdx
api=./packages/api
app=./packages/app
build-args: |
CODE_VERSION=${{ env.CODE_VERSION }}
tags: |
${{ env.LOCAL_IMAGE_NAME_DOCKERHUB }}:${{ env.IMAGE_VERSION }}${{ env.IMAGE_VERSION_SUB_TAG }}-${{ matrix.arch }}
${{ env.NEXT_LOCAL_IMAGE_NAME_DOCKERHUB }}:${{ env.IMAGE_VERSION }}${{ env.IMAGE_VERSION_SUB_TAG }}-${{ matrix.arch }}
push: true
sbom: true
provenance: true
cache-from: type=gha,scope=local-${{ matrix.arch }}
cache-to: type=gha,mode=max,scope=local-${{ matrix.arch }}
publish-local:
name: Publish Local Manifest
needs: [check_version, build-local]
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v4
- name: Load Environment Variables from .env
uses: xom9ikk/dotenv@v2
- name: Login to Docker Hub
uses: docker/login-action@v3
with:
username: ${{ secrets.DOCKER_HUB_USERNAME }}
password: ${{ secrets.DOCKER_HUB_ACCESS_TOKEN }}
- name: Setup Docker Buildx
uses: docker/setup-buildx-action@v3
- name: Create multi-arch manifests
run: |
VERSION="${{ env.IMAGE_VERSION }}${{ env.IMAGE_VERSION_SUB_TAG }}"
MAJOR="${{ env.IMAGE_VERSION }}"
LATEST="${{ env.IMAGE_LATEST_TAG }}"
for IMAGE in "${{ env.LOCAL_IMAGE_NAME_DOCKERHUB }}" "${{ env.NEXT_LOCAL_IMAGE_NAME_DOCKERHUB }}"; do
docker buildx imagetools create \
-t "${IMAGE}:${VERSION}" \
-t "${IMAGE}:${MAJOR}" \
-t "${IMAGE}:${LATEST}" \
"${IMAGE}:${VERSION}-amd64" \
"${IMAGE}:${VERSION}-arm64"
done
# ---------------------------------------------------------------------------
# All-in-One (all-in-one-auth) build each arch natively, then merge
# ---------------------------------------------------------------------------
build-all-in-one:
name: Build All-in-One (${{ matrix.arch }})
needs: [check_changesets, check_version]
if: needs.check_version.outputs.should_release == 'true'
strategy:
fail-fast: true
matrix:
include:
- arch: amd64
platform: linux/amd64
runner: Large-Runner-x64-32
- arch: arm64
platform: linux/arm64
runner: Large-Runner-ARM64-32
runs-on: ${{ matrix.runner }}
steps:
- name: Checkout
uses: actions/checkout@v4
- name: Setup Docker Buildx
uses: docker/setup-buildx-action@v3
- name: Login to Docker Hub
uses: docker/login-action@v3
with:
username: ${{ secrets.DOCKER_HUB_USERNAME }}
password: ${{ secrets.DOCKER_HUB_ACCESS_TOKEN }}
- name: Login to GitHub Container Registry
uses: docker/login-action@v3
with:
registry: ghcr.io
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
- name: Load Environment Variables from .env
uses: xom9ikk/dotenv@v2
- name: Build and Push
uses: docker/build-push-action@v6
with:
file: ./docker/hyperdx/Dockerfile
platforms: ${{ matrix.platform }}
target: all-in-one-auth
build-contexts: |
clickhouse=./docker/clickhouse
otel-collector=./docker/otel-collector
hyperdx=./docker/hyperdx
api=./packages/api
app=./packages/app
build-args: |
CODE_VERSION=${{ env.CODE_VERSION }}
tags: |
${{ env.ALL_IN_ONE_IMAGE_NAME_DOCKERHUB }}:${{ env.IMAGE_VERSION }}${{ env.IMAGE_VERSION_SUB_TAG }}-${{ matrix.arch }}
${{ env.NEXT_ALL_IN_ONE_IMAGE_NAME_DOCKERHUB }}:${{ env.IMAGE_VERSION }}${{ env.IMAGE_VERSION_SUB_TAG }}-${{ matrix.arch }}
push: true
sbom: true
provenance: true
cache-from: type=gha,scope=all-in-one-${{ matrix.arch }}
cache-to: type=gha,mode=max,scope=all-in-one-${{ matrix.arch }}
publish-all-in-one:
name: Publish All-in-One Manifest
needs: [check_version, build-all-in-one]
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v4
- name: Load Environment Variables from .env
uses: xom9ikk/dotenv@v2
- name: Login to Docker Hub
uses: docker/login-action@v3
with:
username: ${{ secrets.DOCKER_HUB_USERNAME }}
password: ${{ secrets.DOCKER_HUB_ACCESS_TOKEN }}
- name: Setup Docker Buildx
uses: docker/setup-buildx-action@v3
- name: Create multi-arch manifests
run: |
VERSION="${{ env.IMAGE_VERSION }}${{ env.IMAGE_VERSION_SUB_TAG }}"
MAJOR="${{ env.IMAGE_VERSION }}"
LATEST="${{ env.IMAGE_LATEST_TAG }}"
for IMAGE in "${{ env.ALL_IN_ONE_IMAGE_NAME_DOCKERHUB }}" "${{ env.NEXT_ALL_IN_ONE_IMAGE_NAME_DOCKERHUB }}"; do
docker buildx imagetools create \
-t "${IMAGE}:${VERSION}" \
-t "${IMAGE}:${MAJOR}" \
-t "${IMAGE}:${LATEST}" \
"${IMAGE}:${VERSION}-amd64" \
"${IMAGE}:${VERSION}-arm64"
done
# ---------------------------------------------------------------------------
# Downstream notifications
# ---------------------------------------------------------------------------
notify_helm_charts:
name: Notify Helm-Charts Downstream
needs:
[
check_changesets,
publish-app,
publish-otel-collector,
publish-local,
publish-all-in-one,
]
runs-on: ubuntu-24.04
if: |
needs.check_changesets.outputs.changeset_outputs_hasChangesets == 'false' &&
needs.publish-app.outputs.app_was_pushed == 'true'
steps:
- name: Checkout
uses: actions/checkout@v4
- name: Load Environment Variables from .env
uses: xom9ikk/dotenv@v2
- name: Notify Helm-Charts Downstream
uses: actions/github-script@v7
continue-on-error: true
env:
TAG: ${{ env.IMAGE_VERSION }}${{ env.IMAGE_VERSION_SUB_TAG }}
with:
github-token: ${{ secrets.CH_BOT_PAT }}
script: |
const { TAG } = process.env;
const result = await github.rest.actions.createWorkflowDispatch({
owner: 'ClickHouse',
repo: 'ClickStack-helm-charts',
workflow_id: '${{ secrets.DOWNSTREAM_HC_WORKFLOW_ID }}',
ref: 'main',
inputs: {
tag: TAG
}
});
notify_ch:
name: Notify CH Downstream
needs:
[
check_changesets,
publish-app,
publish-otel-collector,
publish-local,
publish-all-in-one,
]
runs-on: ubuntu-24.04
# Temporarily disabled:
if: false
# if: |
# needs.check_changesets.outputs.changeset_outputs_hasChangesets == 'false' &&
# needs.publish-app.outputs.app_was_pushed == 'true'
steps:
- name: Checkout
uses: actions/checkout@v4
- name: Load Environment Variables from .env
uses: xom9ikk/dotenv@v2
- name: Get Downstream App Installation Token
id: auth
uses: actions/create-github-app-token@v2
with:
app-id: ${{ secrets.DOWNSTREAM_CH_APP_ID }}
private-key: ${{ secrets.DOWNSTREAM_CH_APP_PRIVATE_KEY }}
owner: ${{ secrets.DOWNSTREAM_CH_OWNER }}
- name: Notify CH Downstream
uses: actions/github-script@v7
continue-on-error: true
env:
TAG: ${{ env.IMAGE_VERSION }}${{ env.IMAGE_VERSION_SUB_TAG }}
with:
github-token: ${{ steps.auth.outputs.token }}
script: |
const { TAG } = process.env;
const result = await github.rest.actions.createWorkflowDispatch({
owner: '${{ secrets.DOWNSTREAM_CH_OWNER }}',
repo: '${{ secrets.DOWNSTREAM_DP_REPO }}',
workflow_id: '${{ secrets.DOWNSTREAM_DP_WORKFLOW_ID }}',
ref: 'main',
inputs: {
tag: TAG
}
});
notify_clickhouse_clickstack:
needs:
[
check_changesets,
publish-app,
publish-otel-collector,
publish-local,
publish-all-in-one,
]
if: needs.publish-app.outputs.app_was_pushed == 'true'
timeout-minutes: 5
runs-on: ubuntu-24.04
steps:
- name: Checkout
uses: actions/checkout@v4
- name: Load Environment Variables from .env
uses: xom9ikk/dotenv@v2
- name: Notify ClickHouse/clickhouse-clickstack Downstream
uses: actions/github-script@v7
continue-on-error: true
env:
TAG: ${{ env.IMAGE_VERSION }}${{ env.IMAGE_VERSION_SUB_TAG }}
with:
github-token: ${{ secrets.CH_BOT_PAT }}
script: |
const { TAG } = process.env;
const result = await github.rest.actions.createWorkflowDispatch({
owner: 'ClickHouse',
repo: 'clickhouse-clickstack',
workflow_id: 'sync-hyperdx.yml',
ref: 'main',
inputs: {
tag: TAG,
}
});
otel-cicd-action:
if: always()
name: OpenTelemetry Export Trace
runs-on: ubuntu-latest
needs:
[
check_changesets,
publish-app,
publish-otel-collector,
publish-local,
publish-all-in-one,
notify_helm_charts,
notify_ch,
notify_clickhouse_clickstack,
]
steps:
- name: Export workflow
uses: corentinmusard/otel-cicd-action@v4
with:
otlpEndpoint: ${{ secrets.OTLP_ENDPOINT }}/v1/traces
otlpHeaders: ${{ secrets.OTLP_HEADERS }}
otelServiceName: 'release-hyperdx-oss-workflow'
githubToken: ${{ secrets.GITHUB_TOKEN }}