mirror of
https://github.com/datahaven-xyz/datahaven
synced 2026-05-24 09:50:01 +00:00
9a5404de82
## Overview This PR consolidates and optimizes the Docker build system, reducing redundancy and improving CI/CD performance. The changes eliminate duplicate Dockerfiles, introduce a flexible build template, and optimize release builds to reuse CI artifacts. ## Changes Summary ### 🐳 Docker Images Restructured **Before:** 5 Dockerfiles with significant overlap **After:** 4 focused images + 1 utility #### Final Structure: 1. **`operator/Dockerfile`** ✨ Updated - **Standard operator image** for CI and release builds - Minimal node image (accepts pre-built binaries) - GHCR: `ghcr.io/datahaven-xyz/datahaven/datahaven` (CI) - DockerHub: `datahavenxyz/datahaven` (releases) 2. **`docker/datahaven-build.Dockerfile`** (moved from `operator/Dockerfile`) - Full source-to-binary build for manual releases - DockerHub: `datahavenxyz/datahaven:{label}` - Supports custom RUSTFLAGS and fast-runtime feature - Only used for manual workflow_dispatch builds 3. **`docker/datahaven-production.Dockerfile`** (kept) - Binary builder for CPU-specific releases - Used by build-prod-binary workflow template - Supports custom target-cpu flags 4. **`docker/datahaven-dev.Dockerfile`** ✨ NEW (local dev only) - **FOR LOCAL DEVELOPMENT/TROUBLESHOOTING ONLY** - Includes debug tools: gdb, strace, vim, sudo - Extra dependencies: librocksdb-dev, curl - RUST_BACKTRACE enabled by default - **DO NOT USE for CI or production builds** 5. **`test/docker/crossbuild-mac-libpq.dockerfile`** (kept) - Utility for macOS → Linux cross-compilation #### Removed (Redundant): - ❌ `docker/datahaven.Dockerfile` → replaced by operator/Dockerfile - ❌ `test/docker/datahaven-node-local.dockerfile` → replaced by datahaven-dev.Dockerfile --- ### 🔄 Workflow Improvements #### Enhanced `publish-docker` Template - Supports both GHCR and DockerHub registries - Flexible inputs: dockerfile, context, build-args, cache scope - Auto-generates OCI-compliant labels - Reduces code duplication (~70 lines → ~15 per workflow) #### Refactored CI Pipeline - **`docker-build-ci`**: Builds `operator/Dockerfile` → GHCR for CI/E2E testing - **`docker-build-release`**: Builds `operator/Dockerfile` → DockerHub (main branch only) - Both CI and release workflows now use the same minimal operator image - Release builds **reuse CI binaries** instead of rebuilding from source #### Optimized Release Workflow The `task-docker-release` workflow now has dual modes: **Mode 1: `workflow_call` (CI - main pushes)** - ✅ Reuses binary from CI's build-operator task - ✅ Uses lightweight `operator/Dockerfile` - ✅ Tags: `latest`, `sha-{short}` - ⚡ **Fast**: ~5 minutes (vs ~30 min previously) **Mode 2: `workflow_dispatch` (Manual)** - ✅ Full source build with `datahaven-build.Dockerfile` - ✅ Custom branch and label selection - ✅ Optional fast-runtime feature - ✅ Tags: `PROD-{label}` or user-defined --- ### 🔧 Additional Optimizations - Copy libpq5 from builder stage instead of reinstalling (smaller, faster) - Remove redundant protobuf-compiler package (use protoc v21.12 directly) - Standardize user UID to 1000 across all runtime images - Consistent OCI labeling and metadata --------- Co-authored-by: Claude <noreply@anthropic.com>
72 lines
No EOL
1.8 KiB
YAML
72 lines
No EOL
1.8 KiB
YAML
#! Main CI Specification for DataHaven Repository
|
|
|
|
name: CI
|
|
|
|
on:
|
|
workflow_dispatch:
|
|
push:
|
|
branches:
|
|
- main
|
|
pull_request:
|
|
branches: [main]
|
|
|
|
permissions:
|
|
contents: read
|
|
packages: write
|
|
|
|
concurrency:
|
|
group: pr-checks-${{ github.workflow }}-${{ github.head_ref || github.run_id }}
|
|
cancel-in-progress: true
|
|
|
|
jobs:
|
|
# First Tier - Build the binary first
|
|
build-operator:
|
|
uses: ./.github/workflows/task-build-operator.yml
|
|
|
|
# First Tier - Other parallel jobs
|
|
ts-build:
|
|
uses: ./.github/workflows/task-ts-build.yml
|
|
ts-lint:
|
|
uses: ./.github/workflows/task-ts-lint.yml
|
|
unit-tests:
|
|
uses: ./.github/workflows/task-rust-tests.yml
|
|
contract-tests:
|
|
uses: ./.github/workflows/task-foundry-tests.yml
|
|
rust-lint:
|
|
uses: ./.github/workflows/task-rust-lint.yml
|
|
|
|
# Second Tier - Jobs that depend on operator build
|
|
check-metadata:
|
|
needs: [build-operator]
|
|
uses: ./.github/workflows/task-check-metadata.yml
|
|
with:
|
|
binary-hash: ${{ needs.build-operator.outputs.binary-hash }}
|
|
|
|
docker-build-ci:
|
|
needs: [build-operator]
|
|
uses: ./.github/workflows/task-docker-ci.yml
|
|
secrets: inherit
|
|
with:
|
|
binary-hash: ${{ needs.build-operator.outputs.binary-hash }}
|
|
|
|
docker-build-release:
|
|
needs: [build-operator]
|
|
if: github.ref == 'refs/heads/main'
|
|
uses: ./.github/workflows/task-docker-release.yml
|
|
secrets: inherit
|
|
with:
|
|
binary-hash: ${{ needs.build-operator.outputs.binary-hash }}
|
|
|
|
moonwall-tests:
|
|
needs: [build-operator]
|
|
uses: ./.github/workflows/task-moonwall-tests.yml
|
|
with:
|
|
binary-hash: ${{ needs.build-operator.outputs.binary-hash }}
|
|
|
|
# Third Tier - E2E tests depend on docker build
|
|
e2e-tests:
|
|
needs: [docker-build-ci]
|
|
uses: ./.github/workflows/task-e2e.yml
|
|
secrets: inherit
|
|
with:
|
|
image-tag: ${{ needs.docker-build-ci.outputs.image-tag }} |