mirror of
https://github.com/datahaven-xyz/datahaven
synced 2026-05-24 09:50:01 +00:00
refactor: Consolidate and optimize Docker image architecture (#233)
## Overview This PR consolidates and optimizes the Docker build system, reducing redundancy and improving CI/CD performance. The changes eliminate duplicate Dockerfiles, introduce a flexible build template, and optimize release builds to reuse CI artifacts. ## Changes Summary ### 🐳 Docker Images Restructured **Before:** 5 Dockerfiles with significant overlap **After:** 4 focused images + 1 utility #### Final Structure: 1. **`operator/Dockerfile`** ✨ Updated - **Standard operator image** for CI and release builds - Minimal node image (accepts pre-built binaries) - GHCR: `ghcr.io/datahaven-xyz/datahaven/datahaven` (CI) - DockerHub: `datahavenxyz/datahaven` (releases) 2. **`docker/datahaven-build.Dockerfile`** (moved from `operator/Dockerfile`) - Full source-to-binary build for manual releases - DockerHub: `datahavenxyz/datahaven:{label}` - Supports custom RUSTFLAGS and fast-runtime feature - Only used for manual workflow_dispatch builds 3. **`docker/datahaven-production.Dockerfile`** (kept) - Binary builder for CPU-specific releases - Used by build-prod-binary workflow template - Supports custom target-cpu flags 4. **`docker/datahaven-dev.Dockerfile`** ✨ NEW (local dev only) - **FOR LOCAL DEVELOPMENT/TROUBLESHOOTING ONLY** - Includes debug tools: gdb, strace, vim, sudo - Extra dependencies: librocksdb-dev, curl - RUST_BACKTRACE enabled by default - **DO NOT USE for CI or production builds** 5. **`test/docker/crossbuild-mac-libpq.dockerfile`** (kept) - Utility for macOS → Linux cross-compilation #### Removed (Redundant): - ❌ `docker/datahaven.Dockerfile` → replaced by operator/Dockerfile - ❌ `test/docker/datahaven-node-local.dockerfile` → replaced by datahaven-dev.Dockerfile --- ### 🔄 Workflow Improvements #### Enhanced `publish-docker` Template - Supports both GHCR and DockerHub registries - Flexible inputs: dockerfile, context, build-args, cache scope - Auto-generates OCI-compliant labels - Reduces code duplication (~70 lines → ~15 per workflow) #### Refactored CI Pipeline - **`docker-build-ci`**: Builds `operator/Dockerfile` → GHCR for CI/E2E testing - **`docker-build-release`**: Builds `operator/Dockerfile` → DockerHub (main branch only) - Both CI and release workflows now use the same minimal operator image - Release builds **reuse CI binaries** instead of rebuilding from source #### Optimized Release Workflow The `task-docker-release` workflow now has dual modes: **Mode 1: `workflow_call` (CI - main pushes)** - ✅ Reuses binary from CI's build-operator task - ✅ Uses lightweight `operator/Dockerfile` - ✅ Tags: `latest`, `sha-{short}` - ⚡ **Fast**: ~5 minutes (vs ~30 min previously) **Mode 2: `workflow_dispatch` (Manual)** - ✅ Full source build with `datahaven-build.Dockerfile` - ✅ Custom branch and label selection - ✅ Optional fast-runtime feature - ✅ Tags: `PROD-{label}` or user-defined --- ### 🔧 Additional Optimizations - Copy libpq5 from builder stage instead of reinstalling (smaller, faster) - Remove redundant protobuf-compiler package (use protoc v21.12 directly) - Standardize user UID to 1000 across all runtime images - Consistent OCI labeling and metadata --------- Co-authored-by: Claude <noreply@anthropic.com>
This commit is contained in:
Steve Degosserie
GitHub
parent
8874a99100
commit
9a5404de82
13 changed files with 528 additions and 387 deletions
106
.github/workflow-templates/publish-docker/action.yml
vendored
106
.github/workflow-templates/publish-docker/action.yml
vendored
|
|
@ -1,69 +1,125 @@
|
|||
name: Publish docker image
|
||||
description: |
|
||||
Publish docker image tags to dockerhub
|
||||
Publish docker image tags to container registry
|
||||
|
||||
inputs:
|
||||
dockerhub_username:
|
||||
description: "Dockerhub username"
|
||||
dockerfile:
|
||||
description: "Path to Dockerfile"
|
||||
required: true
|
||||
dockerhub_password:
|
||||
description: "Dockerhub password"
|
||||
context:
|
||||
description: "Build context path"
|
||||
required: false
|
||||
default: "."
|
||||
registry:
|
||||
description: "Container registry (ghcr.io or docker.io)"
|
||||
required: false
|
||||
default: "docker.io"
|
||||
registry_username:
|
||||
description: "Registry username"
|
||||
required: true
|
||||
registry_password:
|
||||
description: "Registry password"
|
||||
required: true
|
||||
image_tags:
|
||||
description: "Image tags"
|
||||
description: "Image tags (newline or comma-separated)"
|
||||
required: true
|
||||
image_title:
|
||||
description: "Image title"
|
||||
required: true
|
||||
required: false
|
||||
default: "DataHaven Node"
|
||||
image_description:
|
||||
description: "Image description"
|
||||
required: true
|
||||
required: false
|
||||
default: "DataHaven blockchain node"
|
||||
image_url:
|
||||
description: "Image url"
|
||||
required: true
|
||||
required: false
|
||||
default: "https://github.com/datahaven-xyz/datahaven"
|
||||
image_source:
|
||||
description: "Image source"
|
||||
required: true
|
||||
required: false
|
||||
default: "https://github.com/datahaven-xyz/datahaven"
|
||||
image_created:
|
||||
description: "Image creation timestamp"
|
||||
required: true
|
||||
required: false
|
||||
default: ""
|
||||
image_revision:
|
||||
description: "Image revision"
|
||||
required: true
|
||||
description: "Image revision (git sha)"
|
||||
required: false
|
||||
default: ""
|
||||
image_licenses:
|
||||
description: "Image licenses"
|
||||
required: true
|
||||
required: false
|
||||
default: "Apache-2.0"
|
||||
cache_scope:
|
||||
description: "Cache scope for GitHub Actions cache"
|
||||
required: false
|
||||
default: "docker-build"
|
||||
build_args:
|
||||
description: "Build arguments (newline or comma-separated)"
|
||||
required: false
|
||||
default: ""
|
||||
platforms:
|
||||
description: "Target platforms"
|
||||
required: false
|
||||
default: "linux/amd64"
|
||||
|
||||
runs:
|
||||
using: "composite"
|
||||
steps:
|
||||
- name: Set up QEMU
|
||||
uses: docker/setup-qemu-action@v3
|
||||
|
||||
- name: Set up Docker Buildx
|
||||
uses: docker/setup-buildx-action@v3.8.0
|
||||
uses: docker/setup-buildx-action@v3
|
||||
with:
|
||||
version: latest
|
||||
driver-opts: |
|
||||
image=moby/buildkit:master
|
||||
- name: Login to DockerHub
|
||||
|
||||
- name: Login to Container Registry
|
||||
uses: docker/login-action@v3
|
||||
with:
|
||||
username: ${{ inputs.dockerhub_username }}
|
||||
password: ${{ inputs.dockerhub_password }}
|
||||
- name: Build and push datahaven
|
||||
registry: ${{ inputs.registry }}
|
||||
username: ${{ inputs.registry_username }}
|
||||
password: ${{ inputs.registry_password }}
|
||||
|
||||
- name: Prepare labels
|
||||
id: labels
|
||||
shell: bash
|
||||
run: |
|
||||
CREATED="${{ inputs.image_created }}"
|
||||
if [ -z "$CREATED" ]; then
|
||||
CREATED=$(date -u +'%Y-%m-%dT%H:%M:%SZ')
|
||||
fi
|
||||
|
||||
REVISION="${{ inputs.image_revision }}"
|
||||
if [ -z "$REVISION" ]; then
|
||||
REVISION="${{ github.sha }}"
|
||||
fi
|
||||
|
||||
echo "created=$CREATED" >> $GITHUB_OUTPUT
|
||||
echo "revision=$REVISION" >> $GITHUB_OUTPUT
|
||||
|
||||
- name: Build and push image
|
||||
id: docker_build
|
||||
uses: docker/build-push-action@v6
|
||||
with:
|
||||
context: .
|
||||
file: ./docker/datahaven.Dockerfile
|
||||
platforms: linux/amd64
|
||||
context: ${{ inputs.context }}
|
||||
file: ${{ inputs.dockerfile }}
|
||||
platforms: ${{ inputs.platforms }}
|
||||
push: true
|
||||
tags: ${{ inputs.image_tags }}
|
||||
build-args: ${{ inputs.build_args }}
|
||||
cache-from: type=gha,scope=${{ inputs.cache_scope }}
|
||||
cache-to: type=gha,mode=max,scope=${{ inputs.cache_scope }}
|
||||
provenance: mode=max
|
||||
sbom: true
|
||||
labels: |
|
||||
org.opencontainers.image.title=${{ inputs.image_title }}
|
||||
org.opencontainers.image.description=${{ inputs.image_title }}
|
||||
org.opencontainers.image.description=${{ inputs.image_description }}
|
||||
org.opencontainers.image.url=${{ inputs.image_url }}
|
||||
org.opencontainers.image.source=${{ inputs.image_source }}
|
||||
org.opencontainers.image.created=${{ inputs.image_created }}
|
||||
org.opencontainers.image.revision=${{ inputs.image_revision }}
|
||||
org.opencontainers.image.created=${{ steps.labels.outputs.created }}
|
||||
org.opencontainers.image.revision=${{ steps.labels.outputs.revision }}
|
||||
org.opencontainers.image.licenses=${{ inputs.image_licenses }}
|
||||
|
|
|
|||
18
.github/workflows/CI.yml
vendored
18
.github/workflows/CI.yml
vendored
|
|
@ -41,10 +41,18 @@ jobs:
|
|||
uses: ./.github/workflows/task-check-metadata.yml
|
||||
with:
|
||||
binary-hash: ${{ needs.build-operator.outputs.binary-hash }}
|
||||
|
||||
docker-build:
|
||||
|
||||
docker-build-ci:
|
||||
needs: [build-operator]
|
||||
uses: ./.github/workflows/task-docker.yml
|
||||
uses: ./.github/workflows/task-docker-ci.yml
|
||||
secrets: inherit
|
||||
with:
|
||||
binary-hash: ${{ needs.build-operator.outputs.binary-hash }}
|
||||
|
||||
docker-build-release:
|
||||
needs: [build-operator]
|
||||
if: github.ref == 'refs/heads/main'
|
||||
uses: ./.github/workflows/task-docker-release.yml
|
||||
secrets: inherit
|
||||
with:
|
||||
binary-hash: ${{ needs.build-operator.outputs.binary-hash }}
|
||||
|
|
@ -57,8 +65,8 @@ jobs:
|
|||
|
||||
# Third Tier - E2E tests depend on docker build
|
||||
e2e-tests:
|
||||
needs: [docker-build]
|
||||
needs: [docker-build-ci]
|
||||
uses: ./.github/workflows/task-e2e.yml
|
||||
secrets: inherit
|
||||
with:
|
||||
image-tag: ${{ needs.docker-build.outputs.image-tag }}
|
||||
image-tag: ${{ needs.docker-build-ci.outputs.image-tag }}
|
||||
164
.github/workflows/DOCKER-PROD.yml
vendored
164
.github/workflows/DOCKER-PROD.yml
vendored
|
|
@ -1,164 +0,0 @@
|
|||
name: Docker Build & Publish
|
||||
|
||||
on:
|
||||
workflow_dispatch:
|
||||
inputs:
|
||||
label:
|
||||
description: "Label for the Docker image"
|
||||
required: true
|
||||
type: string
|
||||
branch:
|
||||
description: "Branch to checkout and build"
|
||||
required: true
|
||||
type: string
|
||||
fast_runtime:
|
||||
description: "Enable fast runtime features"
|
||||
required: false
|
||||
type: boolean
|
||||
default: false
|
||||
push:
|
||||
branches:
|
||||
- main
|
||||
|
||||
jobs:
|
||||
build-test-push:
|
||||
runs-on: ubuntu-latest
|
||||
outputs:
|
||||
image-tag: ${{ steps.last_tag_extractor.outputs.last_tag_value }}
|
||||
defaults:
|
||||
run:
|
||||
working-directory: operator
|
||||
|
||||
steps:
|
||||
- name: Checkout repository
|
||||
uses: actions/checkout@v4
|
||||
with:
|
||||
ref: ${{ github.event.inputs.branch || github.ref }}
|
||||
|
||||
- uses: ./.github/workflows/actions/cleanup-runner
|
||||
|
||||
- name: Docker meta (dispatch)
|
||||
if: github.event_name == 'workflow_dispatch'
|
||||
id: meta-dispatch
|
||||
uses: docker/metadata-action@v5
|
||||
with:
|
||||
images: datahavenxyz/datahaven
|
||||
flavor: |
|
||||
latest=false
|
||||
tags: |
|
||||
type=raw,value=PROD-${{ github.event.inputs.label }}
|
||||
|
||||
- name: Docker meta (main push)
|
||||
if: github.event_name == 'push' && github.ref == 'refs/heads/main'
|
||||
id: meta-main
|
||||
uses: docker/metadata-action@v5
|
||||
with:
|
||||
images: datahavenxyz/datahaven
|
||||
flavor: |
|
||||
latest=true
|
||||
tags: |
|
||||
type=raw,value=latest
|
||||
|
||||
- name: Extract last tag for job output
|
||||
id: last_tag_extractor
|
||||
run: |
|
||||
if [ "${{ github.event_name }}" == "workflow_dispatch" ]; then
|
||||
echo "last_tag_value=$(echo '${{ steps.meta-dispatch.outputs.json }}' | jq -r '.tags[-1]')" >> $GITHUB_OUTPUT
|
||||
else
|
||||
echo "last_tag_value=$(echo '${{ steps.meta-main.outputs.json }}' | jq -r '.tags[-1]')" >> $GITHUB_OUTPUT
|
||||
fi
|
||||
|
||||
- name: Log Docker Metadata
|
||||
run: |
|
||||
if [ "${{ github.event_name }}" == "workflow_dispatch" ]; then
|
||||
echo "Generated tags: ${{ steps.meta-dispatch.outputs.tags }}"
|
||||
echo "Generated labels: ${{ steps.meta-dispatch.outputs.labels }}"
|
||||
echo "Generated JSON: ${{ steps.meta-dispatch.outputs.json }}"
|
||||
else
|
||||
echo "Generated tags: ${{ steps.meta-main.outputs.tags }}"
|
||||
echo "Generated labels: ${{ steps.meta-main.outputs.labels }}"
|
||||
echo "Generated JSON: ${{ steps.meta-main.outputs.json }}"
|
||||
fi
|
||||
|
||||
- uses: docker/setup-qemu-action@v3
|
||||
- uses: docker/setup-buildx-action@v3
|
||||
- uses: docker/login-action@v3
|
||||
with:
|
||||
username: ${{ secrets.DOCKERHUB_USERNAME }}
|
||||
password: ${{ secrets.DOCKERHUB_TOKEN }}
|
||||
- name: Cache Mount blobs
|
||||
uses: actions/cache@v4
|
||||
id: cache
|
||||
with:
|
||||
path: |
|
||||
**/cargo-registry
|
||||
**/cargo-git
|
||||
key: cache-mount-${{ hashFiles('./operator/Dockerfile') }}-${{ hashFiles('./operator/Cargo.lock') }}-${{hashFiles('./operator/runtime/**/*.rs','./operator/pallets/**/*.rs', './operator/node/**/*.rs')}}
|
||||
restore-keys: |
|
||||
cache-mount-${{ hashFiles('./operator/Dockerfile') }}-${{ hashFiles('./operator/Cargo.lock') }}
|
||||
cache-mount-${{ hashFiles('./operator/Dockerfile') }}
|
||||
cache-mount-
|
||||
- name: Inject cache into docker
|
||||
uses: reproducible-containers/buildkit-cache-dance@v3.1.0
|
||||
with:
|
||||
cache-map: |
|
||||
{
|
||||
"cargo-registry": { "target": "/usr/local/cargo/registry" },
|
||||
"cargo-git": { "target": "/usr/local/cargo/git" }
|
||||
}
|
||||
skip-extraction: ${{ steps.cache.outputs.cache-hit }}
|
||||
- name: Build and push Docker image
|
||||
id: build
|
||||
uses: docker/build-push-action@v5
|
||||
timeout-minutes: 240 # 4 hours
|
||||
with:
|
||||
context: ./operator
|
||||
file: ./operator/Dockerfile
|
||||
push: true
|
||||
tags: ${{ github.event_name == 'workflow_dispatch' && steps.meta-dispatch.outputs.tags || steps.meta-main.outputs.tags }}
|
||||
labels: ${{ github.event_name == 'workflow_dispatch' && steps.meta-dispatch.outputs.labels || steps.meta-main.outputs.labels }}
|
||||
platforms: linux/amd64
|
||||
build-args: |
|
||||
FAST_RUNTIME=${{ github.event_name == 'workflow_dispatch' && github.event.inputs.fast_runtime == 'true' && 'TRUE' || 'FALSE' }}
|
||||
cache-from: type=gha,scope=datahaven-build
|
||||
cache-to: type=gha,mode=max,scope=datahaven-build
|
||||
provenance: mode=max
|
||||
sbom: true
|
||||
- name: Log build cache statistics
|
||||
run: |
|
||||
echo "Build cache statistics:"
|
||||
docker buildx du --verbose
|
||||
|
||||
# --- Smoke tests ---
|
||||
|
||||
- name: Pull and test node --help
|
||||
run: |
|
||||
docker pull ${{ steps.last_tag_extractor.outputs.last_tag_value }}
|
||||
docker run --rm ${{ steps.last_tag_extractor.outputs.last_tag_value }} --help
|
||||
|
||||
- name: Integration test (dev chain starts)
|
||||
run: |
|
||||
docker run --rm -d -p 9944:9944 --name local-dh-node \
|
||||
${{ steps.last_tag_extractor.outputs.last_tag_value }} --dev --unsafe-rpc-external
|
||||
|
||||
- name: Wait for node to be healthy and test
|
||||
run: |
|
||||
echo "Waiting for node to start..."
|
||||
for i in {1..30}; do # Retry for 30 * 5s = 150 seconds
|
||||
if curl --fail --location 'http://127.0.0.1:9944' \
|
||||
--header 'Content-Type: application/json' \
|
||||
--data '{"jsonrpc":"2.0","id":1,"method":"system_chain","params":[]}' ; then
|
||||
echo "Node is healthy!"
|
||||
docker logs local-dh-node --tail 100
|
||||
exit 0
|
||||
fi
|
||||
echo "Attempt $i: Node not ready yet, sleeping 5s..."
|
||||
sleep 5
|
||||
done
|
||||
echo "Node failed to start or respond in time."
|
||||
docker logs local-dh-node --tail 100
|
||||
exit 1
|
||||
|
||||
- name: Cleanup integration test container
|
||||
if: always()
|
||||
run: docker rm -f local-dh-node
|
||||
6
.github/workflows/task-build-operator.yml
vendored
6
.github/workflows/task-build-operator.yml
vendored
|
|
@ -49,9 +49,9 @@ jobs:
|
|||
- name: Prepare binary
|
||||
run: |
|
||||
mkdir -p ./target/ci
|
||||
mkdir -p ./target/x86_64-unknown-linux-gnu/release
|
||||
mkdir -p ../build
|
||||
cp ./target/release/datahaven-node ./target/ci/datahaven-node
|
||||
cp ./target/release/datahaven-node ./target/x86_64-unknown-linux-gnu/release/datahaven-node
|
||||
cp ./target/release/datahaven-node ../build/datahaven-node
|
||||
|
||||
- name: Hash binary
|
||||
id: hash-binary
|
||||
|
|
@ -66,7 +66,7 @@ jobs:
|
|||
uses: actions/upload-artifact@v4
|
||||
with:
|
||||
name: datahaven-node-${{ steps.hash-binary.outputs.datahaven-node-hash }}
|
||||
path: operator/target/x86_64-unknown-linux-gnu/release/datahaven-node
|
||||
path: build/datahaven-node
|
||||
retention-days: 1
|
||||
|
||||
- name: Upload WASM to workflow
|
||||
|
|
|
|||
|
|
@ -1,4 +1,4 @@
|
|||
name: Docker Build & Publish
|
||||
name: Docker Build & Publish (CI)
|
||||
|
||||
on:
|
||||
workflow_dispatch:
|
||||
|
|
@ -23,28 +23,29 @@ permissions:
|
|||
packages: write
|
||||
|
||||
concurrency:
|
||||
group: docker-build-${{ github.ref }}
|
||||
group: docker-build-ci-${{ github.ref }}
|
||||
cancel-in-progress: true
|
||||
|
||||
jobs:
|
||||
build-test-push:
|
||||
runs-on: ubuntu-latest
|
||||
outputs:
|
||||
image-tag: ${{ steps.last_tag_extractor.outputs.image-tag }}
|
||||
image-tag: ${{ steps.extract_tag.outputs.image-tag }}
|
||||
|
||||
steps:
|
||||
- name: Checkout repository
|
||||
uses: actions/checkout@v4
|
||||
|
||||
- name: Download binary artifact
|
||||
uses: actions/download-artifact@v4
|
||||
with:
|
||||
name: datahaven-node-${{ inputs.binary-hash }}
|
||||
path: ./operator/target/x86_64-unknown-linux-gnu/release/
|
||||
path: ./build/
|
||||
|
||||
- name: Prepare binary
|
||||
run: |
|
||||
chmod +x ./operator/target/x86_64-unknown-linux-gnu/release/datahaven-node
|
||||
ls -la ./operator/target/x86_64-unknown-linux-gnu/release/
|
||||
chmod +x ./build/datahaven-node
|
||||
ls -la ./build/
|
||||
|
||||
- name: Docker meta
|
||||
id: meta
|
||||
|
|
@ -61,67 +62,37 @@ jobs:
|
|||
type=ref,event=pr
|
||||
|
||||
- name: Extract tag for job output
|
||||
id: last_tag_extractor
|
||||
id: extract_tag
|
||||
run: |
|
||||
FULL_TAG=$(echo '${{ steps.meta.outputs.json }}' | jq -r '.tags[-1]')
|
||||
TAG_ONLY=$(echo "$FULL_TAG" | sed 's|.*:||')
|
||||
echo "image-tag=$TAG_ONLY" >> $GITHUB_OUTPUT
|
||||
echo "image-name=ghcr.io/datahaven-xyz/datahaven/datahaven:$TAG_ONLY" >> $GITHUB_OUTPUT
|
||||
|
||||
- name: Log Docker Metadata
|
||||
run: |
|
||||
echo "Generated tags: ${{ steps.meta.outputs.tags }}"
|
||||
echo "Generated labels: ${{ steps.meta.outputs.labels }}"
|
||||
echo "Generated JSON: ${{ steps.meta.outputs.json }}"
|
||||
|
||||
- uses: docker/setup-qemu-action@v3
|
||||
- uses: docker/setup-buildx-action@v3
|
||||
with:
|
||||
driver-opts: |
|
||||
image=moby/buildkit:master
|
||||
network=host
|
||||
buildkitd-flags: |
|
||||
--allow-insecure-entitlement network.host
|
||||
--allow-insecure-entitlement security.insecure
|
||||
|
||||
- name: Log in to GHCR
|
||||
uses: docker/login-action@v3
|
||||
with:
|
||||
registry: ghcr.io
|
||||
username: ${{ github.actor }}
|
||||
password: ${{ secrets.GITHUB_TOKEN }}
|
||||
|
||||
- name: Build and push Docker image
|
||||
id: build
|
||||
uses: docker/build-push-action@v5
|
||||
uses: ./.github/workflow-templates/publish-docker
|
||||
with:
|
||||
dockerfile: ./operator/Dockerfile
|
||||
context: .
|
||||
file: ./test/docker/datahaven-node-local.dockerfile
|
||||
push: true
|
||||
tags: ${{ steps.meta.outputs.tags }}
|
||||
labels: ${{ steps.meta.outputs.labels }}
|
||||
platforms: linux/amd64
|
||||
cache-from: type=gha,scope=datahaven-local-build
|
||||
cache-to: type=gha,mode=max,scope=datahaven-local-build
|
||||
provenance: mode=max
|
||||
sbom: true
|
||||
|
||||
- name: Log build cache statistics
|
||||
run: |
|
||||
echo "Build cache statistics:"
|
||||
docker buildx du --verbose
|
||||
registry: ghcr.io
|
||||
registry_username: ${{ github.actor }}
|
||||
registry_password: ${{ secrets.GITHUB_TOKEN }}
|
||||
image_tags: ${{ steps.meta.outputs.tags }}
|
||||
image_title: "DataHaven Node - CI"
|
||||
image_description: "CI build of DataHaven operator node"
|
||||
cache_scope: datahaven-ci-build
|
||||
|
||||
# --- Smoke tests ---
|
||||
|
||||
- name: Pull and test node --help
|
||||
run: |
|
||||
docker pull ${{ steps.last_tag_extractor.outputs.image-name }}
|
||||
docker run --rm ${{ steps.last_tag_extractor.outputs.image-name }} --help
|
||||
docker pull ${{ steps.extract_tag.outputs.image-name }}
|
||||
docker run --rm ${{ steps.extract_tag.outputs.image-name }} --help
|
||||
|
||||
- name: Integration test (dev chain starts)
|
||||
run: |
|
||||
docker run --rm -d -p 9944:9944 --name local-dh-node \
|
||||
${{ steps.last_tag_extractor.outputs.image-name }} --dev --unsafe-rpc-external
|
||||
${{ steps.extract_tag.outputs.image-name }} --dev --unsafe-rpc-external
|
||||
|
||||
- name: Wait for node to be healthy and test
|
||||
run: |
|
||||
|
|
@ -143,4 +114,4 @@ jobs:
|
|||
|
||||
- name: Cleanup integration test container
|
||||
if: always()
|
||||
run: docker rm -f local-dh-node
|
||||
run: docker rm -f local-dh-node
|
||||
190
.github/workflows/task-docker-release.yml
vendored
Normal file
190
.github/workflows/task-docker-release.yml
vendored
Normal file
|
|
@ -0,0 +1,190 @@
|
|||
name: Docker Build & Publish (Release)
|
||||
|
||||
on:
|
||||
workflow_dispatch:
|
||||
inputs:
|
||||
label:
|
||||
description: "Label for the Docker image"
|
||||
required: true
|
||||
type: string
|
||||
branch:
|
||||
description: "Branch to checkout and build"
|
||||
required: true
|
||||
type: string
|
||||
fast_runtime:
|
||||
description: "Enable fast runtime features"
|
||||
required: false
|
||||
type: boolean
|
||||
default: false
|
||||
workflow_call:
|
||||
inputs:
|
||||
binary-hash:
|
||||
description: "The hash of the operator binary (for CI builds)"
|
||||
required: true
|
||||
type: string
|
||||
outputs:
|
||||
image-tag:
|
||||
description: "The tag portion of the docker image (without registry)"
|
||||
value: "${{ jobs.build-test-push.outputs.image-tag }}"
|
||||
|
||||
permissions:
|
||||
contents: read
|
||||
packages: write
|
||||
|
||||
concurrency:
|
||||
group: docker-build-release-${{ github.ref }}
|
||||
cancel-in-progress: true
|
||||
|
||||
jobs:
|
||||
build-test-push:
|
||||
runs-on: ubuntu-latest
|
||||
outputs:
|
||||
image-tag: ${{ steps.extract_tag.outputs.image-tag }}
|
||||
|
||||
steps:
|
||||
- name: Checkout repository
|
||||
uses: actions/checkout@v4
|
||||
with:
|
||||
ref: ${{ github.event.inputs.branch || github.ref }}
|
||||
|
||||
- uses: ./.github/workflows/actions/cleanup-runner
|
||||
if: github.event_name == 'workflow_dispatch'
|
||||
|
||||
# --- Conditional: Download binary for CI builds ---
|
||||
- name: Download binary artifact (CI build)
|
||||
if: github.event_name == 'workflow_call'
|
||||
uses: actions/download-artifact@v4
|
||||
with:
|
||||
name: datahaven-node-${{ inputs.binary-hash }}
|
||||
path: ./build/
|
||||
|
||||
- name: Prepare binary (CI build)
|
||||
if: github.event_name == 'workflow_call'
|
||||
run: |
|
||||
chmod +x ./build/datahaven-node
|
||||
ls -la ./build/
|
||||
|
||||
# --- Docker metadata ---
|
||||
- name: Docker meta (dispatch)
|
||||
if: github.event_name == 'workflow_dispatch'
|
||||
id: meta-dispatch
|
||||
uses: docker/metadata-action@v5
|
||||
with:
|
||||
images: datahavenxyz/datahaven
|
||||
flavor: |
|
||||
latest=false
|
||||
tags: |
|
||||
type=raw,value=${{ github.event.inputs.label }}
|
||||
|
||||
- name: Docker meta (CI - main push)
|
||||
if: github.event_name == 'workflow_call'
|
||||
id: meta-ci
|
||||
uses: docker/metadata-action@v5
|
||||
with:
|
||||
images: datahavenxyz/datahaven
|
||||
flavor: |
|
||||
latest=true
|
||||
tags: |
|
||||
type=raw,value=latest
|
||||
type=sha,format=short,prefix=sha-
|
||||
|
||||
- name: Extract tag for job output
|
||||
id: extract_tag
|
||||
run: |
|
||||
if [ "${{ github.event_name }}" == "workflow_dispatch" ]; then
|
||||
TAG=$(echo '${{ steps.meta-dispatch.outputs.json }}' | jq -r '.tags[-1]')
|
||||
else
|
||||
TAG=$(echo '${{ steps.meta-ci.outputs.json }}' | jq -r '.tags[-1]')
|
||||
fi
|
||||
echo "image-tag=$TAG" >> $GITHUB_OUTPUT
|
||||
|
||||
# --- Conditional: Cargo cache for full builds ---
|
||||
- name: Set up cargo cache (full build)
|
||||
if: github.event_name == 'workflow_dispatch'
|
||||
uses: actions/cache@v4
|
||||
id: cache
|
||||
with:
|
||||
path: |
|
||||
**/cargo-registry
|
||||
**/cargo-git
|
||||
key: cache-mount-${{ hashFiles('./docker/datahaven-build.Dockerfile') }}-${{ hashFiles('./operator/Cargo.lock') }}-${{hashFiles('./operator/runtime/**/*.rs','./operator/pallets/**/*.rs', './operator/node/**/*.rs')}}
|
||||
restore-keys: |
|
||||
cache-mount-${{ hashFiles('./docker/datahaven-build.Dockerfile') }}-${{ hashFiles('./operator/Cargo.lock') }}
|
||||
cache-mount-${{ hashFiles('./docker/datahaven-build.Dockerfile') }}
|
||||
cache-mount-
|
||||
|
||||
- name: Inject cache into docker (full build)
|
||||
if: github.event_name == 'workflow_dispatch'
|
||||
uses: reproducible-containers/buildkit-cache-dance@v3.1.0
|
||||
with:
|
||||
cache-map: |
|
||||
{
|
||||
"cargo-registry": { "target": "/usr/local/cargo/registry" },
|
||||
"cargo-git": { "target": "/usr/local/cargo/git" }
|
||||
}
|
||||
skip-extraction: ${{ steps.cache.outputs.cache-hit }}
|
||||
|
||||
# --- Build and push: Full build (workflow_dispatch) ---
|
||||
- name: Build and push Docker image (full build)
|
||||
if: github.event_name == 'workflow_dispatch'
|
||||
uses: ./.github/workflow-templates/publish-docker
|
||||
with:
|
||||
dockerfile: ./docker/datahaven-build.Dockerfile
|
||||
context: ./operator
|
||||
registry: docker.io
|
||||
registry_username: ${{ secrets.DOCKERHUB_USERNAME }}
|
||||
registry_password: ${{ secrets.DOCKERHUB_TOKEN }}
|
||||
image_tags: ${{ steps.meta-dispatch.outputs.tags }}
|
||||
image_title: "DataHaven Node - Release"
|
||||
image_description: "Release build of DataHaven blockchain node"
|
||||
cache_scope: datahaven-release-build
|
||||
build_args: |
|
||||
FAST_RUNTIME=${{ github.event.inputs.fast_runtime == 'true' && 'TRUE' || 'FALSE' }}
|
||||
|
||||
# --- Build and push: CI binary reuse (workflow_call) ---
|
||||
- name: Build and push Docker image (CI binary)
|
||||
if: github.event_name == 'workflow_call'
|
||||
uses: ./.github/workflow-templates/publish-docker
|
||||
with:
|
||||
dockerfile: ./operator/Dockerfile
|
||||
context: .
|
||||
registry: docker.io
|
||||
registry_username: ${{ secrets.DOCKERHUB_USERNAME }}
|
||||
registry_password: ${{ secrets.DOCKERHUB_TOKEN }}
|
||||
image_tags: ${{ steps.meta-ci.outputs.tags }}
|
||||
image_title: "DataHaven Node - Release"
|
||||
image_description: "Release build of DataHaven operator node"
|
||||
cache_scope: datahaven-release-ci
|
||||
|
||||
# --- Smoke tests ---
|
||||
- name: Pull and test node --help
|
||||
run: |
|
||||
docker pull ${{ steps.extract_tag.outputs.image-tag }}
|
||||
docker run --rm ${{ steps.extract_tag.outputs.image-tag }} --help
|
||||
|
||||
- name: Integration test (dev chain starts)
|
||||
run: |
|
||||
docker run --rm -d -p 9944:9944 --name local-dh-node \
|
||||
${{ steps.extract_tag.outputs.image-tag }} --dev --unsafe-rpc-external
|
||||
|
||||
- name: Wait for node to be healthy and test
|
||||
run: |
|
||||
echo "Waiting for node to start..."
|
||||
for i in {1..30}; do # Retry for 30 * 5s = 150 seconds
|
||||
if curl --fail --location 'http://127.0.0.1:9944' \
|
||||
--header 'Content-Type: application/json' \
|
||||
--data '{"jsonrpc":"2.0","id":1,"method":"system_chain","params":[]}' ; then
|
||||
echo "Node is healthy!"
|
||||
docker logs local-dh-node --tail 100
|
||||
exit 0
|
||||
fi
|
||||
echo "Attempt $i: Node not ready yet, sleeping 5s..."
|
||||
sleep 5
|
||||
done
|
||||
echo "Node failed to start or respond in time."
|
||||
docker logs local-dh-node --tail 100
|
||||
exit 1
|
||||
|
||||
- name: Cleanup integration test container
|
||||
if: always()
|
||||
run: docker rm -f local-dh-node
|
||||
75
docker/datahaven-build.Dockerfile
Normal file
75
docker/datahaven-build.Dockerfile
Normal file
|
|
@ -0,0 +1,75 @@
|
|||
# --- Setup Build Environment ---
|
||||
FROM docker.io/paritytech/ci-unified:bullseye-1.88.0 AS base
|
||||
|
||||
ARG MOLD_VERSION=2.40.4
|
||||
ARG PROTOC_VER=21.12
|
||||
ARG SCCACHE_VERSION=0.10.0
|
||||
ARG FAST_RUNTIME=FALSE
|
||||
RUN apt-get update && apt-get install -y --no-install-recommends \
|
||||
curl \
|
||||
xz-utils \
|
||||
clang \
|
||||
libpq-dev \
|
||||
&& echo "Installing mold v${MOLD_VERSION}..." \
|
||||
&& curl -Lo mold.tar.gz "https://github.com/rui314/mold/releases/download/v${MOLD_VERSION}/mold-${MOLD_VERSION}-x86_64-linux.tar.gz" \
|
||||
&& tar -xf mold.tar.gz --strip-components=1 -C /usr/local \
|
||||
&& rm mold.tar.gz \
|
||||
&& apt-get clean \
|
||||
&& rm -rf /var/lib/apt/lists/* \
|
||||
&& echo "Installing protoc v${PROTOC_VER}..." \
|
||||
&& curl -Lo protoc.zip "https://github.com/protocolbuffers/protobuf/releases/download/v${PROTOC_VER}/protoc-${PROTOC_VER}-linux-x86_64.zip" \
|
||||
&& unzip -q protoc.zip -d /usr/local/ \
|
||||
&& rm protoc.zip \
|
||||
&& echo "Installing sccache v${SCCACHE_VERSION}..." \
|
||||
&& curl -Lo sccache.tar.gz "https://github.com/mozilla/sccache/releases/download/v${SCCACHE_VERSION}/sccache-v${SCCACHE_VERSION}-x86_64-unknown-linux-musl.tar.gz" \
|
||||
&& tar -xf sccache.tar.gz --strip-components=1 -C /usr/local/bin sccache-v${SCCACHE_VERSION}-x86_64-unknown-linux-musl/sccache \
|
||||
&& rm sccache.tar.gz
|
||||
|
||||
RUN cargo install cargo-chef --version 0.1.72 --locked
|
||||
|
||||
ENV RUSTC_WRAPPER=sccache \
|
||||
SCCACHE_DIR=/usr/local/sccache \
|
||||
SCCACHE_CACHE_SIZE=25G \
|
||||
RUSTFLAGS="-Clinker=clang -Clink-arg=-fuse-ld=/usr/local/bin/mold"
|
||||
|
||||
# --- Prepare build plan with cargo-chef ---
|
||||
FROM base AS planner
|
||||
WORKDIR /datahaven
|
||||
COPY . .
|
||||
RUN cargo chef prepare --recipe-path recipe.json
|
||||
|
||||
# --- Build dependencies using cargo-chef ---
|
||||
FROM base AS builder
|
||||
WORKDIR /datahaven
|
||||
COPY --from=planner /datahaven/recipe.json recipe.json
|
||||
RUN --mount=type=cache,target=/usr/local/cargo/registry \
|
||||
--mount=type=cache,target=/usr/local/cargo/git \
|
||||
cargo chef cook --recipe-path recipe.json --release
|
||||
COPY . .
|
||||
RUN --mount=type=cache,target=/usr/local/cargo/registry \
|
||||
--mount=type=cache,target=/usr/local/cargo/git \
|
||||
if [ "$FAST_RUNTIME" = "TRUE" ]; then \
|
||||
cargo build --locked --release --features fast-runtime; \
|
||||
else \
|
||||
cargo build --locked --release; \
|
||||
fi
|
||||
|
||||
# --- Create final lightweight runtime image ---
|
||||
FROM docker.io/parity/base-bin:latest
|
||||
|
||||
COPY --from=builder /usr/lib/x86_64-linux-gnu/libpq.so* /usr/lib/x86_64-linux-gnu/
|
||||
COPY --from=builder /datahaven/target/release/datahaven-node /usr/local/bin
|
||||
|
||||
USER root
|
||||
RUN useradd -m -u 1001 -U -s /bin/sh -d /datahaven datahaven && \
|
||||
mkdir -p /data /datahaven/.local/share && \
|
||||
chown -R datahaven:datahaven /data && \
|
||||
ln -s /data /datahaven/.local/share/datahaven && \
|
||||
/usr/local/bin/datahaven-node --version
|
||||
|
||||
USER datahaven
|
||||
|
||||
EXPOSE 30333 9933 9944 9615
|
||||
VOLUME ["/data"]
|
||||
|
||||
ENTRYPOINT ["/usr/local/bin/datahaven-node"]
|
||||
78
docker/datahaven-dev.Dockerfile
Normal file
78
docker/datahaven-dev.Dockerfile
Normal file
|
|
@ -0,0 +1,78 @@
|
|||
# DataHaven Development/Troubleshooting Image
|
||||
#
|
||||
# This image is ONLY for local development and troubleshooting purposes.
|
||||
# It includes additional debugging tools and dependencies not needed in production.
|
||||
#
|
||||
# DO NOT USE for CI or production builds - use operator/Dockerfile instead.
|
||||
#
|
||||
# Build Args:
|
||||
# DEBUG_MODE - Set to "true" to include debugging tools (default: false)
|
||||
#
|
||||
# Expected Binary Location:
|
||||
# ./operator/target/x86_64-unknown-linux-gnu/release/datahaven-node
|
||||
#
|
||||
# Features:
|
||||
# - Ubuntu base with additional system tools
|
||||
# - librocksdb-dev for local development
|
||||
# - Optional gdb, strace, vim for debugging
|
||||
# - RUST_BACKTRACE enabled by default
|
||||
# - Additional directories (/specs, /storage) for testing
|
||||
|
||||
FROM ubuntu:noble
|
||||
|
||||
LABEL version="0.3.0"
|
||||
LABEL description="DataHaven Node - Development/CI/E2E Testing Build"
|
||||
LABEL maintainer="steve@moonsonglabs.com"
|
||||
|
||||
ARG DEBUG_MODE=false
|
||||
|
||||
# Install runtime dependencies
|
||||
RUN apt-get update && \
|
||||
DEBIAN_FRONTEND=noninteractive apt-get install -y --no-install-recommends \
|
||||
ca-certificates \
|
||||
curl \
|
||||
libpq-dev \
|
||||
librocksdb-dev && \
|
||||
# Optionally install debug tools
|
||||
if [ "$DEBUG_MODE" = "true" ]; then \
|
||||
apt-get install -y --no-install-recommends \
|
||||
sudo \
|
||||
gdb \
|
||||
strace \
|
||||
vim; \
|
||||
fi && \
|
||||
apt-get autoremove -y && \
|
||||
apt-get clean && \
|
||||
find /var/lib/apt/lists/ -type f -not -name lock -delete
|
||||
|
||||
# Create datahaven user and directories
|
||||
RUN useradd -m -u 1000 -U -s /bin/sh -d /datahaven datahaven && \
|
||||
mkdir -p /data /datahaven/.local/share /specs /storage && \
|
||||
chown -R datahaven:datahaven /data /storage && \
|
||||
ln -s /data /datahaven/.local/share/datahaven-node
|
||||
|
||||
# Grant sudo access if debug mode is enabled
|
||||
RUN if [ "$DEBUG_MODE" = "true" ]; then \
|
||||
echo "datahaven ALL=(ALL) NOPASSWD:ALL" >> /etc/sudoers && \
|
||||
chmod -R 777 /storage /data; \
|
||||
fi
|
||||
|
||||
USER datahaven
|
||||
|
||||
# Copy pre-built binary
|
||||
COPY --chown=datahaven:datahaven ./operator/target/x86_64-unknown-linux-gnu/release/datahaven-node /usr/local/bin/datahaven-node
|
||||
RUN chmod uog+x /usr/local/bin/datahaven-node
|
||||
|
||||
# Enable Rust backtraces for better debugging
|
||||
ENV RUST_BACKTRACE=1
|
||||
|
||||
# Expose ports
|
||||
# 30333: p2p networking
|
||||
# 9944: WebSocket/RPC
|
||||
# 9615: Prometheus metrics
|
||||
EXPOSE 30333 9944 9615
|
||||
|
||||
VOLUME ["/data"]
|
||||
|
||||
ENTRYPOINT ["datahaven-node"]
|
||||
CMD ["--tmp"]
|
||||
|
|
@ -1,4 +1,4 @@
|
|||
# Production Node for DataHaven
|
||||
# Production Image for DataHaven
|
||||
#
|
||||
# Requires to run from repository root and to copy the binary in the build folder (part of the release workflow)
|
||||
|
||||
|
|
@ -15,7 +15,7 @@ WORKDIR /
|
|||
|
||||
RUN echo "*** Installing Basic dependencies ***"
|
||||
RUN apt-get update && apt-get install -y ca-certificates && update-ca-certificates
|
||||
RUN apt install --assume-yes git clang curl libpq-dev libssl-dev llvm libudev-dev make protobuf-compiler pkg-config unzip
|
||||
RUN apt install --assume-yes git clang curl libpq-dev libssl-dev llvm libudev-dev make pkg-config unzip
|
||||
|
||||
RUN echo "*** Installing protoc v${PROTOC_VER} ***"
|
||||
RUN curl -Lo /tmp/protoc.zip "https://github.com/protocolbuffers/protobuf/releases/download/v${PROTOC_VER}/protoc-${PROTOC_VER}-linux-x86_64.zip" \
|
||||
|
|
|
|||
|
|
@ -1,34 +0,0 @@
|
|||
# DataHaven Binary
|
||||
#
|
||||
# Requires to run from repository root and to copy the binary in the build folder (part of the release workflow)
|
||||
|
||||
FROM debian:stable AS builder
|
||||
|
||||
RUN apt-get update && apt-get install -y libpq5 ca-certificates && update-ca-certificates
|
||||
|
||||
FROM debian:stable-slim
|
||||
LABEL maintainer="steve@moonsonglabs.com"
|
||||
LABEL description="DataHaven Binary"
|
||||
|
||||
RUN useradd -m -u 1000 -U -s /bin/sh -d /datahaven datahaven && \
|
||||
mkdir -p /datahaven/.local/share && \
|
||||
mkdir /data && \
|
||||
chown -R datahaven:datahaven /data && \
|
||||
ln -s /data /datahaven/.local/share/datahaven && \
|
||||
rm -rf /usr/sbin
|
||||
|
||||
COPY --from=builder /etc/ssl/certs/ca-certificates.crt /etc/ssl/certs/ca-certificates.crt
|
||||
|
||||
USER datahaven
|
||||
|
||||
COPY --chown=datahaven build/* /datahaven
|
||||
RUN chmod uog+x /datahaven/datahaven*
|
||||
|
||||
# 30333 for parachain p2p
|
||||
# 9944 for Websocket & RPC call
|
||||
# 9615 for Prometheus (metrics)
|
||||
EXPOSE 30333 9944 9615
|
||||
|
||||
VOLUME ["/data"]
|
||||
|
||||
ENTRYPOINT ["/datahaven/datahaven-node"]
|
||||
|
|
@ -1,76 +1,77 @@
|
|||
# --- Setup Build Environment ---
|
||||
FROM docker.io/paritytech/ci-unified:bullseye-1.88.0 AS base
|
||||
# DataHaven Operator Image
|
||||
#
|
||||
# This is the standard operator image used for CI and release builds.
|
||||
# It's a minimal image that accepts a pre-built binary.
|
||||
#
|
||||
# Usage:
|
||||
# - CI builds: Binary from build-operator workflow artifact
|
||||
# - Release builds: Binary from build-operator workflow artifact
|
||||
# - Local builds: Binary from local cargo build
|
||||
#
|
||||
# Expected Binary Location:
|
||||
# build/datahaven-node
|
||||
#
|
||||
# Registries:
|
||||
# - GHCR: ghcr.io/datahaven-xyz/datahaven/datahaven (CI)
|
||||
# - DockerHub: datahavenxyz/datahaven (releases)
|
||||
|
||||
ARG MOLD_VERSION=2.40.4
|
||||
ARG PROTOC_VER=21.12
|
||||
ARG SCCACHE_VERSION=0.10.0
|
||||
ARG FAST_RUNTIME=FALSE
|
||||
RUN apt-get update && apt-get install -y --no-install-recommends \
|
||||
curl \
|
||||
xz-utils \
|
||||
clang \
|
||||
libpq-dev \
|
||||
&& echo "Installing mold v${MOLD_VERSION}..." \
|
||||
&& curl -Lo mold.tar.gz "https://github.com/rui314/mold/releases/download/v${MOLD_VERSION}/mold-${MOLD_VERSION}-x86_64-linux.tar.gz" \
|
||||
&& tar -xf mold.tar.gz --strip-components=1 -C /usr/local \
|
||||
&& rm mold.tar.gz \
|
||||
&& apt-get clean \
|
||||
&& rm -rf /var/lib/apt/lists/* \
|
||||
&& echo "Installing protoc v${PROTOC_VER}..." \
|
||||
&& curl -Lo protoc.zip "https://github.com/protocolbuffers/protobuf/releases/download/v${PROTOC_VER}/protoc-${PROTOC_VER}-linux-x86_64.zip" \
|
||||
&& unzip -q protoc.zip -d /usr/local/ \
|
||||
&& rm protoc.zip \
|
||||
&& echo "Installing sccache v${SCCACHE_VERSION}..." \
|
||||
&& curl -Lo sccache.tar.gz "https://github.com/mozilla/sccache/releases/download/v${SCCACHE_VERSION}/sccache-v${SCCACHE_VERSION}-x86_64-unknown-linux-musl.tar.gz" \
|
||||
&& tar -xf sccache.tar.gz --strip-components=1 -C /usr/local/bin sccache-v${SCCACHE_VERSION}-x86_64-unknown-linux-musl/sccache \
|
||||
&& rm sccache.tar.gz
|
||||
FROM debian:stable AS builder
|
||||
|
||||
RUN cargo install cargo-chef --version 0.1.72 --locked
|
||||
# Install CA certificates and libpq5 for the release build
|
||||
RUN apt-get update && \
|
||||
apt-get install -y --no-install-recommends \
|
||||
libpq5 \
|
||||
ca-certificates && \
|
||||
update-ca-certificates && \
|
||||
apt-get clean && \
|
||||
rm -rf /var/lib/apt/lists/*
|
||||
|
||||
ENV RUSTC_WRAPPER=sccache \
|
||||
SCCACHE_DIR=/usr/local/sccache \
|
||||
SCCACHE_CACHE_SIZE=25G \
|
||||
RUSTFLAGS="-Clinker=clang -Clink-arg=-fuse-ld=/usr/local/bin/mold"
|
||||
FROM debian:stable-slim
|
||||
|
||||
# --- Prepare build plan with cargo-chef ---
|
||||
FROM base AS planner
|
||||
WORKDIR /datahaven
|
||||
COPY . .
|
||||
RUN cargo chef prepare --recipe-path recipe.json
|
||||
LABEL version="0.3.0"
|
||||
LABEL description="DataHaven Node - Release Build"
|
||||
LABEL maintainer="steve@moonsonglabs.com"
|
||||
|
||||
# --- Build dependencies using cargo-chef ---
|
||||
FROM base AS builder
|
||||
WORKDIR /datahaven
|
||||
COPY --from=planner /datahaven/recipe.json recipe.json
|
||||
RUN --mount=type=cache,target=/usr/local/cargo/registry \
|
||||
--mount=type=cache,target=/usr/local/cargo/git \
|
||||
cargo chef cook --recipe-path recipe.json --release
|
||||
COPY . .
|
||||
RUN --mount=type=cache,target=/usr/local/cargo/registry \
|
||||
--mount=type=cache,target=/usr/local/cargo/git \
|
||||
if [ "$FAST_RUNTIME" = "TRUE" ]; then \
|
||||
cargo build --locked --release --features fast-runtime; \
|
||||
else \
|
||||
cargo build --locked --release; \
|
||||
fi
|
||||
# Copy CA certificates and shared libraries from builder
|
||||
COPY --from=builder /etc/ssl/certs/ca-certificates.crt /etc/ssl/certs/ca-certificates.crt
|
||||
COPY --from=builder \
|
||||
/lib/x86_64-linux-gnu/libpq.so.5 \
|
||||
/lib/x86_64-linux-gnu/libssl.so.3 \
|
||||
/lib/x86_64-linux-gnu/libcrypto.so.3 \
|
||||
/lib/x86_64-linux-gnu/libgssapi_krb5.so.2 \
|
||||
/lib/x86_64-linux-gnu/libldap.so.2 \
|
||||
/lib/x86_64-linux-gnu/libz.so.1 \
|
||||
/lib/x86_64-linux-gnu/libzstd.so.1 \
|
||||
/lib/x86_64-linux-gnu/libkrb5.so.3 \
|
||||
/lib/x86_64-linux-gnu/libk5crypto.so.3 \
|
||||
/lib/x86_64-linux-gnu/libcom_err.so.2 \
|
||||
/lib/x86_64-linux-gnu/libkrb5support.so.0 \
|
||||
/lib/x86_64-linux-gnu/liblber.so.2 \
|
||||
/lib/x86_64-linux-gnu/libsasl2.so.2 \
|
||||
/lib/x86_64-linux-gnu/libkeyutils.so.1 \
|
||||
/lib/x86_64-linux-gnu/
|
||||
|
||||
# --- Create final lightweight runtime image ---
|
||||
FROM docker.io/parity/base-bin:latest
|
||||
|
||||
RUN apt-get update && apt-get install -y libpq5
|
||||
|
||||
COPY --from=builder /datahaven/target/release/datahaven-node /usr/local/bin
|
||||
|
||||
USER root
|
||||
RUN useradd -m -u 1001 -U -s /bin/sh -d /datahaven datahaven && \
|
||||
mkdir -p /data /datahaven/.local/share && \
|
||||
# Create datahaven user and directories
|
||||
RUN useradd -m -u 1000 -U -s /bin/sh -d /datahaven datahaven && \
|
||||
mkdir -p /datahaven/.local/share /data && \
|
||||
chown -R datahaven:datahaven /data && \
|
||||
ln -s /data /datahaven/.local/share/datahaven && \
|
||||
/usr/local/bin/datahaven-node --version
|
||||
rm -rf /usr/sbin
|
||||
|
||||
USER datahaven
|
||||
|
||||
EXPOSE 30333 9933 9944 9615
|
||||
# Copy pre-built binary
|
||||
COPY --chown=datahaven:datahaven build/* /datahaven
|
||||
|
||||
# Make binary executable
|
||||
RUN chmod uog+x /datahaven/datahaven*
|
||||
|
||||
# Expose ports
|
||||
# 30333: p2p networking
|
||||
# 9944: WebSocket/RPC
|
||||
# 9615: Prometheus metrics
|
||||
EXPOSE 30333 9944 9615
|
||||
|
||||
VOLUME ["/data"]
|
||||
|
||||
ENTRYPOINT ["/usr/local/bin/datahaven-node"]
|
||||
ENTRYPOINT ["/datahaven/datahaven-node"]
|
||||
|
|
|
|||
|
|
@ -1,40 +0,0 @@
|
|||
# DATAHAVEN_NODE DOCKERFILE
|
||||
#
|
||||
# This Dockerfile expects to have the binary already built.
|
||||
# So it just copies the binary into the image and runs it.
|
||||
#
|
||||
# This is done to speed up iterating while running the E2E CLI.
|
||||
#
|
||||
# Requires to run from /test folder and to copy the binary in the build folder
|
||||
|
||||
FROM ubuntu:noble
|
||||
|
||||
LABEL version="0.1.0"
|
||||
LABEL description="DataHaven Node Local Build"
|
||||
|
||||
ENV RUST_BACKTRACE=1
|
||||
|
||||
RUN apt-get update && \
|
||||
DEBIAN_FRONTEND=noninteractive apt-get install -y --no-install-recommends \
|
||||
ca-certificates curl sudo librocksdb-dev libpq-dev && \
|
||||
apt-get autoremove -y && \
|
||||
apt-get clean && \
|
||||
find /var/lib/apt/lists/ -type f -not -name lock -delete && \
|
||||
useradd -m -u 1337 -U -s /bin/sh -d /datahaven datahaven && \
|
||||
mkdir -p /data /datahaven/.local/share /specs /storage && \
|
||||
chown -R datahaven:datahaven /data && \
|
||||
ln -s /data /datahaven/.local/share/datahaven-node && \
|
||||
echo "datahaven ALL=(ALL) NOPASSWD:ALL" >> /etc/sudoers && \
|
||||
chmod -R 777 /storage /data
|
||||
|
||||
USER datahaven
|
||||
|
||||
COPY --chown=datahaven:datahaven ./operator/target/x86_64-unknown-linux-gnu/release/datahaven-node /usr/local/bin/datahaven-node
|
||||
RUN chmod uog+x /usr/local/bin/datahaven-node
|
||||
|
||||
EXPOSE 9333 9944 30333 30334 9615
|
||||
|
||||
VOLUME ["/data"]
|
||||
|
||||
ENTRYPOINT ["datahaven-node"]
|
||||
CMD ["--tmp"]
|
||||
|
|
@ -7,7 +7,7 @@
|
|||
"cli": "bun run cli/index.ts",
|
||||
"fmt": "biome check .",
|
||||
"fmt:fix": "biome check --write .",
|
||||
"build:docker:operator": "docker build --no-cache --platform linux/amd64 -t datahavenxyz/datahaven:local -f ./docker/datahaven-node-local.dockerfile ../.",
|
||||
"build:docker:operator": "docker build --no-cache --platform linux/amd64 -t datahavenxyz/datahaven:local -f ../docker/datahaven-dev.Dockerfile ../.",
|
||||
"generate:wagmi": "wagmi generate",
|
||||
"generate:snowbridge-cfgs": "bun -e \"import {generateSnowbridgeConfigs} from './scripts/gen-snowbridge-cfgs.ts'; await generateSnowbridgeConfigs()\"",
|
||||
"generate:types": "(cd ../operator && cargo build --release) && bun x papi add --wasm \"../operator/target/release/wbuild/datahaven-stagenet-runtime/datahaven_stagenet_runtime.wasm\" datahaven",
|
||||
|
|
|
|||
Loading…
Reference in a new issue