## Overview This PR consolidates and optimizes the Docker build system, reducing redundancy and improving CI/CD performance. The changes eliminate duplicate Dockerfiles, introduce a flexible build template, and optimize release builds to reuse CI artifacts. ## Changes Summary ### 🐳 Docker Images Restructured **Before:** 5 Dockerfiles with significant overlap **After:** 4 focused images + 1 utility #### Final Structure: 1. **`operator/Dockerfile`** ✨ Updated - **Standard operator image** for CI and release builds - Minimal node image (accepts pre-built binaries) - GHCR: `ghcr.io/datahaven-xyz/datahaven/datahaven` (CI) - DockerHub: `datahavenxyz/datahaven` (releases) 2. **`docker/datahaven-build.Dockerfile`** (moved from `operator/Dockerfile`) - Full source-to-binary build for manual releases - DockerHub: `datahavenxyz/datahaven:{label}` - Supports custom RUSTFLAGS and fast-runtime feature - Only used for manual workflow_dispatch builds 3. **`docker/datahaven-production.Dockerfile`** (kept) - Binary builder for CPU-specific releases - Used by build-prod-binary workflow template - Supports custom target-cpu flags 4. **`docker/datahaven-dev.Dockerfile`** ✨ NEW (local dev only) - **FOR LOCAL DEVELOPMENT/TROUBLESHOOTING ONLY** - Includes debug tools: gdb, strace, vim, sudo - Extra dependencies: librocksdb-dev, curl - RUST_BACKTRACE enabled by default - **DO NOT USE for CI or production builds** 5. **`test/docker/crossbuild-mac-libpq.dockerfile`** (kept) - Utility for macOS → Linux cross-compilation #### Removed (Redundant): - ❌ `docker/datahaven.Dockerfile` → replaced by operator/Dockerfile - ❌ `test/docker/datahaven-node-local.dockerfile` → replaced by datahaven-dev.Dockerfile --- ### 🔄 Workflow Improvements #### Enhanced `publish-docker` Template - Supports both GHCR and DockerHub registries - Flexible inputs: dockerfile, context, build-args, cache scope - Auto-generates OCI-compliant labels - Reduces code duplication (~70 lines → ~15 per workflow) #### Refactored CI Pipeline - **`docker-build-ci`**: Builds `operator/Dockerfile` → GHCR for CI/E2E testing - **`docker-build-release`**: Builds `operator/Dockerfile` → DockerHub (main branch only) - Both CI and release workflows now use the same minimal operator image - Release builds **reuse CI binaries** instead of rebuilding from source #### Optimized Release Workflow The `task-docker-release` workflow now has dual modes: **Mode 1: `workflow_call` (CI - main pushes)** - ✅ Reuses binary from CI's build-operator task - ✅ Uses lightweight `operator/Dockerfile` - ✅ Tags: `latest`, `sha-{short}` - ⚡ **Fast**: ~5 minutes (vs ~30 min previously) **Mode 2: `workflow_dispatch` (Manual)** - ✅ Full source build with `datahaven-build.Dockerfile` - ✅ Custom branch and label selection - ✅ Optional fast-runtime feature - ✅ Tags: `PROD-{label}` or user-defined --- ### 🔧 Additional Optimizations - Copy libpq5 from builder stage instead of reinstalling (smaller, faster) - Remove redundant protobuf-compiler package (use protoc v21.12 directly) - Standardize user UID to 1000 across all runtime images - Consistent OCI labeling and metadata --------- Co-authored-by: Claude <noreply@anthropic.com> |
||
|---|---|---|
| .github | ||
| contracts | ||
| deploy | ||
| docker | ||
| operator | ||
| resources | ||
| test | ||
| tools | ||
| .gitignore | ||
| .gitmodules | ||
| biome.json | ||
| CLAUDE.md | ||
| README.md | ||
| taplo.toml | ||
DataHaven 🫎
An EVM-compatible Substrate blockchain secured by EigenLayer, bridging Ethereum and Substrate ecosystems through trustless cross-chain communication.
Overview
DataHaven is an EigenLayer Actively Validated Service (AVS) that combines:
- EVM Compatibility: Full Ethereum support via Frontier pallets for smart contracts and dApps
- EigenLayer Security: Validator set secured by Ethereum's economic security through restaking
- Cross-chain Bridge: Seamless asset and message transfers with Ethereum via Snowbridge
- Dynamic Validators: Operator registry managed on-chain through EigenLayer contracts
- Performance Rewards: Validator incentives distributed cross-chain from Ethereum
Architecture
DataHaven bridges two major blockchain ecosystems:
┌───────────────────────────────────────────────────────────────┐
│ Ethereum (L1) │
│ ┌────────────────────────────────────────────────────────┐ │
│ │ EigenLayer AVS Contracts │ │
│ │ • DataHavenServiceManager (operator lifecycle) │ │
│ │ • RewardsRegistry (performance tracking) │ │
│ │ • VetoableSlasher (misbehavior penalties) │ │
│ └────────────────────────────────────────────────────────┘ │
│ ↕ │
│ Snowbridge Protocol │
└───────────────────────────────────────────────────────────────┘
↕
┌───────────────────────────────────────────────────────────────┐
│ DataHaven (Substrate) │
│ ┌────────────────────────────────────────────────────────┐ │
│ │ Custom Pallets │ │
│ │ • External Validators (sync validator set) │ │
│ │ • Native Transfer (cross-chain tokens) │ │
│ │ • Rewards (distribute validator rewards) │ │
│ │ • Frontier (EVM compatibility) │ │
│ └────────────────────────────────────────────────────────┘ │
└───────────────────────────────────────────────────────────────┘
Repository Structure
datahaven/
├── contracts/ # EigenLayer AVS smart contracts
│ ├── src/ # Service Manager, Rewards Registry, Slasher
│ ├── script/ # Deployment scripts
│ └── test/ # Foundry test suites
├── operator/ # Substrate-based DataHaven node
│ ├── node/ # Node implementation & chain spec
│ ├── pallets/ # Custom pallets (validators, rewards, transfers)
│ └── runtime/ # Runtime configurations (mainnet/stagenet/testnet)
├── test/ # E2E testing framework
│ ├── suites/ # Integration test scenarios
│ ├── framework/ # Test utilities and helpers
│ └── launcher/ # Network deployment automation
├── deploy/ # Kubernetes deployment charts
│ ├── charts/ # Helm charts for nodes and relayers
│ └── environments/ # Environment-specific configurations
├── tools/ # GitHub automation and release scripts
└── .github/ # CI/CD workflows
Each directory contains its own README with detailed information. See:
- contracts/README.md - Smart contract development
- operator/README.md - Node building and runtime development
- test/README.md - E2E testing and network deployment
- deploy/README.md - Kubernetes deployment
- tools/README.md - Development tools
Quick Start
Prerequisites
- Kurtosis - Network orchestration
- Bun v1.2+ - TypeScript runtime
- Docker - Container management
- Foundry - Solidity toolkit
- Rust - For building the operator
- Helm - Kubernetes deployments (optional)
- Zig - For macOS cross-compilation (macOS only)
Launch Local Network
The fastest way to get started is with the interactive CLI:
cd test
bun i # Install dependencies
bun cli launch # Interactive launcher with prompts
This deploys a complete environment including:
- Ethereum network: 2x EL clients (reth), 2x CL clients (lodestar)
- Block explorers: Blockscout (optional), Dora consensus explorer
- DataHaven node: Single validator with fast block times
- AVS contracts: Deployed and configured on Ethereum
- Snowbridge relayers: Bidirectional message passing
For more options and detailed instructions, see the test README.
Run Tests
cd test
bun test:e2e # Run all integration tests
bun test:e2e:parallel # Run with limited concurrency
Development Workflows
Smart Contract Development:
cd contracts
forge build # Compile contracts
forge test # Run contract tests
Node Development:
cd operator
cargo build --release --features fast-runtime
cargo test
./scripts/run-benchmarks.sh
After Making Changes:
cd test
bun generate:wagmi # Regenerate contract bindings
bun generate:types # Regenerate runtime types
Key Features
EVM Compatibility
Full Ethereum Virtual Machine support via Frontier pallets:
- Deploy Solidity smart contracts
- Use existing Ethereum tooling (MetaMask, Hardhat, etc.)
- Compatible with ERC-20, ERC-721, and other standards
EigenLayer Integration
Validator security anchored to Ethereum:
- Operators register via
DataHavenServiceManagercontract - Economic security through ETH restaking
- Slashing protection with veto period via
VetoableSlasher - Performance-based rewards through
RewardsRegistry
Cross-chain Communication
Trustless bridging via Snowbridge:
- Native token transfers between Ethereum ↔ DataHaven
- Cross-chain message passing
- Finality proofs via BEEFY consensus
- Three specialized relayers (beacon, BEEFY, execution)
Dynamic Validator Set
Validator management synchronized with Ethereum:
- EigenLayer operator registry as source of truth
- On-chain validator set updates via External Validators pallet
- Automatic consensus participation changes
- Cross-chain coordination for validator lifecycle
Docker Images
Production images published to DockerHub.
Build optimizations:
- sccache - Rust compilation caching
- cargo-chef - Dependency layer caching
- BuildKit cache mounts - External cache restoration
Build locally:
cd test
bun build:docker:operator # Creates datahavenxyz/datahaven:local
Development Environment
VS Code Configuration
IDE configurations are excluded from version control for personalization, but these settings are recommended for optimal developer experience. Add to your .vscode/settings.json:
Rust Analyzer:
{
"rust-analyzer.linkedProjects": ["./operator/Cargo.toml"],
"rust-analyzer.cargo.allTargets": true,
"rust-analyzer.procMacro.enable": false,
"rust-analyzer.server.extraEnv": {
"CARGO_TARGET_DIR": "target/.rust-analyzer",
"SKIP_WASM_BUILD": 1
},
"rust-analyzer.diagnostics.disabled": ["unresolved-macro-call"],
"rust-analyzer.cargo.buildScripts.enable": false
}
Optimizations:
- Links
operator/directory as the primary Rust project - Disables proc macros and build scripts for faster analysis (Substrate macros are slow)
- Uses dedicated target directory to avoid conflicts
- Skips WASM builds during development
Solidity (Juan Blanco's extension):
{
"solidity.formatter": "forge",
"solidity.compileUsingRemoteVersion": "v0.8.28+commit.7893614a",
"[solidity]": {
"editor.defaultFormatter": "JuanBlanco.solidity"
}
}
Note: Solidity version must match foundry.toml
TypeScript (Biome):
{
"biome.lsp.bin": "test/node_modules/.bin/biome",
"[typescript]": {
"editor.defaultFormatter": "biomejs.biome",
"editor.codeActionsOnSave": {
"source.organizeImports.biome": "always"
}
}
}
CI/CD
Local CI Testing
Run GitHub Actions workflows locally using act:
# Run E2E workflow
act -W .github/workflows/e2e.yml -s GITHUB_TOKEN="$(gh auth token)"
# Run specific job
act -W .github/workflows/e2e.yml -j test-job-name
Automated Workflows
The repository includes GitHub Actions for:
- E2E Testing: Full integration tests on PR and main branch
- Contract Testing: Foundry test suites for smart contracts
- Rust Testing: Unit and integration tests for operator
- Docker Builds: Multi-platform image builds with caching
- Release Automation: Version tagging and changelog generation
See .github/workflows/ for workflow definitions.
Contributing
Development Cycle
- Make Changes: Edit contracts, runtime, or tests
- Run Tests: Component-specific tests (
forge test,cargo test) - Regenerate Types: Update bindings if contracts/runtime changed
- Integration Test: Run E2E tests to verify cross-component behavior
- Code Quality: Format and lint (
cargo fmt,forge fmt,bun fmt:fix)
Common Pitfalls
- Type mismatches: Regenerate with
bun generate:typesafter runtime changes - Contract changes not reflected: Run
bun generate:wagmiafter modifications - Kurtosis issues: Ensure Docker is running and Kurtosis engine is started
- Slow development: Use
--features fast-runtimefor faster block times - Network launch hangs: Check Blockscout - forge output can appear frozen
See CLAUDE.md for detailed development guidance.
License
GPL-3.0 - See LICENSE file for details