Elgato_dark/datahaven
1
0
Fork 0
mirror of https://github.com/datahaven-xyz/datahaven synced 2026-05-23 17:28:23 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions
datahaven/operator/docker-compose.yml

379 lines
11 KiB
YAML
Raw Permalink Normal View History

feat: ✨ Add Docker Compose setup for local DataHaven network (#314) ## 🎯 Overview This PR introduces a comprehensive Docker Compose configuration for running a complete local DataHaven network, making it significantly easier for developers to spin up and test the entire stack locally. ## 🏗️ Architecture ```mermaid graph TB subgraph "DataHaven Network (Docker)" subgraph "Consensus Layer" Alice["🔷 Alice (Validator)<br/>:9944 RPC<br/>4 Keys: GRAN, BABE, IMON, BEEF"] Bob["🔷 Bob (Validator)<br/>:9945 RPC<br/>4 Keys: GRAN, BABE, IMON, BEEF"] Alice <-->|P2P/mDNS| Bob end subgraph "Storage Provider Layer" MSP["💾 MSP (Charlie)<br/>:9946 RPC<br/>1 GiB Storage<br/>1 Key: BCSV"] BSP01["💾 BSP01 (Dave)<br/>:9947 RPC<br/>1 GiB Storage<br/>1 Key: BCSV"] BSP02["💾 BSP02 (Eve)<br/>:9948 RPC<br/>1 GiB Storage<br/>1 Key: BCSV"] end subgraph "Monitoring Layer" Indexer["📊 Indexer<br/>:9949 RPC<br/>Full Mode<br/>No Keys"] Fisherman["🎣 Fisherman (Gustavo)<br/>:9950 RPC<br/>Storage Monitor<br/>1 Key: BCSV"] DB["🗄️ PostgreSQL<br/>:5432<br/>indexer/datahaven"] end Alice -.->|Syncs| MSP Bob -.->|Syncs| MSP Alice -.->|Syncs| BSP01 Bob -.->|Syncs| BSP02 Alice -.->|Syncs| Indexer Bob -.->|Syncs| Fisherman MSP -.->|Monitors| Fisherman BSP01 -.->|Monitors| Fisherman BSP02 -.->|Monitors| Fisherman Indexer -->|Writes| DB Fisherman -->|Writes| DB end style Alice fill:#4a90e2,stroke:#2e5c8a,color:#fff style Bob fill:#4a90e2,stroke:#2e5c8a,color:#fff style MSP fill:#50c878,stroke:#2d7a4a,color:#fff style BSP01 fill:#50c878,stroke:#2d7a4a,color:#fff style BSP02 fill:#50c878,stroke:#2d7a4a,color:#fff style Indexer fill:#f5a623,stroke:#b87818,color:#fff style Fisherman fill:#f5a623,stroke:#b87818,color:#fff style DB fill:#9b59b6,stroke:#6c3a82,color:#fff ``` **Legend:** - 🔷 Validators - Consensus and block production - 💾 Storage Providers - File storage and retrieval - 📊 Indexer - Full blockchain indexing - 🎣 Fisherman - Storage provider monitoring - 🗄️ PostgreSQL - Database for indexer/fisherman - `<-->` P2P Communication | `-.->` Network Sync | `-->` Database Connection ## ✨ What's Included ### Core Network (8 Services) - **2 Validator Nodes** (Alice & Bob) - Consensus and block production - **1 Main Storage Provider (MSP)** - Charlie with 1 GiB storage capacity - **2 Backup Storage Providers (BSPs)** - Dave and Eve with 1 GiB each - **1 StorageHub Indexer** - Full blockchain indexer with PostgreSQL - **1 Fisherman Node** - Gustavo monitoring storage provider behavior - **1 PostgreSQL Database** - Shared database for indexer and fisherman ### Key Features ✅ **Automated Key Injection** - All validator and storage provider keys automatically injected on startup ✅ **Health Checks** - Validators have health checks that verify RPC port is listening before dependent services start ✅ **Orchestrated Startup** - Services start in correct order with health-based dependencies ✅ **Persistent Storage** - Chain data, keystores, and database all persisted in Docker volumes ✅ **Unified Entrypoint Script** - Single script handles all node types (validator, MSP, BSP, fisherman) ✅ **Proper User Permissions** - Root for setup, switches to datahaven user for node execution ✅ **mDNS Peer Discovery** - Nodes automatically discover each other on the Docker network ✅ **Comprehensive Documentation** - Full setup guide, troubleshooting, and verification steps ### 🔄 Startup Sequence The network starts in a carefully orchestrated sequence to ensure stability: 1. **Alice (Validator)** starts first - Injects 4 keys (GRAN, BABE, IMON, BEEF) - Health check waits for RPC port 9944 to be listening - Uses `/proc/net/tcp` for minimal-dependency port checking 2. **Bob (Validator)** waits for Alice to be healthy - Ensures at least one validator is fully operational - Enables block production to start immediately 3. **Storage Providers & Monitoring** wait for both validators to be healthy - MSP, BSP01, BSP02 start after validators are ready - Indexer and Fisherman wait for validators before syncing - PostgreSQL starts independently with its own health check This dependency chain prevents race conditions and ensures reliable network formation. ## 🚀 Quick Start ```bash cd operator # Build the binary (development mode with fast blocks) ./scripts/docker-prepare.sh --fast # Start the entire network docker-compose up -d # View logs docker-compose logs -f # Check service health docker-compose ps # Stop the network docker-compose down -v ``` ## 📋 Port Mappings | Service | RPC/WebSocket | Prometheus | P2P | Database/API | |---------|---------------|------------|-----|--------------| | Alice | 9944 | 9615 | 30333 | - | | Bob | 9945 | 9616 | 30334 | - | | MSP | 9946 | 9617 | 30335 | - | | BSP01 | 9947 | 9618 | 30336 | - | | BSP02 | 9948 | 9619 | 30337 | - | | Indexer | 9949 | 9620 | 30338 | - | | Fisherman | 9950 | 9621 | 30339 | - | | PostgreSQL | - | - | - | 5432 | ## 🔑 Key Injection All cryptographic keys are automatically injected on startup: **Validators (Alice & Bob)** - 4 keys each: - GRANDPA (ed25519) - Finality - BABE (sr25519) - Block authoring - ImOnline (sr25519) - Heartbeat - BEEFY (ecdsa) - Bridge consensus **Storage Providers (MSP, BSPs, Fisherman)** - 1 key each: - BCSV (ecdsa) - Storage provider identity **Indexer** - No keys required (non-validating node) ## 🩺 Health Checks Validator nodes (Alice & Bob) implement health checks to ensure proper startup sequencing: **Health Check Method:** - Reads `/proc/net/tcp` directly to check if RPC port is listening - Zero dependencies - works in minimal debian:stable-slim containers - Converts port to hex and searches for LISTEN state (0A) **Configuration:** - Start period: 30s (allows node initialization) - Interval: 10s (check every 10 seconds) - Timeout: 5s (per health check) - Retries: 5 (must pass 5 consecutive checks) **Benefits:** - Prevents dependent services from starting before validators are ready - Eliminates race conditions during network formation - No additional packages required in Docker image ## 🍎 macOS Requirement **Important:** On Docker Desktop for macOS, you must use the **experimental DockerVMM** virtualization framework: 1. Open Docker Desktop settings 2. Go to "General" tab 3. Enable "Use experimental virtualization framework (DockerVMM)" 4. Restart Docker Desktop The default Apple Virtualization Framework causes networking issues with P2P connections. ## 📁 Files Changed - `operator/docker-compose.yml` - Main orchestration configuration - `operator/scripts/docker-entrypoint.sh` - Unified key injection script - `operator/scripts/docker-prepare.sh` - Binary build helper - `operator/scripts/docker-healthcheck.sh` - Health check script for validators - `operator/DOCKER-COMPOSE.md` - Comprehensive documentation ## 🔍 Testing The configuration has been tested on: - ✅ Docker Desktop for macOS (with DockerVMM) - ✅ Docker on Linux/Ubuntu All nodes successfully: - Inject required keys - Pass health checks - Discover peers via mDNS - Sync blocks and finalize - Connect to PostgreSQL database (indexer/fisherman) ## 📝 Notes - All settings are configured for **local development only** - Uses well-known test seed phrase (⚠️ never use in production!) - RPC exposed without authentication - Unsafe flags enabled for convenience --------- Co-authored-by: Claude <noreply@anthropic.com>
2025-11-22 13:07:46 +00:00
services:
alice:
build:
context: .
dockerfile: Dockerfile
image: datahavenxyz/datahaven:local
platform: linux/amd64
container_name: datahaven-alice
hostname: alice
user: "root" # Run as root to allow key injection, entrypoint switches to datahaven user
networks:
- datahaven-network
ports:
# Alice gets the standard port mappings
- "9944:9944" # WebSocket/RPC
- "9615:9615" # Prometheus metrics
- "30333:30333" # P2P networking
environment:
- NODE_NAME=alice
- NODE_TYPE=validator
- CHAIN=stagenet-local
- SEED=bottom drive obey lake curtain smoke basket hold race lonely fit walk
- RPC_PORT=9944
entrypoint: ["/scripts/docker-entrypoint.sh"]
command:
- --alice
- --chain=stagenet-local
- --unsafe-force-node-key-generation
- --base-path=/data
- --keystore-path=/data/keystore
- --validator
- --discover-local
- --no-prometheus
- --unsafe-rpc-external
- --rpc-cors=all
- --force-authoring
- --no-telemetry
- --enable-offchain-indexing=true
- --pool-type=fork-aware
volumes:
- ./scripts/docker-entrypoint.sh:/scripts/docker-entrypoint.sh:ro
- ./scripts/docker-healthcheck.sh:/scripts/docker-healthcheck.sh:ro
- alice-keystore:/data/keystore
healthcheck:
test: ["CMD-SHELL", "/scripts/docker-healthcheck.sh"]
interval: 10s
timeout: 5s
retries: 5
start_period: 30s
restart: unless-stopped
bob:
build:
context: .
dockerfile: Dockerfile
image: datahavenxyz/datahaven:local
platform: linux/amd64
container_name: datahaven-bob
hostname: bob
user: "root" # Run as root to allow key injection, entrypoint switches to datahaven user
networks:
- datahaven-network
ports:
# Bob gets different ports to avoid conflicts with Alice
- "9945:9944" # WebSocket/RPC
- "9616:9615" # Prometheus metrics
- "30334:30333" # P2P networking
environment:
- NODE_NAME=bob
- NODE_TYPE=validator
- CHAIN=stagenet-local
- SEED=bottom drive obey lake curtain smoke basket hold race lonely fit walk
- RPC_PORT=9944
entrypoint: ["/scripts/docker-entrypoint.sh"]
command:
- --bob
- --chain=stagenet-local
- --unsafe-force-node-key-generation
- --base-path=/data
- --keystore-path=/data/keystore
- --validator
- --discover-local
- --no-prometheus
- --unsafe-rpc-external
- --rpc-cors=all
- --no-telemetry
- --enable-offchain-indexing=true
- --pool-type=fork-aware
volumes:
- ./scripts/docker-entrypoint.sh:/scripts/docker-entrypoint.sh:ro
- ./scripts/docker-healthcheck.sh:/scripts/docker-healthcheck.sh:ro
- bob-keystore:/data/keystore
healthcheck:
test: ["CMD-SHELL", "/scripts/docker-healthcheck.sh"]
interval: 10s
timeout: 5s
retries: 5
start_period: 30s
restart: unless-stopped
depends_on:
alice:
condition: service_healthy
msp:
build:
context: .
dockerfile: Dockerfile
image: datahavenxyz/datahaven:local
platform: linux/amd64
container_name: datahaven-msp
hostname: msp
user: "root" # Run as root to allow key injection, entrypoint switches to datahaven user
networks:
- datahaven-network
ports:
# MSP gets different ports to avoid conflicts with validators
- "9946:9944" # WebSocket/RPC
- "9617:9615" # Prometheus metrics
- "30335:30333" # P2P networking
environment:
- NODE_NAME=charlie
- NODE_TYPE=msp
- SEED=bottom drive obey lake curtain smoke basket hold race lonely fit walk
entrypoint: ["/scripts/docker-entrypoint.sh"]
command:
- --name=msp
- --chain=stagenet-local
- --unsafe-force-node-key-generation
- --base-path=/data
- --keystore-path=/data/keystore
- --discover-local
- --unsafe-rpc-external
- --rpc-cors=all
- --no-prometheus
- --no-telemetry
- --enable-offchain-indexing=true
- --pool-type=fork-aware
- --provider
- --provider-type=msp
- --msp-charging-period=100
- --max-storage-capacity=1073741824
- --jump-capacity=104857600
- --msp-distribute-files
volumes:
- ./scripts/docker-entrypoint.sh:/scripts/docker-entrypoint.sh:ro
- msp-keystore:/data/keystore
restart: unless-stopped
depends_on:
alice:
condition: service_healthy
bob:
condition: service_healthy
bsp01:
build:
context: .
dockerfile: Dockerfile
image: datahavenxyz/datahaven:local
platform: linux/amd64
container_name: datahaven-bsp01
hostname: bsp01
user: "root" # Run as root to allow key injection, entrypoint switches to datahaven user
networks:
- datahaven-network
ports:
# BSP gets different ports to avoid conflicts with validators
- "9947:9944" # WebSocket/RPC
- "9618:9615" # Prometheus metrics
- "30336:30333" # P2P networking
environment:
- NODE_NAME=dave
- NODE_TYPE=bsp
- SEED=bottom drive obey lake curtain smoke basket hold race lonely fit walk
entrypoint: ["/scripts/docker-entrypoint.sh"]
command:
- --name=bsp01
- --chain=stagenet-local
- --unsafe-force-node-key-generation
- --base-path=/data
- --keystore-path=/data/keystore
- --discover-local
- --unsafe-rpc-external
- --rpc-cors=all
- --no-prometheus
- --no-telemetry
- --enable-offchain-indexing=true
- --pool-type=fork-aware
- --provider
- --provider-type=bsp
- --max-storage-capacity=1073741824
- --jump-capacity=104857600
volumes:
- ./scripts/docker-entrypoint.sh:/scripts/docker-entrypoint.sh:ro
- bsp01-keystore:/data/keystore
restart: unless-stopped
depends_on:
alice:
condition: service_healthy
bob:
condition: service_healthy
msp:
condition: service_started
bsp02:
build:
context: .
dockerfile: Dockerfile
image: datahavenxyz/datahaven:local
platform: linux/amd64
container_name: datahaven-bsp02
hostname: bsp02
user: "root" # Run as root to allow key injection, entrypoint switches to datahaven user
networks:
- datahaven-network
ports:
# BSP gets different ports to avoid conflicts with validators
- "9948:9944" # WebSocket/RPC
- "9619:9615" # Prometheus metrics
- "30337:30333" # P2P networking
environment:
- NODE_NAME=eve
- NODE_TYPE=bsp
- SEED=bottom drive obey lake curtain smoke basket hold race lonely fit walk
entrypoint: ["/scripts/docker-entrypoint.sh"]
command:
- --name=bsp02
- --chain=stagenet-local
- --unsafe-force-node-key-generation
- --base-path=/data
- --keystore-path=/data/keystore
- --discover-local
- --unsafe-rpc-external
- --rpc-cors=all
- --no-prometheus
- --no-telemetry
- --enable-offchain-indexing=true
- --pool-type=fork-aware
- --provider
- --provider-type=bsp
- --max-storage-capacity=1073741824
- --jump-capacity=104857600
volumes:
- ./scripts/docker-entrypoint.sh:/scripts/docker-entrypoint.sh:ro
- bsp02-keystore:/data/keystore
restart: unless-stopped
depends_on:
alice:
condition: service_healthy
bob:
condition: service_healthy
msp:
condition: service_started
postgres:
image: postgres:18-alpine
platform: linux/amd64
container_name: datahaven-postgres
hostname: postgres
networks:
- datahaven-network
ports:
- "5432:5432"
environment:
- POSTGRES_USER=indexer
- POSTGRES_PASSWORD=indexer
- POSTGRES_DB=datahaven
volumes:
- postgres-data:/var/lib/postgresql/data
restart: unless-stopped
healthcheck:
test: ["CMD-SHELL", "pg_isready -U indexer -d datahaven"]
interval: 10s
timeout: 5s
retries: 5
indexer:
build:
context: .
dockerfile: Dockerfile
image: datahavenxyz/datahaven:local
platform: linux/amd64
container_name: datahaven-indexer
hostname: indexer
networks:
- datahaven-network
ports:
- "9949:9944" # WebSocket/RPC
- "9620:9615" # Prometheus metrics
- "30338:30333" # P2P networking
environment:
- CHAIN=stagenet-local
command:
- --name=indexer
- --chain=stagenet-local
- --unsafe-force-node-key-generation
- --base-path=/data
- --discover-local
- --unsafe-rpc-external
- --rpc-cors=all
- --no-prometheus
- --no-telemetry
- --enable-offchain-indexing=true
- --pool-type=fork-aware
- --indexer
- --indexer-mode=full
- --indexer-database-url=postgresql://indexer:indexer@postgres:5432/datahaven
volumes:
- indexer-data:/data
restart: unless-stopped
depends_on:
postgres:
condition: service_healthy
alice:
condition: service_healthy
bob:
condition: service_healthy
fisherman:
build:
context: .
dockerfile: Dockerfile
image: datahavenxyz/datahaven:local
platform: linux/amd64
container_name: datahaven-fisherman
hostname: fisherman
user: "root" # Run as root to allow key injection, entrypoint switches to datahaven user
networks:
- datahaven-network
ports:
- "9950:9944" # WebSocket/RPC
- "9621:9615" # Prometheus metrics
- "30339:30333" # P2P networking
environment:
- NODE_NAME=gustavo
- NODE_TYPE=fisherman
- CHAIN=stagenet-local
- SEED=bottom drive obey lake curtain smoke basket hold race lonely fit walk
entrypoint: ["/scripts/docker-entrypoint.sh"]
command:
- --name=fisherman
- --chain=stagenet-local
- --unsafe-force-node-key-generation
- --base-path=/data
- --keystore-path=/data/keystore
- --discover-local
- --unsafe-rpc-external
- --rpc-cors=all
- --no-prometheus
- --no-telemetry
- --enable-offchain-indexing=true
- --pool-type=fork-aware
- --fisherman
- --fisherman-database-url=postgresql://indexer:indexer@postgres:5432/datahaven
volumes:
- ./scripts/docker-entrypoint.sh:/scripts/docker-entrypoint.sh:ro
- fisherman-keystore:/data/keystore
restart: unless-stopped
depends_on:
postgres:
condition: service_healthy
alice:
condition: service_healthy
bob:
condition: service_healthy
networks:
datahaven-network:
driver: bridge
name: datahaven-network
volumes:
alice-keystore:
bob-keystore:
msp-keystore:
bsp01-keystore:
bsp02-keystore:
postgres-data:
indexer-data:
fisherman-keystore:
Reference in a new issue Copy permalink