## Summary This PR adds comprehensive Kubernetes deployment infrastructure for StorageHub components, enabling deployment of the full StorageHub network stack (MSP, BSP, Indexer, and Fisherman nodes) alongside DataHaven nodes in both local and stagenet environments. ### What's Added **1. New Helm Chart: StorageHub MSP Backend API** (`deploy/charts/backend/`) - REST API service for StorageHub operations - Connects to PostgreSQL database for indexed blockchain data - Connects to RPC nodes for real-time blockchain queries - Configurable via TOML configuration file - Supports environment-specific overrides - Includes comprehensive documentation **2. StorageHub Node Deployment Charts** (`deploy/charts/node/storagehub/`) - **MSP Node** (`sh-mspnode`): Main Service Provider nodes with charging capabilities - **BSP Node** (`sh-bspnode`): Backup Service Provider nodes for redundancy - **Indexer Node** (`sh-idxnode`): Full indexing node with PostgreSQL integration - **Fisherman Node** (`sh-fisherman`): Network monitoring and verification node **3. Environment Configurations** - **Local environment** (`deploy/environments/local/`): Development setup with hostpath storage - **Stagenet environment** (`deploy/environments/stagenet/`): Production-like setup with AWS EBS - PostgreSQL database configurations for Indexer and Fisherman nodes - Proper service discovery and network configuration **4. Enhanced CLI Tooling** (`test/cli/`) - New `deploy storagehub` command for deploying StorageHub components - Updated `launch storagehub` command for local testing - Interactive deployment with environment selection - Automatic database provisioning via Bitnami PostgreSQL charts **5. Node Configuration Improvements** - Fork-aware transaction pool for DH boot & validator nodes - Unsafe RPC methods exposed on MSP nodes (for provider operations) - JWT secret support for MSP Backend authentication - ECDSA key scheme for StorageHub BCSV keys (DataHaven compatibility) ### Architecture ``` StorageHub Stack: ├── MSP Nodes (2 replicas) → Storage providers with charging ├── BSP Nodes (2 replicas) → Backup storage providers ├── Indexer Node → Database indexing + PostgreSQL ├── Fisherman Node → Monitoring + PostgreSQL (shared with Indexer) └── MSP Backend API → REST API for StorageHub operations ``` ### Testing **Local Testing**: ```bash cd test bun cli launch storagehub # Interactive launcher # or bun cli deploy storagehub # Deploy via Helm ``` **Stagenet Deployment**: ```bash cd deploy helm install sh-mspnode ./charts/node \ -f ./charts/node/storagehub/sh-mspnode.yaml \ -f ./environments/stagenet/sh-mspnode.yaml \ -n datahaven-stagenet ``` ### Breaking Changes None - This is purely additive infrastructure. ### Migration Notes For existing deployments: 1. DataHaven nodes now use `--pool-type fork-aware` flag 2. Bootnode and validator node configs updated accordingly 3. No action required for existing DataHaven-only deployments |
||
|---|---|---|
| .. | ||
| chainspecs | ||
| charts | ||
| environments | ||
| README.md | ||
DataHaven Deployment
This directory contains Helm charts and configurations for deploying DataHaven nodes and relayers to Kubernetes clusters across various environments (local development, staging, production).
Directory Structure
deploy/
├── charts/ # Helm charts
│ ├── node/ # Node chart
│ │ └── datahaven/ # DataHaven-specific node configurations
│ │ ├── dh-bootnode.yaml
│ │ └── dh-validator.yaml
│ └── relay/ # Relay chart
│ └── snowbridge/ # Snowbridge-specific relay configurations
│ ├── dh-beacon-relay.yaml # Beacon chain relay
│ ├── dh-beefy-relay.yaml # BEEFY consensus relay
│ └── dh-execution-relay.yaml # Execution layer relay
├── environments/ # Environment-specific configurations
│ ├── local/ # Local development environment
│ │ └── values.yaml
│ ├── stagenet/ # Staging environment
│ └── values.yaml
└── scripts/ # Deployment scripts
Prerequisites
- Kubernetes cluster
- kubectl configured
- Helm 3.x installed
Deployment
The recommended way to deploy is using the DataHaven CLI with the deploy command:
cd test && bun cli deploy --e <environment>
Example:
cd test && bun cli deploy --e local
Available environments:
local: Local development environment (minimal resources)stagenet: Staging environment for pre-release testing
Environment Details
Local
- Purpose: Local development and testing
- Replicas: 1 (bootnode + validator)
- Resources: Minimal (256Mi memory, 100m CPU)
- Image: Local Docker builds (
datahavenxyz/datahaven:local) - Storage: Small persistence (1-5Gi)
- Network: Single-node network with fast block times
Stagenet
- Purpose: Pre-production testing and staging
- Replicas: 2+ validators
- Resources: Medium (512Mi memory, 200m CPU)
- Image: Stagenet tags from DockerHub
- Storage: 20Gi+ persistent volumes
- Network: Multi-validator network simulating production
Configuration Structure
The configuration is organized in layers, with later layers overriding earlier ones:
-
Base Configurations (
charts/node/datahaven/):- Base configurations for DataHaven nodes
- Default values for bootnode and validator
-
Environment-Specific Configurations (
environments/<env>/values.yaml):- Environment-specific settings
- Resource configurations
- Image tags
- Replica counts
- Storage configurations
The deployment process:
- Loads base configurations from the respective chart directories
- Applies environment-specific overrides from
environments/<env>/values.yaml - Deploys the components with the merged configuration
Components
Nodes
- Bootnode: Entry point for the network
- Validator: Validates transactions and produces blocks
Relays (Snowbridge)
- Beacon Relay: Relays Ethereum beacon chain finality to DataHaven
- BEEFY Relay: Relays DataHaven BEEFY finality proofs to Ethereum
- Execution Relay: Relays Ethereum execution layer messages to DataHaven
- Solochain Relayers: Relays DataHaven chain operations to the DataHaven AVS
These relayers enable trustless bidirectional token and message passing between Ethereum and DataHaven.
Development Workflow
-
Local Testing:
cd test bun cli launch # Starts local network without K8s -
K8s Deployment:
cd test bun cli deploy --e local -
Building Local Images:
cd test bun build:docker:operator # Builds datahavenxyz/datahaven:local -
Updating Configurations:
- Modify
environments/<env>/values.yamlfor environment-specific changes - Modify chart templates in
charts/for structural changes - Redeploy with
bun cli deploy --e <env>
- Modify
Troubleshooting
- Pods not starting: Check logs with
kubectl logs <pod-name> - Image pull failures: Verify Docker registry access and image tags
- Persistent volume issues: Ensure storage class is available with
kubectl get sc - Network connectivity: Check service endpoints with
kubectl get svc
For more detailed deployment and testing workflows, see the test directory.