From bd746b340cd5361b43240b2eab1ad5af7855ebf3 Mon Sep 17 00:00:00 2001 From: Steve Degosserie <723552+stiiifff@users.noreply.github.com> Date: Tue, 25 Nov 2025 14:15:51 +0100 Subject: [PATCH] =?UTF-8?q?docs:=20=F0=9F=93=9A=20Add=20comprehensive=20no?= =?UTF-8?q?de=20setup=20documentation?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Add documentation for DataHaven and StorageHub node setup and operations: DataHaven nodes: - Bootnode: Peer discovery configuration - Validator: 4 session keys (BABE, GRANDPA, ImOnline, BEEFY) - Full Node: RPC endpoint setup StorageHub nodes: - MSP: Main Storage Provider with 2-step registration - BSP: Backup Storage Provider with 2-step registration - Indexer: PostgreSQL-backed blockchain indexer - Fisherman: Storage provider monitor Each document includes: - CLI flags with descriptions - Key injection requirements - Wallet/funding requirements - On-chain registration extrinsics (using non-privileged 2-step process for MSP/BSP) - Docker and Kubernetes deployment examples - Monitoring and troubleshooting guides 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude --- docs/README.md | 76 +++++ docs/datahaven-bootnode.md | 308 +++++++++++++++++ docs/datahaven-fullnode.md | 439 ++++++++++++++++++++++++ docs/datahaven-validator.md | 451 +++++++++++++++++++++++++ docs/storagehub-bsp.md | 636 +++++++++++++++++++++++++++++++++++ docs/storagehub-fisherman.md | 591 ++++++++++++++++++++++++++++++++ docs/storagehub-indexer.md | 621 ++++++++++++++++++++++++++++++++++ docs/storagehub-msp.md | 586 ++++++++++++++++++++++++++++++++ 8 files changed, 3708 insertions(+) create mode 100644 docs/README.md create mode 100644 docs/datahaven-bootnode.md create mode 100644 docs/datahaven-fullnode.md create mode 100644 docs/datahaven-validator.md create mode 100644 docs/storagehub-bsp.md create mode 100644 docs/storagehub-fisherman.md create mode 100644 docs/storagehub-indexer.md create mode 100644 docs/storagehub-msp.md diff --git a/docs/README.md b/docs/README.md new file mode 100644 index 00000000..e1e81678 --- /dev/null +++ b/docs/README.md @@ -0,0 +1,76 @@ +# DataHaven Node Operations Documentation + +This directory contains comprehensive documentation for setting up and operating DataHaven and StorageHub nodes. + +## Documentation Structure + +### DataHaven Nodes +- [Bootnode Setup](./datahaven-bootnode.md) - Bootnode configuration and operations +- [Validator Setup](./datahaven-validator.md) - Validator node configuration and operations +- [Full Node Setup](./datahaven-fullnode.md) - Full node (RPC) configuration and operations + +### StorageHub Nodes +- [MSP Setup](./storagehub-msp.md) - Main Storage Provider configuration and operations +- [BSP Setup](./storagehub-bsp.md) - Backup Storage Provider configuration and operations +- [Indexer Setup](./storagehub-indexer.md) - Indexer node configuration and operations +- [Fisherman Setup](./storagehub-fisherman.md) - Fisherman node configuration and operations + +## Quick Reference + +### Node Types Overview + +| Node Type | Purpose | Keys Required | On-Chain Registration | Database Required | +|-----------|---------|---------------|----------------------|-------------------| +| **Bootnode** | Network peer discovery | None | No | No | +| **Validator** | Block production & consensus | 4 (BABE, GRANDPA, ImOnline, BEEFY) | Yes (session.setKeys) | No | +| **Full Node** | RPC endpoint, sync only | None | No | No | +| **MSP** | Main storage provider | 1 (BCSV ECDSA) | Yes (2-step: request + confirm) | Optional | +| **BSP** | Backup storage provider | 1 (BCSV ECDSA) | Yes (2-step: request + confirm) | No | +| **Indexer** | Blockchain data indexer | None | No | Yes (PostgreSQL) | +| **Fisherman** | Storage provider monitor | 1 (BCSV ECDSA) | No | Yes (PostgreSQL) | + +### Common CLI Flags + +All node types support standard Substrate flags: +- `--chain ` - Chain specification (dev, local, stagenet-local, testnet-local, mainnet-local) +- `--base-path ` - Base directory for chain data +- `--name ` - Human-readable node name +- `--port ` - P2P network port (default: 30333) +- `--rpc-port ` - WebSocket RPC port (default: 9944) +- `--rpc-external` - Listen on all network interfaces +- `--rpc-cors ` - CORS origins for RPC (default: localhost) +- `--bootnodes ` - Bootstrap nodes for peer discovery + +### Key Types Reference + +| Key Type | Scheme | Purpose | Required For | +|----------|--------|---------|--------------| +| `gran` | ed25519 | GRANDPA finality | Validators | +| `babe` | sr25519 | BABE block authoring | Validators | +| `imon` | sr25519 | ImOnline heartbeat | Validators | +| `beef` | ecdsa | BEEFY bridge consensus | Validators | +| `bcsv` | ecdsa | Storage provider identity | MSP, BSP, Fisherman | + +### Prerequisites + +- [Docker](https://www.docker.com/) - Container runtime +- [Bun](https://bun.sh/) v1.2+ - For testing and tooling +- [Foundry](https://getfoundry.sh/) - For smart contract operations +- [PostgreSQL](https://www.postgresql.org/) - For Indexer and Fisherman nodes + +### Getting Started + +1. Choose your node type from the list above +2. Follow the specific setup guide for that node type +3. Generate or import keys as required +4. Configure CLI flags and environment +5. Start the node +6. Complete on-chain registration (if required) + +### Support & Resources + +- [Main Repository](https://github.com/Moonsong-Labs/datahaven) +- [StorageHub Repository](https://github.com/Moonsong-Labs/storage-hub) +- [E2E Testing Guide](../test/README.md) +- [Docker Compose Guide](../operator/DOCKER-COMPOSE.md) +- [Kubernetes Deployment](../deploy/charts/node/README.md) diff --git a/docs/datahaven-bootnode.md b/docs/datahaven-bootnode.md new file mode 100644 index 00000000..9890abb6 --- /dev/null +++ b/docs/datahaven-bootnode.md @@ -0,0 +1,308 @@ +# DataHaven Bootnode Setup + +## Overview + +A bootnode serves as an entry point for peer discovery in the DataHaven network. It maintains a stable network identity and helps new nodes discover peers. + +## Purpose + +- Provide stable peer discovery endpoint +- Maintain persistent network identity +- Facilitate initial network connections for new nodes +- No participation in consensus or block production + +## Prerequisites + +- DataHaven node binary or Docker image +- Persistent storage for node key +- Open network port (default: 30333) + +## Key Requirements + +### Node Key + +Bootnodes require a **persistent node key** to maintain a stable peer ID. + +#### Generate Node Key + +```bash +# Generate a new node key +datahaven-node key generate-node-key > node-key.txt + +# View the generated peer ID +datahaven-node key inspect-node-key --file node-key.txt +``` + +The output will show: +``` +12D3KooWXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX +``` + +### No Session Keys Required + +Bootnodes do **not** require session keys (BABE, GRANDPA, ImOnline, BEEFY) as they do not participate in consensus. + +## Wallet Requirements + +### No Wallet Required + +Bootnodes do not submit transactions or participate in consensus, so no funded account is needed. + +## CLI Flags + +### Required Flags + +```bash +datahaven-node \ + --chain \ + --name \ + --node-key-file +``` + +### Important Flags + +| Flag | Description | Default | +|------|-------------|---------| +| `--chain ` | Chain specification (stagenet-local, testnet-local, mainnet-local) | Required | +| `--name ` | Human-readable node name | Required | +| `--node-key-file ` | Path to node key file | Required | +| `--base-path ` | Base directory for chain data | `~/.local/share/datahaven-node` | +| `--port ` | P2P network port | `30333` | +| `--listen-addr ` | Listen address for P2P | `/ip4/0.0.0.0/tcp/30333` | +| `--public-addr ` | Public address to advertise | Auto-detected | + +### Optional Flags + +| Flag | Description | +|------|-------------| +| `--no-telemetry` | Disable telemetry reporting | +| `--log ` | Logging targets (e.g., `info,libp2p=debug`) | +| `--unsafe-rpc-external` | Allow external RPC access (not recommended) | + +## Complete Setup Example + +### 1. Generate Node Key + +```bash +mkdir -p /data/bootnode +datahaven-node key generate-node-key > /data/bootnode/node-key.txt +``` + +### 2. Get Peer ID + +```bash +PEER_ID=$(datahaven-node key inspect-node-key --file /data/bootnode/node-key.txt) +echo "Bootnode Peer ID: $PEER_ID" +``` + +### 3. Start Bootnode + +```bash +datahaven-node \ + --chain stagenet-local \ + --name "Bootnode-01" \ + --base-path /data/bootnode \ + --node-key-file /data/bootnode/node-key.txt \ + --port 30333 \ + --listen-addr /ip4/0.0.0.0/tcp/30333 \ + --public-addr /dns/bootnode.example.com/tcp/30333 \ + --no-telemetry +``` + +### 4. Advertise Bootnode Address + +Other nodes can connect using: +```bash +--bootnodes /dns/bootnode.example.com/tcp/30333/p2p/$PEER_ID +``` + +## Docker Deployment + +### Docker Compose + +```yaml +version: '3.8' + +services: + bootnode: + image: datahavenxyz/datahaven:latest + container_name: datahaven-bootnode + ports: + - "30333:30333" + volumes: + - bootnode-data:/data + - ./node-key.txt:/data/node-key.txt:ro + command: + - "--chain=stagenet-local" + - "--name=Bootnode-01" + - "--base-path=/data" + - "--node-key-file=/data/node-key.txt" + - "--port=30333" + - "--listen-addr=/ip4/0.0.0.0/tcp/30333" + - "--no-telemetry" + +volumes: + bootnode-data: +``` + +### Docker Run + +```bash +docker run -d \ + --name datahaven-bootnode \ + -p 30333:30333 \ + -v $(pwd)/bootnode-data:/data \ + -v $(pwd)/node-key.txt:/data/node-key.txt:ro \ + datahavenxyz/datahaven:latest \ + --chain stagenet-local \ + --name "Bootnode-01" \ + --base-path /data \ + --node-key-file /data/node-key.txt \ + --port 30333 \ + --no-telemetry +``` + +## Kubernetes Deployment + +```yaml +apiVersion: v1 +kind: Service +metadata: + name: datahaven-bootnode +spec: + type: LoadBalancer + ports: + - port: 30333 + targetPort: 30333 + name: p2p + selector: + app: datahaven-bootnode + +--- +apiVersion: apps/v1 +kind: StatefulSet +metadata: + name: datahaven-bootnode +spec: + serviceName: datahaven-bootnode + replicas: 1 + selector: + matchLabels: + app: datahaven-bootnode + template: + metadata: + labels: + app: datahaven-bootnode + spec: + containers: + - name: bootnode + image: datahavenxyz/datahaven:latest + ports: + - containerPort: 30333 + name: p2p + volumeMounts: + - name: data + mountPath: /data + - name: node-key + mountPath: /data/node-key.txt + subPath: node-key.txt + readOnly: true + args: + - "--chain=stagenet-local" + - "--name=Bootnode-01" + - "--base-path=/data" + - "--node-key-file=/data/node-key.txt" + - "--port=30333" + - "--no-telemetry" + volumes: + - name: node-key + secret: + secretName: bootnode-node-key + volumeClaimTemplates: + - metadata: + name: data + spec: + accessModes: [ "ReadWriteOnce" ] + resources: + requests: + storage: 100Gi +``` + +## On-Chain Registration + +### Not Required + +Bootnodes do not require any on-chain registration or extrinsics. + +## Monitoring + +### Health Checks + +```bash +# Check peer count +curl -H "Content-Type: application/json" \ + -d '{"id":1, "jsonrpc":"2.0", "method": "system_health"}' \ + http://localhost:9944 + +# Check node info +curl -H "Content-Type: application/json" \ + -d '{"id":1, "jsonrpc":"2.0", "method": "system_localPeerId"}' \ + http://localhost:9944 +``` + +### Logs + +```bash +# View logs with Docker +docker logs -f datahaven-bootnode + +# Filter for connection events +docker logs datahaven-bootnode 2>&1 | grep -i "discovered\|connected" +``` + +## Troubleshooting + +### Issue: Peers Cannot Connect + +**Check:** +1. Port 30333 is open in firewall +2. Public address is correctly configured +3. DNS resolves correctly (if using DNS) +4. Node key file has correct permissions + +### Issue: Node Key Not Found + +**Solution:** +```bash +# Verify node key file exists +ls -la /data/bootnode/node-key.txt + +# Check file permissions +chmod 600 /data/bootnode/node-key.txt +``` + +### Issue: Network Identity Changes + +**Solution:** +Always use `--node-key-file` instead of `--node-key` to ensure the key persists across restarts. + +## Security Considerations + +1. **Node Key Protection**: Keep node key file secure with restricted permissions (600) +2. **RPC Access**: Do not expose RPC publicly on bootnodes +3. **DDoS Protection**: Implement rate limiting at network level +4. **Monitoring**: Set up alerts for unexpected downtime + +## Best Practices + +1. Run multiple bootnodes for redundancy +2. Use DNS names instead of IP addresses for flexibility +3. Monitor peer connections and network health +4. Keep node software updated +5. Backup node key securely + +## Related Documentation + +- [Validator Setup](./datahaven-validator.md) +- [Full Node Setup](./datahaven-fullnode.md) +- [Docker Compose Guide](../operator/DOCKER-COMPOSE.md) diff --git a/docs/datahaven-fullnode.md b/docs/datahaven-fullnode.md new file mode 100644 index 00000000..e4185104 --- /dev/null +++ b/docs/datahaven-fullnode.md @@ -0,0 +1,439 @@ +# DataHaven Full Node Setup + +## Overview + +Full nodes synchronize with the DataHaven network and provide RPC endpoints for applications without participating in consensus or block production. + +## Purpose + +- Synchronize and maintain full blockchain state +- Provide RPC/WebSocket endpoints for applications +- Relay transactions to the network +- Query historical blockchain data +- No participation in consensus or validation + +## Prerequisites + +- DataHaven node binary or Docker image +- Sufficient storage for chain data (200+ GB recommended) +- Stable network connection +- Open network ports (30333, optionally 9944) + +## Key Requirements + +### No Session Keys Required + +Full nodes do **not** require session keys since they don't participate in consensus. + +### Node Key (Optional) + +A node key is optional but recommended for persistent peer identity: + +```bash +# Generate node key +datahaven-node key generate-node-key > /data/fullnode/node-key.txt +``` + +## Wallet Requirements + +### No Wallet Required + +Full nodes do not submit transactions or participate in consensus, so no funded account is needed. + +## CLI Flags + +### Required Flags + +```bash +datahaven-node \ + --chain \ + --name +``` + +### Important Full Node Flags + +| Flag | Description | Default | +|------|-------------|---------| +| `--chain ` | Chain specification (stagenet-local, testnet-local, mainnet-local) | Required | +| `--name ` | Human-readable node name | Required | +| `--base-path ` | Base directory for chain data | `~/.local/share/datahaven-node` | +| `--port ` | P2P network port | `30333` | +| `--rpc-port ` | WebSocket RPC port | `9944` | +| `--rpc-external` | Listen on all network interfaces | Localhost only | +| `--rpc-cors ` | CORS origins for RPC | `localhost` | +| `--rpc-methods ` | RPC methods allowed (`safe`, `unsafe`, `auto`) | `auto` | +| `--bootnodes ` | Bootstrap nodes for peer discovery | None | + +### Pruning and Storage Flags + +| Flag | Description | Default | +|------|-------------|---------| +| `--pruning ` | State pruning mode (`archive`, ``) | `256` blocks | +| `--blocks-pruning ` | Block pruning mode (`archive`, `archive-canonical`, ``) | `archive-canonical` | +| `--state-cache-size ` | State cache size in MB | `67108864` (64 GB) | + +### Network Flags + +| Flag | Description | +|------|-------------| +| `--public-addr ` | Public address to advertise | +| `--listen-addr ` | Listen address for P2P | +| `--reserved-nodes ` | Reserved peer addresses | +| `--reserved-only` | Only connect to reserved nodes | +| `--no-private-ip` | Disable private IP discovery | + +### Optional Flags + +| Flag | Description | +|------|-------------| +| `--prometheus-external` | Expose Prometheus metrics externally | +| `--prometheus-port ` | Prometheus metrics port (default: 9615) | +| `--telemetry-url ` | Telemetry endpoint | +| `--log ` | Logging verbosity (e.g., `info,libp2p=debug`) | +| `--max-runtime-instances ` | Max WASM runtime instances | +| `--execution ` | Execution strategy (`native`, `wasm`, `both`) | + +## Complete Setup Examples + +### 1. Basic Full Node + +```bash +datahaven-node \ + --chain stagenet-local \ + --name "FullNode-01" \ + --base-path /data/fullnode \ + --port 30333 \ + --rpc-port 9944 \ + --rpc-external \ + --rpc-cors all \ + --bootnodes /dns/bootnode.example.com/tcp/30333/p2p/12D3KooW... +``` + +### 2. Archive Node + +```bash +datahaven-node \ + --chain stagenet-local \ + --name "ArchiveNode-01" \ + --base-path /data/archive \ + --pruning archive \ + --blocks-pruning archive \ + --port 30333 \ + --rpc-port 9944 \ + --rpc-external \ + --rpc-cors all \ + --rpc-methods safe +``` + +### 3. RPC Node with High Performance + +```bash +datahaven-node \ + --chain stagenet-local \ + --name "RPC-Node-01" \ + --base-path /data/rpc \ + --port 30333 \ + --rpc-port 9944 \ + --rpc-external \ + --rpc-cors all \ + --rpc-methods safe \ + --state-cache-size 134217728 \ + --max-runtime-instances 8 \ + --execution wasm +``` + +## Docker Deployment + +### Docker Compose + +```yaml +version: '3.8' + +services: + fullnode: + image: datahavenxyz/datahaven:latest + container_name: datahaven-fullnode + ports: + - "30333:30333" + - "9944:9944" + - "9615:9615" # Prometheus metrics + volumes: + - fullnode-data:/data + command: + - "--chain=stagenet-local" + - "--name=FullNode-01" + - "--base-path=/data" + - "--port=30333" + - "--rpc-port=9944" + - "--rpc-external" + - "--rpc-cors=all" + - "--rpc-methods=safe" + - "--prometheus-external" + - "--prometheus-port=9615" + - "--bootnodes=/dns/bootnode/tcp/30333/p2p/12D3KooW..." + restart: unless-stopped + +volumes: + fullnode-data: +``` + +### Docker Run + +```bash +docker run -d \ + --name datahaven-fullnode \ + -p 30333:30333 \ + -p 9944:9944 \ + -v $(pwd)/fullnode-data:/data \ + datahavenxyz/datahaven:latest \ + --chain stagenet-local \ + --name "FullNode-01" \ + --base-path /data \ + --port 30333 \ + --rpc-port 9944 \ + --rpc-external \ + --rpc-cors all \ + --rpc-methods safe +``` + +## Kubernetes Deployment + +```yaml +apiVersion: v1 +kind: Service +metadata: + name: datahaven-fullnode +spec: + type: LoadBalancer + ports: + - port: 30333 + targetPort: 30333 + name: p2p + - port: 9944 + targetPort: 9944 + name: rpc + - port: 9615 + targetPort: 9615 + name: metrics + selector: + app: datahaven-fullnode + +--- +apiVersion: apps/v1 +kind: StatefulSet +metadata: + name: datahaven-fullnode +spec: + serviceName: datahaven-fullnode + replicas: 1 + selector: + matchLabels: + app: datahaven-fullnode + template: + metadata: + labels: + app: datahaven-fullnode + spec: + containers: + - name: fullnode + image: datahavenxyz/datahaven:latest + ports: + - containerPort: 30333 + name: p2p + - containerPort: 9944 + name: rpc + - containerPort: 9615 + name: metrics + volumeMounts: + - name: data + mountPath: /data + resources: + requests: + memory: "8Gi" + cpu: "2" + limits: + memory: "16Gi" + cpu: "4" + args: + - "--chain=stagenet-local" + - "--name=FullNode-01" + - "--base-path=/data" + - "--port=30333" + - "--rpc-port=9944" + - "--rpc-external" + - "--rpc-cors=all" + - "--rpc-methods=safe" + - "--prometheus-external" + volumeClaimTemplates: + - metadata: + name: data + spec: + accessModes: [ "ReadWriteOnce" ] + resources: + requests: + storage: 200Gi +``` + +## On-Chain Registration + +### Not Required + +Full nodes do not require any on-chain registration or extrinsics. + +## Monitoring + +### Health Checks + +```bash +# Check node health +curl -s -H "Content-Type: application/json" \ + -d '{"id":1, "jsonrpc":"2.0", "method": "system_health"}' \ + http://localhost:9944 | jq + +# Check sync status +curl -s -H "Content-Type: application/json" \ + -d '{"id":1, "jsonrpc":"2.0", "method": "system_syncState"}' \ + http://localhost:9944 | jq + +# Check peer count +curl -s -H "Content-Type: application/json" \ + -d '{"id":1, "jsonrpc":"2.0", "method": "system_peers"}' \ + http://localhost:9944 | jq +``` + +### Prometheus Metrics + +Access at `http://localhost:9615/metrics` when `--prometheus-external` is enabled. + +Key metrics: +- `substrate_block_height` - Current block height +- `substrate_finalized_height` - Finalized block height +- `substrate_peers_count` - Connected peer count +- `substrate_ready_transactions_number` - Pending transactions +- `substrate_sync_blocks_total` - Total blocks synced + +### Log Monitoring + +```bash +# View logs with Docker +docker logs -f datahaven-fullnode + +# Filter for errors +docker logs datahaven-fullnode 2>&1 | grep -i error + +# Check sync progress +docker logs datahaven-fullnode 2>&1 | grep -i "Imported\|Syncing" +``` + +## RPC Usage Examples + +### Query Chain Data + +```bash +# Get latest block +curl -s -H "Content-Type: application/json" \ + -d '{"id":1, "jsonrpc":"2.0", "method": "chain_getBlock"}' \ + http://localhost:9944 | jq + +# Get account balance +curl -s -H "Content-Type: application/json" \ + -d '{"id":1, "jsonrpc":"2.0", "method": "system_accountNextIndex", "params":["0x..."]}' \ + http://localhost:9944 | jq +``` + +### Submit Transactions + +```bash +# Submit extrinsic +curl -s -H "Content-Type: application/json" \ + -d '{"id":1, "jsonrpc":"2.0", "method": "author_submitExtrinsic", "params":["0x..."]}' \ + http://localhost:9944 | jq +``` + +## Troubleshooting + +### Issue: Slow Sync Speed + +**Solutions:** +1. Increase `--max-runtime-instances` to 8-16 +2. Increase `--state-cache-size` (requires more RAM) +3. Use faster storage (NVMe SSD) +4. Add more `--bootnodes` for better peer discovery + +### Issue: High Memory Usage + +**Solutions:** +1. Reduce `--state-cache-size` +2. Enable pruning (remove `--pruning archive`) +3. Reduce `--max-runtime-instances` + +### Issue: RPC Connection Refused + +**Check:** +1. `--rpc-external` flag is set +2. Port 9944 is open in firewall +3. `--rpc-cors` includes your origin +4. Node is fully started (check logs) + +### Issue: No Peers Connecting + +**Solutions:** +1. Verify bootnode addresses are correct +2. Check port 30333 is open +3. Use `--listen-addr /ip4/0.0.0.0/tcp/30333` +4. Check firewall rules + +## Performance Tuning + +### For RPC Workloads + +```bash +datahaven-node \ + --rpc-methods safe \ + --rpc-max-connections 1000 \ + --state-cache-size 134217728 \ + --max-runtime-instances 16 \ + --execution wasm +``` + +### For Archive Node + +```bash +datahaven-node \ + --pruning archive \ + --blocks-pruning archive \ + --state-cache-size 268435456 +``` + +### Resource Requirements + +| Node Type | CPU | RAM | Storage | Network | +|-----------|-----|-----|---------|---------| +| Full Node (Pruned) | 2-4 cores | 8-16 GB | 100-200 GB | 100 Mbps | +| Archive Node | 4-8 cores | 16-32 GB | 500+ GB | 100 Mbps | +| RPC Node (High Traffic) | 8-16 cores | 32-64 GB | 200-500 GB | 1 Gbps | + +## Security Considerations + +1. **RPC Security**: Use `--rpc-methods safe` for public endpoints +2. **CORS**: Restrict `--rpc-cors` to specific domains in production +3. **Rate Limiting**: Implement reverse proxy with rate limiting +4. **Firewall**: Restrict RPC access to known IPs +5. **Monitoring**: Set up alerts for unusual activity + +## Best Practices + +1. Use dedicated server for production RPC nodes +2. Enable Prometheus metrics for monitoring +3. Regular backups of chain data +4. Use load balancer for multiple RPC nodes +5. Keep node software updated +6. Monitor disk space usage +7. Implement log rotation + +## Related Documentation + +- [Bootnode Setup](./datahaven-bootnode.md) +- [Validator Setup](./datahaven-validator.md) +- [Docker Compose Guide](../operator/DOCKER-COMPOSE.md) +- [Kubernetes Deployment](../deploy/charts/node/README.md) diff --git a/docs/datahaven-validator.md b/docs/datahaven-validator.md new file mode 100644 index 00000000..932eade2 --- /dev/null +++ b/docs/datahaven-validator.md @@ -0,0 +1,451 @@ +# DataHaven Validator Node Setup + +## Overview + +Validator nodes participate in consensus, produce blocks, and secure the DataHaven network through EigenLayer AVS integration. + +## Purpose + +- Participate in BABE block production +- Sign GRANDPA finality votes +- Submit ImOnline heartbeats +- Participate in BEEFY bridge consensus +- Earn rewards for block production + +## Prerequisites + +- DataHaven node binary or Docker image +- Funded account with staking balance +- Persistent storage for chain data +- Stable network connection +- Open network ports (30333, optionally 9944) + +## Key Requirements + +### Session Keys (4 Required) + +Validators require **four session keys** for different consensus mechanisms: + +| Key Type | Scheme | Purpose | +|----------|--------|---------| +| `gran` | ed25519 | GRANDPA finality gadget | +| `babe` | sr25519 | BABE block authoring | +| `imon` | sr25519 | ImOnline validator heartbeat | +| `beef` | ecdsa | BEEFY bridge consensus | + +### Generate Session Keys + +#### Method 1: Using RPC (Recommended) + +```bash +# Start node first, then generate keys via RPC +curl -H "Content-Type: application/json" \ + -d '{"id":1, "jsonrpc":"2.0", "method": "author_rotateKeys"}' \ + http://localhost:9944 + +# Returns: "0x" +``` + +#### Method 2: CLI Key Insertion + +```bash +# Generate seed phrase first +SEED=$(datahaven-node key generate | grep "Secret phrase" | cut -d'`' -f2) + +# Insert GRANDPA key (ed25519) +datahaven-node key insert \ + --base-path /data/validator \ + --chain stagenet-local \ + --key-type gran \ + --scheme ed25519 \ + --suri "$SEED" + +# Insert BABE key (sr25519) +datahaven-node key insert \ + --base-path /data/validator \ + --chain stagenet-local \ + --key-type babe \ + --scheme sr25519 \ + --suri "$SEED" + +# Insert ImOnline key (sr25519) +datahaven-node key insert \ + --base-path /data/validator \ + --chain stagenet-local \ + --key-type imon \ + --scheme sr25519 \ + --suri "$SEED" + +# Insert BEEFY key (ecdsa) +datahaven-node key insert \ + --base-path /data/validator \ + --chain stagenet-local \ + --key-type beef \ + --scheme ecdsa \ + --suri "$SEED" +``` + +#### Method 3: Docker Entrypoint (Automated) + +Set environment variables and let the Docker entrypoint inject keys: + +```bash +export NODE_TYPE=validator +export NODE_NAME=Alice +export SEED="your seed phrase here" +export CHAIN=stagenet-local +``` + +The entrypoint script (`operator/scripts/docker-entrypoint.sh`) automatically injects all 4 keys. + +## Wallet Requirements + +### Controller Account + +- **Purpose**: Controls validator operations +- **Required Balance**: Minimum staking amount + transaction fees +- **Funding**: Must be funded before validator registration + +### Generate Controller Account + +```bash +# Generate new account +datahaven-node key generate + +# Output: +# Secret phrase: +# Network ID: substrate +# Secret seed: 0x... +# Public key (hex): 0x... +# Account ID: 0x... (20-byte Ethereum-style address) +# SS58 Address: ... +``` + +## CLI Flags + +### Required Flags + +```bash +datahaven-node \ + --chain \ + --validator \ + --name +``` + +### Important Validator Flags + +| Flag | Description | Default | +|------|-------------|---------| +| `--chain ` | Chain specification | Required | +| `--validator` | Run as validator | Required | +| `--name ` | Node name | Required | +| `--base-path ` | Base directory for data | `~/.local/share/datahaven-node` | +| `--port ` | P2P port | `30333` | +| `--rpc-port ` | WebSocket RPC port | `9944` | +| `--bootnodes ` | Bootstrap nodes | None | + +### Optional Flags + +| Flag | Description | +|------|-------------| +| `--rpc-external` | Listen on all interfaces | +| `--rpc-cors ` | CORS origins (e.g., `all` or `http://localhost:3000`) | +| `--prometheus-external` | Expose Prometheus metrics externally | +| `--telemetry-url ` | Telemetry endpoint | +| `--log ` | Logging verbosity (e.g., `info,runtime=debug`) | +| `--unsafe-force-node-key-generation` | Generate node key (dev only) | + +## Complete Setup Example + +### 1. Generate Keys + +```bash +# Generate controller account +SEED="your secure seed phrase here" +echo $SEED | datahaven-node key inspect + +# Start node to generate session keys via RPC +datahaven-node \ + --chain stagenet-local \ + --base-path /tmp/validator \ + --validator \ + --name "TempValidator" \ + --rpc-port 9944 & + +# Wait for node to start +sleep 10 + +# Generate session keys +SESSION_KEYS=$(curl -s -H "Content-Type: application/json" \ + -d '{"id":1, "jsonrpc":"2.0", "method": "author_rotateKeys"}' \ + http://localhost:9944 | jq -r '.result') + +echo "Session Keys: $SESSION_KEYS" + +# Stop temporary node +pkill -f datahaven-node +``` + +### 2. Fund Controller Account + +```bash +# Get account address from seed +CONTROLLER_ADDR=$(echo $SEED | datahaven-node key inspect --output-type json | jq -r '.ss58PublicKey') + +# Fund account via faucet or transfer from another account +# Minimum: Staking amount + fees (e.g., 1000 HAVE + 10 HAVE fees) +``` + +### 3. Start Validator Node + +```bash +datahaven-node \ + --chain stagenet-local \ + --validator \ + --name "Validator-01" \ + --base-path /data/validator \ + --port 30333 \ + --rpc-port 9944 \ + --bootnodes /dns/bootnode.example.com/tcp/30333/p2p/12D3KooW... \ + --telemetry-url "wss://telemetry.polkadot.io/submit/ 0" \ + --log info +``` + +### 4. Register Validator On-Chain + +See [On-Chain Registration](#on-chain-registration) section below. + +## Docker Deployment + +### Docker Compose + +```yaml +version: '3.8' + +services: + validator: + image: datahavenxyz/datahaven:latest + container_name: datahaven-validator + environment: + NODE_TYPE: validator + NODE_NAME: Alice + SEED: "your seed phrase here" + CHAIN: stagenet-local + KEYSTORE_PATH: /data/keystore + ports: + - "30333:30333" + - "9944:9944" + volumes: + - validator-data:/data + command: + - "--chain=stagenet-local" + - "--validator" + - "--name=Validator-01" + - "--base-path=/data" + - "--keystore-path=/data/keystore" + - "--port=30333" + - "--rpc-port=9944" + - "--rpc-external" + - "--rpc-cors=all" + +volumes: + validator-data: +``` + +## Kubernetes Deployment + +See `deploy/charts/node/values.yaml` for full Helm configuration. + +```yaml +apiVersion: apps/v1 +kind: StatefulSet +metadata: + name: datahaven-validator +spec: + serviceName: datahaven-validator + replicas: 1 + selector: + matchLabels: + app: datahaven-validator + template: + metadata: + labels: + app: datahaven-validator + spec: + containers: + - name: validator + image: datahavenxyz/datahaven:latest + env: + - name: NODE_TYPE + value: "validator" + - name: NODE_NAME + value: "Alice" + - name: SEED + valueFrom: + secretKeyRef: + name: validator-seed + key: seed + - name: CHAIN + value: "stagenet-local" + ports: + - containerPort: 30333 + name: p2p + - containerPort: 9944 + name: rpc + volumeMounts: + - name: data + mountPath: /data + args: + - "--chain=stagenet-local" + - "--validator" + - "--name=Validator-01" + - "--base-path=/data" + - "--port=30333" + - "--rpc-port=9944" + volumeClaimTemplates: + - metadata: + name: data + spec: + accessModes: [ "ReadWriteOnce" ] + resources: + requests: + storage: 200Gi +``` + +## On-Chain Registration + +### Step 1: Set Session Keys + +Using Polkadot.js Apps or TypeScript: + +```typescript +import { ApiPromise, WsProvider } from '@polkadot/api'; +import { Keyring } from '@polkadot/keyring'; + +const api = await ApiPromise.create({ + provider: new WsProvider('ws://localhost:9944') +}); + +const keyring = new Keyring({ type: 'ethereum' }); +const controller = keyring.addFromUri('your seed phrase'); + +// Set session keys (from author_rotateKeys RPC) +const sessionKeys = '0x...'; // Combined public keys hex + +const setKeysTx = api.tx.session.setKeys(sessionKeys, []); +await setKeysTx.signAndSend(controller); +``` + +### Step 2: Register with EigenLayer AVS + +Validators must register with the DataHaven AVS smart contract on Ethereum. + +```solidity +// DataHavenServiceManager.sol +function registerOperatorToAVS( + address operator, + ISignatureUtils.SignatureWithSaltAndExpiry memory operatorSignature +) external; +``` + +See `contracts/` directory and `test/scripts/` for registration scripts. + +### Step 3: Verify Registration + +```bash +# Check session keys +curl -s -H "Content-Type: application/json" \ + -d '{"id":1, "jsonrpc":"2.0", "method": "author_hasSessionKeys", "params":["0x..."]}' \ + http://localhost:9944 + +# Check validator status +curl -s -H "Content-Type: application/json" \ + -d '{"id":1, "jsonrpc":"2.0", "method": "system_health"}' \ + http://localhost:9944 +``` + +## Monitoring + +### Key Metrics + +- Block production rate +- Finality lag +- Peer count +- Session key validity +- ImOnline heartbeats + +### Prometheus Metrics + +```bash +# Enable Prometheus endpoint +datahaven-node --validator --prometheus-external --prometheus-port 9615 + +# Access metrics +curl http://localhost:9615/metrics +``` + +### Health Checks + +```bash +# System health +curl -s -H "Content-Type: application/json" \ + -d '{"id":1, "jsonrpc":"2.0", "method": "system_health"}' \ + http://localhost:9944 | jq + +# Chain info +curl -s -H "Content-Type: application/json" \ + -d '{"id":1, "jsonrpc":"2.0", "method": "system_chain"}' \ + http://localhost:9944 | jq +``` + +## Troubleshooting + +### Issue: Not Producing Blocks + +**Check:** +1. Session keys are set on-chain +2. Account is in the validator set +3. Node is fully synced +4. Session keys match on-chain registration + +### Issue: Session Keys Lost + +**Solution:** +```bash +# Rotate keys and re-register +curl -H "Content-Type: application/json" \ + -d '{"id":1, "jsonrpc":"2.0", "method": "author_rotateKeys"}' \ + http://localhost:9944 + +# Then submit new keys via session.setKeys extrinsic +``` + +### Issue: Not in Active Validator Set + +**Check:** +1. Sufficient stake amount +2. Not slashed +3. Session transition period +4. Maximum validator count not exceeded + +## Security Considerations + +1. **Key Management**: Store seed phrase securely offline +2. **Network Security**: Use firewall to restrict RPC access +3. **High Availability**: Implement monitoring and alerting +4. **Slashing Prevention**: Monitor validator performance +5. **Backup Strategy**: Regular backups of keystores + +## Best Practices + +1. Monitor network connectivity +2. Keep node software updated +3. Test key rotation procedures +4. Document incident response procedures + +## Related Documentation + +- [Bootnode Setup](./datahaven-bootnode.md) +- [Full Node Setup](./datahaven-fullnode.md) +- [EigenLayer AVS Integration](../contracts/README.md) +- [Rewards System](../operator/pallets/external-validators/README.md) diff --git a/docs/storagehub-bsp.md b/docs/storagehub-bsp.md new file mode 100644 index 00000000..76de01fa --- /dev/null +++ b/docs/storagehub-bsp.md @@ -0,0 +1,636 @@ +# StorageHub Backup Storage Provider (BSP) Setup + +## Overview + +Backup Storage Providers (BSPs) provide redundant storage for files in the StorageHub network, receiving files from Main Storage Providers (MSPs) and submitting proofs of storage. + +## Purpose + +- Store backup copies of files +- Submit proofs of storage periodically +- Charge fees from users for backup storage +- Handle bucket migrations +- Serve file download requests as backup +- Ensure data redundancy and availability + +## Prerequisites + +- DataHaven node binary or Docker image +- Funded account with sufficient balance for deposits +- Storage capacity (minimum 2 data units, recommended 10+ GiB) +- Stable network connection +- Open network ports (30333, optionally 9944) + +## Key Requirements + +### BCSV Key (ECDSA - 1 Required) + +BSPs require **one BCSV key** for storage provider identity. + +| Key Type | Scheme | Purpose | +|----------|--------|---------| +| `bcsv` | ecdsa | Storage provider identity and signing | + +### Generate BCSV Key + +#### Method 1: CLI Key Insertion + +```bash +# Generate seed phrase +SEED=$(datahaven-node key generate | grep "Secret phrase" | cut -d'`' -f2) + +# Insert BCSV key (ecdsa) +datahaven-node key insert \ + --base-path /data/bsp \ + --chain stagenet-local \ + --key-type bcsv \ + --scheme ecdsa \ + --suri "$SEED" +``` + +#### Method 2: Docker Entrypoint (Automated) + +Set environment variables: + +```bash +export NODE_TYPE=bsp +export NODE_NAME=bsp01 +export SEED="your seed phrase here" +export CHAIN=stagenet-local +``` + +The entrypoint script automatically injects the BCSV key. + +## Wallet Requirements + +### Provider Account + +- **Purpose**: BSP registration, transaction fees, and deposits +- **Required Balance**: + - Minimum deposit: 100 HAVE (SpMinDeposit) + - Deposit per data unit: 2 HAVE per unit + - Transaction fees: ~10 HAVE + - **Recommended**: 200+ HAVE for initial setup +- **Funding**: Must be funded **before** BSP registration +- **Account Type**: Ethereum-style 20-byte address (AccountId20) + +### Generate Provider Account + +```bash +# Generate new account from seed +SEED="your secure seed phrase here" +echo $SEED | datahaven-node key inspect --output-type json | jq + +# Derive BSP account +echo "$SEED" | datahaven-node key inspect --output-type json | jq -r '.ss58PublicKey' +``` + +## CLI Flags + +### Required Flags + +```bash +datahaven-node \ + --chain \ + --provider \ + --provider-type bsp \ + --max-storage-capacity \ + --jump-capacity +``` + +### Core Provider Flags + +| Flag | Description | Required | Default | +|------|-------------|----------|---------| +| `--provider` | Enable storage provider mode | Yes | false | +| `--provider-type bsp` | Set provider type to BSP | Yes | None | +| `--max-storage-capacity ` | Maximum storage capacity | Yes | None | +| `--jump-capacity ` | Jump capacity for new storage | Yes | None | +| `--storage-layer ` | Storage backend (`rocksdb` or `memory`) | No | `memory` | +| `--storage-path ` | Storage path (required if rocksdb) | No | None | + +**Example Values:** +- `--max-storage-capacity 10737418240` (10 GiB) +- `--jump-capacity 1073741824` (1 GiB) + +### BSP-Specific Task Flags + +| Flag | Description | Default | +|------|-------------|---------| +| `--bsp-upload-file-task` | Enable file upload from MSP task | false | +| `--bsp-upload-file-max-try-count ` | Max retries for file uploads | 5 | +| `--bsp-upload-file-max-tip ` | Max tip for upload file extrinsics | 0 | +| `--bsp-move-bucket-task` | Enable bucket migration task | false | +| `--bsp-move-bucket-grace-period ` | Grace period after bucket move | 300 | +| `--bsp-charge-fees-task` | Enable automatic fee charging | false | +| `--bsp-charge-fees-min-debt ` | Minimum debt threshold to charge | 0 | +| `--bsp-submit-proof-task` | Enable proof submission task | false | +| `--bsp-submit-proof-max-attempts ` | Max attempts to submit proof | 3 | + +### Remote File Handling Flags + +| Flag | Description | Default | +|------|-------------|---------| +| `--max-file-size ` | Maximum file size | 10737418240 (10 GB) | +| `--connection-timeout ` | Connection timeout | 30 | +| `--read-timeout ` | Read timeout | 300 | +| `--follow-redirects ` | Follow HTTP redirects | true | +| `--max-redirects ` | Maximum redirects | 10 | +| `--user-agent ` | HTTP user agent | "StorageHub-Client/1.0" | +| `--chunk-size ` | Upload/download chunk size | 8192 (8 KB) | +| `--chunks-buffer ` | Number of chunks to buffer | 512 | + +### Operational Flags + +| Flag | Description | Default | +|------|-------------|---------| +| `--extrinsic-retry-timeout ` | Extrinsic retry timeout | 60 | +| `--sync-mode-min-blocks-behind ` | Min blocks behind for sync mode | 5 | +| `--check-for-pending-proofs-period ` | Period to check pending proofs | 4 | +| `--max-blocks-behind-to-catch-up-root-changes ` | Max blocks to process for root changes | 10 | + +## Complete Setup Example + +### 1. Generate Keys and Account + +```bash +# Generate seed phrase +SEED="your secure seed phrase here" + +# Derive BSP account +BSP_ACCOUNT=$(echo "$SEED" | datahaven-node key inspect --output-type json | jq -r '.ss58PublicKey') +echo "BSP Account: $BSP_ACCOUNT" + +# Insert BCSV key +datahaven-node key insert \ + --base-path /data/bsp \ + --chain stagenet-local \ + --key-type bcsv \ + --scheme ecdsa \ + --suri "$SEED" +``` + +### 2. Fund Provider Account + +```bash +# Transfer funds to BSP account +# Minimum: 200 HAVE (100 deposit + 100 for operations) + +# Using Polkadot.js or a funded account, send HAVE tokens to $BSP_ACCOUNT +``` + +### 3. Start BSP Node + +```bash +datahaven-node \ + --chain stagenet-local \ + --name "BSP01" \ + --base-path /data/bsp \ + --provider \ + --provider-type bsp \ + --max-storage-capacity 10737418240 \ + --jump-capacity 1073741824 \ + --storage-layer rocksdb \ + --storage-path /data/bsp/storage \ + --bsp-upload-file-task \ + --bsp-move-bucket-task \ + --bsp-charge-fees-task \ + --bsp-submit-proof-task \ + --port 30333 \ + --rpc-port 9946 \ + --bootnodes /dns/bootnode.example.com/tcp/30333/p2p/12D3KooW... +``` + +### 4. Register BSP On-Chain + +See [On-Chain Registration](#on-chain-registration) section below. + +## Docker Deployment + +### Docker Compose + +```yaml +version: '3.8' + +services: + bsp: + image: datahavenxyz/datahaven:latest + container_name: storagehub-bsp + environment: + NODE_TYPE: bsp + NODE_NAME: bsp01 + SEED: "your seed phrase here" + CHAIN: stagenet-local + KEYSTORE_PATH: /data/keystore + ports: + - "30334:30333" + - "9946:9946" + volumes: + - bsp-data:/data + - bsp-storage:/data/storage + command: + - "--chain=stagenet-local" + - "--name=BSP01" + - "--base-path=/data" + - "--keystore-path=/data/keystore" + - "--provider" + - "--provider-type=bsp" + - "--max-storage-capacity=10737418240" + - "--jump-capacity=1073741824" + - "--storage-layer=rocksdb" + - "--storage-path=/data/storage" + - "--bsp-upload-file-task" + - "--bsp-move-bucket-task" + - "--bsp-charge-fees-task" + - "--bsp-submit-proof-task" + - "--port=30333" + - "--rpc-port=9946" + restart: unless-stopped + +volumes: + bsp-data: + bsp-storage: +``` + +## Kubernetes Deployment + +```yaml +apiVersion: apps/v1 +kind: StatefulSet +metadata: + name: storagehub-bsp +spec: + serviceName: storagehub-bsp + replicas: 1 + selector: + matchLabels: + app: storagehub-bsp + template: + metadata: + labels: + app: storagehub-bsp + spec: + containers: + - name: bsp + image: datahavenxyz/datahaven:latest + env: + - name: NODE_TYPE + value: "bsp" + - name: NODE_NAME + value: "bsp01" + - name: SEED + valueFrom: + secretKeyRef: + name: bsp-seed + key: seed + ports: + - containerPort: 30333 + name: p2p + - containerPort: 9946 + name: rpc + volumeMounts: + - name: data + mountPath: /data + - name: storage + mountPath: /data/storage + resources: + requests: + memory: "4Gi" + cpu: "2" + limits: + memory: "8Gi" + cpu: "4" + args: + - "--chain=stagenet-local" + - "--provider" + - "--provider-type=bsp" + - "--max-storage-capacity=10737418240" + - "--jump-capacity=1073741824" + - "--storage-layer=rocksdb" + - "--storage-path=/data/storage" + - "--bsp-upload-file-task" + - "--bsp-move-bucket-task" + - "--bsp-charge-fees-task" + - "--bsp-submit-proof-task" + volumeClaimTemplates: + - metadata: + name: data + spec: + accessModes: [ "ReadWriteOnce" ] + resources: + requests: + storage: 100Gi + - metadata: + name: storage + spec: + accessModes: [ "ReadWriteOnce" ] + resources: + requests: + storage: 500Gi +``` + +## On-Chain Registration + +### BSP Registration Process + +BSPs must be registered on-chain via the `Providers` pallet using a **2-step process**: + +1. **Step 1**: Call `request_bsp_sign_up` - Initiates registration and reserves deposit +2. **Step 2**: Call `confirm_sign_up` - Completes registration after randomness verification + +This two-step mechanism ensures security and prevents manipulation of provider IDs through randomness. + +### Step 1: Request BSP Sign Up + +```typescript +import { createClient } from 'polkadot-api'; +import { getWsProvider } from 'polkadot-api/ws-provider/web'; +import { withPolkadotSdkCompat } from 'polkadot-api/polkadot-sdk-compat'; +import { datahaven } from '@polkadot-api/descriptors'; +import { Binary } from 'polkadot-api'; + +// Connect to DataHaven node +const client = createClient( + withPolkadotSdkCompat(getWsProvider('ws://localhost:9944')) +); +const typedApi = client.getTypedApi(datahaven); + +// BSP signer (using your BCSV key account) +const bspSigner = /* your polkadot-api signer */; + +// BSP configuration +const capacity = BigInt(10_737_418_240); // 10 GiB in bytes +const multiaddresses = [ + '/ip4/127.0.0.1/tcp/30333', + '/dns/bsp01.example.com/tcp/30333' +].map(addr => Binary.fromText(addr)); + +// Step 1: Request BSP sign up +const requestTx = typedApi.tx.Providers.request_bsp_sign_up({ + capacity: capacity, + multiaddresses: multiaddresses, + payment_account: bspSigner.publicKey // Account receiving payments +}); + +// Sign and submit the request +const requestResult = await requestTx.signAndSubmit(bspSigner); +console.log('BSP sign-up requested. Waiting for finalization...'); + +await requestResult.finalized(); +console.log('Request finalized! Deposit has been reserved.'); +``` + +**What Happens in Step 1:** +- Validates multiaddresses format +- Calculates required deposit based on capacity (`SpMinDeposit + capacity * DepositPerData`) +- Verifies account has sufficient balance +- **Holds (reserves) the deposit** from your account +- Creates a pending sign-up request +- Emits `BspRequestSignUpSuccess` event + +### Step 2: Confirm Sign Up + +After requesting, you must wait for sufficient randomness to be available (controlled by `MaxBlocksForRandomness` parameter, typically 2 hours on mainnet). + +```typescript +// Step 2: Confirm the sign-up (after waiting for randomness) +const confirmTx = typedApi.tx.Providers.confirm_sign_up({ + provider_account: undefined // Optional: omit to use signer's account +}); + +// Sign and submit confirmation +const confirmResult = await confirmTx.signAndSubmit(bspSigner); +console.log('Confirming BSP registration...'); + +await confirmResult.finalized(); +console.log('BSP registration confirmed and active!'); +``` + +**What Happens in Step 2:** +- Verifies randomness is sufficiently fresh +- Checks request hasn't expired +- Generates Provider ID using randomness +- Registers BSP in the system +- Applies sign-up lock period (90 days on testnet/mainnet via `BspSignUpLockPeriod`) +- Emits `BspSignUpSuccess` event +- Deposit remains held for duration of BSP operation + +### Timing Requirements + +| Parameter | Testnet | Mainnet | Description | +|-----------|---------|---------|-------------| +| Min wait time | ~2 minutes | ~2 hours | Wait after `request_bsp_sign_up` for randomness | +| Max wait time | Set by `MaxBlocksForRandomness` | Typically 2 hours | Request expires if not confirmed in time | +| Sign-up lock | 90 days | 90 days | Period before BSP can sign off after registration | + +### Verify Registration + +```typescript +// Check BSP registration status +const bspAccount = bspSigner.publicKey; + +const registeredBspId = await typedApi.query.Providers.AccountIdToBackupStorageProviderId.getValue( + bspAccount +); + +if (registeredBspId) { + console.log('Registered BSP ID:', registeredBspId); + + // Get full BSP details + const bspInfo = await typedApi.query.Providers.BackupStorageProviders.getValue( + registeredBspId + ); + console.log('BSP Info:', bspInfo); +} else { + console.log('BSP not yet registered or confirmation pending'); +} +``` + +### Cancel Pending Request + +If you change your mind before confirming: + +```typescript +const cancelTx = typedApi.tx.Providers.cancel_sign_up(); +await cancelTx.signAndSubmit(bspSigner); +console.log('Sign-up request cancelled, deposit returned'); +``` + +### Development/Testing: Force Sign Up (Requires Sudo) + +For development and testing environments with sudo access, you can bypass the 2-step process: + +```typescript +// Single-step registration for testing (requires sudo) +const sudoSigner = /* sudo account signer */; + +const bspCall = typedApi.tx.Providers.force_bsp_sign_up({ + who: bspAccount, + bsp_id: /* pre-generated provider ID */, + capacity: BigInt(10_737_418_240), + multiaddresses: multiaddresses, + payment_account: bspAccount, + weight: undefined // Optional weight parameter +}); + +const sudoTx = typedApi.tx.Sudo.sudo({ call: bspCall.decodedCall }); +await sudoTx.signAndSubmit(sudoSigner); +``` + +### Registration Parameters + +| Parameter | Type | Description | Example | +|-----------|------|-------------|---------| +| `capacity` | StorageDataUnit | Storage capacity in bytes | `10737418240` (10 GiB) | +| `multiaddresses` | Vec | P2P network addresses | `[Binary.fromText("/ip4/...")]` | +| `payment_account` | AccountId | Account receiving payments | `0x...` (20-byte) | + +### Deposit Requirements + +- **Base Deposit**: 100 HAVE (`SpMinDeposit`) +- **Per Data Unit**: 2 HAVE per unit (`DepositPerData`) +- **Total for 10 GiB**: ~100 HAVE + (10 GiB in units × 2 HAVE) + +The deposit is **held (reserved)** from your account when you call `request_bsp_sign_up` and remains held while you operate as a BSP. + +## Monitoring + +### Health Checks + +```bash +# Check node health +curl -s -H "Content-Type: application/json" \ + -d '{"id":1, "jsonrpc":"2.0", "method": "system_health"}' \ + http://localhost:9946 | jq + +# Check provider status +curl -s -H "Content-Type: application/json" \ + -d '{"id":1, "jsonrpc":"2.0", "method": "storageprovider_getStatus"}' \ + http://localhost:9946 | jq +``` + +### Key Metrics to Monitor + +- Storage capacity usage +- Number of stored files +- Proof submission success rate +- File upload success rate from MSPs +- Fee collection status +- Bucket migration status + +### Logs + +```bash +# View BSP logs +docker logs -f storagehub-bsp + +# Filter for storage events +docker logs storagehub-bsp 2>&1 | grep -i "storage\|proof\|file" + +# Monitor proof submissions +docker logs storagehub-bsp 2>&1 | grep -i "proof" +``` + +## Proof Submission + +### Automatic Proof Submission + +BSPs automatically submit proofs when `--bsp-submit-proof-task` is enabled. + +### Proof Submission Flow + +1. **Challenge Received**: BSP receives storage proof challenge from ProofsDealer pallet +2. **Proof Generation**: BSP generates Merkle proof for challenged data +3. **Proof Submission**: BSP submits proof via `proofsDealer.submitProof` extrinsic +4. **Verification**: ProofsDealer pallet verifies proof on-chain +5. **Reward/Penalty**: BSP receives reward for valid proof or penalty for invalid/missing proof + +### Monitor Proof Submission + +```bash +# Check pending proofs +curl -s -H "Content-Type: application/json" \ + -d '{"id":1, "jsonrpc":"2.0", "method": "storageprovider_getPendingProofs"}' \ + http://localhost:9946 | jq +``` + +## Troubleshooting + +### Issue: Registration Failed + +**Check:** +1. Account has sufficient balance (200+ HAVE) +2. BCSV key is correctly inserted +3. Capacity meets minimum (2 data units) +4. Provider ID is correctly calculated + +### Issue: Not Receiving Files from MSP + +**Check:** +1. BSP is registered on-chain +2. `--bsp-upload-file-task` flag is enabled +3. Storage capacity not exceeded +4. Node is fully synced +5. Network connectivity to MSPs + +### Issue: Proof Submission Failing + +**Check:** +1. `--bsp-submit-proof-task` flag is enabled +2. Node is fully synced +3. Sufficient balance for transaction fees +4. Files are correctly stored and accessible +5. Check logs for specific errors + +### Issue: Fee Charging Not Working + +**Check:** +1. `--bsp-charge-fees-task` flag is enabled +2. Users have sufficient debt to charge +3. Node is synced and connected + +## Security Considerations + +1. **Key Management**: Store seed phrase securely offline +2. **Storage Security**: Encrypt storage at rest +3. **Network Security**: Use firewall to restrict access +4. **Proof Integrity**: Ensure storage backend reliability +5. **Backup Strategy**: Regular backups of stored data + +## Best Practices + +1. Use production-grade storage (NVMe SSD recommended) +2. Monitor storage capacity proactively +3. Enable all BSP tasks for full functionality +4. Keep node software updated +5. Implement monitoring and alerting for proof submissions +6. Set reasonable `bsp-submit-proof-max-attempts` (3-5) +7. Document operational procedures +8. Monitor network connectivity to MSPs + +## Performance Considerations + +### Resource Requirements + +| Component | Minimum | Recommended | +|-----------|---------|-------------| +| CPU | 2 cores | 4 cores | +| RAM | 4 GB | 8 GB | +| Storage (Chain Data) | 100 GB | 200 GB | +| Storage (Files) | 10 GB | 500+ GB | +| Network | 100 Mbps | 1 Gbps | + +### Storage Backend Comparison + +| Backend | Pros | Cons | Use Case | +|---------|------|------|----------| +| `memory` | Fast, simple | Not persistent | Testing only | +| `rocksdb` | Persistent, production-ready | Slower than memory | Production | + +## Related Documentation + +- [MSP Setup](./storagehub-msp.md) +- [Indexer Setup](./storagehub-indexer.md) +- [Fisherman Setup](./storagehub-fisherman.md) +- [StorageHub Pallets](https://github.com/Moonsong-Labs/storage-hub) +- [Proofs Dealer Pallet](https://github.com/Moonsong-Labs/storage-hub/tree/main/pallets/proofs-dealer) diff --git a/docs/storagehub-fisherman.md b/docs/storagehub-fisherman.md new file mode 100644 index 00000000..777a4aa9 --- /dev/null +++ b/docs/storagehub-fisherman.md @@ -0,0 +1,591 @@ +# StorageHub Fisherman Node Setup + +## Overview + +Fisherman nodes monitor and validate storage provider behavior, detecting violations and submitting challenges to ensure network integrity. + +## Purpose + +- Monitor storage provider behavior and compliance +- Detect storage proof violations +- Validate provider availability +- Submit challenges for non-compliant providers +- Ensure data integrity and provider accountability +- Earn rewards for successful violation detection + +## Prerequisites + +- DataHaven node binary or Docker image +- Funded account with sufficient balance for challenges +- PostgreSQL 14+ database (can share with Indexer) +- Sufficient storage for database (100+ GB recommended) +- Stable network connection +- Open network ports (30333, optionally 9944) + +## Key Requirements + +### BCSV Key (ECDSA - 1 Required) + +Fishermen require **one BCSV key** for signing challenge submissions. + +| Key Type | Scheme | Purpose | +|----------|--------|---------| +| `bcsv` | ecdsa | Fisherman identity and challenge signing | + +### Generate BCSV Key + +#### Method 1: CLI Key Insertion + +```bash +# Generate seed phrase +SEED=$(datahaven-node key generate | grep "Secret phrase" | cut -d'`' -f2) + +# Insert BCSV key (ecdsa) +datahaven-node key insert \ + --base-path /data/fisherman \ + --chain stagenet-local \ + --key-type bcsv \ + --scheme ecdsa \ + --suri "$SEED//Gustavo" +``` + +#### Method 2: Docker Entrypoint (Automated) + +Set environment variables: + +```bash +export NODE_TYPE=fisherman +export NODE_NAME=Gustavo +export SEED="your seed phrase here" +export CHAIN=stagenet-local +``` + +The entrypoint script automatically injects the BCSV key. + +## Wallet Requirements + +### Fisherman Account + +- **Purpose**: Challenge submission and transaction fees +- **Required Balance**: + - Transaction fees: ~10 HAVE per challenge + - **Recommended**: 100+ HAVE for continuous operations +- **Funding**: Must be funded to submit challenges +- **Account Type**: Ethereum-style 20-byte address (AccountId20) + +### Generate Fisherman Account + +```bash +# Generate new account from seed +SEED="your secure seed phrase here" +echo $SEED | datahaven-node key inspect --output-type json | jq + +# Derive Fisherman account (common derivation: //Gustavo) +echo "$SEED//Gustavo" | datahaven-node key inspect --output-type json | jq -r '.ss58PublicKey' +``` + +## Database Requirements + +### PostgreSQL Setup + +Fisherman nodes **require** a PostgreSQL database, which can be shared with an Indexer node. + +#### Install PostgreSQL + +```bash +# Ubuntu/Debian +sudo apt update +sudo apt install postgresql-14 postgresql-contrib + +# macOS +brew install postgresql@14 + +# Docker +docker run -d \ + --name fisherman-postgres \ + -e POSTGRES_PASSWORD=indexer \ + -e POSTGRES_USER=indexer \ + -e POSTGRES_DB=datahaven \ + -p 5432:5432 \ + -v fisherman-db:/var/lib/postgresql/data \ + postgres:14 +``` + +#### Database Connection String + +``` +postgresql://indexer:indexer@localhost:5432/datahaven +``` + +## CLI Flags + +### Required Flags + +```bash +datahaven-node \ + --chain \ + --fisherman \ + --fisherman-database-url +``` + +### Core Fisherman Flags + +| Flag | Description | Required | Default | +|------|-------------|----------|---------| +| `--fisherman` | Enable fisherman service | Yes | false | +| `--fisherman-database-url ` | PostgreSQL connection URL | Yes* | None | +| `--fisherman-incomplete-sync-max ` | Max incomplete sync requests to process | No | 10000 | +| `--fisherman-incomplete-sync-page-size ` | Page size for pagination | No | 256 | +| `--fisherman-sync-mode-min-blocks-behind ` | Min blocks behind for sync mode | No | 5 | + +*Can also use `FISHERMAN_DATABASE_URL` environment variable + +### Standard Node Flags + +| Flag | Description | Default | +|------|-------------|---------| +| `--chain ` | Chain specification | Required | +| `--name ` | Node name | Required | +| `--base-path ` | Base directory for chain data | `~/.local/share/datahaven-node` | +| `--port ` | P2P port | `30333` | +| `--rpc-port ` | WebSocket RPC port | `9944` | +| `--bootnodes ` | Bootstrap nodes | None | + +### Optional Flags + +| Flag | Description | +|------|-------------| +| `--pruning ` | State pruning mode | +| `--prometheus-external` | Expose Prometheus metrics | +| `--log ` | Logging verbosity | + +## Important Constraints + +### Cannot Run with Lite Indexer + +**CRITICAL**: Fisherman nodes **cannot** be run alongside an Indexer node in `lite` mode. They require either: +- A separate full Indexer node +- An Indexer in `fishing` mode +- An Indexer in `full` mode + +### Cannot Run as Provider Simultaneously + +A node **cannot** run as both a fisherman and a storage provider (MSP/BSP) at the same time. + +## Complete Setup Examples + +### 1. Generate Keys and Account + +```bash +# Generate seed phrase +SEED="your secure seed phrase here" + +# Derive Fisherman account +FISHERMAN_ACCOUNT=$(echo "$SEED//Gustavo" | datahaven-node key inspect --output-type json | jq -r '.ss58PublicKey') +echo "Fisherman Account: $FISHERMAN_ACCOUNT" + +# Insert BCSV key +datahaven-node key insert \ + --base-path /data/fisherman \ + --chain stagenet-local \ + --key-type bcsv \ + --scheme ecdsa \ + --suri "$SEED//Gustavo" +``` + +### 2. Fund Fisherman Account + +```bash +# Transfer funds to Fisherman account +# Minimum: 100 HAVE for continuous operations + +# Using Polkadot.js or a funded account, send HAVE tokens to $FISHERMAN_ACCOUNT +``` + +### 3. Setup Database + +```bash +# Start PostgreSQL with Docker +docker run -d \ + --name fisherman-postgres \ + -e POSTGRES_PASSWORD=indexer \ + -e POSTGRES_USER=indexer \ + -e POSTGRES_DB=datahaven \ + -p 5432:5432 \ + -v fisherman-db:/var/lib/postgresql/data \ + postgres:14 + +# Verify connection +psql postgresql://indexer:indexer@localhost:5432/datahaven -c "SELECT version();" +``` + +### 4. Start Fisherman Node + +```bash +datahaven-node \ + --chain stagenet-local \ + --name "Fisherman-Gustavo" \ + --base-path /data/fisherman \ + --fisherman \ + --fisherman-database-url postgresql://indexer:indexer@localhost:5432/datahaven \ + --fisherman-incomplete-sync-max 10000 \ + --fisherman-incomplete-sync-page-size 256 \ + --fisherman-sync-mode-min-blocks-behind 5 \ + --port 30333 \ + --rpc-port 9948 \ + --bootnodes /dns/bootnode.example.com/tcp/30333/p2p/12D3KooW... +``` + +## Docker Deployment + +### Docker Compose (Full Stack) + +```yaml +version: '3.8' + +services: + postgres: + image: postgres:14 + container_name: fisherman-postgres + environment: + POSTGRES_DB: datahaven + POSTGRES_USER: indexer + POSTGRES_PASSWORD: indexer + ports: + - "5432:5432" + volumes: + - fisherman-db:/var/lib/postgresql/data + healthcheck: + test: ["CMD-SHELL", "pg_isready -U indexer -d datahaven"] + interval: 10s + timeout: 5s + retries: 5 + + indexer: + image: datahavenxyz/datahaven:latest + container_name: storagehub-indexer + depends_on: + postgres: + condition: service_healthy + environment: + INDEXER_DATABASE_URL: postgresql://indexer:indexer@postgres:5432/datahaven + ports: + - "30335:30333" + - "9947:9947" + volumes: + - indexer-data:/data + command: + - "--chain=stagenet-local" + - "--name=Indexer-Fishing" + - "--base-path=/data" + - "--indexer" + - "--indexer-mode=fishing" + - "--port=30333" + - "--rpc-port=9947" + restart: unless-stopped + + fisherman: + image: datahavenxyz/datahaven:latest + container_name: storagehub-fisherman + depends_on: + postgres: + condition: service_healthy + indexer: + condition: service_started + environment: + NODE_TYPE: fisherman + NODE_NAME: Gustavo + SEED: "your seed phrase here" + CHAIN: stagenet-local + KEYSTORE_PATH: /data/keystore + FISHERMAN_DATABASE_URL: postgresql://indexer:indexer@postgres:5432/datahaven + ports: + - "30336:30333" + - "9948:9948" + volumes: + - fisherman-data:/data + command: + - "--chain=stagenet-local" + - "--name=Fisherman-Gustavo" + - "--base-path=/data" + - "--keystore-path=/data/keystore" + - "--fisherman" + - "--fisherman-incomplete-sync-max=10000" + - "--fisherman-incomplete-sync-page-size=256" + - "--port=30333" + - "--rpc-port=9948" + restart: unless-stopped + +volumes: + fisherman-db: + indexer-data: + fisherman-data: +``` + +## Kubernetes Deployment + +```yaml +apiVersion: apps/v1 +kind: StatefulSet +metadata: + name: storagehub-fisherman +spec: + serviceName: storagehub-fisherman + replicas: 1 + selector: + matchLabels: + app: storagehub-fisherman + template: + metadata: + labels: + app: storagehub-fisherman + spec: + containers: + - name: fisherman + image: datahavenxyz/datahaven:latest + env: + - name: NODE_TYPE + value: "fisherman" + - name: NODE_NAME + value: "Gustavo" + - name: SEED + valueFrom: + secretKeyRef: + name: fisherman-seed + key: seed + - name: FISHERMAN_DATABASE_URL + value: postgresql://indexer:indexer@fisherman-postgres:5432/datahaven + ports: + - containerPort: 30333 + name: p2p + - containerPort: 9948 + name: rpc + volumeMounts: + - name: data + mountPath: /data + resources: + requests: + memory: "4Gi" + cpu: "2" + limits: + memory: "8Gi" + cpu: "4" + args: + - "--chain=stagenet-local" + - "--name=Fisherman-Gustavo" + - "--base-path=/data" + - "--fisherman" + - "--fisherman-incomplete-sync-max=10000" + - "--fisherman-incomplete-sync-page-size=256" + - "--port=30333" + - "--rpc-port=9948" + volumeClaimTemplates: + - metadata: + name: data + spec: + accessModes: [ "ReadWriteOnce" ] + resources: + requests: + storage: 100Gi +``` + +## On-Chain Registration + +### Not Required + +Fisherman nodes do not require on-chain registration. They operate autonomously by monitoring blockchain data and submitting challenges as needed. + +## Fisherman Operations + +### Challenge Submission Flow + +1. **Monitor**: Fisherman monitors blockchain data via database +2. **Detect**: Identifies storage provider violations: + - Missing proofs + - Invalid proofs + - Storage capacity violations + - Availability issues +3. **Verify**: Validates violation independently +4. **Challenge**: Submits challenge extrinsic to ProofsDealer pallet +5. **Reward**: Receives reward if challenge is validated + +### Types of Violations Detected + +| Violation Type | Description | Extrinsic | +|----------------|-------------|-----------| +| Missing Proof | Provider failed to submit proof | `proofsDealer.challengeMissingProof` | +| Invalid Proof | Submitted proof is invalid | `proofsDealer.challengeInvalidProof` | +| Over Capacity | Provider exceeds declared capacity | `providers.challengeCapacity` | +| Unavailable | Provider is unreachable | `providers.challengeAvailability` | + +### Reward System + +- Successful challenges earn rewards from slashed provider deposits +- Failed challenges may result in fisherman penalties +- Reward amount depends on violation severity + +## Monitoring + +### Health Checks + +```bash +# Check node health +curl -s -H "Content-Type: application/json" \ + -d '{"id":1, "jsonrpc":"2.0", "method": "system_health"}' \ + http://localhost:9948 | jq + +# Check fisherman status +curl -s -H "Content-Type: application/json" \ + -d '{"id":1, "jsonrpc":"2.0", "method": "fisherman_getStatus"}' \ + http://localhost:9948 | jq +``` + +### Database Queries + +```sql +-- Get recent challenges submitted +SELECT * FROM challenges +WHERE fisherman_account = '0x...' +ORDER BY block_number DESC +LIMIT 10; + +-- Get successful challenges +SELECT * FROM challenges +WHERE fisherman_account = '0x...' + AND status = 'validated' +ORDER BY block_number DESC; + +-- Get violation statistics +SELECT violation_type, COUNT(*) as count +FROM challenges +WHERE fisherman_account = '0x...' +GROUP BY violation_type; +``` + +### Key Metrics + +- Number of challenges submitted +- Challenge success rate +- Rewards earned +- Violations detected by type +- Account balance (for fees) + +### Logs + +```bash +# View Fisherman logs +docker logs -f storagehub-fisherman + +# Filter for challenge events +docker logs storagehub-fisherman 2>&1 | grep -i "challenge\|violation" + +# Monitor successful challenges +docker logs storagehub-fisherman 2>&1 | grep -i "challenge.*success" +``` + +## Troubleshooting + +### Issue: Database Connection Failed + +**Check:** +1. PostgreSQL is running: `docker ps | grep postgres` +2. Connection string is correct +3. Database is accessible from fisherman node +4. Indexer has populated database + +### Issue: Not Detecting Violations + +**Check:** +1. Indexer node is running and synced +2. Indexer mode is `fishing` or `full` (not `lite`) +3. Database has recent data +4. Fisherman account has sufficient balance +5. BCSV key is correctly inserted + +### Issue: Challenge Submission Failing + +**Check:** +1. Account has sufficient balance for fees +2. BCSV key is valid and inserted +3. Node is fully synced +4. Violation is still valid (not already challenged) +5. Check logs for specific error messages + +### Issue: No Rewards Received + +**Check:** +1. Challenges were validated successfully +2. Reward distribution period has passed +3. Check on-chain events for reward distribution +4. Verify fisherman account address + +## Security Considerations + +1. **Key Management**: Store seed phrase securely offline +2. **Account Security**: Monitor balance for unexpected drops +3. **Database Security**: Secure database access +4. **Network Security**: Use firewall to restrict access +5. **False Positives**: Ensure validation logic is accurate + +## Best Practices + +1. Run alongside a dedicated Indexer node +2. Monitor account balance and set up auto-refill +3. Set reasonable `incomplete-sync-max` to avoid overload +4. Keep node software updated +5. Implement monitoring and alerting +6. Document operational procedures +7. Test challenge submission in development environment +8. Monitor provider behavior patterns + +## Performance Considerations + +### Resource Requirements + +| Component | Minimum | Recommended | +|-----------|---------|-------------| +| CPU | 2 cores | 4 cores | +| RAM | 4 GB | 8 GB | +| Storage (Chain Data) | 100 GB | 200 GB | +| Storage (Database) | Shared with Indexer | Shared with Indexer | +| Network | 100 Mbps | 1 Gbps | + +### Tuning Parameters + +```bash +# For high-volume monitoring +--fisherman-incomplete-sync-max 20000 \ +--fisherman-incomplete-sync-page-size 512 \ +--fisherman-sync-mode-min-blocks-behind 3 +``` + +## Economic Considerations + +### Operational Costs + +- **Transaction Fees**: ~10 HAVE per challenge +- **False Challenge Penalty**: Varies by violation type +- **Monitoring Costs**: Infrastructure costs + +### Revenue Potential + +- **Successful Challenges**: Rewards from slashed deposits +- **Volume**: Depends on network size and provider behavior +- **Competition**: Multiple fishermen may detect same violations + +### Break-Even Analysis + +``` +Monthly Revenue = (Successful Challenges × Reward per Challenge) +Monthly Costs = (Infrastructure Costs + Transaction Fees) +Net Profit = Monthly Revenue - Monthly Costs +``` + +## Related Documentation + +- [MSP Setup](./storagehub-msp.md) +- [BSP Setup](./storagehub-bsp.md) +- [Indexer Setup](./storagehub-indexer.md) +- [StorageHub Pallets](https://github.com/Moonsong-Labs/storage-hub) +- [Proofs Dealer Pallet](https://github.com/Moonsong-Labs/storage-hub/tree/main/pallets/proofs-dealer) +- [Docker Compose Guide](../operator/DOCKER-COMPOSE.md) diff --git a/docs/storagehub-indexer.md b/docs/storagehub-indexer.md new file mode 100644 index 00000000..5be9a1e8 --- /dev/null +++ b/docs/storagehub-indexer.md @@ -0,0 +1,621 @@ +# StorageHub Indexer Node Setup + +## Overview + +Indexer nodes index blockchain data into a PostgreSQL database, enabling efficient querying of storage operations, file metadata, and provider activities. + +## Purpose + +- Index blockchain data to PostgreSQL database +- Enable efficient querying of storage operations +- Support fisherman node operations +- Provide historical data analysis +- Track file system events and provider activities + +## Prerequisites + +- DataHaven node binary or Docker image +- PostgreSQL 14+ database server +- Sufficient storage for database (100+ GB recommended) +- Stable network connection +- Open network ports (30333, optionally 9944) + +## Key Requirements + +### No Session Keys Required + +Indexer nodes do **not** require session keys as they are non-signing nodes that only observe and index blockchain data. + +### No BCSV Key Required + +Indexer nodes do not participate in storage operations, so no BCSV key is needed. + +## Wallet Requirements + +### No Wallet Required + +Indexer nodes do not submit transactions, so no funded account is needed. + +## Database Requirements + +### PostgreSQL Setup + +#### Install PostgreSQL + +```bash +# Ubuntu/Debian +sudo apt update +sudo apt install postgresql-14 postgresql-contrib + +# macOS +brew install postgresql@14 + +# Docker +docker run -d \ + --name indexer-postgres \ + -e POSTGRES_PASSWORD=indexer \ + -e POSTGRES_USER=indexer \ + -e POSTGRES_DB=datahaven \ + -p 5432:5432 \ + -v indexer-db:/var/lib/postgresql/data \ + postgres:14 +``` + +#### Create Database + +```bash +# Connect to PostgreSQL +psql -U postgres + +# Create database and user +CREATE DATABASE datahaven; +CREATE USER indexer WITH ENCRYPTED PASSWORD 'indexer'; +GRANT ALL PRIVILEGES ON DATABASE datahaven TO indexer; +\q +``` + +#### Database Connection String + +``` +postgresql://indexer:indexer@localhost:5432/datahaven +``` + +## CLI Flags + +### Required Flags + +```bash +datahaven-node \ + --chain \ + --indexer \ + --indexer-database-url +``` + +### Core Indexer Flags + +| Flag | Description | Required | Default | +|------|-------------|----------|---------| +| `--indexer` | Enable indexer service | Yes | false | +| `--indexer-database-url ` | PostgreSQL connection URL | Yes* | None | +| `--indexer-mode ` | Indexer mode (`full`, `lite`, `fishing`) | No | `full` | + +*Can also use `INDEXER_DATABASE_URL` environment variable + +### Indexer Modes + +| Mode | Description | Data Indexed | Use Case | +|------|-------------|--------------|----------| +| `full` | Index all blockchain data | All events, storage, metadata | Complete historical data | +| `lite` | Index essential storage data | Storage operations, files, providers | Storage-focused queries | +| `fishing` | Index data for fisherman | Provider challenges, proofs, violations | Fisherman operations | + +### Standard Node Flags + +| Flag | Description | Default | +|------|-------------|---------| +| `--chain ` | Chain specification | Required | +| `--name ` | Node name | Required | +| `--base-path ` | Base directory for chain data | `~/.local/share/datahaven-node` | +| `--port ` | P2P port | `30333` | +| `--rpc-port ` | WebSocket RPC port | `9944` | +| `--bootnodes ` | Bootstrap nodes | None | + +### Optional Flags + +| Flag | Description | +|------|-------------| +| `--pruning ` | State pruning mode (recommend `archive` for indexer) | +| `--blocks-pruning ` | Block pruning mode (recommend `archive`) | +| `--prometheus-external` | Expose Prometheus metrics | +| `--log ` | Logging verbosity | + +## Complete Setup Examples + +### 1. Setup Database + +```bash +# Start PostgreSQL with Docker +docker run -d \ + --name indexer-postgres \ + -e POSTGRES_PASSWORD=indexer \ + -e POSTGRES_USER=indexer \ + -e POSTGRES_DB=datahaven \ + -p 5432:5432 \ + -v indexer-db:/var/lib/postgresql/data \ + postgres:14 + +# Verify connection +psql postgresql://indexer:indexer@localhost:5432/datahaven -c "SELECT version();" +``` + +### 2. Start Indexer Node (Full Mode) + +```bash +datahaven-node \ + --chain stagenet-local \ + --name "Indexer-Full" \ + --base-path /data/indexer \ + --indexer \ + --indexer-mode full \ + --indexer-database-url postgresql://indexer:indexer@localhost:5432/datahaven \ + --pruning archive \ + --blocks-pruning archive \ + --port 30333 \ + --rpc-port 9947 \ + --bootnodes /dns/bootnode.example.com/tcp/30333/p2p/12D3KooW... +``` + +### 3. Start Indexer Node (Lite Mode) + +```bash +datahaven-node \ + --chain stagenet-local \ + --name "Indexer-Lite" \ + --base-path /data/indexer-lite \ + --indexer \ + --indexer-mode lite \ + --indexer-database-url postgresql://indexer:indexer@localhost:5432/datahaven \ + --port 30333 \ + --rpc-port 9947 +``` + +### 4. Start Indexer Node (Fishing Mode) + +```bash +datahaven-node \ + --chain stagenet-local \ + --name "Indexer-Fishing" \ + --base-path /data/indexer-fishing \ + --indexer \ + --indexer-mode fishing \ + --indexer-database-url postgresql://indexer:indexer@localhost:5432/datahaven \ + --port 30333 \ + --rpc-port 9947 +``` + +## Docker Deployment + +### Docker Compose (Full Stack) + +```yaml +version: '3.8' + +services: + postgres: + image: postgres:14 + container_name: indexer-postgres + environment: + POSTGRES_DB: datahaven + POSTGRES_USER: indexer + POSTGRES_PASSWORD: indexer + ports: + - "5432:5432" + volumes: + - indexer-db:/var/lib/postgresql/data + healthcheck: + test: ["CMD-SHELL", "pg_isready -U indexer -d datahaven"] + interval: 10s + timeout: 5s + retries: 5 + + indexer: + image: datahavenxyz/datahaven:latest + container_name: storagehub-indexer + depends_on: + postgres: + condition: service_healthy + environment: + INDEXER_DATABASE_URL: postgresql://indexer:indexer@postgres:5432/datahaven + ports: + - "30335:30333" + - "9947:9947" + volumes: + - indexer-data:/data + command: + - "--chain=stagenet-local" + - "--name=Indexer-Full" + - "--base-path=/data" + - "--indexer" + - "--indexer-mode=full" + - "--pruning=archive" + - "--blocks-pruning=archive" + - "--port=30333" + - "--rpc-port=9947" + - "--rpc-external" + restart: unless-stopped + +volumes: + indexer-db: + indexer-data: +``` + +### Docker Run + +```bash +# Start PostgreSQL +docker run -d \ + --name indexer-postgres \ + -e POSTGRES_PASSWORD=indexer \ + -e POSTGRES_USER=indexer \ + -e POSTGRES_DB=datahaven \ + -p 5432:5432 \ + postgres:14 + +# Wait for PostgreSQL to be ready +sleep 5 + +# Start Indexer +docker run -d \ + --name storagehub-indexer \ + --link indexer-postgres:postgres \ + -e INDEXER_DATABASE_URL=postgresql://indexer:indexer@postgres:5432/datahaven \ + -p 30333:30333 \ + -p 9947:9947 \ + -v $(pwd)/indexer-data:/data \ + datahavenxyz/datahaven:latest \ + --chain stagenet-local \ + --name "Indexer-Full" \ + --base-path /data \ + --indexer \ + --indexer-mode full \ + --port 30333 \ + --rpc-port 9947 +``` + +## Kubernetes Deployment + +```yaml +apiVersion: v1 +kind: Service +metadata: + name: indexer-postgres +spec: + ports: + - port: 5432 + targetPort: 5432 + selector: + app: indexer-postgres + +--- +apiVersion: apps/v1 +kind: StatefulSet +metadata: + name: indexer-postgres +spec: + serviceName: indexer-postgres + replicas: 1 + selector: + matchLabels: + app: indexer-postgres + template: + metadata: + labels: + app: indexer-postgres + spec: + containers: + - name: postgres + image: postgres:14 + env: + - name: POSTGRES_DB + value: datahaven + - name: POSTGRES_USER + value: indexer + - name: POSTGRES_PASSWORD + valueFrom: + secretKeyRef: + name: indexer-db-secret + key: password + ports: + - containerPort: 5432 + name: postgres + volumeMounts: + - name: data + mountPath: /var/lib/postgresql/data + volumeClaimTemplates: + - metadata: + name: data + spec: + accessModes: [ "ReadWriteOnce" ] + resources: + requests: + storage: 200Gi + +--- +apiVersion: apps/v1 +kind: StatefulSet +metadata: + name: storagehub-indexer +spec: + serviceName: storagehub-indexer + replicas: 1 + selector: + matchLabels: + app: storagehub-indexer + template: + metadata: + labels: + app: storagehub-indexer + spec: + containers: + - name: indexer + image: datahavenxyz/datahaven:latest + env: + - name: INDEXER_DATABASE_URL + value: postgresql://indexer:indexer@indexer-postgres:5432/datahaven + ports: + - containerPort: 30333 + name: p2p + - containerPort: 9947 + name: rpc + volumeMounts: + - name: data + mountPath: /data + resources: + requests: + memory: "8Gi" + cpu: "4" + limits: + memory: "16Gi" + cpu: "8" + args: + - "--chain=stagenet-local" + - "--name=Indexer-Full" + - "--base-path=/data" + - "--indexer" + - "--indexer-mode=full" + - "--pruning=archive" + - "--blocks-pruning=archive" + - "--port=30333" + - "--rpc-port=9947" + volumeClaimTemplates: + - metadata: + name: data + spec: + accessModes: [ "ReadWriteOnce" ] + resources: + requests: + storage: 300Gi +``` + +## On-Chain Registration + +### Not Required + +Indexer nodes do not require any on-chain registration or extrinsics. + +## Database Schema + +### Key Tables (Generated Automatically) + +The indexer automatically creates and manages database tables: + +- **blocks**: Block headers and metadata +- **extrinsics**: Extrinsic data per block +- **events**: Blockchain events +- **storage_providers**: MSP/BSP registration data +- **files**: File metadata and storage information +- **buckets**: Bucket ownership and configuration +- **proofs**: Proof submissions and challenges +- **payment_streams**: Payment stream data + +### Query Examples + +```sql +-- Get all MSPs +SELECT * FROM storage_providers WHERE provider_type = 'msp'; + +-- Get files stored by a specific MSP +SELECT * FROM files WHERE msp_id = '0x...'; + +-- Get recent proof submissions +SELECT * FROM proofs ORDER BY block_number DESC LIMIT 10; + +-- Get total storage capacity by provider type +SELECT provider_type, SUM(capacity) as total_capacity +FROM storage_providers +GROUP BY provider_type; +``` + +## Monitoring + +### Health Checks + +```bash +# Check node health +curl -s -H "Content-Type: application/json" \ + -d '{"id":1, "jsonrpc":"2.0", "method": "system_health"}' \ + http://localhost:9947 | jq + +# Check sync status +curl -s -H "Content-Type: application/json" \ + -d '{"id":1, "jsonrpc":"2.0", "method": "system_syncState"}' \ + http://localhost:9947 | jq +``` + +### Database Health + +```bash +# Check database connection +psql postgresql://indexer:indexer@localhost:5432/datahaven -c "SELECT COUNT(*) FROM blocks;" + +# Check database size +psql postgresql://indexer:indexer@localhost:5432/datahaven -c "SELECT pg_size_pretty(pg_database_size('datahaven'));" + +# Check table sizes +psql postgresql://indexer:indexer@localhost:5432/datahaven -c " +SELECT + schemaname, + tablename, + pg_size_pretty(pg_total_relation_size(schemaname||'.'||tablename)) AS size +FROM pg_tables +WHERE schemaname = 'public' +ORDER BY pg_total_relation_size(schemaname||'.'||tablename) DESC;" +``` + +### Key Metrics + +- Indexing lag (blocks behind chain tip) +- Database size and growth rate +- Query performance +- Connection pool usage +- Disk I/O performance + +## Troubleshooting + +### Issue: Database Connection Failed + +**Check:** +1. PostgreSQL is running: `docker ps | grep postgres` +2. Connection string is correct +3. Database credentials are valid +4. Network connectivity between node and database +5. PostgreSQL logs: `docker logs indexer-postgres` + +### Issue: Slow Indexing + +**Solutions:** +1. Optimize PostgreSQL configuration: + ```sql + ALTER SYSTEM SET shared_buffers = '4GB'; + ALTER SYSTEM SET effective_cache_size = '12GB'; + ALTER SYSTEM SET maintenance_work_mem = '1GB'; + ALTER SYSTEM SET checkpoint_completion_target = 0.9; + ALTER SYSTEM SET wal_buffers = '16MB'; + ALTER SYSTEM SET default_statistics_target = 100; + ``` +2. Add indexes to frequently queried columns +3. Use faster storage (NVMe SSD) +4. Increase database connection pool size + +### Issue: Database Running Out of Space + +**Solutions:** +1. Enable PostgreSQL auto-vacuum: `ALTER TABLE SET (autovacuum_enabled = true);` +2. Manual vacuum: `VACUUM FULL;` +3. Archive old data +4. Increase disk space + +### Issue: Indexer Not Catching Up + +**Check:** +1. Node is fully synced: Check `system_syncState` +2. Database has sufficient resources +3. No errors in indexer logs +4. PostgreSQL is not overloaded + +## Performance Tuning + +### PostgreSQL Configuration + +Edit `postgresql.conf`: + +```ini +# Memory +shared_buffers = 4GB +effective_cache_size = 12GB +maintenance_work_mem = 1GB +work_mem = 256MB + +# Checkpoints +checkpoint_completion_target = 0.9 +wal_buffers = 16MB +max_wal_size = 4GB + +# Connections +max_connections = 200 + +# Query Performance +random_page_cost = 1.1 # For SSD +effective_io_concurrency = 200 + +# Statistics +default_statistics_target = 100 +``` + +### Indexer Node Configuration + +```bash +datahaven-node \ + --indexer \ + --pruning archive \ + --blocks-pruning archive \ + --state-cache-size 268435456 \ # 256 MB + --max-runtime-instances 8 +``` + +### Resource Requirements + +| Mode | CPU | RAM | Storage (Chain) | Storage (DB) | Network | +|------|-----|-----|-----------------|--------------|---------| +| Full | 4-8 cores | 16-32 GB | 200 GB | 200-500 GB | 100 Mbps | +| Lite | 2-4 cores | 8-16 GB | 100 GB | 50-100 GB | 100 Mbps | +| Fishing | 2-4 cores | 8-16 GB | 100 GB | 50-100 GB | 100 Mbps | + +## Security Considerations + +1. **Database Security**: Use strong passwords, restrict network access +2. **Connection Encryption**: Use SSL for PostgreSQL connections +3. **Access Control**: Limit database access to indexer node only +4. **Backup Strategy**: Regular database backups +5. **Monitoring**: Set up alerts for connection failures + +## Best Practices + +1. Use dedicated PostgreSQL server for production +2. Enable regular database backups (daily recommended) +3. Monitor database size and plan for growth +4. Use archive mode for complete historical data +5. Implement connection pooling (e.g., PgBouncer) +6. Regular database maintenance (VACUUM, ANALYZE) +7. Set up monitoring and alerting +8. Document backup/restore procedures + +## Backup and Restore + +### Backup Database + +```bash +# Full backup +pg_dump -U indexer -h localhost datahaven > datahaven-backup-$(date +%Y%m%d).sql + +# Compressed backup +pg_dump -U indexer -h localhost datahaven | gzip > datahaven-backup-$(date +%Y%m%d).sql.gz +``` + +### Restore Database + +```bash +# Restore from backup +psql -U indexer -h localhost datahaven < datahaven-backup-20250124.sql + +# Restore from compressed backup +gunzip -c datahaven-backup-20250124.sql.gz | psql -U indexer -h localhost datahaven +``` + +## Related Documentation + +- [MSP Setup](./storagehub-msp.md) +- [BSP Setup](./storagehub-bsp.md) +- [Fisherman Setup](./storagehub-fisherman.md) +- [PostgreSQL Documentation](https://www.postgresql.org/docs/14/) +- [Docker Compose Guide](../operator/DOCKER-COMPOSE.md) diff --git a/docs/storagehub-msp.md b/docs/storagehub-msp.md new file mode 100644 index 00000000..18fb9450 --- /dev/null +++ b/docs/storagehub-msp.md @@ -0,0 +1,586 @@ +# StorageHub Main Storage Provider (MSP) Setup + +## Overview + +Main Storage Providers (MSPs) are primary storage providers in the StorageHub network that manage user data, buckets, and coordinate with Backup Storage Providers (BSPs). + +## Purpose + +- Store and manage user files and buckets +- Charge storage fees from users +- Distribute files to BSPs for redundancy +- Manage bucket migrations +- Serve file download requests +- Submit proofs of storage + +## Prerequisites + +- DataHaven node binary or Docker image +- Funded account with sufficient balance for deposits +- Storage capacity (minimum 2 data units, recommended 10+ GiB) +- Stable network connection +- Open network ports (30333, optionally 9944) +- Optional: PostgreSQL database for advanced features + +## Key Requirements + +### BCSV Key (ECDSA - 1 Required) + +MSPs require **one BCSV key** for storage provider identity. + +| Key Type | Scheme | Purpose | +|----------|--------|---------| +| `bcsv` | ecdsa | Storage provider identity and signing | + +### Generate BCSV Key + +#### Method 1: CLI Key Insertion + +```bash +# Generate seed phrase +SEED=$(datahaven-node key generate | grep "Secret phrase" | cut -d'`' -f2) + +# Insert BCSV key (ecdsa) +datahaven-node key insert \ + --base-path /data/msp \ + --chain stagenet-local \ + --key-type bcsv \ + --scheme ecdsa \ + --suri "$SEED" +``` + +#### Method 2: Docker Entrypoint (Automated) + +Set environment variables: + +```bash +export NODE_TYPE=msp +export NODE_NAME=msp01 +export SEED="your seed phrase here" +export CHAIN=stagenet-local +``` + +The entrypoint script automatically injects the BCSV key. + +## Wallet Requirements + +### Provider Account + +- **Purpose**: MSP registration, transaction fees, and deposits +- **Required Balance**: + - Minimum deposit: 100 HAVE (SpMinDeposit) + - Deposit per data unit: 2 HAVE per unit + - Transaction fees: ~10 HAVE + - **Recommended**: 200+ HAVE for initial setup +- **Funding**: Must be funded **before** MSP registration +- **Account Type**: Ethereum-style 20-byte address (AccountId20) + +### Generate Provider Account + +```bash +# Generate new account from seed +SEED="your secure seed phrase here" +echo $SEED | datahaven-node key inspect --output-type json | jq + +# Derive MSP account +echo "$SEED//my_awesome_msp" | datahaven-node key inspect --output-type json | jq -r '.ss58PublicKey' +``` + +## CLI Flags + +### Required Flags + +```bash +datahaven-node \ + --chain \ + --provider \ + --provider-type msp \ + --max-storage-capacity \ + --jump-capacity \ + --msp-charging-period +``` + +### Core Provider Flags + +| Flag | Description | Required | Default | +|------|-------------|----------|---------| +| `--provider` | Enable storage provider mode | Yes | false | +| `--provider-type msp` | Set provider type to MSP | Yes | None | +| `--max-storage-capacity ` | Maximum storage capacity | Yes | None | +| `--jump-capacity ` | Jump capacity for new storage | Yes | None | +| `--msp-charging-period ` | Fee charging period in blocks | Yes | None | +| `--storage-layer ` | Storage backend (`rocksdb` or `memory`) | No | `memory` | +| `--storage-path ` | Storage path (required if rocksdb) | No | None | + +**Example Values:** +- `--max-storage-capacity 10737418240` (10 GiB) +- `--jump-capacity 1073741824` (1 GiB) +- `--msp-charging-period 100` (100 blocks) + +### MSP-Specific Task Flags + +| Flag | Description | Default | +|------|-------------|---------| +| `--msp-charge-fees-task` | Enable automatic fee charging | false | +| `--msp-charge-fees-min-debt ` | Minimum debt threshold to charge | 0 | +| `--msp-move-bucket-task` | Enable bucket migration task | false | +| `--msp-move-bucket-max-try-count ` | Max retries for bucket moves | 5 | +| `--msp-move-bucket-max-tip ` | Max tip for move bucket extrinsics | 0 | +| `--msp-distribute-files` | Enable file distribution to BSPs | false | + +### Remote File Handling Flags + +| Flag | Description | Default | +|------|-------------|---------| +| `--max-file-size ` | Maximum file size | 10737418240 (10 GB) | +| `--connection-timeout ` | Connection timeout | 30 | +| `--read-timeout ` | Read timeout | 300 | +| `--follow-redirects ` | Follow HTTP redirects | true | +| `--max-redirects ` | Maximum redirects | 10 | +| `--user-agent ` | HTTP user agent | "StorageHub-Client/1.0" | +| `--chunk-size ` | Upload/download chunk size | 8192 (8 KB) | +| `--chunks-buffer ` | Number of chunks to buffer | 512 | + +### Operational Flags + +| Flag | Description | Default | +|------|-------------|---------| +| `--extrinsic-retry-timeout ` | Extrinsic retry timeout | 60 | +| `--sync-mode-min-blocks-behind ` | Min blocks behind for sync mode | 5 | +| `--check-for-pending-proofs-period ` | Period to check pending proofs | 4 | +| `--max-blocks-behind-to-catch-up-root-changes ` | Max blocks to process for root changes | 10 | + +## Complete Setup Example + +### 1. Generate Keys and Account + +```bash +# Generate seed phrase +SEED="your secure seed phrase here" + +# Derive MSP account +MSP_ACCOUNT=$(echo "$SEED//msp01" | datahaven-node key inspect --output-type json | jq -r '.ss58PublicKey') +echo "MSP Account: $MSP_ACCOUNT" + +# Insert BCSV key +datahaven-node key insert \ + --base-path /data/msp \ + --chain stagenet-local \ + --key-type bcsv \ + --scheme ecdsa \ + --suri "$SEED" +``` + +### 2. Fund Provider Account + +```bash +# Transfer funds to MSP account +# Minimum: 200 HAVE (100 deposit + 100 for operations) + +# Using Polkadot.js or a funded account, send HAVE tokens to $MSP_ACCOUNT +``` + +### 3. Start MSP Node + +```bash +datahaven-node \ + --chain stagenet-local \ + --name "MSP01" \ + --base-path /data/msp \ + --provider \ + --provider-type msp \ + --max-storage-capacity 10737418240 \ + --jump-capacity 1073741824 \ + --msp-charging-period 100 \ + --storage-layer rocksdb \ + --storage-path /data/msp/storage \ + --msp-charge-fees-task \ + --msp-move-bucket-task \ + --msp-distribute-files \ + --port 30333 \ + --rpc-port 9945 \ + --bootnodes /dns/bootnode.example.com/tcp/30333/p2p/12D3KooW... +``` + +### 4. Register MSP On-Chain + +See [On-Chain Registration](#on-chain-registration) section below. + +## Docker Deployment + +### Docker Compose + +```yaml +version: '3.8' + +services: + msp: + image: datahavenxyz/datahaven:latest + container_name: storagehub-msp + environment: + NODE_TYPE: msp + NODE_NAME: msp01 + SEED: "your seed phrase here" + CHAIN: stagenet-local + KEYSTORE_PATH: /data/keystore + ports: + - "30333:30333" + - "9945:9945" + volumes: + - msp-data:/data + - msp-storage:/data/storage + command: + - "--chain=stagenet-local" + - "--name=MSP01" + - "--base-path=/data" + - "--keystore-path=/data/keystore" + - "--provider" + - "--provider-type=msp" + - "--max-storage-capacity=10737418240" + - "--jump-capacity=1073741824" + - "--msp-charging-period=100" + - "--storage-layer=rocksdb" + - "--storage-path=/data/storage" + - "--msp-charge-fees-task" + - "--msp-move-bucket-task" + - "--msp-distribute-files" + - "--port=30333" + - "--rpc-port=9945" + restart: unless-stopped + +volumes: + msp-data: + msp-storage: +``` + +## Kubernetes Deployment + +```yaml +apiVersion: apps/v1 +kind: StatefulSet +metadata: + name: storagehub-msp +spec: + serviceName: storagehub-msp + replicas: 1 + selector: + matchLabels: + app: storagehub-msp + template: + metadata: + labels: + app: storagehub-msp + spec: + containers: + - name: msp + image: datahavenxyz/datahaven:latest + env: + - name: NODE_TYPE + value: "msp" + - name: NODE_NAME + value: "MSP01" + - name: SEED + valueFrom: + secretKeyRef: + name: msp-seed + key: seed + ports: + - containerPort: 30333 + name: p2p + - containerPort: 9945 + name: rpc + volumeMounts: + - name: data + mountPath: /data + - name: storage + mountPath: /data/storage + resources: + requests: + memory: "4Gi" + cpu: "2" + limits: + memory: "8Gi" + cpu: "4" + args: + - "--chain=stagenet-local" + - "--provider" + - "--provider-type=msp" + - "--max-storage-capacity=10737418240" + - "--jump-capacity=1073741824" + - "--msp-charging-period=100" + - "--storage-layer=rocksdb" + - "--storage-path=/data/storage" + - "--msp-charge-fees-task" + - "--msp-move-bucket-task" + - "--msp-distribute-files" + volumeClaimTemplates: + - metadata: + name: data + spec: + accessModes: [ "ReadWriteOnce" ] + resources: + requests: + storage: 100Gi + - metadata: + name: storage + spec: + accessModes: [ "ReadWriteOnce" ] + resources: + requests: + storage: 500Gi +``` + +## On-Chain Registration + +### MSP Registration Process + +MSPs must be registered on-chain via the `Providers` pallet using a **2-step process**: + +1. **Step 1**: Call `request_msp_sign_up` - Initiates registration and reserves deposit +2. **Step 2**: Call `confirm_sign_up` - Completes registration after randomness verification + +This two-step mechanism ensures security and prevents manipulation of provider IDs through randomness. + +### Step 1: Request MSP Sign Up + +```typescript +import { createClient } from 'polkadot-api'; +import { getWsProvider } from 'polkadot-api/ws-provider/web'; +import { withPolkadotSdkCompat } from 'polkadot-api/polkadot-sdk-compat'; +import { datahaven } from '@polkadot-api/descriptors'; +import { Binary } from 'polkadot-api'; + +// Connect to DataHaven node +const client = createClient( + withPolkadotSdkCompat(getWsProvider('ws://localhost:9944')) +); +const typedApi = client.getTypedApi(datahaven); + +// MSP signer (using your BCSV key account) +const mspSigner = /* your polkadot-api signer */; + +// MSP configuration +const capacity = BigInt(10_737_418_240); // 10 GiB in bytes +const multiaddresses = [ + '/ip4/127.0.0.1/tcp/30333', + '/dns/msp01.example.com/tcp/30333' +].map(addr => Binary.fromText(addr)); + +// Step 1: Request MSP sign up +const requestTx = typedApi.tx.Providers.request_msp_sign_up({ + capacity: capacity, + multiaddresses: multiaddresses, + value_prop_price_per_giga_unit_of_data_per_block: BigInt(18_520_000_000), + commitment: Binary.fromText('msp01'), + value_prop_max_data_limit: BigInt(1_073_741_824), + payment_account: mspSigner.publicKey // Account receiving payments +}); + +// Sign and submit the request +const requestResult = await requestTx.signAndSubmit(mspSigner); +console.log('MSP sign-up requested. Waiting for finalization...'); + +await requestResult.finalized(); +console.log('Request finalized! Deposit has been reserved.'); +``` + +**What Happens in Step 1:** +- Validates multiaddresses format +- Calculates required deposit based on capacity (`SpMinDeposit + capacity * DepositPerData`) +- Verifies account has sufficient balance +- **Holds (reserves) the deposit** from your account +- Creates a pending sign-up request +- Emits `MspRequestSignUpSuccess` event + +### Step 2: Confirm Sign Up + +After requesting, you must wait for sufficient randomness to be available (controlled by `MaxBlocksForRandomness` parameter, typically 2 hours on mainnet). + +```typescript +// Step 2: Confirm the sign-up (after waiting for randomness) +const confirmTx = typedApi.tx.Providers.confirm_sign_up({ + provider_account: undefined // Optional: omit to use signer's account +}); + +// Sign and submit confirmation +const confirmResult = await confirmTx.signAndSubmit(mspSigner); +console.log('Confirming MSP registration...'); + +await confirmResult.finalized(); +console.log('MSP registration confirmed and active!'); +``` + +**What Happens in Step 2:** +- Verifies randomness is sufficiently fresh +- Checks request hasn't expired +- Generates Provider ID using randomness +- Registers MSP in the system +- Emits `MspSignUpSuccess` event +- Deposit remains held for duration of MSP operation + +### Timing Requirements + +| Parameter | Testnet | Mainnet | Description | +|-----------|---------|---------|-------------| +| Min wait time | ~2 minutes | ~2 hours | Wait after `request_msp_sign_up` for randomness | +| Max wait time | Set by `MaxBlocksForRandomness` | Typically 2 hours | Request expires if not confirmed in time | + +### Verify Registration + +```typescript +// Check MSP registration status +const mspAccount = mspSigner.publicKey; + +const registeredMspId = await typedApi.query.Providers.AccountIdToMainStorageProviderId.getValue( + mspAccount +); + +if (registeredMspId) { + console.log('Registered MSP ID:', registeredMspId); + + // Get full MSP details + const mspInfo = await typedApi.query.Providers.MainStorageProviders.getValue( + registeredMspId + ); + console.log('MSP Info:', mspInfo); +} else { + console.log('MSP not yet registered or confirmation pending'); +} +``` + +### Cancel Pending Request + +If you change your mind before confirming: + +```typescript +const cancelTx = typedApi.tx.Providers.cancel_sign_up(); +await cancelTx.signAndSubmit(mspSigner); +console.log('Sign-up request cancelled, deposit returned'); +``` + +### Development/Testing: Force Sign Up (Requires Sudo) + +For development and testing environments with sudo access, you can bypass the 2-step process: + +```typescript +// Single-step registration for testing (requires sudo) +const sudoSigner = /* sudo account signer */; + +const mspCall = typedApi.tx.Providers.force_msp_sign_up({ + who: mspAccount, + msp_id: /* pre-generated provider ID */, + capacity: BigInt(10_737_418_240), + value_prop_price_per_giga_unit_of_data_per_block: BigInt(18_520_000_000), + multiaddresses: multiaddresses, + commitment: Binary.fromText('msp01'), + value_prop_max_data_limit: BigInt(1_073_741_824), + payment_account: mspAccount +}); + +const sudoTx = typedApi.tx.Sudo.sudo({ call: mspCall.decodedCall }); +await sudoTx.signAndSubmit(sudoSigner); +``` + +### Registration Parameters + +| Parameter | Type | Description | Example | +|-----------|------|-------------|---------| +| `capacity` | StorageDataUnit | Storage capacity in bytes | `10737418240` (10 GiB) | +| `multiaddresses` | Vec | P2P network addresses | `[Binary.fromText("/ip4/...")]` | +| `value_prop_price_per_giga_unit_of_data_per_block` | Balance | Price per GiB per block | `18520000000` | +| `commitment` | Bytes | Service commitment identifier | `Binary.fromText("msp01")` | +| `value_prop_max_data_limit` | StorageDataUnit | Max data per value prop | `1073741824` (1 GiB) | +| `payment_account` | AccountId | Account receiving payments | `0x...` (20-byte) | + +### Deposit Requirements + +- **Base Deposit**: 100 HAVE (`SpMinDeposit`) +- **Per Data Unit**: 2 HAVE per unit (`DepositPerData`) +- **Total for 10 GiB**: ~100 HAVE + (10 GiB in units × 2 HAVE) + +The deposit is **held (reserved)** from your account when you call `request_msp_sign_up` and remains held while you operate as an MSP. + +## Monitoring + +### Health Checks + +```bash +# Check node health +curl -s -H "Content-Type: application/json" \ + -d '{"id":1, "jsonrpc":"2.0", "method": "system_health"}' \ + http://localhost:9945 | jq + +# Check provider status +curl -s -H "Content-Type: application/json" \ + -d '{"id":1, "jsonrpc":"2.0", "method": "storageprovider_getStatus"}' \ + http://localhost:9945 | jq +``` + +### Key Metrics to Monitor + +- Storage capacity usage +- Number of stored files +- Fee collection status +- Proof submission success rate +- Bucket migration status +- BSP distribution success rate + +### Logs + +```bash +# View MSP logs +docker logs -f storagehub-msp + +# Filter for storage events +docker logs storagehub-msp 2>&1 | grep -i "storage\|bucket\|file" +``` + +## Troubleshooting + +### Issue: Registration Failed + +**Check:** +1. Account has sufficient balance (200+ HAVE) +2. BCSV key is correctly inserted +3. Capacity meets minimum (2 data units) +4. Provider ID is correctly calculated + +### Issue: Not Accepting Files + +**Check:** +1. MSP is registered on-chain +2. Storage capacity not exceeded +3. Node is fully synced +4. RPC endpoint is accessible + +### Issue: Fee Charging Not Working + +**Check:** +1. `--msp-charge-fees-task` flag is enabled +2. `--msp-charging-period` matches on-chain value +3. Users have sufficient debt to charge + +## Security Considerations + +1. **Key Management**: Store seed phrase securely offline +2. **Storage Security**: Encrypt storage at rest +3. **Network Security**: Use firewall to restrict access +4. **Access Control**: Limit RPC access to trusted sources +5. **Backup Strategy**: Regular backups of stored data + +## Best Practices + +1. Use production-grade storage (NVMe SSD recommended) +2. Monitor storage capacity proactively +3. Enable all MSP tasks for full functionality +4. Set reasonable `msp-charging-period` (100-1000 blocks) +5. Keep node software updated +6. Implement monitoring and alerting +7. Document operational procedures + +## Related Documentation + +- [BSP Setup](./storagehub-bsp.md) +- [Indexer Setup](./storagehub-indexer.md) +- [Fisherman Setup](./storagehub-fisherman.md) +- [StorageHub Pallets](https://github.com/Moonsong-Labs/storage-hub)