Elgato_dark/datahaven
1
0
Fork 0
mirror of https://github.com/datahaven-xyz/datahaven synced 2026-05-24 01:38:32 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions
410 commits 43 branches 56 tags 33 MiB
Commit graph

2 commits

Author SHA1 Message Date
Steve Degosserie
5988691a2f
feat: ✨ Add deployment charts for StorageHub MSP, BSP & Indexer nodes (Local & Stagenet envs) (#160) 
## Summary

This PR adds comprehensive Kubernetes deployment infrastructure for
StorageHub components, enabling deployment of the full StorageHub
network stack (MSP, BSP, Indexer, and Fisherman nodes) alongside
DataHaven nodes in both local and stagenet environments.

### What's Added

**1. New Helm Chart: StorageHub MSP Backend API**
(`deploy/charts/backend/`)
- REST API service for StorageHub operations
- Connects to PostgreSQL database for indexed blockchain data
- Connects to RPC nodes for real-time blockchain queries
- Configurable via TOML configuration file
- Supports environment-specific overrides
- Includes comprehensive documentation

**2. StorageHub Node Deployment Charts**
(`deploy/charts/node/storagehub/`)
- **MSP Node** (`sh-mspnode`): Main Service Provider nodes with charging
capabilities
- **BSP Node** (`sh-bspnode`): Backup Service Provider nodes for
redundancy
- **Indexer Node** (`sh-idxnode`): Full indexing node with PostgreSQL
integration
- **Fisherman Node** (`sh-fisherman`): Network monitoring and
verification node

**3. Environment Configurations**
- **Local environment** (`deploy/environments/local/`): Development
setup with hostpath storage
- **Stagenet environment** (`deploy/environments/stagenet/`):
Production-like setup with AWS EBS
- PostgreSQL database configurations for Indexer and Fisherman nodes
- Proper service discovery and network configuration

**4. Enhanced CLI Tooling** (`test/cli/`)
- New `deploy storagehub` command for deploying StorageHub components
- Updated `launch storagehub` command for local testing
- Interactive deployment with environment selection
- Automatic database provisioning via Bitnami PostgreSQL charts

**5. Node Configuration Improvements**
- Fork-aware transaction pool for DH boot & validator nodes
- Unsafe RPC methods exposed on MSP nodes (for provider operations)
- JWT secret support for MSP Backend authentication
- ECDSA key scheme for StorageHub BCSV keys (DataHaven compatibility)

### Architecture

```
StorageHub Stack:
├── MSP Nodes (2 replicas) → Storage providers with charging
├── BSP Nodes (2 replicas) → Backup storage providers
├── Indexer Node → Database indexing + PostgreSQL
├── Fisherman Node → Monitoring + PostgreSQL (shared with Indexer)
└── MSP Backend API → REST API for StorageHub operations
```

### Testing

**Local Testing**:
```bash
cd test
bun cli launch storagehub  # Interactive launcher
# or
bun cli deploy storagehub  # Deploy via Helm
```

**Stagenet Deployment**:
```bash
cd deploy
helm install sh-mspnode ./charts/node \
  -f ./charts/node/storagehub/sh-mspnode.yaml \
  -f ./environments/stagenet/sh-mspnode.yaml \
  -n datahaven-stagenet
```

### Breaking Changes

None - This is purely additive infrastructure.

### Migration Notes

For existing deployments:
1. DataHaven nodes now use `--pool-type fork-aware` flag
2. Bootnode and validator node configs updated accordingly
3. No action required for existing DataHaven-only deployments
 2025-10-21 23:18:50 +03:00
Steve Degosserie
1f0cd6de27
feat: ✨ Enhanced Helm deployment with ingress management and solochain support (#115) 
This PR enhances the Helm-based deployment system with several key
improvements organized into the following areas:

## New Features

### Ingress Management
- **Ingress per replica**: Added `ingress-per-replica` chart template
that automatically generates an ingress for each node replica, exposing
individual pod instances
- **Traefik integration**: Added chart values to deploy Traefik as the
ingress controller for local K8s clusters, enabling proper ingress
testing (requires adding names to `/etc/hosts`)

### Solochain Support
- **New relay chart**: Added dedicated Helm chart for Solochain relay
deployment
- **CLI integration**: Updated DataHaven CLI to deploy both Execution
and Solochain relayers

## Configuration Improvements

### Environment Structure
- **Modular configs**: Refactored environmental configurations from
single `values.yaml` files into separate component-specific overrides
for better organization

### Node Configuration
- **Base config updates**: Improved base configurations for bootnode and
validator nodes
- **Network protocol**: Reverted from litep2p back to libp2p to resolve
node communication issues on Stagenet
- **Archive node routing**: In Stagenet, relayers now connect to the
bootnode (configured as archive node) instead of validator nodes

## Storage & Deployment

### Persistent Storage
- **Relayer database**: Added support for persistent volumes to store
relayer databases instead of using ephemeral storage

### Deployment Scripts
- **Documentation cleanup**: Removed obsolete test deploy.sh script and
updated the deployment README with clearer instructions

These changes provide a more robust, scalable, and maintainable
deployment system for DataHaven infrastructure.

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **New Features**
* Added support for deploying new relayer types: Solochain and Execution
relayers, with dedicated configuration and secret management.
* Introduced per-replica ingress configuration for node deployments,
allowing each replica to have its own ingress resource and hostname.
* Added persistent storage options for relay data, configurable via
storage path, class, and size.
* Added new deployment configuration files for local and stagenet
environments, including Traefik ingress controller setup.
* Introduced a new relay category, "Solochain Relayers," for standalone
chain operations and cross-chain communication.

* **Improvements**
* Updated deployment configurations to use container-specific
environment YAML files for more granular control.
* Enhanced relay and node configurations with new flags and backend
options, including dynamic peer ID generation and automatic bootnode
discovery.
* Updated relayer endpoints to consistently use the bootnode for
connections.
* Refined relay configuration files for improved structure, clarity, and
endpoint management.

* **Bug Fixes**
* Corrected deployment logic to reference the correct
environment-specific configuration files during Helm deployments.

* **Documentation**
* Simplified and updated deployment documentation to focus on CLI-based
deployment, removing outdated manual instructions and adding a concise
overview of components and relayer types.

* **Chores**
* Removed deprecated deployment scripts and outdated configuration files
to streamline the deployment process.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->

---------

Co-authored-by: Claude <noreply@anthropic.com>
 2025-08-11 14:30:01 +03:00