An EVM compatible Substrate chain, powered by StorageHub and secured by EigenLayer
Find a file
Steve Degosserie 1f0cd6de27
feat: Enhanced Helm deployment with ingress management and solochain support (#115)
This PR enhances the Helm-based deployment system with several key
improvements organized into the following areas:

## New Features

### Ingress Management
- **Ingress per replica**: Added `ingress-per-replica` chart template
that automatically generates an ingress for each node replica, exposing
individual pod instances
- **Traefik integration**: Added chart values to deploy Traefik as the
ingress controller for local K8s clusters, enabling proper ingress
testing (requires adding names to `/etc/hosts`)

### Solochain Support
- **New relay chart**: Added dedicated Helm chart for Solochain relay
deployment
- **CLI integration**: Updated DataHaven CLI to deploy both Execution
and Solochain relayers

## Configuration Improvements

### Environment Structure
- **Modular configs**: Refactored environmental configurations from
single `values.yaml` files into separate component-specific overrides
for better organization

### Node Configuration
- **Base config updates**: Improved base configurations for bootnode and
validator nodes
- **Network protocol**: Reverted from litep2p back to libp2p to resolve
node communication issues on Stagenet
- **Archive node routing**: In Stagenet, relayers now connect to the
bootnode (configured as archive node) instead of validator nodes

## Storage & Deployment

### Persistent Storage
- **Relayer database**: Added support for persistent volumes to store
relayer databases instead of using ephemeral storage

### Deployment Scripts
- **Documentation cleanup**: Removed obsolete test deploy.sh script and
updated the deployment README with clearer instructions

These changes provide a more robust, scalable, and maintainable
deployment system for DataHaven infrastructure.

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **New Features**
* Added support for deploying new relayer types: Solochain and Execution
relayers, with dedicated configuration and secret management.
* Introduced per-replica ingress configuration for node deployments,
allowing each replica to have its own ingress resource and hostname.
* Added persistent storage options for relay data, configurable via
storage path, class, and size.
* Added new deployment configuration files for local and stagenet
environments, including Traefik ingress controller setup.
* Introduced a new relay category, "Solochain Relayers," for standalone
chain operations and cross-chain communication.

* **Improvements**
* Updated deployment configurations to use container-specific
environment YAML files for more granular control.
* Enhanced relay and node configurations with new flags and backend
options, including dynamic peer ID generation and automatic bootnode
discovery.
* Updated relayer endpoints to consistently use the bootnode for
connections.
* Refined relay configuration files for improved structure, clarity, and
endpoint management.

* **Bug Fixes**
* Corrected deployment logic to reference the correct
environment-specific configuration files during Helm deployments.

* **Documentation**
* Simplified and updated deployment documentation to focus on CLI-based
deployment, removing outdated manual instructions and adding a concise
overview of components and relayer types.

* **Chores**
* Removed deprecated deployment scripts and outdated configuration files
to streamline the deployment process.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->

---------

Co-authored-by: Claude <noreply@anthropic.com>
2025-08-11 14:30:01 +03:00
.github/workflows test: 🏗️ Setup e2e testing framework (#104) 2025-07-16 18:51:07 +02:00
contracts refactor: remove duplicate code for calculating merkle root and proof (#124) 2025-07-28 18:34:28 +02:00
deploy feat: Enhanced Helm deployment with ingress management and solochain support (#115) 2025-08-11 14:30:01 +03:00
operator fix: 🔨 Fix testnet Identity pallet's configuration (#127) 2025-08-07 15:54:32 +03:00
resources test: Add E2E Tests (#36) 2025-04-14 16:22:43 -03:00
test feat: Enhanced Helm deployment with ingress management and solochain support (#115) 2025-08-11 14:30:01 +03:00
.gitignore ci: 👷 Add CI to check PAPI metadata (#107) 2025-06-19 19:12:04 -03:00
.gitmodules build: Change Snowbridge contracts dependency from upstream to fork (#18) 2025-03-28 15:49:43 -03:00
biome.json fix: 🚨 Add error in TS for missing awaits (#81) 2025-05-19 22:28:43 +00:00
CLAUDE.md feat: 🧑‍💻 Add CLAUDE.md file for Claude Code (#102) 2025-06-16 15:02:11 -03:00
README.md feat: 🚀 Add deploy command to CLI (#87) 2025-06-12 10:24:03 +02:00
taplo.toml ci: 🐳 Start Publishing Docker Images (#64) 2025-05-08 20:32:55 -03:00

DataHaven 🫎

An EVM compatible Substrate chain, powered by StorageHub and secured by EigenLayer.

Repo Structure

datahaven/
├── .github/ # GitHub Actions workflows.
├── contracts/ # Implementation of the DataHaven AVS (Autonomous Verifiable Service) smart contracts to interact with EigenLayer.
├── operator/ # DataHaven node based on Substrate. The "Operator" in EigenLayer terms.
├── test/ # Integration tests for the AVS and Operator.
├── resources/ # Miscellaneous resources for the DataHaven project.
└── README.md

E2E CLI

This repo comes with a CLI for launching a local DataHaven network, packaged with:

  1. A full Ethereum network with:
    • 2 x Execution Layer clients (e.g., reth)
    • 2 x Consensus Layer clients (e.g., lodestar)
    • Blockscout Explorer services for EL (if enabled with --blockscout)
    • Dora Explorer service for CL
    • Contracts deployed and configured for the DataHaven network.
  2. A DataHaven solochain.
  3. Snowbridge relayers for cross-chain communication.

To launch the network, follow the instructions in the test README.

Docker

This repo publishes images to DockerHub.

Tip

If you cannot see this repo you must be added to the permission list for the private repo.

To aid with speed it employs the following:

  • sccache: De-facto caching tool to speed up rust builds.
  • cargo chef: A method of caching building the dependencies as a docker layer to cut down compilation times.
  • buildx cache mounts: Using buildx's new feature to mount an externally restored cache into a container.
  • cache dance: Weird workaround (endorsed by docker themselves) to inject caches into containers and return the result back to the CI.

To run a docker image locally (moonsonglabs/datahaven:local), from the /test folder run:

bun build:docker:operator

Working with IDEs

VS Code (and its forks)

IDE configurations are ignored from this repo's version control, to allow for personalisation. However, there are a few key configurations that we suggest for a better experience. Here are the key suggested configurations to add to your .vscode/settings.json file:

Rust

{
  "rust-analyzer.linkedProjects": ["./operator/Cargo.toml"],
  "rust-analyzer.cargo.allTargets": true,
  "rust-analyzer.procMacro.enable": false,
  "rust-analyzer.server.extraEnv": {
    "CARGO_TARGET_DIR": "target/.rust-analyzer",
    "SKIP_WASM_BUILD": 1
  },
  "rust-analyzer.diagnostics.disabled": ["unresolved-macro-call"],
  "rust-analyzer.cargo.buildScripts.enable": false
}

These settings optimise Rust Analyzer for the DataHaven codebase:

  • Marks the operator/ folder as a linked project for analysis. The root of this repo is a workspace, and this is the rust project that should be analysed by rust-analyzer.
  • Disables proc macros and build scripts to improve performance. Otherwise, Substrate's proc macros will make iterative checks from rust-analyzer unbearably slow.
  • Sets a dedicated target directory for Rust Analyzer to avoid conflicts with other build targets like release builds.
  • Disables WASM builds during analysis for faster feedback.

Solidity

For Juan Blanco's Solidity Extension, add the following to your .vscode/settings.json file:

{
  "solidity.formatter": "forge",
  "solidity.compileUsingRemoteVersion": "v0.8.28+commit.7893614a",
  "[solidity]": {
    "editor.defaultFormatter": "JuanBlanco.solidity"
  }
}

These settings configure Solidity support:

  • Uses Forge as the formatter for consistency with the project's tooling.
  • Sets a specific Solidity version for compilation. This one should match the version used in foundry.toml.
  • Sets the Solidity extension as the default formatter.

Typescript

This repo uses Biome for TypeScript linting and formatting. To make the extension work nicely with this repo, add the following to your .vscode/settings.json file:

{
  "biome.lsp.bin": "test/node_modules/.bin/biome",
  "[typescript]": {
    "editor.defaultFormatter": "biomejs.biome",
    "editor.codeActionsOnSave": {
      "source.organizeImports.biome": "always"
    }
  }
}
  • Sets the Biome binary to the one in the test/ folder.
  • Sets Biome as the default formatter for TypeScript.
  • Sets Biome to always organise imports on save.

CI

Using the act binary, you can run GitHub Actions locally.

For example, to run the entire e2e workflow:

act -W .github/workflows/e2e.yml -s GITHUB_TOKEN="$(gh auth token)"

Which results in:

INFO[0000] Using docker host 'unix:///var/run/docker.sock', and daemon socket 'unix:///var/run/docker.sock'
INFO[0000] Start server on http://192.168.1.97:34567
[E2E - Kurtosis Deploy and Verify/kurtosis] ⭐ Run Set up job
[E2E - Kurtosis Deploy and Verify/kurtosis] 🚀  Start image=catthehacker/ubuntu:rust-24.04
[E2E - Kurtosis Deploy and Verify/kurtosis]   🐳  docker pull image=catthehacker/ubuntu:rust-24.04 platform= username= forcePull=true
[E2E - Kurtosis Deploy and Verify/kurtosis] using DockerAuthConfig authentication for docker pull
[E2E - Kurtosis Deploy and Verify/kurtosis]   🐳  docker create image=catthehacker/ubuntu:rust-24.04 platform= entrypoint=["tail" "-f" "/dev/null"] cmd=[] network="host"
[E2E - Kurtosis Deploy and Verify/kurtosis]   🐳  docker run image=catthehacker/ubuntu:rust-24.04 platform= entrypoint=["tail" "-f" "/dev/null"] cmd=[] network="host"
[E2E - Kurtosis Deploy and Verify/kurtosis]   🐳  docker exec cmd=[node --no-warnings -e console.log(process.execPath)] user= workdir=
[E2E - Kurtosis Deploy and Verify/kurtosis]   ✅  Success - Set up job
[E2E - Kurtosis Deploy and Verify/kurtosis]   ☁  git clone 'https://github.com/oven-sh/setup-bun' # ref=v2
...
[E2E - Kurtosis Deploy and Verify/kurtosis]   ✅  Success - Post Install Foundry [212.864597ms]
[E2E - Kurtosis Deploy and Verify/kurtosis] ⭐ Run Complete job
[E2E - Kurtosis Deploy and Verify/kurtosis] Cleaning up container for job kurtosis
[E2E - Kurtosis Deploy and Verify/kurtosis]   ✅  Success - Complete job
[E2E - Kurtosis Deploy and Verify/kurtosis] 🏁  Job succeeded