This PR upgrades the StorageHub dependencies to tag
[v0.0.6-alpha](https://github.com/Moonsong-Labs/storage-hub/releases/tag/v0.0.6-alpha).
This includes the fix to connect through TLS to a Postgres DB (allowing
connecting to an AWS hosted DB for instance), and a fix for a missing
indexer DB migration.
Additionally, it adds a new runtime API.
EDIT (previously breaking changes):
As of the new version, the name of a column in the indexer DB has
changed. This can affect the functionality of nodes running a Postgres
DB with the old schema. A
[migration](05d269a26d)
is included in the new
[tag](https://github.com/Moonsong-Labs/storage-hub/commits/v0.0.6-alpha/),
so no need to mark it as breaking.
---------
Co-authored-by: Ahmad Kaouk <56095276+ahmadkaouk@users.noreply.github.com>
Co-authored-by: Gonza Montiel <gonzamontiel@users.noreply.github.com>
Co-authored-by: Steve Degosserie <723552+stiiifff@users.noreply.github.com>
## Summary
- rename the FRAME alias for `pallet_evm` from `Evm` to `EVM` across the
mainnet, stagenet, and testnet runtimes
- adjust benchmarks, configuration modules, genesis builders, and
runtime tests to rely on the new alias
- keep precompile genesis setup and proxy/precompile tests aligned with
the updated names
## Context
Frontier’s `StorageOverrideHandler` (see
`fc_storage::StorageQuerier::account_code`) reads contract bytecode from
`pallet_evm::AccountCodes` using the constant `PALLET_EVM = b"EVM"` to
build the storage key:
`twox_128("EVM") ++ twox_128("AccountCodes") ++ …`
Our runtimes exported `pallet_evm` as `Evm`, so substrate stored
bytecode under the *camel-cased* prefix (`twox_128("Evm")`). Every call
that ultimately hits the storage override—including `eth_getCode`,
`eth_call`, and state queries during replay—therefore failed to locate
code for *any* account (deployed contracts and precompiles alike).
Renaming the alias to `EVM` realigns the storage prefix with Frontier’s
expectations so the override layers can pull bytecode correctly.
## Testing
- `cargo check -p datahaven-node`
- `cargo build --release -p datahaven-node`
- `eth_getCode 0x0000000000000000000000000000000000000802` → returns
`0x60006000fd`
## Storage Migration
Renaming a pallet alias changes the storage prefix for all pallet data.
Without migration, existing EVM data (smart contracts, account codes,
storage) would become inaccessible.
**Migration details:**
- **Type**: Multi-Block Migration (MBM)
- **Storage migrated**: `AccountCodes`, `AccountCodesMetadata`,
`AccountStorages`
- **Migration ID**: `datahaven-evm-mbm` (version 0 → 1)
**Testing the migration:**
```bash
# Build runtime with try-runtime
cargo build --release --features try-runtime -p
datahaven-stagenet-runtime
# Test against stagenet
try-runtime \
--runtime
./target/release/wbuild/datahaven-stagenet-runtime/datahaven_stagenet_runtime.wasm
\
on-runtime-upgrade \
--blocktime 6000 \
--checks all \
--disable-spec-version-check \
live --uri wss://dh-validator-0.datahaven-kt.xyz
```
Test results from stagenet:
- ✅ Migration completes in 1 block
- ✅ PoV size: ~5.3 KB
- ✅ Weight consumption: <0.1% of block capacity
- ✅ All 39 keys successfully migrated
## ⚠️ Breaking Changes ⚠️
If you are manually computing storage keys for the EVM pallet (e.g., directly querying chain state), you must update your code to use the new storage prefix:
- Old prefix: twox128("Evm") = 0x8b90cb...
- New prefix: twox128("EVM") = 0x6a5e91...
All EVM-facing interfaces remain unchanged.
This PR restructures and port the block validation test suite from
Moonbeam and add the necessary infrastructure for contract-based
testing.
### Test Suites Added
**Block Validation Suite 1** (`test-block-1.ts`) - *Refactored from
original `test-block.ts`*
- T01: Validates block number increments correctly after manual block
creation
- T02: Checks block timestamps are valid and within expected bounds
- T03: Verifies complete block structure including gasLimit, difficulty,
receiptsRoot, transactionsRoot, logsBloom, and other Ethereum-compatible
fields
- T04: Confirms blocks are retrievable by hash
- T05: Confirms blocks are retrievable by number
**Block Validation Suite 2** (`test-block-2.ts`) - *New*
- T01: Verifies block number persistence across test cases
- T02: Validates parent-child block hash linkage in the chain
**Block Gas Limits Suite** (`test-block-gas.ts`) - *New*
- T01-T06: Tests all three transaction types (legacy, eip1559, eip2930)
can deploy contracts at max extrinsic gas limit, and correctly reject
transactions exceeding that limit
- T07: Deploys `BlockVariables` contract and verifies runtime gas limit
is accessible from within contract execution
**Block Genesis Suite** (`test-block-genesis.ts`) - *New*
- T01: Validates genesis block (block 0) contains correct
Ethereum-compatible structure and empty transaction/uncle lists
- T02: Confirms genesis block is retrievable by hash
### Infrastructure Additions
**Contract Deployment Helpers**
- `fetchCompiledContract`: Loads compiled Solidity artifacts with ABI
and bytecode from JSON output
- `deployCompiledContract`: Handles contract deployment with support for
legacy, EIP-1559, and EIP-2930 transaction types, including gas
parameter configuration
**Solidity Test Fixtures**
- `BlockVariables.sol`: Exposes `block.gaslimit`, `block.chainid`,
`block.number` via view functions for runtime validation
- `Fibonacci.sol`: Provides `fib2(n)` pure function for computational
gas testing
- `MultiplyBy7.sol`: Minimal pure function contract for basic deployment
testing
**Enhanced Runtime Constants**
- Added `EXTRINSIC_GAS_LIMIT` (52M), `BLOCK_WEIGHT_LIMIT` (2T),
`MAX_POV_SIZE` (10MB) for all three environments
- Constants derived from and documented against Rust runtime
configuration in `operator/runtime/*/src/lib.rs`
### Dependencies
- Added `solc@0.8.30` and supporting packages (`yargs`,
`command-exists`, `js-sha3`, `memorystream`) for local Solidity
compilation
## Add E2E validator-set update flow
- feat: `test/utils/validators.ts` for on-demand validator
orchestration.
- feat: `test/suites/validator-set-update.test.ts` covering allowlist →
register → update.
- some minor launcher updates: avoid docker cache, add `--platform` when
building datahaven image, avoid sending validator-set update on launch.
- Helpers: ABI shortcut in `test/utils/contracts.ts`; config tweaks in
`test/configs/validator-set.json`.
- Minor cleanup/formatting across `test/launcher/*`,
`test/scripts/setup-validators.ts`, and related tests.
- added `keepAlive` flag to `BaseTestSuite`, in order to avoid tearing
down the network while debugging. Defaults, obviously, to false.
- added a `failOnTomeout` option on to waitForDataHavenEvents() so the
test fails of the timeout is reached and no event was captured.
### Coverage
- The test simulates an scenario in which we have two active authorities
(alice and bob), which are running, and registered as operators, which
is the normal state after the chain launches. Then:
- It launches two more nodes (charlie and dave)
- It add the nodes to allowlist and register them as operators
- It sends the validator set update message
- Checks that the validator update message was propagated through the
gateway and arrived the external-validators pallet
- Checks that the chain continues producing blocks
### Notes
The last test case has a timeout of 10 minutes. This is to respect
propagation times of the message through the relayers. We are testing
that the external validators pallet actually updated the validator set.
Locally, I could expect 5~6 minutes, I just wanted to be on the safe
side. CI is passing showing that this was enough indeed.
---------
Co-authored-by: Steve Degosserie <723552+stiiifff@users.noreply.github.com>
Co-authored-by: Ahmad Kaouk <56095276+ahmadkaouk@users.noreply.github.com>
### Description
This PR introduces the **Moonwall** end-to-end (E2E) testing framework.
The primary motivation for this is to enable the porting of existing
Mobeam tests into the `DataHaven` repository.
### Key Changes
* **Node Manual Sealing:**
* Introduced a `--sealing=manual` flag for the `datahaven-node`. When
enabled, blocks are only produced on demand via an RPC call. This is the
core mechanism that allows for deterministic tests.
* **Moonwall Framework Integration:**
* Added `@moonwall/cli` and `@moonwall/util` dependencies to the
`test/package.json`.
* A new `test/moonwall.config.json` file configures the test
environment, defining how Moonwall should launch the `datahaven-node`
with the manual sealing flag.
* Added a `moonwall:test` script to `package.json` for running the
tests.
* **CI Workflow:**
* A new reusable workflow, `.github/workflows/task-moonwall-tests.yml`,
has been created to handle the setup, execution, and reporting of
Moonwall tests.
* The main `CI.yml` now includes a `moonwall-tests` job that runs after
the `build-operator` job, ensuring it always tests the correct,
freshly-built binary.
* **Example Test Suite:**
* A new test suite, `test/datahaven/suites/dev/test-block.ts`, had been
copied from moonbeam.
### How to Run Locally
1. Navigate to the `test` directory.
2. Install dependencies: `bun install`
3. Run the tests: `bun run moonwall:test`
---------
Co-authored-by: undercover-cactus <lola@moonsonglabs.com>
In this PR we update StorageHub to its latest tage `v0.0.4-alpha`. It
includes so minor changes on the FilesystemAPI.
Its also force to specify the `maintenance_mode` options for fisherman.
For now we are not allowing this mode so we just skip entirely.
---------
Co-authored-by: Steve Degosserie <723552+stiiifff@users.noreply.github.com>
## Overview
This PR integrates Substrate's `pallet-migrations` into all DataHaven
runtimes (mainnet, stagenet, testnet) to enable robust multi-block
migration capabilities. This infrastructure allows complex runtime
upgrades to be executed across multiple blocks while maintaining chain
stability and providing governance controls.
## What Changed
### Core Integration
- **Added `pallet-migrations` dependency** across all runtime
configurations
- **Integrated migration pallet** as pallet index 39 in all runtimes
- **Created shared migration configuration** in
`datahaven-runtime-common`
### Runtime Configuration
- **Mainnet, Stagenet, and Testnet** now include identical migration
configurations
- **MaxServiceWeight** parameter set to 75% of max block weight for safe
migration execution
- **Migration cursor limits** configured (65KB max cursor, 256B max
identifier)
- **Failure handling** configured to freeze the chain on migration
failures (similar to Moonbeam's maintenance mode)
## Future Work
- [ ] Add custom failure handler (safe mode) to replace chain freeze
- [ ] Generate DataHaven-specific benchmarks for migration weights
---------
Co-authored-by: undercover-cactus <lola@moonsonglabs.com>
### PR Description
Add a comprehensive end-to-end test that validates rewards distribution
across the full system (chain → bridge → execution environment).
### Use cases covered
- Verify the rewards infrastructure is correctly deployed and reachable.
- Detect the end-of-era rewards emission and capture its essential data.
- Confirm the cross-chain delivery and execution of the rewards message.
- Ensure the rewards registry updates with the new root and can be
queried.
- Generate per-validator proofs for claiming rewards.
- Successfully claim rewards for a validator and validate the payout is
reflected.
- Prevent a second (double) claim for the same index with a proper
rejection.
---------
Co-authored-by: Steve Degosserie <723552+stiiifff@users.noreply.github.com>
## Summary
This PR resolves all CI failures following the migration to self-hosted
GitHub runners (`DH-Testing` group) by eliminating sudo dependencies and
fixing Docker connectivity issues.
## Key Changes
### 🔧 **Eliminated sudo requirements across all workflows**
- **Setup Environment**: Installed mold linker and system dependencies
in userspace without sudo
- **Tool Installation**: Replaced apt/system package installations with
direct binary downloads:
- Kurtosis: Direct binary download from GitHub releases (v1.10.3)
- Taplo: Direct binary installation for Cargo.toml formatting
- cargo-nextest: Using `cargo install` instead of GitHub action
(v0.9.100)
- **Runner Cleanup**: Skipped cleanup-runner action entirely on
self-hosted runners (bare-metal manages disk space externally)
### 🐳 **Fixed Docker connectivity for E2E tests**
- **Enhanced dockerode configuration** with robust fallback logic for
different socket locations
- **Added DOCKER_HOST environment variable** to E2E workflow for
consistent Docker daemon access
- **Implemented connection testing** with detailed error diagnostics for
troubleshooting
- **Resolves FailedToOpenSocket errors** by supporting multiple socket
paths and connection methods
### 🏷️ **Workflow optimizations**
- **Label-based targeting**: All heavy workloads (Rust builds, E2E
tests) now run on `DH-Testing` runners
- **Dependency management**: Used `install-deps: false` flag instead of
hardcoded runner detection
- **Permission fixes**: Corrected Docker build permissions and GHCR
organization names
---------
Co-authored-by: Claude <noreply@anthropic.com>
This PR updates the project dependencies as follows:
- **Polkadot SDK**: from the `stable2412` (moving) branch to the
`polkadot-stable2412-6` (fixed) tag.
- **StorageHub**: to revision
`f8281283b6003a3009a32431ed0f3cd628561d6b`, which also depends on
Polkadot SDK `polkadot-stable2412-6`.
- **Frontier**: revision `75329a2df49e2cc7981485392c31160929d1bd48n`
which, likewise, depends on Polkadot SDK `polkadot-stable2412-6`.
## Key changes
- Add CLI command to update AVS metadata URI on-chain via
`updateAVSMetadataURI` function. Use:
- `bun cli contracts update-metadata --chain <...> --uri <...>`
- Support for multiple chains (tipicaly holesky, hoodi, or a mainnet)
- Tx confirmation and gas usage reporting
---------
Co-authored-by: Steve Degosserie <723552+stiiifff@users.noreply.github.com>
## Summary
This PR resolves all CI failures following the migration to the new
DataHaven Github & Docker Hub organizations, and correctly leverage
self-hosted GitHub runners (`DH-runners` group) by eliminating sudo
dependencies.
## Key Changes
### 🔧 **Eliminated sudo requirements across all workflows**
- **Setup Environment**: Installed mold linker and system dependencies
in userspace without sudo
- **Tool Installation**: Replaced apt/system package installations with
direct binary downloads:
- Kurtosis: Direct binary download from GitHub releases (v1.10.3)
- Taplo: Direct binary installation for Cargo.toml formatting
- cargo-nextest: Using `cargo install` instead of GitHub action
(v0.9.100)
- **Runner Cleanup**: Skipped cleanup-runner action entirely on
self-hosted runners (bare-metal manages disk space externally)
### 🏷️ **Workflow optimizations**
- **Group-based targeting**: All heavy workloads (Rust builds, E2E
tests) now run on `DH-runners` runners
- **Dependency management**: Used `install-deps: false` flag instead of
hardcoded runner detection
Co-authored-by: Claude <noreply@anthropic.com>
### Summary
- **Add** `test/suites/native-token-transfer.test.ts` focused on the
HAVE native token lifecycle via Snowbridge v2.
- **Validate** registration, DataHaven → Ethereum mints, Ethereum →
DataHaven unlocks, event emission, and 1:1 backing invariant.
### Tests added
- should register DataHaven native token on Ethereum
- should transfer tokens from DataHaven to Ethereum
- should maintain 1:1 backing ratio
- should emit transfer events
- should transfer tokens from Ethereum to DataHaven (Snowbridge v2)
### What the suite covers
- **Registration**: Sudo-registers the native token; confirms
`ForeignTokenRegistered` on the Gateway; verifies ERC-20 metadata
(`HAVE`/`wHAVE`, 18 decimals).
- **DataHaven → Ethereum**: Executes `transfer_to_ethereum`; asserts
Substrate events (`TokensLocked`, `TokensTransferredToEthereum`);
observes Ethereum `Transfer` mint (from zero address); validates sender
balance delta, sovereign account increase, and ERC-20 recipient credit.
- **Backing invariant**: Ensures sovereign account balance ≥ ERC-20
total supply.
- **Event emission**: Confirms key Substrate events without polling
delays.
- **Ethereum → DataHaven**: Approves and calls `Gateway.sendToken`; if
unsupported locally, the test skips; otherwise asserts burn on Ethereum
and unlock on DataHaven with corresponding balance deltas.
---------
Co-authored-by: Steve Degosserie <723552+stiiifff@users.noreply.github.com>
## Summary
This PR introduces support for deploying Datahaven contracts to
different chains (hoodi, holesky, mainnet), as well as a new cli command
to manage this deployment separately from the regular deployment, while
maintaining compatibility with it.
#### New CLI command
- **`bun cli contracts deploy`** - Deploy contracts to supported chains
(Hoodi, Holesky, Mainnet)
- **`bun cli contracts status`** - Check deployment configuration and
status
- **`bun cli contracts verify`** - Verify contracts on block explorers
- Commands need the chain parameter: `--chain <hoodi | holesky |
mainnet>`
- Right now only `hoodi` and `holesky` are supported
### Deployment
#### Hoodi & Holesky Network Support
- Added **DeployBase.s.sol** as common ground for
**DeployTestnet.s.sol** (also new) and **DeployLocal.s.sol** (existing).
- **Hoodi configuration** (`contracts/config/hoodi.json`) with deployed
EigenLayer contract addresses to reference.
- **Holesky configuration** (`contracts/config/hoodi.json`) with
deployed EigenLayer contract addresses to reference.
#### Contracts being deployed
- **DataHaven**: ServiceManager, VetoableSlasher, RewardsRegistry
- **Snowbridge**: BeefyClient, AgentExecutor, Gateway, RewardsAgent
- **EigenLayer**: References existing deployed contracts (not
re-deployed)
#### Deployment files
When the deployment is done, a new file under `contracts/deployments` is
generated with the addresses of the deployed contracts, for each chain
(it will be overriden per chain if run multiple times). So we would have
one `anvil.json`, `hoodi.json`, `holesky.json`, etc, with the addresses
of the deployed contracts for reference and for later verification.
#### Todo
- [x] Test compatibility with existing `bun cli launch` and `bun cli
deploy` commands
#### For follow-up PRs
- Fix verification issue with `foundry verify-contracts` when specifying
the `chain` or `chain-id` parameter, needed for hoodi
(https://github.com/foundry-rs/foundry/issues/7466).
- Add `redeploy` feature to only override implementation contract and
leave the proxy address untouched
## Usage Examples
```bash
# Deploy to Hoodi network
bun cli contracts deploy --chain hoodi
# Check deployment status
bun cli contracts status --chain hoodi
# Verify contracts on block explorer
bun cli contracts verify --chain hoodi
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Added deployment and configuration support for new networks "hoodi"
and "holesky", including new configuration and deployment files.
* Introduced a CLI tool for managing contract deployments, status
checks, and verification across supported chains.
* Added example environment configuration and comprehensive deployment
documentation.
* Enabled contract verification and status reporting via the CLI with
support for block explorer integration.
* **Improvements**
* Refactored deployment scripts for modularity, supporting both local
and testnet environments.
* Centralized and extended configuration loading to support additional
contract addresses and network parameters.
* Enhanced deployment utilities and typings to support multi-network
deployments.
* **Bug Fixes**
* Improved input validation and error handling in CLI commands and
deployment scripts.
* Added explicit handling for zero address in operator strategy
retrieval.
* **Chores**
* Updated documentation and configuration templates for easier
onboarding and deployment management.
* Improved logging and output formatting for deployment and verification
processes.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
---------
Co-authored-by: Steve Degosserie <723552+stiiifff@users.noreply.github.com>
Co-authored-by: Ahmad Kaouk <56095276+ahmadkaouk@users.noreply.github.com>
## Summary
This PR integrates the Substrate Proxy pallet into DataHaven runtimes
(testnet, stagenet, mainnet) with comprehensive test coverage. The proxy
pallet enables account delegation functionality, allowing accounts to
authorize other accounts to execute calls on their behalf with
configurable permissions.
## Changes
### Proxy Pallet Integration
- **Added proxy pallet** to all three DataHaven runtimes (testnet,
stagenet, mainnet)
- **Configured custom ProxyType enum** with DataHaven-specific proxy
types:
- `Any` - Unrestricted proxy access
- `NonTransfer` - All calls except balance transfers
- `Governance` - Governance and utility calls only
- `Staking` - Staking operations (placeholder)
- `CancelProxy` - Proxy announcement cancellation
- `Balances` - Balance transfer operations only
- `IdentityJudgement` - Identity judgement operations
- `SudoOnly` - Privileged sudo operations only
### Runtime Configuration
- **Integrated pallet-proxy** into runtime construction macros
- **Configured proxy deposits** (base + per-proxy fees)
- **Set proxy limits** (maximum proxies per account)
- **Implemented InstanceFilter** for call filtering per proxy type
- **Added proxy pallet to runtime APIs** and metadata
### Comprehensive Test Suite
- `operator/runtime/testnet/tests/proxy.rs` - 24 comprehensive proxy
tests
- `operator/runtime/stagenet/tests/proxy.rs` - 24 comprehensive proxy
tests
- `operator/runtime/mainnet/tests/proxy.rs` - 24 comprehensive proxy
tests
- Updated `lib.rs` files in all three runtimes to include proxy test
modules
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- New Features
- Enable account proxies across mainnet, stagenet, and testnet with
configurable types, delays, and per-type call permissions.
- Support anonymous (pure) proxies, proxy announcements with delays, and
deposit/limit parameters for proxy management.
- Tests
- Add comprehensive integration tests covering proxy lifecycle,
filtering, pure proxies, announcements, batching, chaining, multisig,
identity, and sudo paths.
- Test builder now supports optional sudo setup.
- Chores
- Add benchmarking, weights, and try-runtime support for proxies.
- Update internal package metadata version.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
---------
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Ahmad Kaouk <ahmadkaouk.93@gmail.com>
Co-authored-by: Ahmad Kaouk <56095276+ahmadkaouk@users.noreply.github.com>
This PR enhances the Helm-based deployment system with several key
improvements organized into the following areas:
## New Features
### Ingress Management
- **Ingress per replica**: Added `ingress-per-replica` chart template
that automatically generates an ingress for each node replica, exposing
individual pod instances
- **Traefik integration**: Added chart values to deploy Traefik as the
ingress controller for local K8s clusters, enabling proper ingress
testing (requires adding names to `/etc/hosts`)
### Solochain Support
- **New relay chart**: Added dedicated Helm chart for Solochain relay
deployment
- **CLI integration**: Updated DataHaven CLI to deploy both Execution
and Solochain relayers
## Configuration Improvements
### Environment Structure
- **Modular configs**: Refactored environmental configurations from
single `values.yaml` files into separate component-specific overrides
for better organization
### Node Configuration
- **Base config updates**: Improved base configurations for bootnode and
validator nodes
- **Network protocol**: Reverted from litep2p back to libp2p to resolve
node communication issues on Stagenet
- **Archive node routing**: In Stagenet, relayers now connect to the
bootnode (configured as archive node) instead of validator nodes
## Storage & Deployment
### Persistent Storage
- **Relayer database**: Added support for persistent volumes to store
relayer databases instead of using ephemeral storage
### Deployment Scripts
- **Documentation cleanup**: Removed obsolete test deploy.sh script and
updated the deployment README with clearer instructions
These changes provide a more robust, scalable, and maintainable
deployment system for DataHaven infrastructure.
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Added support for deploying new relayer types: Solochain and Execution
relayers, with dedicated configuration and secret management.
* Introduced per-replica ingress configuration for node deployments,
allowing each replica to have its own ingress resource and hostname.
* Added persistent storage options for relay data, configurable via
storage path, class, and size.
* Added new deployment configuration files for local and stagenet
environments, including Traefik ingress controller setup.
* Introduced a new relay category, "Solochain Relayers," for standalone
chain operations and cross-chain communication.
* **Improvements**
* Updated deployment configurations to use container-specific
environment YAML files for more granular control.
* Enhanced relay and node configurations with new flags and backend
options, including dynamic peer ID generation and automatic bootnode
discovery.
* Updated relayer endpoints to consistently use the bootnode for
connections.
* Refined relay configuration files for improved structure, clarity, and
endpoint management.
* **Bug Fixes**
* Corrected deployment logic to reference the correct
environment-specific configuration files during Helm deployments.
* **Documentation**
* Simplified and updated deployment documentation to focus on CLI-based
deployment, removing outdated manual instructions and adding a concise
overview of components and relayer types.
* **Chores**
* Removed deprecated deployment scripts and outdated configuration files
to streamline the deployment process.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
---------
Co-authored-by: Claude <noreply@anthropic.com>
* Add `run_benchmarks.sh` script to run runtime benchmarks
* Sets up benchmark configs and directory structure to store weights
(`operator/runtime/<RUNTIME>/weights`)
* (naive) fixes to some benchmarks:
* `pallet_datahaven_native_transfer`:
* use a mock for `NativeTokenId`
* look at the balance difference of the treasury instead of the total
(this makes the benchmark agnostic to genesis setup)
* `snowbridge_pallet_system` / `snowbridge_pallet_system_v2` use native
token xcm location vs relay chain one. Add missing benchmark methods and
update fixture with valid data.
* `snowbridge_pallet_ethereum_client`: update fixtures with valid data
* `snowbrige_pallet_inbound_queue_v2`: set EthereumGatewayAddress when
initializing storage on benchmark and use a mock message processor ( as
fixture has `CreateAsset` payload which is not supported in the
`EigenLayerMessageProcessor`)
* `snowbridge_pallet_outbound_queue_v2`: add missing
`submit_delivery_receipt` benchmark which required a dedicated fixture
(all copied from the upstream pallet)
* `pallet_treasury`: Use an `ExistentialDeposit` of `1` on benchmark,
else payout fails.
* `pallet_transaction_payment`: Use a custom `WeightToFee` that makes
the Fee small, else account in benchmark cannot pay for fees (It is
funded a multiplier of `ExistentialDeposit` and is expected for that to
be enough, but it's not in our particular setup).
* comment out `pallet_identity` and `pallet_im_online` due to
incompatibilities (to be addressed later)
* Basic benchmark run to set `WeightInfo` from `weights` in configs
(real run should be done later using target hardware)
---------
Co-authored-by: Ahmad Kaouk <ahmadkaouk.93@gmail.com>
Co-authored-by: Tobi Demeco <50408393+TDemeco@users.noreply.github.com>
Co-authored-by: undercover-cactus <lola@moonsonglabs.com>
Co-authored-by: TDemeco <tdemeco@itba.edu.ar>
## Summary
This PR introduces comprehensive event waiting utilities for both
DataHaven (Substrate) and Ethereum chains, providing a unified
interface for handling blockchain events in E2E tests.
## What's New
- **Event Utilities** (`test/utils/events.ts`): New utilities for
waiting on blockchain events
- `waitForDataHavenEvent`: Type-safe event waiting for Substrate chain
events
- `waitForEthereumEvent`: Event waiting for Ethereum contract events
- Graceful timeout handling (returns null instead of throwing)
- Support for event filtering, callbacks, and custom timeouts
- **Documentation** (`test/docs/event-utilities-guide.md`):
Comprehensive guide covering usage examples for both DataHaven and
Ethereum.
## Test Plan
- [ ] New event utilities work as expected
- [ ] Event filtering works correctly for both chains
- [ ] Timeout handling behaves as documented
- [ ] Parallel event waiting with `Promise.all()` works
---------
Co-authored-by: Claude <noreply@anthropic.com>
## Summary
This PR add some error example for troubleshooting and an instruction to
purge minikube (can be useful to restart fresh).
It also replaces the command `kurtosis cluster set docker.k8s` by
`kurtosis cluster set <pick-a-cluster-option>` to force the reader to
actually read the different options and not just copy/paste the command
in the terminal. The options here matters to complete the deployment.
### Problem
Introducing `--network` should make easy to container nodes to find each
other. But this change was made half-way for the relayers, and it was
using the external port to find the first datahaven node (usually
Alice). So:
- In cli launch, Alice node port mapping was left to random port `-p
9944` instead of `-p 9944:9944`.
- Relayers couldn't connect to DataHaven nodes because they were using
the external WS port (now random) instead of hitting the internal port
(which for a cli launch we actually fix it to 9944).
### Solution
- [x] **Fixed Docker port mapping**: Explicit `-p 9944:9944` for Alice
node under network `cli-launch`
- [x] **Enhanced container spec**: Added `internalPorts` tracking to
`LaunchedNetwork`
- [x] **Fixed relayer connections**: Use internal ports for container
communication
## Implement E2E Testing Framework with Isolated Networks
### Summary
Refactors the existing E2E testing infrastructure to provide isolated
test environments with parallel execution support. Each test suite now
runs in its own network namespace, preventing resource conflicts.
### Key Changes
- **New Testing Framework** (`test/framework/`): Base classes for test
lifecycle management with automatic setup/teardown
- **Launcher Module** (`test/launcher/`): Extracted network
orchestration logic from CLI handlers for reusability
- **Parallel Execution**: Added `test-parallel.ts` script with
concurrency limits to prevent resource exhaustion
- **Test Isolation**: Each suite gets unique network IDs (format:
`suiteName-timestamp`) and Docker networks
- **Improved Test Organization**: Migrated tests to new framework,
deprecated old test structure
### Test Improvements
- Added 4 new test suites demonstrating framework usage. :
- `contracts.test.ts` - Smart contract deployment/interaction
- `datahaven-substrate.test.ts` - Substrate API operations
- `cross-chain.test.ts` - Snowbridge cross-chain messaging
- `ethereum-basic.test.ts` - Ethereum network operations
> [!WARNING]
The test suites themselves are bad and shouldn't be consider examples of
good tests. They were AI generated just to test the concurrency of test
runners
### Documentation
- Added comprehensive framework overview (`E2E_FRAMEWORK_OVERVIEW.md`)
- Updated README with parallel testing commands
- Added test patterns and best practices
### Breaking Changes
- Old test suites moved to `e2e - DEPRECATED/` directory
- Test execution now requires extending `BaseTestSuite` class
### Testing
Run tests with: `bun test:e2e` or `bun test:e2e:parallel` (with
concurrency limits)
### TODO
- [ ] Implement good test examples.
- [ ] Implement useful test utils (like waiting for an event to show up
in DataHaven or Ethereum).
- [ ] Enforce tests with CI (currently cannot be done due to
intermittent error when sending a transaction with PAPI).
---------
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: undercover-cactus <lola@moonsonglabs.com>
## Summary of changes
- We decided to remove the topics and nonce from the massage encoding
since we don't use them (original commit:
ee2a3f2fd4).
- Besides, we already have a nonce at the Snowbridge message level
f4ab5c2b2e/operator/primitives/snowbridge/inbound-queue/src/v2/message.rs (L105)
- I had to recreate the static test for _encoding_ (happens in
[DataHavenSnowbridgeMessages.sol](d12d40634f/contracts/src/libraries/DataHavenSnowbridgeMessages.sol)
) / _decoding_ (happens in
[operator/primitives/bridge/src/lib.rs)](f9f9cc65fe/operator/primitives/bridge/src/lib.rs).
Now it matches the current structure. The idea is that now we can test
that we don't break the decoding in followup refactoring.
- Fixes a problem with EigenLayer validator addresses. In all our
contracts we were using `bytes32` to refer to a Solochain validator
address. But on our Substrate change we actually expect AccountId20, so
only 20 bytes. This was causing the decoding to fail.
- I opted for the minimal change that would be to take the right-most 20
bytes to send that to our chain. But we might want aswell to limit our
EigenLayer contracts to be only 20 bytes long. @ahmadkaouk showcase this
[here](92a34c273c)
- Adds a bash script to run the static test. The test will compile the
contracts, run the encoding test, compile the operator, and run the
decoding test. This saves a huge amount of time since we don't need to
run the full e2e setup. The way of running it is the following:
```bash
cd operator/test/scripts
./test_message_encoding.sh
```
- As a consequence of this PR, the execution relayer now works properly.
EDIT:
> [!IMPORTANT]
**We decided to use 20-byte addresses in our contracts**. So what is
stated above is not valid anymore.
The change implies that the mapping from Ethereum addresses to bytes32
addresses now it's a mapping as follows:
dd3ba99ac0/contracts/src/DataHavenServiceManager.sol (L51-L52)
I've updated helper functions, tests, etc to be compliant with this
change. The execution relayer and beefy relayer look stable now.
---------
Co-authored-by: Ahmad Kaouk <ahmadkaouk.93@gmail.com>
Co-authored-by: Ahmad Kaouk <56095276+ahmadkaouk@users.noreply.github.com>
# Description
This PR implements a comprehensive overhaul of the `RewardsRegistry`
contract to maintain complete history of reward merkle roots while
providing index-based claim tracking for operators. The new architecture
enables operators to claim rewards from any historical merkle root
instead of only the latest one. To do so, it:
- Adds the `merkleRootHistory` storage array to the contract, in which
we keep all rewards roots that ever came from the DataHaven side.
- Adds the `operatorClaimedByIndex` storage map to the contract, in
which we keep track, for each validator and root index, if it has
claimed it or not.
- This works even for new validators, since theoretically with this
system you could argue they could claim older roots that they were not a
part of which would be catastrophic, but they could never draft a
correct proof for those to claim them.
- Keeps some of the interface from before the overhaul, to have quick
access to the latest rewards merkle root through `getLatestMerkleRoot()`
and to claim rewards for it with `claimRewards()`. This is because the
expected behaviour is for validators to claim their rewards every era.
- Adds a way to batch claim rewards with `claimRewardsBatch()`. This
function allows a validator to claim rewards for multiple root indices
in one call by providing multiple proofs, useful if the validator has
fallen behind claims and has to catch up, although special care will
have to be taken by it to avoid reaching the gas limit of a transaction.
## Storage Efficiency Analysis
One might think this solution is not as storage-efficient as other
solutions that we can think of (I even had two other alternatives in
mind as well), but a simple back-of-the-envelope calculation gives us
peace of mind that the impact of this solution on the overal state size
of the chain is negligible:
### Assumptions (Worst Case Scenario):
- 1,000 validators (actual estimate for DataHaven: ~50/100 validators)
- 6-hour eras (most-likely scenario, following what Polkadot does:
~24-hour eras)
- Which means 4 merkle root updates per day
### Annual Storage Requirements:
- Merkle Root History: **46,720 bytes/year**
- 4 roots/day × 32 bytes/root × 365 days/year = 46,720 bytes/year
- Operator Claim Tracking: **~1.46 MB/year**
- 1,000 operators × 1 boolean/(operator * root index) × 1 byte/boolean ×
4 root indices/day × 365 days/year = 1,460,000 bytes/year
- **Total: ~1.5 MB/year**
This represents negligible storage overhead compared to the significant
operational benefits gained.
## TODO
Since we want to allow the operators/validators to only have to interact
with the AVS contract (that's why the `claimRewards` functions have the
`onlyAVS` modifier), we still have to:
- [x] Add the required functions to the AVS to allow operators to claim
their rewards.
- [x] Adds comprehensive unit tests for them.
---------
Co-authored-by: Steve Degosserie <723552+stiiifff@users.noreply.github.com>
Co-authored-by: Ahmad Kaouk <56095276+ahmadkaouk@users.noreply.github.com>
Co-authored-by: Ahmad Kaouk <ahmadkaouk.93@gmail.com>
This PR solves an issue where the solochain relayer was fatally crashing
because the DataHaven chain initialized with an empty
`RewardsRegistryAddress` parameter. This caused all initial reward
merkle root update messages to target the zero address before the
parameter could be properly set by the launch command of the CLI. When
the relayer tried to call the update function on the zero address (which
isn't a valid contract), the transactions reverted and crashed the
relayer.
The band-aid fix implemented is to set the actual `RewardsRegistry`
contract address as the default value for the `RewardsRegistryAddress`
parameter (since we know it as it's consistent between our CLI runs).
This ensures initial messages have a valid target contract address from
startup, preventing the fatal crashes.
Again, this is a temporary fix until we implement a more robust solution
for parameter initialization, since it won't hold up if any changes
happen to our contracts.
---------
Co-authored-by: Gonza Montiel <gon.montiel@gmail.com>
- Added a parameter `--all` to `bun cli launch` (now coherent with `bun
cli stop`)
- Equivalent to run `bun cli launch --d --bd --lk --dc --fv --sv --uv
--sp --r --cn`
- Removed `bun start:all` command
- Added a kurtosis cluster type check
- The problem was that we now support deploying and launching the CLI
tool with different types of clusters. If you're deploying, your
kurtosis cluster most likely needs to point to a `kubernetes` type
cluster, if you're running `bun cli launch` you need to use native
docker containers. This PR adds a check and warns the user to point to
the right config.
- *Disclaimer*: we don't know the cluster name of the user so we can't
force it to be anything 🫠
This PR contains improvements to the DataHaven deployment
infrastructure:
1. Directory restructure: Moved from `deployment/` to `deploy/` (more
common for K8s / Helm -based deployment configs).
2. Added **local environment** support: updated CLI to support deploying
to a local K8s cluster.
3. Manual deployment script: `deploy/scripts/deploy.sh` for manual
deployments.
4. Environment-specific configurations: Structured values files for each
environment.
5. Chart organization: Renamed bridges-common-relay to relay for
clarity.
---------
Co-authored-by: Gonza Montiel <gon.montiel@gmail.com>
Co-authored-by: Gonza Montiel <gonzamontiel@users.noreply.github.com>
This PR integrates the Substrate FRAME Treasury pallet across all three
DataHaven runtime environments (`testnet`, `mainnet`, `stagenet`) with a
custom fee allocation mechanism and comprehensive test coverage.
### Key Changes
#### Treasury Pallet Integration:
- Added Treasury pallet to all three runtimes (testnet, mainnet,
stagenet) with 20% fee allocation and 80% burn mechanism.
- Implemented dynamic fee proportion control via
`FeesTreasuryProportion` runtime parameter.
- Integrated treasury with custom fee handlers:
`DealWithEthereumBaseFees`, `DealWithEthereumPriorityFees`, and
`DealWithSubstrateFeesAndTip`.
#### Comprehensive Testing Infrastructure:
- Created robust test architecture with session management and validator
setup across all runtimes
- Added block author management utilities for treasury testing requiring
pallet_authorship integration
- Implemented 15 total tests (5 tests × 3 runtimes) covering:
- EVM transaction fee allocation with/without priority fees
- Substrate fee and tip handling validation
- Treasury spending functionality via sudo
- Complete fee flow verification with actual network base fee
calculations
#### Technnical Implementation
Fee Allocation Logic:
- Base fees: 20% to treasury, 80% burned
- Priority fees: 100% to block author
- Substrate tips: 100% to block author
The implementation is based on
https://github.com/moonbeam-foundation/moonbeam/pull/3120 in
[Moonbeam](https://github.com/moonbeam-foundation/moonbeam), and
assisted by Claude Code.
Add CI check for Polkadot-API metadata freshness
This PR adds a new CI workflow that ensures the Polkadot-API metadata
file (`test/.papi/metadata/datahaven.scale`)
is kept up-to-date when runtime changes are made.
Changes:
- Added task-check-metadata.yml workflow that:
- Reuses the WASM artifact from the build-operator job (no duplicate
compilation)
- Runs `bun x papi add` to regenerate metadata
- Fails if the metadata file has uncommitted changes
- Integrated the check into `CI.yml` as a second-tier job alongside
`docker-build`
Why:
- Prevents TypeScript type definitions from becoming out of sync with
the runtime
- Reminds developers to run `bun generate:types:fast` when making
runtime changes
- Ensures consistent type safety across the codebase
The check provides clear error messages with instructions when metadata
is outdated.
