## Summary
Renames the rewardsInitiator state variable, modifier, internal check
function, setRewardsInitiator function, and RewardsInitiatorSet event in
DataHavenServiceManager to their snowbridgeInitiator-prefixed
equivalents, to better reflect the role of this address.
## Motivation
The previous rewardsInitiator naming was misleading — the address
filling this role is specifically the Snowbridge relayer/gateway.
Renaming it end-to-end clarifies intent and aligns the codebase with the
actual architecture.
## Changes
* DataHavenServiceManager.sol: renamed state variable, modifier,
internal check, setRewardsInitiator → -> setSnowbridgeInitiator,
RewardsInitiatorSet -> SnowbridgeInitiatorSet
* IDataHavenServiceManager.sol: updated event, function signature, and
NatSpec
* Deploy scripts & configs: updated field names across all environments
(anvil, testnet, stagenet, mainnet)
## Summary
- **Add `--environment` option to `contracts upgrade` command**,
aligning it with `deploy`, `verify`, and other contracts subcommands.
Deployment file lookups now use `buildNetworkId(chain, environment)`
(e.g. `stagenet-hoodi.json`).
- **Fix ProxyAdmin address written as `address(0)` in deployment JSON
for live deployments.** `_createServiceManagerProxy` now returns the
actual ProxyAdmin so it propagates to `_outputDeployedAddresses`.
- **Fix ProxyAdmin ownership transfer using wrong address in
`DeployLive`.** `transferOwnership` now uses `params.avsOwner` (from
`AVS_OWNER_ADDRESS`) instead of `_avsOwner` (from
`AVS_OWNER_PRIVATE_KEY` with Anvil default fallback).
- **Add ProxyAdmin to the `contracts verify` list** so it is verified on
block explorers.
- **Log AVS owner address** before the proxy upgrade transaction for
signer verification.
- **Handle forge receipt-fetch failures gracefully** during proxy
upgrades — downgrade to a warning with the tx hash instead of a hard
error when the RPC returns null for a receipt.
## Stagenet upgrade to v0.20.0
AVS contracts were upgraded to version v0.20.0 on Stagenet (Hoodi).
Updated contract addresses:
| Contract | Address |
|---|---|
| ServiceManager (Proxy) | `0xED73cCaF067cebC706B2B3a6cf2b9af2c696c6d3`
|
| ServiceManagerImplementation |
`0x0Af4a129D0F3d57B5bD51CAB323EA114C28c064a` |
| ProxyAdmin | `0xeb1a705e1aa96e6a6329d8a8eb0f5ec38eb7b69d` |
| BeefyClient | `0xE65dc4eCA2Fd428361076e1f204731224CeB4292` |
| AgentExecutor | `0x35d3FdCB19A246a1763421168dF69dA3dE207063` |
| Gateway | `0xE9352f1488F12bFEd722c133C129ca5F467463d1` |
| RewardsAgent | `0x2E039a88838241d1Ac738cf2e3C5763ba12571e7` |
| DelegationManager | `0x867837a9722C512e0862d8c2E15b8bE220E8b87d` |
| StrategyManager | `0xeE45e76ddbEDdA2918b8C7E3035cd37Eab3b5D41` |
| AVSDirectory | `0xD58f6844f79eB1fbd9f7091d05f7cb30d3363926` |
| RewardsCoordinator | `0x29e8572678e0c272350aa0b4B8f304E47EBcd5e7` |
| AllocationManager | `0x95a7431400F362F3647a69535C5666cA0133CAA0` |
| PermissionController | `0xdcCF401fD121d8C542E96BC1d0078884422aFAD2` |
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: Ahmad Kaouk <56095276+ahmadkaouk@users.noreply.github.com>
## Summary
This PR rename the `rewardsAgentOrigin`, `rewardsMessageOrigin`, etc...
into a less specific less now that the Snowbrige Agent is also being
used to relay slashing messages.
This PR also have a fix to register the Agent address instead of the
operator node address to check the sender of the message. Without this
fix we could never relay rewards or execute slashing because we would
get an error regarding the message.
## What changed
* Removing the prefix `rewards` everytime we were refering the
snowbridge agent (to clarify that the agent is not only being used by
the reward feature)
* Fix the deployment script to register the `agentAddress` as the
required sender for relaying substrate message
## What is missing
[ ] ~~Rename `onlyRewardsInitiator` and `rewardsInitiator` in the
`DatahavenServiceManager.sol ` for something that would include
slashing~~ This should be done in another PR.
[x] Check the Testnet Deploy script to make sure it is using the agent
address
---------
Co-authored-by: Ahmad Kaouk <56095276+ahmadkaouk@users.noreply.github.com>
## Contracts upgrade command with simple version tracking
This PR aims to take the most minimal changes from #438 to make the
upgrade command available.
So it adds the `bun cli contracts upgrade` command for deploying a new
`DataHavenServiceManager` implementation and upgrading the proxy, and
includes a simple version tracking via a `contracts/VERSION` file.
### Contracts
**`DataHavenServiceManager.sol`**
- Added `_version` storage variable
- Added `DATAHAVEN_VERSION()` view function,
- Added `updateVersion(string)` function gated by `onlyProxyAdmin`
- Added `VersionUpdated` event
- The version is set at initialization and updated atomically with proxy
upgrades via `upgradeAndCall`.
### CLI
**`bun cli contracts upgrade`** works in two modes: _dry-run_ or
_execute_.
**Dry-run (default)**
Deploys the new implementation on-chain (signed by the deployer key),
then prints a ready-to-submit JSON payload for the multisig to execute
the proxy upgrade. No AVS owner key required.
```bash
# Uses version from contracts/VERSION (standard workflow)
bun cli contracts upgrade --chain hoodi
# Override version for this upgrade only (warns if it differs from contracts/VERSION)
bun cli contracts upgrade --chain hoodi --target x.y.z
```
Example output:
```json
{
"to": "0xProxyAdmin...",
"value": "0",
"data": "0x...",
"description": "Upgrade ServiceManager proxy to 0xNewImpl... and set version to 1.1.0"
}
```
**Execute mode (`--execute`)**
Deploys the implementation and broadcasts the proxy upgrade + version
update in a single atomic `upgradeAndCall` transaction. Requires
`AVS_OWNER_PRIVATE_KEY`. Used mostly for testing.
```bash
bun cli contracts upgrade --chain anvil --execute
```
---
### Expected flow
- Bump mannually contracts/VERSION (e.g., 1.1.0)
- Run bun cli contracts upgrade --chain anvil|hoodi|mainnet
## Era-targeted validator set submission with dedicated submitter role
> **Note:** This PR includes a detailed specification at
[`specs/validator-set-submission/validator-set-submission.md`](https://github.com/datahaven-xyz/datahaven/blob/feat/validator-set-submitter/specs/validator-set-submission/validator-set-submission.md)
that covers the design rationale, submission lifecycle, era-targeting
rules, and failure modes. Reading the spec first will make the contract,
pallet, and daemon changes easier to follow.
### Summary
- Introduce a dedicated `validatorSetSubmitter` role on
`DataHavenServiceManager`, separating validator set submission authority
from the contract owner
- Replace the unscoped `sendNewValidatorSet` with
`sendNewValidatorSetForEra`, which encodes a `targetEra` into the
Snowbridge message payload
- Add server-side era validation in the `external-validators` pallet to
reject stale, duplicate, or out-of-range submissions
- Add a long-running TypeScript daemon that watches session changes and
automatically submits each era's validator set at the right time
### Contract changes (`contracts/`)
- **New `validatorSetSubmitter` storage slot** — set during `initialize`
and rotatable via `setValidatorSetSubmitter` (owner-only). The storage
gap is decremented accordingly.
- **`sendNewValidatorSet` → `sendNewValidatorSetForEra`** — accepts a
`uint64 targetEra` parameter and is restricted to
`onlyValidatorSetSubmitter` instead of `onlyOwner`.
- **`buildNewValidatorSetMessageForEra`** — the
`NewValidatorSetPayload.externalIndex` is now caller-supplied instead of
hardcoded to `0`.
- **New events** — `ValidatorSetSubmitterUpdated`,
`ValidatorSetMessageSubmitted`.
- **New error** — `OnlyValidatorSetSubmitter`.
- **New test suite** — `ValidatorSetSubmitter.t.sol` covering submitter
set/rotate, access control, era encoding, and legacy function removal.
### Pallet changes (`operator/`)
- **`validate_target_era`** in `external-validators` — enforces
`activeEra < targetEra <= activeEra + 1` and `targetEra > ExternalIndex`
(dedup guard).
- **New errors** — `TargetEraTooOld`, `TargetEraTooNew`,
`DuplicateOrStaleTargetEra`.
- **Tests** — five new test cases for era boundary conditions (next-era
acceptance, old-era rejection, too-new rejection, duplicate rejection,
genesis behavior). Existing `era_hooks_with_external_index` test updated
to use valid target eras.
- **Runtime test fixes** — `external_index: 0` → `1` in
mainnet/stagenet/testnet EigenLayer message processor tests to satisfy
the new validation.
### Validator set submitter daemon
(`test/tools/validator-set-submitter/`)
- Event-driven service that subscribes to finalized
`Session.CurrentIndex` via Polkadot-API `watchValue`.
- Submits once per era during the last session, targeting `ActiveEra +
1`.
- Tracks submitted eras to avoid duplicates; skips if `ExternalIndex`
already covers the target.
- Startup self-checks: Ethereum connectivity, DataHaven connectivity,
on-chain submitter authorization.
- Supports `--dry-run` mode and YAML configuration.
- Graceful shutdown on `SIGINT`/`SIGTERM`.
### Test & tooling updates
- **E2E test** (`validator-set-update.test.ts`) — calls
`sendNewValidatorSetForEra` with a computed `targetEra` and filters the
substrate event by `external_index`.
- **`update-validator-set.ts` script** — accepts `--target-era` flag;
defaults to era 1 for fresh networks.
- **CLI launch** — wires validator set update as an interactive step
after relayer launch.
- **`package.json`** — new `submitter` and `submitter:dry-run` scripts.
- Regenerated contract bindings, PAPI metadata, state-diff, and storage
layout snapshots.
### Test plan
- [x] `forge test` — passes, including new `ValidatorSetSubmitter.t.sol`
- [x] `cargo test` — passes, including new era-validation tests in
`external-validators`
- [x] `bun test:e2e` — validator-set-update suite passes with
era-targeted flow
- [x] Manual: run submitter daemon against local network (`bun
submitter`), verify it submits once per era at the correct session
## ⚠️ Breaking Changes ⚠️
- **`sendNewValidatorSet` removed** — replaced by
`sendNewValidatorSetForEra(uint64 targetEra, ...)`. Callers must now
supply a `targetEra` parameter.
- **Access control changed** — validator set submission is now
restricted to the `validatorSetSubmitter` role instead of the contract
`owner`. The submitter address is set during `initialize` and rotatable
via `setValidatorSetSubmitter` (owner-only).
- **`external-validators` pallet now validates `targetEra`** — messages
with a stale, duplicate, or out-of-range `external_index` are rejected
on-chain. Existing integrations sending `external_index: 0` will fail
validation.
---------
Co-authored-by: Cursor <cursoragent@cursor.com>
## Summary
- Add stagenet-hoodi deployment artifacts (contract addresses, rewards
info) and update Snowbridge config with latest validator set
- Fix the `bun cli contracts verify` command to correctly verify all
deployed contracts, including proxy contracts and Snowbridge
dependencies
- Fix the `bun cli contracts update-metadata` command to use the correct
config file when `--environment` is specified
## Contract verification fixes
The verification CLI hardcoded all contract source paths as
`src/<Name>.sol`, which failed for:
- **Snowbridge contracts** (Gateway, BeefyClient, AgentExecutor) — these
live in `lib/snowbridge/contracts/src/`
- **Gateway proxy** — the `Gateway` deployment address is actually a
`GatewayProxy`, not the Gateway implementation. The implementation
address needs to be resolved from the ERC1967 storage slot
- **ServiceManager proxy** — was not being verified at all
Changes:
- Added `contractPath` field to `ContractToVerify` so each contract
specifies its source location relative to the contracts directory
- Added `guessConstructorArgs` option for proxy contracts with complex
encoded init data (uses forge's `--guess-constructor-args`)
- Gateway is now verified as two separate contracts: Gateway
Implementation (address resolved from ERC1967 proxy slot) and
GatewayProxy
- ServiceManager proxy is now verified as `TransparentUpgradeableProxy`
## Update-metadata fix
The `update-metadata` command was ignoring the `--environment` flag when
selecting the deployments file:
1. The handler received a pre-built networkId (`"stagenet-hoodi"`) as
the chain parameter, which `getChainDeploymentParams` couldn't resolve
(falling back to anvil). Now chain and environment are passed
separately.
2. Commander.js routed `--environment` to the parent contracts command,
leaving `options.environment` undefined on the subcommand. Added the
same parent-fallback logic already used for `--chain`.
## Test plan
- [x] `bun typecheck` passes
- [x] Ran `bun cli contracts verify --chain hoodi --environment
stagenet` — all contracts verified successfully on Etherscan
- [x] `bun cli contracts update-metadata --chain hoodi --environment
stagenet` now reads the correct `stagenet-hoodi.json` deployments file
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
## Summary
We are now launching the MSP backend when starting stpragehub services.
In this PR, we also fix the MSP and BSP node configuration and register
it with the correct keys.
## What changed
* Added a launch Backend MSP function that is called when launching
storage hub services
* Fix the wrong genesis error message in storagehub node by removing the
`--chain dev` flags (so it can be launch of the same network as our
local datahaven nodes).
* Use the correct keys to register MSP and BSP. We were injecting
different keys that the one we used for MSP and BSP registration leading
to the MSP and BSP node to never fully register as storage providers.
---------
Co-authored-by: Ahmad Kaouk <56095276+ahmadkaouk@users.noreply.github.com>
## Summary
- Add multi-environment deployment support (stagenet, testnet, mainnet)
to CLI and contracts
- Configure stagenet and testnet runtimes with correct genesis hashes
and Snowbridge Agent IDs
- Add CLI commands for BEEFY checkpoint updates and rewards origin
computation
- Add ETH validator strategies (native beacon chain ETH + LSTs) to all
config files
## Changes
### Runtime Configuration
**Stagenet Runtime:**
- Set `StagenetGenesisHash` to DataHaven stagenet genesis hash
- Configure `RewardsAgentOrigin` with computed Snowbridge Agent ID
- Add tests verifying rewards account derivation and agent ID
computation
**Testnet Runtime:**
- Set `TestnetGenesisHash` to DataHaven testnet genesis hash
- Configure `RewardsAgentOrigin` with computed Snowbridge Agent ID
- Add tests verifying rewards account derivation and agent ID
computation
The Rewards Agent ID is computed following Snowbridge's location
description pattern:
```
blake2_256(SCALE_ENCODE("GlobalConsensus", ByGenesis(genesis), "AccountKey20", rewards_account))
```
### CLI Enhancements
- All contracts subcommands (`status`, `deploy`, `verify`,
`update-metadata`) now accept `--environment` option
- Config and deployment files use environment-prefixed naming (e.g.,
`stagenet-hoodi.json`, `testnet-hoodi.json`)
- New `update-beefy-checkpoint` command that:
- Connects to a live DataHaven chain via WebSocket RPC
- Fetches all BEEFY data at the same finalized block for consistency
- Uses parallel queries with `Promise.all` for better performance
- Computes authority hashes (keccak256 of Ethereum addresses derived
from BEEFY public keys)
- Uses Snowbridge's quorum formula `n - floor((n-1)/3)` for strictly >
2/3 majority
- New `update-rewards-origin` command that computes the Snowbridge Agent
ID for the rewards pallet
- Centralized validation via `contractsPreActionHook` for all contract
commands
- Environment validation against allowlist (`stagenet`, `testnet`,
`mainnet`)
### Contract Changes
- Network validation uses explicit allowlist instead of suffix matching
- Added `initialValidatorSetId` and `nextValidatorSetId` fields to
`SnowbridgeConfig` struct
- `DeployBase.s.sol` now uses config values for validator set IDs
instead of hardcoded 0/1
- `DeployParams.s.sol` loads validator set IDs from config with
backwards compatibility
### Validator Strategies
Added ETH-equivalent strategies to allow validators to stake using
native ETH or LSTs:
**All Networks:**
- `0xbeaC0eeEeeeeEEeEeEEEEeeEEeEeeeEeeEEBEaC0` - Native beacon chain ETH
(virtual strategy)
**Hoodi Testnet:**
- `0xf8a1a66130d614c7360e868576d5e59203475fe0` - stETH
- `0x24579aD4fe83aC53546E5c2D3dF5F85D6383420d` - WETH
**Ethereum Mainnet:**
- `0x93c4b944D05dfe6df7645A86cd2206016c51564D` - stETH
- `0x1BeE69b7dFFfA4E2d53C2a2Df135C388AD25dCD2` - rETH
- `0x54945180dB7943c0ed0FEE7EdaB2Bd24620256bc` - cbETH
### Config Files
- `stagenet-hoodi.json` - Hoodi testnet with stagenet EigenLayer
addresses
- `testnet-hoodi.json` - Hoodi testnet with testnet EigenLayer addresses
- `mainnet-ethereum.json` - Ethereum mainnet with mainnet EigenLayer
addresses
- Removed `hoodi.json` (replaced by environment-prefixed files)
## Usage
```bash
# Deploy to stagenet on Hoodi
bun cli contracts deploy --chain hoodi --environment stagenet
# Update BEEFY checkpoint from live chain
bun cli contracts update-beefy-checkpoint \
--chain hoodi \
--environment stagenet \
--rpc-url wss://services.datahaven-dev.network/stagenet
# Compute rewards origin for a chain
bun cli contracts update-rewards-origin \
--chain hoodi \
--environment stagenet \
--rpc-url wss://services.datahaven-dev.network/stagenet
# Check deployment status
bun cli contracts status --chain hoodi --environment stagenet
```
---------
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
## Summary
This PR improve the generating state workflow. It will also check for
outdated state-diff.json and add a practical script to easily generate a
new one.
The way we generate state has also been changed to make it work with
macOS M1 system. We don't run the tool in the container anymore but
instead directly on the machine.
## What changes
* A check-generated-state.js script was added to quickly look for
outdated test
* The check was added in the CI
* A generate-contracts.ts script was added to easily generate the new
state with the new instructions to run on MacOS
---------
Co-authored-by: Gonza Montiel <gon.montiel@gmail.com>
Co-authored-by: Ahmad Kaouk <56095276+ahmadkaouk@users.noreply.github.com>
Co-authored-by: Steve Degosserie <723552+stiiifff@users.noreply.github.com>
Co-authored-by: Gonza Montiel <gonzamontiel@users.noreply.github.com>
## Summary
This PR remove the middlewares contracts from eigen layer. Instead we
are planning to use the eigne layer contract directly. It also removes
the tests related to the middleware slasher code and the mock contract
used in it.
## Motivation
When slashing an operator in the Dathaven we are going through the
substrate slashing pallet already implemented. It already allow to
configure a slashing window and/or to cancel a slashing. In the future
it will also be compatible with a government pallet. This part of code
is therefore redundant. For the same reason we remove the tests because
we are not using the slashing middleware contracts.
## What changed
* Remove the slasher middleware files
* Remove the tests related to the middleware slasher file
This PR significantly refactors and improves the end-to-end testing
framework and infrastructure. The primary focus was on simplifying the
test suites, improving reliability through better resource management,
and hardening the relayer infrastructure.
All E2E tests are now passing on the CI and demonstrate consistent
reliability when run locally.
### Key Changes
#### 1. E2E Test Suite Refactor & Cleanup
* **Simplified Test Logic**: Heavily refactored the core test suites
(`native-token-transfer.test.ts`, `rewards-message.test.ts`, and
`validator-set-update.test.ts`). The new implementation is much cleaner,
utilizing shared helpers to reduce boilerplate.
* **Utility Consolidation**: Removed redundant utility files
(`storage.ts`, `rewards-helpers.ts`) and simplified `events.ts`. Event
waiting now uses `rxjs` for Substrate and native `viem` watchers for
Ethereum, which is more robust and easier to maintain.
* **Better Connector Management**: Unified the creation and cleanup of
test clients in `ConnectorFactory`. It now handles the lifecycle of
WebSocket connections more gracefully, including clearing the
`socketClientCache` to prevent reconnection noise during teardown.
#### 2. Infrastructure & Stability
* **Relayer Relaunch Policy**: Added a restart policy for Snowbridge
relayer containers. They are now configured with `--restart
on-failure:5`, ensuring that relayers automatically relaunch if they
crash during the sensitive initialization phase.
* **WebSocket Integration**:
* Updated the `ConnectorFactory` to prefer **WebSockets** for the
Ethereum public client, which is essential for efficient, event-heavy
E2E testing.
* Enhanced `launchKurtosisNetwork` to correctly identify and register
the Execution Layer's WebSocket endpoint from Kurtosis.
* **Disabled Contract Injection**: This PR temporarily disables the
automatic injection of contracts into the genesis state by default.
* *Reason*: I encountered issues generating a valid `state-diff.json`
for the latest contract versions. Even after applying several
workarounds, the injected state remained unstable. As a result, I've
reverted to manual contract deployment during the launch sequence for
better reliability for now.
#### 3. Documentation & Maintenance
* Removed obsolete documentation (`event-utilities-guide.md`) that no
longer reflects the simplified event-handling API.
* Cleaned up `test/launcher/validators.ts` and moved logic into more
appropriate helpers.
---------
Co-authored-by: Steve Degosserie <723552+stiiifff@users.noreply.github.com>
## Summary
Optimizes the DataHaven parameter configuration during E2E test
infrastructure setup by batching multiple extrinsics into a single
transaction, reducing setup time.
## Problem
Setting runtime parameters required 5 separate
`Parameters.set_parameter` calls, each waiting for block finality. This
created unnecessary delays during infrastructure setup since each call
blocked sequentially.
## Solution
- **Batch parameter updates:** Combine all `Parameters.set_parameter
`calls into a single `Utility.batch_all` transaction wrapped in
`Sudo.sudo`.
- **~5× faster parameter setup:** Only wait for finality once instead of
5 separate times
- **Code simplification:** Refactored parameter handling code, removing
~190 lines of unnecessary abstractions and complexity
# Enable AVS owner workflow
Until now, the deployer of the contracts and the owner of the deployed
contracts where the same account. Even if we allowed a different owner
to be specified, we were using the same. For this reason, a private key
was required, so after the deployment we could execute owned
transactions needed for the CLI.
In this PR we:
- Add a mechanism to the CLI to specify a different owner account other
than the deployer via `--avs-owner-address`
- Add CLI flags `--avs-owner-key` and`--execute-owner-transactions` so
account ownership vs. immediate execution is explicit and deferred. If
both previous parameters are provided, the CLI will execute the
transactions using the private key provided.
- Allow DataHaven AVS deploy scripts to toggle owner-call execution via
an env flag `TX_EXECUTION`
- Add documentation on how the new parameters work in `test/README.md`
and `test/docs/deployment.md`.
---------
Co-authored-by: Steve Degosserie <723552+stiiifff@users.noreply.github.com>
Co-authored-by: Ahmad Kaouk <56095276+ahmadkaouk@users.noreply.github.com>
## Summary
This PR fixes several issues with the CLI deploy-contracts command to
properly support local Anvil deployments and improves the overall
contract deployment workflow.
### Key fixes:
- Add support for anvil chain in the CLI deploy contracts command
- Rename PRIVATE_KEY to DEPLOYER_PRIVATE_KEY for consistency and clarity
across the deployment flow
- Fix EigenLayer contract status display for local/anvil chains by
reading addresses from the deployments file instead of config
- Fix runShellCommandWithLogger to properly throw errors on command
failure
- Correct totalSteps in DeployTestnet.s.sol from 2 to 4
### Housekeeping:
- Update .gitignore to ignore the entire broadcast/ folder
(autogenerated Foundry artifacts)
- Streamline contracts/README.md with clearer structure and deployment
instructions
## Add StorageHub nodes to CLI
This PR adds StorageHub node infrastructure to the CLI and fixes CLI
flag handling and improves the CLI logic a bit.
### Fix: CLI safeguards
- Prevents contract deployment and validator operations when `--ndc`
flag is used
- Skips Kurtosis service display in summary when `--nlk` flag is used
### Feat: Dockerized Storage Hub Nodes
- Adds 5 Docker containers: PostgreSQL, MSP, BSP, Indexer, and Fisherman
nodes
- New CLI flags: `--storagehub` `--no-storagehub` to control StorageHub
nodes launch
- Automatic provider funding and registration (Charleth for MSP and
Dorothy for BSP)
- Exposes nodes on ports 9945-9948 for local development
**TODO**
- [x] MSP & BSP associated pre-funded account.
- [x] Call `forceMspSignUp` and `forceBspSignUp` extrinsics
---------
Co-authored-by: Steve Degosserie <723552+stiiifff@users.noreply.github.com>
## Changes
Fixes `bun cli launch --all` command failing when using locally built
Docker images.
### What changed
- Check local Docker images first before querying Docker Hub in
`checkTagExists`
Co-authored-by: Steve Degosserie <723552+stiiifff@users.noreply.github.com>
## Add E2E validator-set update flow
- feat: `test/utils/validators.ts` for on-demand validator
orchestration.
- feat: `test/suites/validator-set-update.test.ts` covering allowlist →
register → update.
- some minor launcher updates: avoid docker cache, add `--platform` when
building datahaven image, avoid sending validator-set update on launch.
- Helpers: ABI shortcut in `test/utils/contracts.ts`; config tweaks in
`test/configs/validator-set.json`.
- Minor cleanup/formatting across `test/launcher/*`,
`test/scripts/setup-validators.ts`, and related tests.
- added `keepAlive` flag to `BaseTestSuite`, in order to avoid tearing
down the network while debugging. Defaults, obviously, to false.
- added a `failOnTomeout` option on to waitForDataHavenEvents() so the
test fails of the timeout is reached and no event was captured.
### Coverage
- The test simulates an scenario in which we have two active authorities
(alice and bob), which are running, and registered as operators, which
is the normal state after the chain launches. Then:
- It launches two more nodes (charlie and dave)
- It add the nodes to allowlist and register them as operators
- It sends the validator set update message
- Checks that the validator update message was propagated through the
gateway and arrived the external-validators pallet
- Checks that the chain continues producing blocks
### Notes
The last test case has a timeout of 10 minutes. This is to respect
propagation times of the message through the relayers. We are testing
that the external validators pallet actually updated the validator set.
Locally, I could expect 5~6 minutes, I just wanted to be on the safe
side. CI is passing showing that this was enough indeed.
---------
Co-authored-by: Steve Degosserie <723552+stiiifff@users.noreply.github.com>
Co-authored-by: Ahmad Kaouk <56095276+ahmadkaouk@users.noreply.github.com>
## Key changes
- Add CLI command to update AVS metadata URI on-chain via
`updateAVSMetadataURI` function. Use:
- `bun cli contracts update-metadata --chain <...> --uri <...>`
- Support for multiple chains (tipicaly holesky, hoodi, or a mainnet)
- Tx confirmation and gas usage reporting
---------
Co-authored-by: Steve Degosserie <723552+stiiifff@users.noreply.github.com>
## Summary
This PR resolves all CI failures following the migration to the new
DataHaven Github & Docker Hub organizations, and correctly leverage
self-hosted GitHub runners (`DH-runners` group) by eliminating sudo
dependencies.
## Key Changes
### 🔧 **Eliminated sudo requirements across all workflows**
- **Setup Environment**: Installed mold linker and system dependencies
in userspace without sudo
- **Tool Installation**: Replaced apt/system package installations with
direct binary downloads:
- Kurtosis: Direct binary download from GitHub releases (v1.10.3)
- Taplo: Direct binary installation for Cargo.toml formatting
- cargo-nextest: Using `cargo install` instead of GitHub action
(v0.9.100)
- **Runner Cleanup**: Skipped cleanup-runner action entirely on
self-hosted runners (bare-metal manages disk space externally)
### 🏷️ **Workflow optimizations**
- **Group-based targeting**: All heavy workloads (Rust builds, E2E
tests) now run on `DH-runners` runners
- **Dependency management**: Used `install-deps: false` flag instead of
hardcoded runner detection
Co-authored-by: Claude <noreply@anthropic.com>
## Summary
This PR introduces support for deploying Datahaven contracts to
different chains (hoodi, holesky, mainnet), as well as a new cli command
to manage this deployment separately from the regular deployment, while
maintaining compatibility with it.
#### New CLI command
- **`bun cli contracts deploy`** - Deploy contracts to supported chains
(Hoodi, Holesky, Mainnet)
- **`bun cli contracts status`** - Check deployment configuration and
status
- **`bun cli contracts verify`** - Verify contracts on block explorers
- Commands need the chain parameter: `--chain <hoodi | holesky |
mainnet>`
- Right now only `hoodi` and `holesky` are supported
### Deployment
#### Hoodi & Holesky Network Support
- Added **DeployBase.s.sol** as common ground for
**DeployTestnet.s.sol** (also new) and **DeployLocal.s.sol** (existing).
- **Hoodi configuration** (`contracts/config/hoodi.json`) with deployed
EigenLayer contract addresses to reference.
- **Holesky configuration** (`contracts/config/hoodi.json`) with
deployed EigenLayer contract addresses to reference.
#### Contracts being deployed
- **DataHaven**: ServiceManager, VetoableSlasher, RewardsRegistry
- **Snowbridge**: BeefyClient, AgentExecutor, Gateway, RewardsAgent
- **EigenLayer**: References existing deployed contracts (not
re-deployed)
#### Deployment files
When the deployment is done, a new file under `contracts/deployments` is
generated with the addresses of the deployed contracts, for each chain
(it will be overriden per chain if run multiple times). So we would have
one `anvil.json`, `hoodi.json`, `holesky.json`, etc, with the addresses
of the deployed contracts for reference and for later verification.
#### Todo
- [x] Test compatibility with existing `bun cli launch` and `bun cli
deploy` commands
#### For follow-up PRs
- Fix verification issue with `foundry verify-contracts` when specifying
the `chain` or `chain-id` parameter, needed for hoodi
(https://github.com/foundry-rs/foundry/issues/7466).
- Add `redeploy` feature to only override implementation contract and
leave the proxy address untouched
## Usage Examples
```bash
# Deploy to Hoodi network
bun cli contracts deploy --chain hoodi
# Check deployment status
bun cli contracts status --chain hoodi
# Verify contracts on block explorer
bun cli contracts verify --chain hoodi
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Added deployment and configuration support for new networks "hoodi"
and "holesky", including new configuration and deployment files.
* Introduced a CLI tool for managing contract deployments, status
checks, and verification across supported chains.
* Added example environment configuration and comprehensive deployment
documentation.
* Enabled contract verification and status reporting via the CLI with
support for block explorer integration.
* **Improvements**
* Refactored deployment scripts for modularity, supporting both local
and testnet environments.
* Centralized and extended configuration loading to support additional
contract addresses and network parameters.
* Enhanced deployment utilities and typings to support multi-network
deployments.
* **Bug Fixes**
* Improved input validation and error handling in CLI commands and
deployment scripts.
* Added explicit handling for zero address in operator strategy
retrieval.
* **Chores**
* Updated documentation and configuration templates for easier
onboarding and deployment management.
* Improved logging and output formatting for deployment and verification
processes.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
---------
Co-authored-by: Steve Degosserie <723552+stiiifff@users.noreply.github.com>
Co-authored-by: Ahmad Kaouk <56095276+ahmadkaouk@users.noreply.github.com>
This PR enhances the Helm-based deployment system with several key
improvements organized into the following areas:
## New Features
### Ingress Management
- **Ingress per replica**: Added `ingress-per-replica` chart template
that automatically generates an ingress for each node replica, exposing
individual pod instances
- **Traefik integration**: Added chart values to deploy Traefik as the
ingress controller for local K8s clusters, enabling proper ingress
testing (requires adding names to `/etc/hosts`)
### Solochain Support
- **New relay chart**: Added dedicated Helm chart for Solochain relay
deployment
- **CLI integration**: Updated DataHaven CLI to deploy both Execution
and Solochain relayers
## Configuration Improvements
### Environment Structure
- **Modular configs**: Refactored environmental configurations from
single `values.yaml` files into separate component-specific overrides
for better organization
### Node Configuration
- **Base config updates**: Improved base configurations for bootnode and
validator nodes
- **Network protocol**: Reverted from litep2p back to libp2p to resolve
node communication issues on Stagenet
- **Archive node routing**: In Stagenet, relayers now connect to the
bootnode (configured as archive node) instead of validator nodes
## Storage & Deployment
### Persistent Storage
- **Relayer database**: Added support for persistent volumes to store
relayer databases instead of using ephemeral storage
### Deployment Scripts
- **Documentation cleanup**: Removed obsolete test deploy.sh script and
updated the deployment README with clearer instructions
These changes provide a more robust, scalable, and maintainable
deployment system for DataHaven infrastructure.
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Added support for deploying new relayer types: Solochain and Execution
relayers, with dedicated configuration and secret management.
* Introduced per-replica ingress configuration for node deployments,
allowing each replica to have its own ingress resource and hostname.
* Added persistent storage options for relay data, configurable via
storage path, class, and size.
* Added new deployment configuration files for local and stagenet
environments, including Traefik ingress controller setup.
* Introduced a new relay category, "Solochain Relayers," for standalone
chain operations and cross-chain communication.
* **Improvements**
* Updated deployment configurations to use container-specific
environment YAML files for more granular control.
* Enhanced relay and node configurations with new flags and backend
options, including dynamic peer ID generation and automatic bootnode
discovery.
* Updated relayer endpoints to consistently use the bootnode for
connections.
* Refined relay configuration files for improved structure, clarity, and
endpoint management.
* **Bug Fixes**
* Corrected deployment logic to reference the correct
environment-specific configuration files during Helm deployments.
* **Documentation**
* Simplified and updated deployment documentation to focus on CLI-based
deployment, removing outdated manual instructions and adding a concise
overview of components and relayer types.
* **Chores**
* Removed deprecated deployment scripts and outdated configuration files
to streamline the deployment process.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
---------
Co-authored-by: Claude <noreply@anthropic.com>
### Problem
Introducing `--network` should make easy to container nodes to find each
other. But this change was made half-way for the relayers, and it was
using the external port to find the first datahaven node (usually
Alice). So:
- In cli launch, Alice node port mapping was left to random port `-p
9944` instead of `-p 9944:9944`.
- Relayers couldn't connect to DataHaven nodes because they were using
the external WS port (now random) instead of hitting the internal port
(which for a cli launch we actually fix it to 9944).
### Solution
- [x] **Fixed Docker port mapping**: Explicit `-p 9944:9944` for Alice
node under network `cli-launch`
- [x] **Enhanced container spec**: Added `internalPorts` tracking to
`LaunchedNetwork`
- [x] **Fixed relayer connections**: Use internal ports for container
communication
## Implement E2E Testing Framework with Isolated Networks
### Summary
Refactors the existing E2E testing infrastructure to provide isolated
test environments with parallel execution support. Each test suite now
runs in its own network namespace, preventing resource conflicts.
### Key Changes
- **New Testing Framework** (`test/framework/`): Base classes for test
lifecycle management with automatic setup/teardown
- **Launcher Module** (`test/launcher/`): Extracted network
orchestration logic from CLI handlers for reusability
- **Parallel Execution**: Added `test-parallel.ts` script with
concurrency limits to prevent resource exhaustion
- **Test Isolation**: Each suite gets unique network IDs (format:
`suiteName-timestamp`) and Docker networks
- **Improved Test Organization**: Migrated tests to new framework,
deprecated old test structure
### Test Improvements
- Added 4 new test suites demonstrating framework usage. :
- `contracts.test.ts` - Smart contract deployment/interaction
- `datahaven-substrate.test.ts` - Substrate API operations
- `cross-chain.test.ts` - Snowbridge cross-chain messaging
- `ethereum-basic.test.ts` - Ethereum network operations
> [!WARNING]
The test suites themselves are bad and shouldn't be consider examples of
good tests. They were AI generated just to test the concurrency of test
runners
### Documentation
- Added comprehensive framework overview (`E2E_FRAMEWORK_OVERVIEW.md`)
- Updated README with parallel testing commands
- Added test patterns and best practices
### Breaking Changes
- Old test suites moved to `e2e - DEPRECATED/` directory
- Test execution now requires extending `BaseTestSuite` class
### Testing
Run tests with: `bun test:e2e` or `bun test:e2e:parallel` (with
concurrency limits)
### TODO
- [ ] Implement good test examples.
- [ ] Implement useful test utils (like waiting for an event to show up
in DataHaven or Ethereum).
- [ ] Enforce tests with CI (currently cannot be done due to
intermittent error when sending a transaction with PAPI).
---------
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: undercover-cactus <lola@moonsonglabs.com>
## Summary of changes
- We decided to remove the topics and nonce from the massage encoding
since we don't use them (original commit:
ee2a3f2fd4).
- Besides, we already have a nonce at the Snowbridge message level
f4ab5c2b2e/operator/primitives/snowbridge/inbound-queue/src/v2/message.rs (L105)
- I had to recreate the static test for _encoding_ (happens in
[DataHavenSnowbridgeMessages.sol](d12d40634f/contracts/src/libraries/DataHavenSnowbridgeMessages.sol)
) / _decoding_ (happens in
[operator/primitives/bridge/src/lib.rs)](f9f9cc65fe/operator/primitives/bridge/src/lib.rs).
Now it matches the current structure. The idea is that now we can test
that we don't break the decoding in followup refactoring.
- Fixes a problem with EigenLayer validator addresses. In all our
contracts we were using `bytes32` to refer to a Solochain validator
address. But on our Substrate change we actually expect AccountId20, so
only 20 bytes. This was causing the decoding to fail.
- I opted for the minimal change that would be to take the right-most 20
bytes to send that to our chain. But we might want aswell to limit our
EigenLayer contracts to be only 20 bytes long. @ahmadkaouk showcase this
[here](92a34c273c)
- Adds a bash script to run the static test. The test will compile the
contracts, run the encoding test, compile the operator, and run the
decoding test. This saves a huge amount of time since we don't need to
run the full e2e setup. The way of running it is the following:
```bash
cd operator/test/scripts
./test_message_encoding.sh
```
- As a consequence of this PR, the execution relayer now works properly.
EDIT:
> [!IMPORTANT]
**We decided to use 20-byte addresses in our contracts**. So what is
stated above is not valid anymore.
The change implies that the mapping from Ethereum addresses to bytes32
addresses now it's a mapping as follows:
dd3ba99ac0/contracts/src/DataHavenServiceManager.sol (L51-L52)
I've updated helper functions, tests, etc to be compliant with this
change. The execution relayer and beefy relayer look stable now.
---------
Co-authored-by: Ahmad Kaouk <ahmadkaouk.93@gmail.com>
Co-authored-by: Ahmad Kaouk <56095276+ahmadkaouk@users.noreply.github.com>
This PR solves an issue where the solochain relayer was fatally crashing
because the DataHaven chain initialized with an empty
`RewardsRegistryAddress` parameter. This caused all initial reward
merkle root update messages to target the zero address before the
parameter could be properly set by the launch command of the CLI. When
the relayer tried to call the update function on the zero address (which
isn't a valid contract), the transactions reverted and crashed the
relayer.
The band-aid fix implemented is to set the actual `RewardsRegistry`
contract address as the default value for the `RewardsRegistryAddress`
parameter (since we know it as it's consistent between our CLI runs).
This ensures initial messages have a valid target contract address from
startup, preventing the fatal crashes.
Again, this is a temporary fix until we implement a more robust solution
for parameter initialization, since it won't hold up if any changes
happen to our contracts.
---------
Co-authored-by: Gonza Montiel <gon.montiel@gmail.com>
- Added a parameter `--all` to `bun cli launch` (now coherent with `bun
cli stop`)
- Equivalent to run `bun cli launch --d --bd --lk --dc --fv --sv --uv
--sp --r --cn`
- Removed `bun start:all` command
- Added a kurtosis cluster type check
- The problem was that we now support deploying and launching the CLI
tool with different types of clusters. If you're deploying, your
kurtosis cluster most likely needs to point to a `kubernetes` type
cluster, if you're running `bun cli launch` you need to use native
docker containers. This PR adds a check and warns the user to point to
the right config.
- *Disclaimer*: we don't know the cluster name of the user so we can't
force it to be anything 🫠
This PR contains improvements to the DataHaven deployment
infrastructure:
1. Directory restructure: Moved from `deployment/` to `deploy/` (more
common for K8s / Helm -based deployment configs).
2. Added **local environment** support: updated CLI to support deploying
to a local K8s cluster.
3. Manual deployment script: `deploy/scripts/deploy.sh` for manual
deployments.
4. Environment-specific configurations: Structured values files for each
environment.
5. Chart organization: Renamed bridges-common-relay to relay for
clarity.
---------
Co-authored-by: Gonza Montiel <gon.montiel@gmail.com>
Co-authored-by: Gonza Montiel <gonzamontiel@users.noreply.github.com>
This PR adds a quick check to see if the image to use for the relayer is
a locally-built one (by checking that it ends with `:local`) and avoids
pulling from Docker Hub if so. This fixes the issue in which we were
unable to run the CLI with a local relayer image since it was always
trying to pull from the corresponding repository which of course didn't
exist for local images.
### New Features
1. Add the `deploy` command to our CLI.
1. Conditionally deploys kurtosis eth network if we're in `stagenet`
environment.
2. Deploys DH nodes.
3. Deploys contracts (all of them). In `mainnet` and `testnet` it
shouldn't deploy EL contracts, but for now that's not implemented.
4. Configures parameters, validators and relayers in the same way as
`launch`.
5. Currently, it only deploys `beefy` and `beacon` relayers, `execution`
and `solochain` relayers are pending for a subsequent PR.
2. Add `waitFor` utility function that receives a lambda.
### Refactors
1. Several common functionalities used both by the `launch` and `deploy`
command have been moved to the `cli/handlers/common` directory, from
where both commands use them. These include
1. Checks for installed dependencies.
2. Common constants.
3. The `LaunchedNetwork` class has been moved to this directory.
4. DataHaven nodes common functions.
5. Kurtosis common functions.
6. Relayer common functions.
7. Kubernetes functions (although only used by `deploy`, it seemed
fitting to have it here still).
8. Remove CLI questions and separator prints from `deploy-contracts.ts`
and `set-datahaven-parameters.ts` scripts. These things should be in the
`cli/launch` folder, which consumes the functions in these scripts.
9. Remove `setParametersFromCollection` from `utils` folder and put it
in `cli`.
10. Create base snowbridge relayer configs for `local` and `stagenet` as
two separate directories.
### Fixes
1. Sets the default time of the `deploy` command to 6s as Lodestar is
slower than Lighthouse.
2. In `runShellCommandWithLogger` only print `stderr` if the command
fails.
### Additional Minor Changes
1. K8s secret key names changed from `dh-beefy-relay-eth-key` to
`dh-beefy-relay-ethereum-key` and `dh-beacon-relay-sub-key` to
`dh-beacon-relay-substrate-key`, for simplicity in the deployment
script.
11. Update suggested configs for `.vscode` configs.
---------
Co-authored-by: Steve Degosserie <723552+stiiifff@users.noreply.github.com>
This PR renames all usage of `moonsonglabs/snowbridge-relayer` to
`moonsonglabs/snowbridge-relay` to adapt to the refactor done in [PR
#18](https://github.com/Moonsong-Labs/snowbridge/pull/18) of our
Snowbridge fork. The aforementioned PR should be merged before this one,
although caution is not really needed since the CI would fail otherwise.
Lighthouse consensus layer client has a bug when building the next sync
committee merkle proof, that is preventing the Snowbridge relayer from
updating the next sync committee in the Ethereum Beacon Client pallet,
on the DataHaven Substrate chain. See issue
[here](https://github.com/sigp/lighthouse/issues/7552).
As a consequence, we're moving to the lodestar implementation.
## This PR includes:
- Running the execution relayer on the CLI
- Modifying the Payload generation in the `DataHavenServiceManager.sol`
- Modified the `EigenLayerMessageProcessor` to work with the
ValidatorSet update message, but for this change we are loosing the
generic message type (it was the only way to make it work so far).
- Adds a `--no-wait` argument to the cli launch and stop commands to
bootstrap faster.
### Testing the Snowbridge message encoding / decoding
- Added`MessageEncoding.t.sol` is documented and explains how to
generate bin data to use in the rust test.
- Added a Rust unit test to `EigenLayerMessageProcessor` to compare the
message encoding/decoding taking the bytes from a file (previously
generated with some mock data).
Specifically, we want that:
3cbca0db6d/contracts/src/libraries/DataHavenSnowbridgeMessages.sol (L78-L85)
Generates the right bytes encoding for
0e2c9cd518/operator/primitives/bridge/src/lib.rs (L51)
If the test passes, it's very likely that the CLI will also pass, if
not, then we might wanna check something else is missing.
### Breaking change ⚠️
For compatibility reasons with Snowbridge contracts (they call specific
extrinsics of specific pallets), I had to rename:
- `InboundQueueV2` -> `EthereumInboundQueueV2`
- `OutboundQueueV2` -> `EthereumOutboundQueueV2`
## For follow up PRs:
- Add an automated way of generating the Solidity bytes fo testing, so
we don't need to maintain the MessageEncoding.t.sol and generate the
binary data manually.
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **New Features**
- Added support for the "execution" relayer type in relay configuration,
parsing, and CLI launch utilities.
- Introduced a Solidity test contract for encoding and logging validator
set messages.
- Added a comprehensive "start:all" script to streamline launching and
setup processes.
- **Enhancements**
- Improved message encoding for validator set updates, aligning with new
struct field names and message formats.
- Updated relay configuration schema and validation to support execution
relayers and OFAC settings.
- Increased beacon datastore capacity and adjusted relay scheduling
parameters in configuration files.
- **Refactor**
- Renamed runtime type aliases for inbound/outbound queues to more
descriptive names across mainnet, stagenet, and testnet.
- Centralized and streamlined validator set update logic in CLI
utilities.
- Centralized message decoding logic and improved visibility of message
fields in Rust components.
- **Bug Fixes**
- Improved error handling and decoding logic for message processing in
Rust components.
- **Tests**
- Added Rust and Solidity tests for message encoding and processing
validation.
- **Chores**
- Updated dependencies and feature flags in Rust project configuration.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
---------
Co-authored-by: Facundo Farall <37149322+ffarall@users.noreply.github.com>
This PR:
- Adds launching the new `solochain` relayer in the relayers' script,
with its config files and schemas.
- Updates the relayer launching to use switch-case logic since we are
now going to be running 4 relayers, making it cleaner.
- Minor CLI fixes (adding additional args to build command for Linux,
improving naming, deleting idle containers instead of only active ones).
- Deletes unused files `substrate-relay.json` (now
`solochain-relay.json`), `gen-snowbridge-cfgs.ts` and
`snowbridge-relayer.ts` (now Snowbridge binary and docker image
generation can be done exclusively from our [Snowbridge
repo](https://github.com/Moonsong-Labs/snowbridge))
- Updates the `UniversalLocation` of our stagenet runtime to be under
the global consensus for XCM instead of actually being the global
consensus. This makes it so we can actually use the Snowbridge System
pallets to queue up messages. For mainnet we are going to want to have
our own Network ID instead of using Polkadot's.
> [!WARNING]
> ~~All in all these changes allows us to run the solochain relayer, but
it won't work without the refactor that's being worked on in
https://github.com/Moonsong-Labs/snowbridge/pull/14. I'd advise waiting
for that PR to be merged before merging this one.~~ MERGED ✅
This PRs extracts [this
commit](74c0c0be0a)
from @TDemeco's PR to add a way to include parameters as part of the
CLI.
### Key changes
- CLI tool to set DataHaven runtime parameters via JSON configuration
- Supports both interactive prompts and command-line flags
- Type-safe parameter parsing and validation
- Already adds the parameters for the `EthereumGatewayAddress`, that
otherwise we would need to add manually to the node using an explorer.
---------
Co-authored-by: TDemeco <tdemeco@itba.edu.ar>
## Changes
- New option: `--kurtosis-enclave-name` to allow you to specify a new
ethereum network with a different enclave name. Neccesary step for
setting up local testing with multiple enclaves running at once
- Refactor: all single dash options (e.g. `-i`) have been renamed to
have double dashes `--` for consistency others
- sad times: CLI now must be invoked `bun cli launch` since we have
multiple fns now
- Package Update
- New Function: `stop` which allows you to kill all components (eth, dh,
relayers) or only a single part
- Gonza's mac target build fix
- Misc fixes to ci like caching and rate limits
## Additional Comment
The CLI needs multiple commands and this PR adds the first new one
`stop`.
This syntax is faily extensible so @ffarall 's k8 command can follow
suite.
Originally we were going to have an `exec` command to expose internal
fns to be callable by command line, i.e. generate beacon checkpoint -
however the need for it has since been superceded by the deploy to k8
command that's going to be added. I've left the code in place though so
when another usecase comes up we can just plug that in.
---------
Co-authored-by: Facundo Farall <37149322+ffarall@users.noreply.github.com>
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **Documentation**
- Added detailed IDE configuration recommendations for Rust, Solidity,
and TypeScript in the README to enhance developer experience.
- **Chores**
- Updated Biome configuration files and package dependencies to the
latest schema and version.
- Refined code formatting, linting, and import organization settings for
consistency across the project.
- **Refactor**
- Reordered import statements in multiple files for improved
readability.
- Simplified function signatures and ensured proper async handling in
utility scripts.
- **Bug Fixes**
- Ensured proper completion of asynchronous operations in shell utility
functions.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
---------
Co-authored-by: Gonza Montiel <gonzamontiel@users.noreply.github.com>
## Changes
- Latest changes to have working relayer 🎉 component
- Changed spawning snowbridge relayers to docker containers
- Small logging output changes
- Refactoring to `LaunchedNetwork` class
- new flag `--bd` `--build-datahaven` which will build a local docker
container which is **much** quicker than the proper CI build (which uses
a controlled build enviroment)
- new bun script `start:e2e:local`, which is everything that
`start:e2e:ci` has, but with building local docker container and
log_level debug set
---
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
## Summary by CodeRabbit
- **New Features**
- Added support for launching and managing relayer and DataHaven
services using Docker containers and networks.
- Introduced a CLI option to specify the relayer Docker image tag
instead of a binary path.
- **Improvements**
- Enhanced log messages with clearer text and expressive emojis for
better user feedback.
- Improved summary display by removing relayer services from the output.
- Updated build scripts to consistently enable the "fast-runtime"
feature for cross-platform builds.
- Refined validation and error reporting for checkpoint data parsing.
- **Bug Fixes**
- Improved Docker container cleanup and network management during
service launch and teardown.
- **Chores**
- Updated and refactored npm scripts for Docker operations and
end-to-end test cleanup.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
---------
Co-authored-by: Facundo Farall <37149322+ffarall@users.noreply.github.com>
In this PR:
1. Add new `datahaven-node-local.dockerfile` for building a local image
with the locally built DataHaven Node binary. This severely improves
iteration speed on running `bun cli` if there are changes in the DH
node. Previously, it relied on the published dockerfile, which builds
the Cargo project inside of it, taking +20m even with no changes.
2. Building this local dockerfile is integrated to the CLI, which now
also asks if the user wants to rebuild the local docker image of the DH
node.
3. A new script `cargo-crossbuild` is added, to be able to build the
DataHaven node Cargo project both from Mac and Linux, with the target
being `x86_64-unknown-linux-gnu`. For building from Mac, it uses `cargo
zigbuild`, so `zig` is now a dependency. Building for this target is
needed because the docker image is an Ubuntu image, so it will need to
run a linux binary.
4. Added `zig` as dependency in docs.
5. CI still uses the docker image built by the CI itself, which builds
the Cargo project inside of it. The CI can take advantage of caching for
this.
> [!NOTE]
> This is `Part 3` of the ongoing _Docker Series._
## New Additions:
- Launching Datahaven network will spin up containers, as opposed to
native binaries
- `stop:docker` script to kill all dh containers
- `e2e` test suite for datahaven solochain network
- Contains reference test file that uses papi for storage queries,
submitting exts, runtime calls (good job on that facu and tobi)
- Added new utils:
- `waitForLog()` to wait for log lines in docker container logs
- `createPapiConnectors()` helper for test cases to build and connect to
dh network
- `getPapiSigner()` helper to return a papi compatible signer using our
prefunded accounts (alith by default)
- `sendTxn()` helper to submit txn and wait for block inclusion, instead
of finalization, which std library provides
## Changes:
> [!CAUTION]
> Launching native binaries for datahaven no longer supported.
- Datahaven binary location cli option changed to `-i,
--datahaven-image-tag`
- To locally run this you'll need a datahaven docker image handy, you'll
need to either:
- Point to remote dockerhub e.g. `moonsonglabs/datahaven:main` (must be
logged in and have permission)
- Build this locally with `bun build:docker:operator`
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **New Features**
- Added end-to-end tests for the Datahaven solochain, including runtime
API queries, storage lookups, extrinsic submissions, and event
listening.
- Introduced CLI option to specify the Datahaven Docker image tag, with
a default value.
- Added CLI option to disable the Relayer.
- Provided new scripts to stop Docker containers associated with
Datahaven.
- Added utility functions for Docker log monitoring and container
startup checks.
- Introduced utilities for interacting with the Datahaven Polkadot API.
- **Improvements**
- Switched Datahaven network launch from local binaries to Docker
containers.
- Enhanced cache accuracy in build workflows by including Rust source
files in cache keys.
- Improved build performance with TypeScript incremental build options.
- Increased timeout for end-to-end tests for better reliability.
- Updated CLI version to 0.2.0.
- Modified Dockerfile build to enable the `fast-runtime` feature.
- Extended network launch summary to include relayer and container
details.
- **Bug Fixes**
- Fixed cleanup logic by tracking and preparing for forced removal of
Docker containers after tests.
- **Chores**
- Updated workflow steps for Docker image handling and network checks.
- Adjusted scripts and workflow logic for improved Docker and test
management.
- Removed top-level disk usage summaries from cleanup workflow for
streamlined reporting.
- Enhanced shell command utility to support asynchronous wait during
execution.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
---------
Co-authored-by: Facundo Farall <37149322+ffarall@users.noreply.github.com>
This PR:
- Adds a check to make sure the BEEFY RPC endpoint is ready before
spinning up the BEEFY relayer, otherwise it would just fail and crash.
- Adds the `--enable-offchain-indexing=true` flag to the Datahaven nodes
run when starting up the E2E infra. This is needed because otherwise
nodes can't be queried by the relayer to generate the required proofs
since they would not store the MMR leafs/nodes/root, so the relayer
would just crash.
- Updates the way we were generating the merkle root of the validator
set.
- The BEEFY pallet (and as such, the relayer) generate the validator set
merkle tree by getting the validator list and treating it as an already
ordered set of merkle leafs, hashing each pair in succession without
caring what each leaf value is.
- Meanwhile, the OpenZeppelin crypto library (and as such, the
EigenLayer contracts) also gets the merkle leafs list but hashes each
pair of leafs in value order.
- This means that, for example, if the list of leafs is: ["0x124",
"0x123"]
- BEEFY would do `hash("0x124123")` to get the value of the parent node.
- OZ and EigenLayer would do `hash("0x123124")` to get the value of the
parent node.
- This created a mismatch between what the BEEFY relayer was expecting
and what was actually calculated in our script. A way to obtain a merkle
tree using the BEEFY way was added to solve this.
- Updates the authority set to not be a hardcoded array of keys, now the
BEEFY keys are obtained by directly querying the runtime before
deploying the BEEFY contracts.
- Renames a few files from `flamingo` to `datahaven`.
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **New Features**
- Added support for calculating and generating Merkle roots and proofs
without sorting, enabling new use cases for validator set management.
- Introduced dynamic fetching and configuration of Datahaven validator
authorities during network launch.
- Added readiness check for the BEEFY protocol before relayer startup,
improving reliability of relayer operations.
- **Bug Fixes**
- Fixed indentation issues in configuration files for improved
readability and consistency.
- **Chores**
- Updated validator lists and addresses in configuration files.
- Enhanced e2e test scripts and added new commands for relayer and
minimal test scenarios.
- Added new dependency to test package.
- Updated version strings in package metadata.
- **Refactor**
- Improved logging and configuration handling during network and relayer
launches.
- Simplified import statements and removed unused code.
- **Style**
- Reformatted configuration and TypeScript files for better readability.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
---------
Co-authored-by: Tim B <79199034+timbrinded@users.noreply.github.com>
Co-authored-by: Facundo Farall <37149322+ffarall@users.noreply.github.com>
