## Era-targeted validator set submission with dedicated submitter role
> **Note:** This PR includes a detailed specification at
[`specs/validator-set-submission/validator-set-submission.md`](https://github.com/datahaven-xyz/datahaven/blob/feat/validator-set-submitter/specs/validator-set-submission/validator-set-submission.md)
that covers the design rationale, submission lifecycle, era-targeting
rules, and failure modes. Reading the spec first will make the contract,
pallet, and daemon changes easier to follow.
### Summary
- Introduce a dedicated `validatorSetSubmitter` role on
`DataHavenServiceManager`, separating validator set submission authority
from the contract owner
- Replace the unscoped `sendNewValidatorSet` with
`sendNewValidatorSetForEra`, which encodes a `targetEra` into the
Snowbridge message payload
- Add server-side era validation in the `external-validators` pallet to
reject stale, duplicate, or out-of-range submissions
- Add a long-running TypeScript daemon that watches session changes and
automatically submits each era's validator set at the right time
### Contract changes (`contracts/`)
- **New `validatorSetSubmitter` storage slot** — set during `initialize`
and rotatable via `setValidatorSetSubmitter` (owner-only). The storage
gap is decremented accordingly.
- **`sendNewValidatorSet` → `sendNewValidatorSetForEra`** — accepts a
`uint64 targetEra` parameter and is restricted to
`onlyValidatorSetSubmitter` instead of `onlyOwner`.
- **`buildNewValidatorSetMessageForEra`** — the
`NewValidatorSetPayload.externalIndex` is now caller-supplied instead of
hardcoded to `0`.
- **New events** — `ValidatorSetSubmitterUpdated`,
`ValidatorSetMessageSubmitted`.
- **New error** — `OnlyValidatorSetSubmitter`.
- **New test suite** — `ValidatorSetSubmitter.t.sol` covering submitter
set/rotate, access control, era encoding, and legacy function removal.
### Pallet changes (`operator/`)
- **`validate_target_era`** in `external-validators` — enforces
`activeEra < targetEra <= activeEra + 1` and `targetEra > ExternalIndex`
(dedup guard).
- **New errors** — `TargetEraTooOld`, `TargetEraTooNew`,
`DuplicateOrStaleTargetEra`.
- **Tests** — five new test cases for era boundary conditions (next-era
acceptance, old-era rejection, too-new rejection, duplicate rejection,
genesis behavior). Existing `era_hooks_with_external_index` test updated
to use valid target eras.
- **Runtime test fixes** — `external_index: 0` → `1` in
mainnet/stagenet/testnet EigenLayer message processor tests to satisfy
the new validation.
### Validator set submitter daemon
(`test/tools/validator-set-submitter/`)
- Event-driven service that subscribes to finalized
`Session.CurrentIndex` via Polkadot-API `watchValue`.
- Submits once per era during the last session, targeting `ActiveEra +
1`.
- Tracks submitted eras to avoid duplicates; skips if `ExternalIndex`
already covers the target.
- Startup self-checks: Ethereum connectivity, DataHaven connectivity,
on-chain submitter authorization.
- Supports `--dry-run` mode and YAML configuration.
- Graceful shutdown on `SIGINT`/`SIGTERM`.
### Test & tooling updates
- **E2E test** (`validator-set-update.test.ts`) — calls
`sendNewValidatorSetForEra` with a computed `targetEra` and filters the
substrate event by `external_index`.
- **`update-validator-set.ts` script** — accepts `--target-era` flag;
defaults to era 1 for fresh networks.
- **CLI launch** — wires validator set update as an interactive step
after relayer launch.
- **`package.json`** — new `submitter` and `submitter:dry-run` scripts.
- Regenerated contract bindings, PAPI metadata, state-diff, and storage
layout snapshots.
### Test plan
- [x] `forge test` — passes, including new `ValidatorSetSubmitter.t.sol`
- [x] `cargo test` — passes, including new era-validation tests in
`external-validators`
- [x] `bun test:e2e` — validator-set-update suite passes with
era-targeted flow
- [x] Manual: run submitter daemon against local network (`bun
submitter`), verify it submits once per era at the correct session
## ⚠️ Breaking Changes ⚠️
- **`sendNewValidatorSet` removed** — replaced by
`sendNewValidatorSetForEra(uint64 targetEra, ...)`. Callers must now
supply a `targetEra` parameter.
- **Access control changed** — validator set submission is now
restricted to the `validatorSetSubmitter` role instead of the contract
`owner`. The submitter address is set during `initialize` and rotatable
via `setValidatorSetSubmitter` (owner-only).
- **`external-validators` pallet now validates `targetEra`** — messages
with a stale, duplicate, or out-of-range `external_index` are rejected
on-chain. Existing integrations sending `external_index: 0` will fail
validation.
---------
Co-authored-by: Cursor <cursoragent@cursor.com>
This PR significantly refactors and improves the end-to-end testing
framework and infrastructure. The primary focus was on simplifying the
test suites, improving reliability through better resource management,
and hardening the relayer infrastructure.
All E2E tests are now passing on the CI and demonstrate consistent
reliability when run locally.
### Key Changes
#### 1. E2E Test Suite Refactor & Cleanup
* **Simplified Test Logic**: Heavily refactored the core test suites
(`native-token-transfer.test.ts`, `rewards-message.test.ts`, and
`validator-set-update.test.ts`). The new implementation is much cleaner,
utilizing shared helpers to reduce boilerplate.
* **Utility Consolidation**: Removed redundant utility files
(`storage.ts`, `rewards-helpers.ts`) and simplified `events.ts`. Event
waiting now uses `rxjs` for Substrate and native `viem` watchers for
Ethereum, which is more robust and easier to maintain.
* **Better Connector Management**: Unified the creation and cleanup of
test clients in `ConnectorFactory`. It now handles the lifecycle of
WebSocket connections more gracefully, including clearing the
`socketClientCache` to prevent reconnection noise during teardown.
#### 2. Infrastructure & Stability
* **Relayer Relaunch Policy**: Added a restart policy for Snowbridge
relayer containers. They are now configured with `--restart
on-failure:5`, ensuring that relayers automatically relaunch if they
crash during the sensitive initialization phase.
* **WebSocket Integration**:
* Updated the `ConnectorFactory` to prefer **WebSockets** for the
Ethereum public client, which is essential for efficient, event-heavy
E2E testing.
* Enhanced `launchKurtosisNetwork` to correctly identify and register
the Execution Layer's WebSocket endpoint from Kurtosis.
* **Disabled Contract Injection**: This PR temporarily disables the
automatic injection of contracts into the genesis state by default.
* *Reason*: I encountered issues generating a valid `state-diff.json`
for the latest contract versions. Even after applying several
workarounds, the injected state remained unstable. As a result, I've
reverted to manual contract deployment during the launch sequence for
better reliability for now.
#### 3. Documentation & Maintenance
* Removed obsolete documentation (`event-utilities-guide.md`) that no
longer reflects the simplified event-handling API.
* Cleaned up `test/launcher/validators.ts` and moved logic into more
appropriate helpers.
---------
Co-authored-by: Steve Degosserie <723552+stiiifff@users.noreply.github.com>
## Summary
Optimizes the DataHaven parameter configuration during E2E test
infrastructure setup by batching multiple extrinsics into a single
transaction, reducing setup time.
## Problem
Setting runtime parameters required 5 separate
`Parameters.set_parameter` calls, each waiting for block finality. This
created unnecessary delays during infrastructure setup since each call
blocked sequentially.
## Solution
- **Batch parameter updates:** Combine all `Parameters.set_parameter
`calls into a single `Utility.batch_all` transaction wrapped in
`Sudo.sudo`.
- **~5× faster parameter setup:** Only wait for finality once instead of
5 separate times
- **Code simplification:** Refactored parameter handling code, removing
~190 lines of unnecessary abstractions and complexity
### New Features
1. Add the `deploy` command to our CLI.
1. Conditionally deploys kurtosis eth network if we're in `stagenet`
environment.
2. Deploys DH nodes.
3. Deploys contracts (all of them). In `mainnet` and `testnet` it
shouldn't deploy EL contracts, but for now that's not implemented.
4. Configures parameters, validators and relayers in the same way as
`launch`.
5. Currently, it only deploys `beefy` and `beacon` relayers, `execution`
and `solochain` relayers are pending for a subsequent PR.
2. Add `waitFor` utility function that receives a lambda.
### Refactors
1. Several common functionalities used both by the `launch` and `deploy`
command have been moved to the `cli/handlers/common` directory, from
where both commands use them. These include
1. Checks for installed dependencies.
2. Common constants.
3. The `LaunchedNetwork` class has been moved to this directory.
4. DataHaven nodes common functions.
5. Kurtosis common functions.
6. Relayer common functions.
7. Kubernetes functions (although only used by `deploy`, it seemed
fitting to have it here still).
8. Remove CLI questions and separator prints from `deploy-contracts.ts`
and `set-datahaven-parameters.ts` scripts. These things should be in the
`cli/launch` folder, which consumes the functions in these scripts.
9. Remove `setParametersFromCollection` from `utils` folder and put it
in `cli`.
10. Create base snowbridge relayer configs for `local` and `stagenet` as
two separate directories.
### Fixes
1. Sets the default time of the `deploy` command to 6s as Lodestar is
slower than Lighthouse.
2. In `runShellCommandWithLogger` only print `stderr` if the command
fails.
### Additional Minor Changes
1. K8s secret key names changed from `dh-beefy-relay-eth-key` to
`dh-beefy-relay-ethereum-key` and `dh-beacon-relay-sub-key` to
`dh-beacon-relay-substrate-key`, for simplicity in the deployment
script.
11. Update suggested configs for `.vscode` configs.
---------
Co-authored-by: Steve Degosserie <723552+stiiifff@users.noreply.github.com>
This PR:
- Adds launching the new `solochain` relayer in the relayers' script,
with its config files and schemas.
- Updates the relayer launching to use switch-case logic since we are
now going to be running 4 relayers, making it cleaner.
- Minor CLI fixes (adding additional args to build command for Linux,
improving naming, deleting idle containers instead of only active ones).
- Deletes unused files `substrate-relay.json` (now
`solochain-relay.json`), `gen-snowbridge-cfgs.ts` and
`snowbridge-relayer.ts` (now Snowbridge binary and docker image
generation can be done exclusively from our [Snowbridge
repo](https://github.com/Moonsong-Labs/snowbridge))
- Updates the `UniversalLocation` of our stagenet runtime to be under
the global consensus for XCM instead of actually being the global
consensus. This makes it so we can actually use the Snowbridge System
pallets to queue up messages. For mainnet we are going to want to have
our own Network ID instead of using Polkadot's.
> [!WARNING]
> ~~All in all these changes allows us to run the solochain relayer, but
it won't work without the refactor that's being worked on in
https://github.com/Moonsong-Labs/snowbridge/pull/14. I'd advise waiting
for that PR to be merged before merging this one.~~ MERGED ✅
## Changes
- New CI file for making Docker Prod images
- Changed E2E tests use an image built from a local dockerfile
- Some cargo build options to make it quicker
- Fix the cache hit rate
- added `tsgo` preview to the project 😎
- Can be invoked with `bun tsgo` to typecheck
- Install in IDE
[VSCode](https://code.visualstudio.com/docs/configure/extensions/extension-marketplace)
& [Zed](https://github.com/zed-extensions/tsgo) for super-fast inline
typechecking (as you type basically)
## Context
This PR attempts to make the frankly unacceptable CI times better. This
achieves that aim by making a crappy image for day-to-day usage and let
the prod issue take ages since that will be infrequently used. The
reason why the original design didn't work for us is because: 1) we are
using the free GH runners 2) when we goto baremetal runners we'll lose
our rapid caching abilities which make using docker cheap.
Also, we add `tsgo` support to improve devex. The improvement is
astounding.
```sh
hyperfine -n tsc "bun tsc --incremental false --extendedDiagnostics" -n tsgo "bun tsgo --incremental false --extendedDiagnostics"
Benchmark 1: tsc
Time (mean ± σ): 5.500 s ± 0.221 s [User: 8.939 s, System: 0.400 s]
Range (min … max): 5.196 s … 5.845 s 10 runs
Benchmark 2: tsgo
Time (mean ± σ): 99.1 ms ± 8.4 ms [User: 392.8 ms, System: 54.1 ms]
Range (min … max): 88.3 ms … 116.0 ms 29 runs
Summary
tsgo ran
55.48 ± 5.22 times faster than tsc
```
This PRs extracts [this
commit](74c0c0be0a)
from @TDemeco's PR to add a way to include parameters as part of the
CLI.
### Key changes
- CLI tool to set DataHaven runtime parameters via JSON configuration
- Supports both interactive prompts and command-line flags
- Type-safe parameter parsing and validation
- Already adds the parameters for the `EthereumGatewayAddress`, that
otherwise we would need to add manually to the node using an explorer.
---------
Co-authored-by: TDemeco <tdemeco@itba.edu.ar>
