## Add E2E validator-set update flow
- feat: `test/utils/validators.ts` for on-demand validator
orchestration.
- feat: `test/suites/validator-set-update.test.ts` covering allowlist →
register → update.
- some minor launcher updates: avoid docker cache, add `--platform` when
building datahaven image, avoid sending validator-set update on launch.
- Helpers: ABI shortcut in `test/utils/contracts.ts`; config tweaks in
`test/configs/validator-set.json`.
- Minor cleanup/formatting across `test/launcher/*`,
`test/scripts/setup-validators.ts`, and related tests.
- added `keepAlive` flag to `BaseTestSuite`, in order to avoid tearing
down the network while debugging. Defaults, obviously, to false.
- added a `failOnTomeout` option on to waitForDataHavenEvents() so the
test fails of the timeout is reached and no event was captured.
### Coverage
- The test simulates an scenario in which we have two active authorities
(alice and bob), which are running, and registered as operators, which
is the normal state after the chain launches. Then:
- It launches two more nodes (charlie and dave)
- It add the nodes to allowlist and register them as operators
- It sends the validator set update message
- Checks that the validator update message was propagated through the
gateway and arrived the external-validators pallet
- Checks that the chain continues producing blocks
### Notes
The last test case has a timeout of 10 minutes. This is to respect
propagation times of the message through the relayers. We are testing
that the external validators pallet actually updated the validator set.
Locally, I could expect 5~6 minutes, I just wanted to be on the safe
side. CI is passing showing that this was enough indeed.
---------
Co-authored-by: Steve Degosserie <723552+stiiifff@users.noreply.github.com>
Co-authored-by: Ahmad Kaouk <56095276+ahmadkaouk@users.noreply.github.com>
## Summary
This PR introduces comprehensive event waiting utilities for both
DataHaven (Substrate) and Ethereum chains, providing a unified
interface for handling blockchain events in E2E tests.
## What's New
- **Event Utilities** (`test/utils/events.ts`): New utilities for
waiting on blockchain events
- `waitForDataHavenEvent`: Type-safe event waiting for Substrate chain
events
- `waitForEthereumEvent`: Event waiting for Ethereum contract events
- Graceful timeout handling (returns null instead of throwing)
- Support for event filtering, callbacks, and custom timeouts
- **Documentation** (`test/docs/event-utilities-guide.md`):
Comprehensive guide covering usage examples for both DataHaven and
Ethereum.
## Test Plan
- [ ] New event utilities work as expected
- [ ] Event filtering works correctly for both chains
- [ ] Timeout handling behaves as documented
- [ ] Parallel event waiting with `Promise.all()` works
---------
Co-authored-by: Claude <noreply@anthropic.com>
## Implement E2E Testing Framework with Isolated Networks
### Summary
Refactors the existing E2E testing infrastructure to provide isolated
test environments with parallel execution support. Each test suite now
runs in its own network namespace, preventing resource conflicts.
### Key Changes
- **New Testing Framework** (`test/framework/`): Base classes for test
lifecycle management with automatic setup/teardown
- **Launcher Module** (`test/launcher/`): Extracted network
orchestration logic from CLI handlers for reusability
- **Parallel Execution**: Added `test-parallel.ts` script with
concurrency limits to prevent resource exhaustion
- **Test Isolation**: Each suite gets unique network IDs (format:
`suiteName-timestamp`) and Docker networks
- **Improved Test Organization**: Migrated tests to new framework,
deprecated old test structure
### Test Improvements
- Added 4 new test suites demonstrating framework usage. :
- `contracts.test.ts` - Smart contract deployment/interaction
- `datahaven-substrate.test.ts` - Substrate API operations
- `cross-chain.test.ts` - Snowbridge cross-chain messaging
- `ethereum-basic.test.ts` - Ethereum network operations
> [!WARNING]
The test suites themselves are bad and shouldn't be consider examples of
good tests. They were AI generated just to test the concurrency of test
runners
### Documentation
- Added comprehensive framework overview (`E2E_FRAMEWORK_OVERVIEW.md`)
- Updated README with parallel testing commands
- Added test patterns and best practices
### Breaking Changes
- Old test suites moved to `e2e - DEPRECATED/` directory
- Test execution now requires extending `BaseTestSuite` class
### Testing
Run tests with: `bun test:e2e` or `bun test:e2e:parallel` (with
concurrency limits)
### TODO
- [ ] Implement good test examples.
- [ ] Implement useful test utils (like waiting for an event to show up
in DataHaven or Ethereum).
- [ ] Enforce tests with CI (currently cannot be done due to
intermittent error when sending a transaction with PAPI).
---------
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: undercover-cactus <lola@moonsonglabs.com>
In this PR:
1. Add [Polkadot API](https://papi.how/) support to `test/` directory
(i.e. the e2e CLI and testing framework) to be able to interact with the
Substrate chain.
1. This allows typed interactions with transactions, constants, storage,
runtime APIs, and non-typed interaction with RPC methods.
2. Types are autogenerated when running `bun i`, from the
`datahaven.scale` file that is part of this repo's version control.
2. Add new utilities file to `papi` related functionalities. For the
time being, generating a new signer from a private key.
3. ~Add a new step to the CLI that sends a transaction to the DataHaven
network. _*THIS SHOULD BE REMOVED SOON, IT'S JUST FOR TESTING
PURPOSES*_~
1. Both steps that send test transactions have been removed from the
CLI, for convenience and being error prone. Their scripts remain usable
for testing purposes if needed.
4. Removes the `apis.rs` files from the runtime definitions. Having them
in a separate file meant that the runtime was not including the Runtime
APIs in the metadata blob, preventing `papi` from creating types from
it. This change can be reapplied after upgrading to `polkadot-sdk`
`stable-2503`. More info
[here](https://github.com/paritytech/polkadot-sdk/issues/6659).
5. Add script to re-generate types, and corresponding docs.
6. Makes logger synchronous to avoid prints happening before logs.
Eventually our CI will be required to run two private blockchains
locally plus associated relayers.
This PR is to prepare for this fate by improving run times and
refactoring our existing CIs so they are a bit easier to reason about.
### Refactors
- **_We now run ALL CIs on every PR!_** This is so that we decomplexify
the logic around conditional builds and fetching built binaries from
another source. This reduces the surface area of code we have to
maintain at the cost of execution time
- This penalty is ameliorated by a layered caching system. At best, it
will be less than a minute to complete a build since everything will be
cached. On GH runners this is about 6 minutes sadly.
- We will no longer be at risk of important CIs being skipped
erroneously which hide true failures.
- Caching is a low-risk approach because at worst it has to build from
scratch. A bad cache hit will never imply the wrong thing gets build
since cargo is smart enough to just throw away any inappropriate build
artefacts.
- `setup-rust` action created so we have a unified way of setting up
runner and unifying our approach to caching
- Use a unique caching key for different activities and it will fallback
to shared cache if no matches
- we are using `mainnet` kurtosis config so that it works with relayer
assumptions
### Additions
- We can specify the ethereum block time via a new cli arg `--slot-time
<seconds>`
- We can specify arbitrary network_param args which get passed into the
generated yaml
- e.g. giving `bun cli --kurtosis-network-args="pet=cat food=fish" will
add:
```yml
network_params:
# existing params...
pet: cat
food: fish
```
- We now have the ability to programmatically modify the yaml
- This means we are back down to a single `minimal.yml` kurtosis config
so we dont have to maintain changes between them
- Flow is: `add new cli arg` -> `add if() block which mutates yaml` ->
`profit`
---------
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Facundo Farall <37149322+ffarall@users.noreply.github.com>
## Summary
This PR adds statically typed bindings for contracts. This allows you to
write E2E tests with full completions in TS.
## Additions
- `ts-build.yml` New CI, this will make sure that if there's changes
made to the contracts that the contract-bindings are up to date.
- `package.json` script changes
- `start:e2e:ci` - Designed to be run with all options specified since
CIs are famously bad with iteractive CLI prompts
- `test:e2e` - added timeout
- `generate:wagmi` - This generates the smart contract bindings for our
tests
- New Function Helpers:
- `generateRandomAccount()` Returns a viem account type object for a
random account. Useful for tests where you want idempotency on a long
lived network since the state is probabilistically fresh
- `getContractInstance()` Returns a viem contract instance that allows
you to read/write to the deployed contract. You should get full type
inference here for the methods available and parameters required.
### Example
```ts
it("avs() can be read from contract instance", async () => {
const value = await instance.read.avs();
expect(isAddress(value), "AVS getter should return an address").toBeTrue();
});
```
---------
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Facundo Farall <37149322+ffarall@users.noreply.github.com>
## Human Written Description
This PR adds the following to the E2E CLI:
- Relayer config generation for: `beacon-relay` `beefy-relay`
- The other two relayer types to be added later
- Relayers don't actually work yet
- By default turned off, this requires a binary to be present in:
`<repo_root>/operator/target/release` dir
- Datahaven network launching
- DH network is using default `local` network chain spec
- Launched with 5 nodes since our authority set is 6 large (and you need
2/3 + 1 of set size
> [!NOTE]
> Both the relayer and the DH node binaries are being run as local
processes TEMPORARILY. This means that logging is done in a very
rudimentary way (we pipe to a file whilst the CLI is running).
>
> This means that when the CLI finishes **the log files will no longer
be written to**.
> This is temporary since spawning binaries is a stop gap solution until
docker images available.
---
> [!IMPORTANT]
> The following is AI generated slop describing this PR's changes:
**Key Changes:**
* **CLI Enhancements (`test/cli/index.ts`):**
* Added options `--datahaven` and `--datahaven-bin-path` to enable
launching local DataHaven nodes.
* Added options `--relayer` and `--relayer-bin-path` to enable launching
Snowbridge relayers (Beefy and Beacon).
* Added negation flags (`--no-fund-validators`, `--no-setup-validators`,
`--no-update-validator-set`) for more granular control over validator
setup steps.
* Added `--skip-cleaning` option to preserve Kurtosis state between
runs.
* Added a pre-action hook (`launchPreActionHook`) to validate flag
combinations (e.g., `--verified` requires `--blockscout`).
* **New CLI Handlers (`test/cli/handlers/launch/`):**
* `datahaven.ts`: Logic for spawning DataHaven node processes using the
specified binary. Manages ports and process cleanup.
* `relayer.ts`: Logic for configuring and spawning Snowbridge relayer
processes (Beefy and Beacon). Reads contract deployment addresses,
updates relayer config templates, and uses specified private keys.
Manages log files and process cleanup.
* `summary.ts`: Generates and displays the table of running services
(including dynamically launched DataHaven nodes) and their endpoints.
* `validator.ts`: Extracted validator funding, setup, and set update
logic into its own handler.
* `index.ts`: Orchestrates the launch sequence based on CLI options,
calling the appropriate handlers. Includes a `LaunchedNetwork` class to
track spawned processes, file descriptors, and node ports for cleanup.
* **Updated `package.json` Scripts:**
* Added `start:e2e:minrelayer` script for a minimal setup including
relayers and DataHaven nodes.
* Modified `stop:e2e` to include `pkill datahaven` for proper cleanup.
* Added `stop:e2e:quick` to only stop the Kurtosis enclave without full
cleaning.
* **Updated `launch-kurtosis.ts`:** Modified to use new Kurtosis utility
functions and added a `skipCleaning` option.
* **New Utility Functions:**
* `test/utils/kurtosis.ts`: Functions to inspect Kurtosis services
(`getServiceFromKurtosis`, `getPortFromKurtosis`,
`getServicesFromKurtosis`).
* `test/utils/parser.ts`: Zod schemas and parsing functions for
Snowbridge relayer configurations.
* **Constants & Minor Updates:**
* Added `SUBSTRATE_FUNDED_ACCOUNTS` to `test/utils/constants.ts`.
* Updated `tsconfig.json` include paths.
* Refactored `test/utils/docker.ts` (though now largely superseded by
Kurtosis utils).
* Updated logging in `test/scripts/send-txn.ts`.
**Reasoning:**
This PR significantly expands the E2E testing capabilities by allowing
developers to easily launch and integrate local DataHaven nodes and
Snowbridge relayers into the test network, facilitating more
comprehensive integration testing. The CLI refactoring makes managing
these complex setups more robust and user-friendly.
WARNING: This PR changes the kurtosis package to use the one from
upstream, not our fork, as it was not working at the moment. This should
be changed when fixed.
In this PR:
1. Turn `launch-kurtosis` script into a CLI, which parses parameters and
can interactively run multiple steps.
2. Separate steps of such CLI into their own scripts.
1. New script created `launch-kurtosis`, which detects if an enclave is
running and prompts to relaunch it if it is.
2. New script created `deploy-contracts` to deploy all contracts. It can
optionally verify them as well.
3. Each step can be interactively opted in/out, or choose whether to run
it via CLI params.
4. The CLI offers a help command as well.
5. Cleanup logs of CLI. Logs of internal commands ran can be printed
with LOG_LEVEL=debug. In case of error, they are always printed out.
This PR adds the basic framework for E2E and the greater typescript
testing framework for the repo.
### Contents
- CI workflow for running E2E tests on push to main, PRs and manual
- Test suite for E2E
- Currently only read-only tests
- Using `bun:test` for time being but we can always change in future
- We are using the logging library `pino` so we can help with debugging
- set environment variable `LOG_LEVEL` to get extra information on
helpers
- Local `utils` namespace for all our test suites for easy importing
- Has helpers which interact with both the docker driver and also
blockscout backend
- Allows us to fetch deployed smart contracts by their name or address
and interact with them
---------
Co-authored-by: Facundo Farall <37149322+ffarall@users.noreply.github.com>
