This PR:
1. Generally improves the logging of the testing CLI, making the logs
more concise and easier to follow, with clearer sections and
separations.
2. Launches DataHaven solochain nodes at the beginning not the end.
3. Prompts the user if they want to launch DataHaven nodes and
Snowbridge Relayers.
---------
Co-authored-by: Tim B <79199034+timbrinded@users.noreply.github.com>
Eventually our CI will be required to run two private blockchains
locally plus associated relayers.
This PR is to prepare for this fate by improving run times and
refactoring our existing CIs so they are a bit easier to reason about.
### Refactors
- **_We now run ALL CIs on every PR!_** This is so that we decomplexify
the logic around conditional builds and fetching built binaries from
another source. This reduces the surface area of code we have to
maintain at the cost of execution time
- This penalty is ameliorated by a layered caching system. At best, it
will be less than a minute to complete a build since everything will be
cached. On GH runners this is about 6 minutes sadly.
- We will no longer be at risk of important CIs being skipped
erroneously which hide true failures.
- Caching is a low-risk approach because at worst it has to build from
scratch. A bad cache hit will never imply the wrong thing gets build
since cargo is smart enough to just throw away any inappropriate build
artefacts.
- `setup-rust` action created so we have a unified way of setting up
runner and unifying our approach to caching
- Use a unique caching key for different activities and it will fallback
to shared cache if no matches
- we are using `mainnet` kurtosis config so that it works with relayer
assumptions
### Additions
- We can specify the ethereum block time via a new cli arg `--slot-time
<seconds>`
- We can specify arbitrary network_param args which get passed into the
generated yaml
- e.g. giving `bun cli --kurtosis-network-args="pet=cat food=fish" will
add:
```yml
network_params:
# existing params...
pet: cat
food: fish
```
- We now have the ability to programmatically modify the yaml
- This means we are back down to a single `minimal.yml` kurtosis config
so we dont have to maintain changes between them
- Flow is: `add new cli arg` -> `add if() block which mutates yaml` ->
`profit`
---------
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Facundo Farall <37149322+ffarall@users.noreply.github.com>
## Human Written Description
This PR adds the following to the E2E CLI:
- Relayer config generation for: `beacon-relay` `beefy-relay`
- The other two relayer types to be added later
- Relayers don't actually work yet
- By default turned off, this requires a binary to be present in:
`<repo_root>/operator/target/release` dir
- Datahaven network launching
- DH network is using default `local` network chain spec
- Launched with 5 nodes since our authority set is 6 large (and you need
2/3 + 1 of set size
> [!NOTE]
> Both the relayer and the DH node binaries are being run as local
processes TEMPORARILY. This means that logging is done in a very
rudimentary way (we pipe to a file whilst the CLI is running).
>
> This means that when the CLI finishes **the log files will no longer
be written to**.
> This is temporary since spawning binaries is a stop gap solution until
docker images available.
---
> [!IMPORTANT]
> The following is AI generated slop describing this PR's changes:
**Key Changes:**
* **CLI Enhancements (`test/cli/index.ts`):**
* Added options `--datahaven` and `--datahaven-bin-path` to enable
launching local DataHaven nodes.
* Added options `--relayer` and `--relayer-bin-path` to enable launching
Snowbridge relayers (Beefy and Beacon).
* Added negation flags (`--no-fund-validators`, `--no-setup-validators`,
`--no-update-validator-set`) for more granular control over validator
setup steps.
* Added `--skip-cleaning` option to preserve Kurtosis state between
runs.
* Added a pre-action hook (`launchPreActionHook`) to validate flag
combinations (e.g., `--verified` requires `--blockscout`).
* **New CLI Handlers (`test/cli/handlers/launch/`):**
* `datahaven.ts`: Logic for spawning DataHaven node processes using the
specified binary. Manages ports and process cleanup.
* `relayer.ts`: Logic for configuring and spawning Snowbridge relayer
processes (Beefy and Beacon). Reads contract deployment addresses,
updates relayer config templates, and uses specified private keys.
Manages log files and process cleanup.
* `summary.ts`: Generates and displays the table of running services
(including dynamically launched DataHaven nodes) and their endpoints.
* `validator.ts`: Extracted validator funding, setup, and set update
logic into its own handler.
* `index.ts`: Orchestrates the launch sequence based on CLI options,
calling the appropriate handlers. Includes a `LaunchedNetwork` class to
track spawned processes, file descriptors, and node ports for cleanup.
* **Updated `package.json` Scripts:**
* Added `start:e2e:minrelayer` script for a minimal setup including
relayers and DataHaven nodes.
* Modified `stop:e2e` to include `pkill datahaven` for proper cleanup.
* Added `stop:e2e:quick` to only stop the Kurtosis enclave without full
cleaning.
* **Updated `launch-kurtosis.ts`:** Modified to use new Kurtosis utility
functions and added a `skipCleaning` option.
* **New Utility Functions:**
* `test/utils/kurtosis.ts`: Functions to inspect Kurtosis services
(`getServiceFromKurtosis`, `getPortFromKurtosis`,
`getServicesFromKurtosis`).
* `test/utils/parser.ts`: Zod schemas and parsing functions for
Snowbridge relayer configurations.
* **Constants & Minor Updates:**
* Added `SUBSTRATE_FUNDED_ACCOUNTS` to `test/utils/constants.ts`.
* Updated `tsconfig.json` include paths.
* Refactored `test/utils/docker.ts` (though now largely superseded by
Kurtosis utils).
* Updated logging in `test/scripts/send-txn.ts`.
**Reasoning:**
This PR significantly expands the E2E testing capabilities by allowing
developers to easily launch and integrate local DataHaven nodes and
Snowbridge relayers into the test network, facilitating more
comprehensive integration testing. The CLI refactoring makes managing
these complex setups more robust and user-friendly.
This PR adds the `setup-validators` Typescript script that, given an
already started up network, sets up a new validator set and sends it
through Snowbridge's Gateway to the solochain. To accomplish that
purpose, this PR:
- Modifies the `DeployLocal` script to save in the `anvil.json` file not
only the deployed strategies' addresses but also the owner of each
strategy's underlying token. This owner is used as the source of funds
to transfer tokens to other validators so they can register under that
strategy.
- Adds an `OPERATOR_SOLOCHAIN_ADDRESS` to the `Accounts` utility script
contract. This address is the one used as the Solochain address when
registering a new Operator.
- Updates the `SignUpOperator` (which I believe is now deprecated since
we have multiple Operator Sets) and `SignUpOperatorBase` scripts to
adapt to both aforementioned changes.
- Updates the `ELScriptStorage` script to save the new extra information
of each deployed strategy (the creator of the underlying token) in
storage.
- Adds a `validator-set.json` file which contains the validators that
should be registered in the AVS and sent to the Solochain network
through the Snowbridge Gateway when starting any integration test. This
is currently hardcoded but could be generated in any other way, giving
us flexibility for testing.
- Adds both a Markdown file and a Excalidraw diagram showcasing both how
the setup of integration tests work and possible integration tests that
will be added in a future PR. This list is not exahustive as there are
many more scenarios we will want to test using integration tests.
- Updates the `e2e-cli.ts` script to execute the validator setup when
bootstrapping the network used for integration testing.
---------
Co-authored-by: Facundo Farall <37149322+ffarall@users.noreply.github.com>
