Elgato_dark/datahaven
1
0
Fork 0
mirror of https://github.com/datahaven-xyz/datahaven synced 2026-05-24 09:50:01 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions
datahaven/test/package.json

85 lines
3.9 KiB
JSON
Raw Permalink Normal View History

test: ⚙️ Add Kurtosis Launch Script (#34) This PR adds a script one-liner to spin up an ethereum network, with attached explorers and deploys our `/contracts` folder into there. --------- Co-authored-by: Facundo Farall <37149322+ffarall@users.noreply.github.com>
2025-04-08 15:42:45 +00:00
{
"name": "@datahaven/e2e-test",
"module": "index.ts",
"type": "module",
"private": true,
"scripts": {
test: ⚙️ Parse & Generate Relayer Configs (#54) ## Human Written Description This PR adds the following to the E2E CLI: - Relayer config generation for: `beacon-relay` `beefy-relay` - The other two relayer types to be added later - Relayers don't actually work yet - By default turned off, this requires a binary to be present in: `<repo_root>/operator/target/release` dir - Datahaven network launching - DH network is using default `local` network chain spec - Launched with 5 nodes since our authority set is 6 large (and you need 2/3 + 1 of set size > [!NOTE] > Both the relayer and the DH node binaries are being run as local processes TEMPORARILY. This means that logging is done in a very rudimentary way (we pipe to a file whilst the CLI is running). > > This means that when the CLI finishes **the log files will no longer be written to**. > This is temporary since spawning binaries is a stop gap solution until docker images available. --- > [!IMPORTANT] > The following is AI generated slop describing this PR's changes: **Key Changes:** * **CLI Enhancements (`test/cli/index.ts`):** * Added options `--datahaven` and `--datahaven-bin-path` to enable launching local DataHaven nodes. * Added options `--relayer` and `--relayer-bin-path` to enable launching Snowbridge relayers (Beefy and Beacon). * Added negation flags (`--no-fund-validators`, `--no-setup-validators`, `--no-update-validator-set`) for more granular control over validator setup steps. * Added `--skip-cleaning` option to preserve Kurtosis state between runs. * Added a pre-action hook (`launchPreActionHook`) to validate flag combinations (e.g., `--verified` requires `--blockscout`). * **New CLI Handlers (`test/cli/handlers/launch/`):** * `datahaven.ts`: Logic for spawning DataHaven node processes using the specified binary. Manages ports and process cleanup. * `relayer.ts`: Logic for configuring and spawning Snowbridge relayer processes (Beefy and Beacon). Reads contract deployment addresses, updates relayer config templates, and uses specified private keys. Manages log files and process cleanup. * `summary.ts`: Generates and displays the table of running services (including dynamically launched DataHaven nodes) and their endpoints. * `validator.ts`: Extracted validator funding, setup, and set update logic into its own handler. * `index.ts`: Orchestrates the launch sequence based on CLI options, calling the appropriate handlers. Includes a `LaunchedNetwork` class to track spawned processes, file descriptors, and node ports for cleanup. * **Updated `package.json` Scripts:** * Added `start:e2e:minrelayer` script for a minimal setup including relayers and DataHaven nodes. * Modified `stop:e2e` to include `pkill datahaven` for proper cleanup. * Added `stop:e2e:quick` to only stop the Kurtosis enclave without full cleaning. * **Updated `launch-kurtosis.ts`:** Modified to use new Kurtosis utility functions and added a `skipCleaning` option. * **New Utility Functions:** * `test/utils/kurtosis.ts`: Functions to inspect Kurtosis services (`getServiceFromKurtosis`, `getPortFromKurtosis`, `getServicesFromKurtosis`). * `test/utils/parser.ts`: Zod schemas and parsing functions for Snowbridge relayer configurations. * **Constants & Minor Updates:** * Added `SUBSTRATE_FUNDED_ACCOUNTS` to `test/utils/constants.ts`. * Updated `tsconfig.json` include paths. * Refactored `test/utils/docker.ts` (though now largely superseded by Kurtosis utils). * Updated logging in `test/scripts/send-txn.ts`. **Reasoning:** This PR significantly expands the E2E testing capabilities by allowing developers to easily launch and integrate local DataHaven nodes and Snowbridge relayers into the test network, facilitating more comprehensive integration testing. The CLI refactoring makes managing these complex setups more robust and user-friendly.
2025-04-29 12:24:00 +00:00
"cli": "bun run cli/index.ts",
test: ⚙️ Add Kurtosis Launch Script (#34) This PR adds a script one-liner to spin up an ethereum network, with attached explorers and deploys our `/contracts` folder into there. --------- Co-authored-by: Facundo Farall <37149322+ffarall@users.noreply.github.com>
2025-04-08 15:42:45 +00:00
"fmt": "biome check .",
"fmt:fix": "biome check --write .",
refactor: Consolidate and optimize Docker image architecture (#233) ## Overview This PR consolidates and optimizes the Docker build system, reducing redundancy and improving CI/CD performance. The changes eliminate duplicate Dockerfiles, introduce a flexible build template, and optimize release builds to reuse CI artifacts. ## Changes Summary ### 🐳 Docker Images Restructured **Before:** 5 Dockerfiles with significant overlap **After:** 4 focused images + 1 utility #### Final Structure: 1. **`operator/Dockerfile`** ✨ Updated - **Standard operator image** for CI and release builds - Minimal node image (accepts pre-built binaries) - GHCR: `ghcr.io/datahaven-xyz/datahaven/datahaven` (CI) - DockerHub: `datahavenxyz/datahaven` (releases) 2. **`docker/datahaven-build.Dockerfile`** (moved from `operator/Dockerfile`) - Full source-to-binary build for manual releases - DockerHub: `datahavenxyz/datahaven:{label}` - Supports custom RUSTFLAGS and fast-runtime feature - Only used for manual workflow_dispatch builds 3. **`docker/datahaven-production.Dockerfile`** (kept) - Binary builder for CPU-specific releases - Used by build-prod-binary workflow template - Supports custom target-cpu flags 4. **`docker/datahaven-dev.Dockerfile`** ✨ NEW (local dev only) - **FOR LOCAL DEVELOPMENT/TROUBLESHOOTING ONLY** - Includes debug tools: gdb, strace, vim, sudo - Extra dependencies: librocksdb-dev, curl - RUST_BACKTRACE enabled by default - **DO NOT USE for CI or production builds** 5. **`test/docker/crossbuild-mac-libpq.dockerfile`** (kept) - Utility for macOS → Linux cross-compilation #### Removed (Redundant): - ❌ `docker/datahaven.Dockerfile` → replaced by operator/Dockerfile - ❌ `test/docker/datahaven-node-local.dockerfile` → replaced by datahaven-dev.Dockerfile --- ### 🔄 Workflow Improvements #### Enhanced `publish-docker` Template - Supports both GHCR and DockerHub registries - Flexible inputs: dockerfile, context, build-args, cache scope - Auto-generates OCI-compliant labels - Reduces code duplication (~70 lines → ~15 per workflow) #### Refactored CI Pipeline - **`docker-build-ci`**: Builds `operator/Dockerfile` → GHCR for CI/E2E testing - **`docker-build-release`**: Builds `operator/Dockerfile` → DockerHub (main branch only) - Both CI and release workflows now use the same minimal operator image - Release builds **reuse CI binaries** instead of rebuilding from source #### Optimized Release Workflow The `task-docker-release` workflow now has dual modes: **Mode 1: `workflow_call` (CI - main pushes)** - ✅ Reuses binary from CI's build-operator task - ✅ Uses lightweight `operator/Dockerfile` - ✅ Tags: `latest`, `sha-{short}` - ⚡ **Fast**: ~5 minutes (vs ~30 min previously) **Mode 2: `workflow_dispatch` (Manual)** - ✅ Full source build with `datahaven-build.Dockerfile` - ✅ Custom branch and label selection - ✅ Optional fast-runtime feature - ✅ Tags: `PROD-{label}` or user-defined --- ### 🔧 Additional Optimizations - Copy libpq5 from builder stage instead of reinstalling (smaller, faster) - Remove redundant protobuf-compiler package (use protoc v21.12 directly) - Standardize user UID to 1000 across all runtime images - Consistent OCI labeling and metadata --------- Co-authored-by: Claude <noreply@anthropic.com>
2025-10-14 23:33:20 +00:00
"build:docker:operator": "docker build --no-cache --platform linux/amd64 -t datahavenxyz/datahaven:local -f ../docker/datahaven-dev.Dockerfile ../.",
test: 🧙 Generate Type Bindings for Contracts (#58) ## Summary This PR adds statically typed bindings for contracts. This allows you to write E2E tests with full completions in TS. ## Additions - `ts-build.yml` New CI, this will make sure that if there's changes made to the contracts that the contract-bindings are up to date. - `package.json` script changes - `start:e2e:ci` - Designed to be run with all options specified since CIs are famously bad with iteractive CLI prompts - `test:e2e` - added timeout - `generate:wagmi` - This generates the smart contract bindings for our tests - New Function Helpers: - `generateRandomAccount()` Returns a viem account type object for a random account. Useful for tests where you want idempotency on a long lived network since the state is probabilistically fresh - `getContractInstance()` Returns a viem contract instance that allows you to read/write to the deployed contract. You should get full type inference here for the methods available and parameters required. ### Example ```ts it("avs() can be read from contract instance", async () => { const value = await instance.read.avs(); expect(isAddress(value), "AVS getter should return an address").toBeTrue(); }); ``` --------- Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> Co-authored-by: Facundo Farall <37149322+ffarall@users.noreply.github.com>
2025-05-01 10:14:19 +00:00
"generate:wagmi": "wagmi generate",
test: 🧪 Initial Relay Work (#52) Small changes to help progress us closer to relayer support --------- Co-authored-by: Facundo Farall <37149322+ffarall@users.noreply.github.com>
2025-04-23 15:28:19 +00:00
"generate:snowbridge-cfgs": "bun -e \"import {generateSnowbridgeConfigs} from './scripts/gen-snowbridge-cfgs.ts'; await generateSnowbridgeConfigs()\"",
feat: ✨ Add Polkadot API support to CLI and e2e testing infra (#68) In this PR: 1. Add [Polkadot API](https://papi.how/) support to `test/` directory (i.e. the e2e CLI and testing framework) to be able to interact with the Substrate chain. 1. This allows typed interactions with transactions, constants, storage, runtime APIs, and non-typed interaction with RPC methods. 2. Types are autogenerated when running `bun i`, from the `datahaven.scale` file that is part of this repo's version control. 2. Add new utilities file to `papi` related functionalities. For the time being, generating a new signer from a private key. 3. ~Add a new step to the CLI that sends a transaction to the DataHaven network. _*THIS SHOULD BE REMOVED SOON, IT'S JUST FOR TESTING PURPOSES*_~ 1. Both steps that send test transactions have been removed from the CLI, for convenience and being error prone. Their scripts remain usable for testing purposes if needed. 4. Removes the `apis.rs` files from the runtime definitions. Having them in a separate file meant that the runtime was not including the Runtime APIs in the metadata blob, preventing `papi` from creating types from it. This change can be reapplied after upgrading to `polkadot-sdk` `stable-2503`. More info [here](https://github.com/paritytech/polkadot-sdk/issues/6659). 5. Add script to re-generate types, and corresponding docs. 6. Makes logger synchronous to avoid prints happening before logs.
2025-05-13 03:03:21 +00:00
"generate:types": "(cd ../operator && cargo build --release) && bun x papi add --wasm \"../operator/target/release/wbuild/datahaven-stagenet-runtime/datahaven_stagenet_runtime.wasm\" datahaven",
refactor: :truck: rename relayer image to adapt to snowbridge repo refactor (#92) This PR renames all usage of `moonsonglabs/snowbridge-relayer` to `moonsonglabs/snowbridge-relay` to adapt to the refactor done in [PR #18](https://github.com/Moonsong-Labs/snowbridge/pull/18) of our Snowbridge fork. The aforementioned PR should be merged before this one, although caution is not really needed since the CI would fail otherwise.
2025-06-10 17:02:14 +00:00
"generate:types:fast": "(cd ../operator && cargo build --release --features fast-runtime) && bun x papi add --wasm \"../operator/target/release/wbuild/datahaven-stagenet-runtime/datahaven_stagenet_runtime.wasm\" datahaven",
test: ✨ New CLI functions (#84) ## Changes - New option: `--kurtosis-enclave-name` to allow you to specify a new ethereum network with a different enclave name. Neccesary step for setting up local testing with multiple enclaves running at once - Refactor: all single dash options (e.g. `-i`) have been renamed to have double dashes `--` for consistency others - sad times: CLI now must be invoked `bun cli launch` since we have multiple fns now - Package Update - New Function: `stop` which allows you to kill all components (eth, dh, relayers) or only a single part - Gonza's mac target build fix - Misc fixes to ci like caching and rate limits ## Additional Comment The CLI needs multiple commands and this PR adds the first new one `stop`. This syntax is faily extensible so @ffarall 's k8 command can follow suite. Originally we were going to have an `exec` command to expose internal fns to be callable by command line, i.e. generate beacon checkpoint - however the need for it has since been superceded by the deploy to k8 command that's going to be added. I've left the code in place though so when another usecase comes up we can just plug that in. --------- Co-authored-by: Facundo Farall <37149322+ffarall@users.noreply.github.com>
2025-05-22 20:02:12 +00:00
"start:e2e:verified": "bun cli launch --verified --blockscout --deploy-contracts --setup-validators --update-validator-set --fund-validators",
fix: :bug: Use lodestar instead of lighthouse CL client (#91) Lighthouse consensus layer client has a bug when building the next sync committee merkle proof, that is preventing the Snowbridge relayer from updating the next sync committee in the Ethereum Beacon Client pallet, on the DataHaven Substrate chain. See issue [here](https://github.com/sigp/lighthouse/issues/7552). As a consequence, we're moving to the lodestar implementation.
2025-06-09 15:29:31 +00:00
"start:e2e:verified:relayers": "bun cli launch --verified --blockscout --deploy-contracts --setup-validators --update-validator-set --fund-validators --relayer --datahaven",
test: 🐳 Add Docker relay support to CLI (#74) ## Changes - Latest changes to have working relayer :tada: component - Changed spawning snowbridge relayers to docker containers - Small logging output changes - Refactoring to `LaunchedNetwork` class - new flag `--bd` `--build-datahaven` which will build a local docker container which is **much** quicker than the proper CI build (which uses a controlled build enviroment) - new bun script `start:e2e:local`, which is everything that `start:e2e:ci` has, but with building local docker container and log_level debug set --- <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit ## Summary by CodeRabbit - **New Features** - Added support for launching and managing relayer and DataHaven services using Docker containers and networks. - Introduced a CLI option to specify the relayer Docker image tag instead of a binary path. - **Improvements** - Enhanced log messages with clearer text and expressive emojis for better user feedback. - Improved summary display by removing relayer services from the output. - Updated build scripts to consistently enable the "fast-runtime" feature for cross-platform builds. - Refined validation and error reporting for checkpoint data parsing. - **Bug Fixes** - Improved Docker container cleanup and network management during service launch and teardown. - **Chores** - Updated and refactored npm scripts for Docker operations and end-to-end test cleanup. <!-- end of auto-generated comment: release notes by coderabbit.ai --> --------- Co-authored-by: Facundo Farall <37149322+ffarall@users.noreply.github.com>
2025-05-18 23:31:46 +00:00
"start:e2e:local": "LOG_LEVEL=debug bun start:e2e:ci --bd",
feat: 🚀 Add `deploy` command to CLI (#87) ### New Features 1. Add the `deploy` command to our CLI. 1. Conditionally deploys kurtosis eth network if we're in `stagenet` environment. 2. Deploys DH nodes. 3. Deploys contracts (all of them). In `mainnet` and `testnet` it shouldn't deploy EL contracts, but for now that's not implemented. 4. Configures parameters, validators and relayers in the same way as `launch`. 5. Currently, it only deploys `beefy` and `beacon` relayers, `execution` and `solochain` relayers are pending for a subsequent PR. 2. Add `waitFor` utility function that receives a lambda. ### Refactors 1. Several common functionalities used both by the `launch` and `deploy` command have been moved to the `cli/handlers/common` directory, from where both commands use them. These include 1. Checks for installed dependencies. 2. Common constants. 3. The `LaunchedNetwork` class has been moved to this directory. 4. DataHaven nodes common functions. 5. Kurtosis common functions. 6. Relayer common functions. 7. Kubernetes functions (although only used by `deploy`, it seemed fitting to have it here still). 8. Remove CLI questions and separator prints from `deploy-contracts.ts` and `set-datahaven-parameters.ts` scripts. These things should be in the `cli/launch` folder, which consumes the functions in these scripts. 9. Remove `setParametersFromCollection` from `utils` folder and put it in `cli`. 10. Create base snowbridge relayer configs for `local` and `stagenet` as two separate directories. ### Fixes 1. Sets the default time of the `deploy` command to 6s as Lodestar is slower than Lighthouse. 2. In `runShellCommandWithLogger` only print `stderr` if the command fails. ### Additional Minor Changes 1. K8s secret key names changed from `dh-beefy-relay-eth-key` to `dh-beefy-relay-ethereum-key` and `dh-beacon-relay-sub-key` to `dh-beacon-relay-substrate-key`, for simplicity in the deployment script. 11. Update suggested configs for `.vscode` configs. --------- Co-authored-by: Steve Degosserie <723552+stiiifff@users.noreply.github.com>
2025-06-12 08:24:03 +00:00
"start:e2e:ci": "bun cli launch --datahaven --no-build-datahaven --launch-kurtosis --deploy-contracts --fund-validators --setup-validators --update-validator-set --set-parameters --relayer --clean-network --slot-time 2",
test: ✨ New CLI functions (#84) ## Changes - New option: `--kurtosis-enclave-name` to allow you to specify a new ethereum network with a different enclave name. Neccesary step for setting up local testing with multiple enclaves running at once - Refactor: all single dash options (e.g. `-i`) have been renamed to have double dashes `--` for consistency others - sad times: CLI now must be invoked `bun cli launch` since we have multiple fns now - Package Update - New Function: `stop` which allows you to kill all components (eth, dh, relayers) or only a single part - Gonza's mac target build fix - Misc fixes to ci like caching and rate limits ## Additional Comment The CLI needs multiple commands and this PR adds the first new one `stop`. This syntax is faily extensible so @ffarall 's k8 command can follow suite. Originally we were going to have an `exec` command to expose internal fns to be callable by command line, i.e. generate beacon checkpoint - however the need for it has since been superceded by the deploy to k8 command that's going to be added. I've left the code in place though so when another usecase comes up we can just plug that in. --------- Co-authored-by: Facundo Farall <37149322+ffarall@users.noreply.github.com>
2025-05-22 20:02:12 +00:00
"start:e2e:minrelayer": "bun cli launch --relayer --deploy-contracts --no-setup-validators --no-update-validator-set --no-fund-validators --datahaven",
fix: 🐛 CLI fixes for independently running parts of it (#85)
2025-05-21 17:01:12 +00:00
"stop:docker:datahaven": "docker rm -f $(docker ps -aq --filter name='^datahaven-') 2>/dev/null || true; docker network rm datahaven-net || true",
refactor: :truck: rename relayer image to adapt to snowbridge repo refactor (#92) This PR renames all usage of `moonsonglabs/snowbridge-relayer` to `moonsonglabs/snowbridge-relay` to adapt to the refactor done in [PR #18](https://github.com/Moonsong-Labs/snowbridge/pull/18) of our Snowbridge fork. The aforementioned PR should be merged before this one, although caution is not really needed since the CI would fail otherwise.
2025-06-10 17:02:14 +00:00
"stop:docker:relayer": "docker rm -f $(docker ps -aq --filter name='^snowbridge-') 2>/dev/null || true",
test: ✨ New CLI functions (#84) ## Changes - New option: `--kurtosis-enclave-name` to allow you to specify a new ethereum network with a different enclave name. Neccesary step for setting up local testing with multiple enclaves running at once - Refactor: all single dash options (e.g. `-i`) have been renamed to have double dashes `--` for consistency others - sad times: CLI now must be invoked `bun cli launch` since we have multiple fns now - Package Update - New Function: `stop` which allows you to kill all components (eth, dh, relayers) or only a single part - Gonza's mac target build fix - Misc fixes to ci like caching and rate limits ## Additional Comment The CLI needs multiple commands and this PR adds the first new one `stop`. This syntax is faily extensible so @ffarall 's k8 command can follow suite. Originally we were going to have an `exec` command to expose internal fns to be callable by command line, i.e. generate beacon checkpoint - however the need for it has since been superceded by the deploy to k8 command that's going to be added. I've left the code in place though so when another usecase comes up we can just plug that in. --------- Co-authored-by: Facundo Farall <37149322+ffarall@users.noreply.github.com>
2025-05-22 20:02:12 +00:00
"stop:e2e": "bun cli stop --all",
"stop:dh": "bun cli stop --datahaven --no-enclave --no-relayer ",
"stop:sb": "bun cli stop --relayer --no-datahaven --no-enclave",
"stop:eth": "bun cli stop --enclave --no-datahaven --no-relayer",
"stop:engine": "bun cli stop --kurtosisEngine --no-datahaven --no-relayer --no-enclave",
test: 🏗️ Setup e2e testing framework (#104) ## Implement E2E Testing Framework with Isolated Networks ### Summary Refactors the existing E2E testing infrastructure to provide isolated test environments with parallel execution support. Each test suite now runs in its own network namespace, preventing resource conflicts. ### Key Changes - **New Testing Framework** (`test/framework/`): Base classes for test lifecycle management with automatic setup/teardown - **Launcher Module** (`test/launcher/`): Extracted network orchestration logic from CLI handlers for reusability - **Parallel Execution**: Added `test-parallel.ts` script with concurrency limits to prevent resource exhaustion - **Test Isolation**: Each suite gets unique network IDs (format: `suiteName-timestamp`) and Docker networks - **Improved Test Organization**: Migrated tests to new framework, deprecated old test structure ### Test Improvements - Added 4 new test suites demonstrating framework usage. : - `contracts.test.ts` - Smart contract deployment/interaction - `datahaven-substrate.test.ts` - Substrate API operations - `cross-chain.test.ts` - Snowbridge cross-chain messaging - `ethereum-basic.test.ts` - Ethereum network operations > [!WARNING] The test suites themselves are bad and shouldn't be consider examples of good tests. They were AI generated just to test the concurrency of test runners ### Documentation - Added comprehensive framework overview (`E2E_FRAMEWORK_OVERVIEW.md`) - Updated README with parallel testing commands - Added test patterns and best practices ### Breaking Changes - Old test suites moved to `e2e - DEPRECATED/` directory - Test execution now requires extending `BaseTestSuite` class ### Testing Run tests with: `bun test:e2e` or `bun test:e2e:parallel` (with concurrency limits) ### TODO - [ ] Implement good test examples. - [ ] Implement useful test utils (like waiting for an event to show up in DataHaven or Ethereum). - [ ] Enforce tests with CI (currently cannot be done due to intermittent error when sending a transaction with PAPI). --------- Co-authored-by: Claude <noreply@anthropic.com> Co-authored-by: undercover-cactus <lola@moonsonglabs.com>
2025-07-16 16:51:07 +00:00
"test:e2e": "bun test ./suites --timeout 900000",
test: port Moonwall block validation suite (#201) This PR restructures and port the block validation test suite from Moonbeam and add the necessary infrastructure for contract-based testing. ### Test Suites Added **Block Validation Suite 1** (`test-block-1.ts`) - *Refactored from original `test-block.ts`* - T01: Validates block number increments correctly after manual block creation - T02: Checks block timestamps are valid and within expected bounds - T03: Verifies complete block structure including gasLimit, difficulty, receiptsRoot, transactionsRoot, logsBloom, and other Ethereum-compatible fields - T04: Confirms blocks are retrievable by hash - T05: Confirms blocks are retrievable by number **Block Validation Suite 2** (`test-block-2.ts`) - *New* - T01: Verifies block number persistence across test cases - T02: Validates parent-child block hash linkage in the chain **Block Gas Limits Suite** (`test-block-gas.ts`) - *New* - T01-T06: Tests all three transaction types (legacy, eip1559, eip2930) can deploy contracts at max extrinsic gas limit, and correctly reject transactions exceeding that limit - T07: Deploys `BlockVariables` contract and verifies runtime gas limit is accessible from within contract execution **Block Genesis Suite** (`test-block-genesis.ts`) - *New* - T01: Validates genesis block (block 0) contains correct Ethereum-compatible structure and empty transaction/uncle lists - T02: Confirms genesis block is retrievable by hash ### Infrastructure Additions **Contract Deployment Helpers** - `fetchCompiledContract`: Loads compiled Solidity artifacts with ABI and bytecode from JSON output - `deployCompiledContract`: Handles contract deployment with support for legacy, EIP-1559, and EIP-2930 transaction types, including gas parameter configuration **Solidity Test Fixtures** - `BlockVariables.sol`: Exposes `block.gaslimit`, `block.chainid`, `block.number` via view functions for runtime validation - `Fibonacci.sol`: Provides `fib2(n)` pure function for computational gas testing - `MultiplyBy7.sol`: Minimal pure function contract for basic deployment testing **Enhanced Runtime Constants** - Added `EXTRINSIC_GAS_LIMIT` (52M), `BLOCK_WEIGHT_LIMIT` (2T), `MAX_POV_SIZE` (10MB) for all three environments - Constants derived from and documented against Rust runtime configuration in `operator/runtime/*/src/lib.rs` ### Dependencies - Added `solc@0.8.30` and supporting packages (`yargs`, `command-exists`, `js-sha3`, `memorystream`) for local Solidity compilation
2025-10-06 07:47:35 +00:00
"compile:contracts": "bun x tsx scripts/compile-contracts.ts compile",
test: 🏗️ Setup e2e testing framework (#104) ## Implement E2E Testing Framework with Isolated Networks ### Summary Refactors the existing E2E testing infrastructure to provide isolated test environments with parallel execution support. Each test suite now runs in its own network namespace, preventing resource conflicts. ### Key Changes - **New Testing Framework** (`test/framework/`): Base classes for test lifecycle management with automatic setup/teardown - **Launcher Module** (`test/launcher/`): Extracted network orchestration logic from CLI handlers for reusability - **Parallel Execution**: Added `test-parallel.ts` script with concurrency limits to prevent resource exhaustion - **Test Isolation**: Each suite gets unique network IDs (format: `suiteName-timestamp`) and Docker networks - **Improved Test Organization**: Migrated tests to new framework, deprecated old test structure ### Test Improvements - Added 4 new test suites demonstrating framework usage. : - `contracts.test.ts` - Smart contract deployment/interaction - `datahaven-substrate.test.ts` - Substrate API operations - `cross-chain.test.ts` - Snowbridge cross-chain messaging - `ethereum-basic.test.ts` - Ethereum network operations > [!WARNING] The test suites themselves are bad and shouldn't be consider examples of good tests. They were AI generated just to test the concurrency of test runners ### Documentation - Added comprehensive framework overview (`E2E_FRAMEWORK_OVERVIEW.md`) - Updated README with parallel testing commands - Added test patterns and best practices ### Breaking Changes - Old test suites moved to `e2e - DEPRECATED/` directory - Test execution now requires extending `BaseTestSuite` class ### Testing Run tests with: `bun test:e2e` or `bun test:e2e:parallel` (with concurrency limits) ### TODO - [ ] Implement good test examples. - [ ] Implement useful test utils (like waiting for an event to show up in DataHaven or Ethereum). - [ ] Enforce tests with CI (currently cannot be done due to intermittent error when sending a transaction with PAPI). --------- Co-authored-by: Claude <noreply@anthropic.com> Co-authored-by: undercover-cactus <lola@moonsonglabs.com>
2025-07-16 16:51:07 +00:00
"test:e2e:parallel": "bun scripts/test-parallel.ts",
test: Integrate moonwall (#185) ### Description This PR introduces the **Moonwall** end-to-end (E2E) testing framework. The primary motivation for this is to enable the porting of existing Mobeam tests into the `DataHaven` repository. ### Key Changes * **Node Manual Sealing:** * Introduced a `--sealing=manual` flag for the `datahaven-node`. When enabled, blocks are only produced on demand via an RPC call. This is the core mechanism that allows for deterministic tests. * **Moonwall Framework Integration:** * Added `@moonwall/cli` and `@moonwall/util` dependencies to the `test/package.json`. * A new `test/moonwall.config.json` file configures the test environment, defining how Moonwall should launch the `datahaven-node` with the manual sealing flag. * Added a `moonwall:test` script to `package.json` for running the tests. * **CI Workflow:** * A new reusable workflow, `.github/workflows/task-moonwall-tests.yml`, has been created to handle the setup, execution, and reporting of Moonwall tests. * The main `CI.yml` now includes a `moonwall-tests` job that runs after the `build-operator` job, ensuring it always tests the correct, freshly-built binary. * **Example Test Suite:** * A new test suite, `test/datahaven/suites/dev/test-block.ts`, had been copied from moonbeam. ### How to Run Locally 1. Navigate to the `test` directory. 2. Install dependencies: `bun install` 3. Run the tests: `bun run moonwall:test` --------- Co-authored-by: undercover-cactus <lola@moonsonglabs.com>
2025-09-30 14:47:39 +00:00
"moonwall:test": "moonwall test dev_datahaven",
"moonwall:run": "moonwall run dev_datahaven",
feat: ✨ Add Polkadot API support to CLI and e2e testing infra (#68) In this PR: 1. Add [Polkadot API](https://papi.how/) support to `test/` directory (i.e. the e2e CLI and testing framework) to be able to interact with the Substrate chain. 1. This allows typed interactions with transactions, constants, storage, runtime APIs, and non-typed interaction with RPC methods. 2. Types are autogenerated when running `bun i`, from the `datahaven.scale` file that is part of this repo's version control. 2. Add new utilities file to `papi` related functionalities. For the time being, generating a new signer from a private key. 3. ~Add a new step to the CLI that sends a transaction to the DataHaven network. _*THIS SHOULD BE REMOVED SOON, IT'S JUST FOR TESTING PURPOSES*_~ 1. Both steps that send test transactions have been removed from the CLI, for convenience and being error prone. Their scripts remain usable for testing purposes if needed. 4. Removes the `apis.rs` files from the runtime definitions. Having them in a separate file meant that the runtime was not including the Runtime APIs in the metadata blob, preventing `papi` from creating types from it. This change can be reapplied after upgrading to `polkadot-sdk` `stable-2503`. More info [here](https://github.com/paritytech/polkadot-sdk/issues/6659). 5. Add script to re-generate types, and corresponding docs. 6. Makes logger synchronous to avoid prints happening before logs.
2025-05-13 03:03:21 +00:00
"typecheck": "tsc --noEmit",
refactor: 🐳 Improve docker caching (again) (#86) ## Changes - New CI file for making Docker Prod images - Changed E2E tests use an image built from a local dockerfile - Some cargo build options to make it quicker - Fix the cache hit rate - added `tsgo` preview to the project 😎 - Can be invoked with `bun tsgo` to typecheck - Install in IDE [VSCode](https://code.visualstudio.com/docs/configure/extensions/extension-marketplace) & [Zed](https://github.com/zed-extensions/tsgo) for super-fast inline typechecking (as you type basically) ## Context This PR attempts to make the frankly unacceptable CI times better. This achieves that aim by making a crappy image for day-to-day usage and let the prod issue take ages since that will be infrequently used. The reason why the original design didn't work for us is because: 1) we are using the free GH runners 2) when we goto baremetal runners we'll lose our rapid caching abilities which make using docker cheap. Also, we add `tsgo` support to improve devex. The improvement is astounding. ```sh hyperfine -n tsc "bun tsc --incremental false --extendedDiagnostics" -n tsgo "bun tsgo --incremental false --extendedDiagnostics" Benchmark 1: tsc Time (mean ± σ): 5.500 s ± 0.221 s [User: 8.939 s, System: 0.400 s] Range (min … max): 5.196 s … 5.845 s 10 runs Benchmark 2: tsgo Time (mean ± σ): 99.1 ms ± 8.4 ms [User: 392.8 ms, System: 54.1 ms] Range (min … max): 88.3 ms … 116.0 ms 29 runs Summary tsgo ran 55.48 ± 5.22 times faster than tsc ```
2025-05-27 16:14:15 +00:00
"tsgo": "tsgo tsc --noEmit --pretty --skipLibCheck",
feat: ✨ Add Polkadot API support to CLI and e2e testing infra (#68) In this PR: 1. Add [Polkadot API](https://papi.how/) support to `test/` directory (i.e. the e2e CLI and testing framework) to be able to interact with the Substrate chain. 1. This allows typed interactions with transactions, constants, storage, runtime APIs, and non-typed interaction with RPC methods. 2. Types are autogenerated when running `bun i`, from the `datahaven.scale` file that is part of this repo's version control. 2. Add new utilities file to `papi` related functionalities. For the time being, generating a new signer from a private key. 3. ~Add a new step to the CLI that sends a transaction to the DataHaven network. _*THIS SHOULD BE REMOVED SOON, IT'S JUST FOR TESTING PURPOSES*_~ 1. Both steps that send test transactions have been removed from the CLI, for convenience and being error prone. Their scripts remain usable for testing purposes if needed. 4. Removes the `apis.rs` files from the runtime definitions. Having them in a separate file meant that the runtime was not including the Runtime APIs in the metadata blob, preventing `papi` from creating types from it. This change can be reapplied after upgrading to `polkadot-sdk` `stable-2503`. More info [here](https://github.com/paritytech/polkadot-sdk/issues/6659). 5. Add script to re-generate types, and corresponding docs. 6. Makes logger synchronous to avoid prints happening before logs.
2025-05-13 03:03:21 +00:00
"postinstall": "papi"
test: ⚙️ Add Kurtosis Launch Script (#34) This PR adds a script one-liner to spin up an ethereum network, with attached explorers and deploys our `/contracts` folder into there. --------- Co-authored-by: Facundo Farall <37149322+ffarall@users.noreply.github.com>
2025-04-08 15:42:45 +00:00
},
"devDependencies": {
refactor: 🐳 Improve docker caching (again) (#86) ## Changes - New CI file for making Docker Prod images - Changed E2E tests use an image built from a local dockerfile - Some cargo build options to make it quicker - Fix the cache hit rate - added `tsgo` preview to the project 😎 - Can be invoked with `bun tsgo` to typecheck - Install in IDE [VSCode](https://code.visualstudio.com/docs/configure/extensions/extension-marketplace) & [Zed](https://github.com/zed-extensions/tsgo) for super-fast inline typechecking (as you type basically) ## Context This PR attempts to make the frankly unacceptable CI times better. This achieves that aim by making a crappy image for day-to-day usage and let the prod issue take ages since that will be infrequently used. The reason why the original design didn't work for us is because: 1) we are using the free GH runners 2) when we goto baremetal runners we'll lose our rapid caching abilities which make using docker cheap. Also, we add `tsgo` support to improve devex. The improvement is astounding. ```sh hyperfine -n tsc "bun tsc --incremental false --extendedDiagnostics" -n tsgo "bun tsgo --incremental false --extendedDiagnostics" Benchmark 1: tsc Time (mean ± σ): 5.500 s ± 0.221 s [User: 8.939 s, System: 0.400 s] Range (min … max): 5.196 s … 5.845 s 10 runs Benchmark 2: tsgo Time (mean ± σ): 99.1 ms ± 8.4 ms [User: 392.8 ms, System: 54.1 ms] Range (min … max): 88.3 ms … 116.0 ms 29 runs Summary tsgo ran 55.48 ± 5.22 times faster than tsc ```
2025-05-27 16:14:15 +00:00
"@types/bun": "latest",
test: port Moonwall block validation suite (#201) This PR restructures and port the block validation test suite from Moonbeam and add the necessary infrastructure for contract-based testing. ### Test Suites Added **Block Validation Suite 1** (`test-block-1.ts`) - *Refactored from original `test-block.ts`* - T01: Validates block number increments correctly after manual block creation - T02: Checks block timestamps are valid and within expected bounds - T03: Verifies complete block structure including gasLimit, difficulty, receiptsRoot, transactionsRoot, logsBloom, and other Ethereum-compatible fields - T04: Confirms blocks are retrievable by hash - T05: Confirms blocks are retrievable by number **Block Validation Suite 2** (`test-block-2.ts`) - *New* - T01: Verifies block number persistence across test cases - T02: Validates parent-child block hash linkage in the chain **Block Gas Limits Suite** (`test-block-gas.ts`) - *New* - T01-T06: Tests all three transaction types (legacy, eip1559, eip2930) can deploy contracts at max extrinsic gas limit, and correctly reject transactions exceeding that limit - T07: Deploys `BlockVariables` contract and verifies runtime gas limit is accessible from within contract execution **Block Genesis Suite** (`test-block-genesis.ts`) - *New* - T01: Validates genesis block (block 0) contains correct Ethereum-compatible structure and empty transaction/uncle lists - T02: Confirms genesis block is retrievable by hash ### Infrastructure Additions **Contract Deployment Helpers** - `fetchCompiledContract`: Loads compiled Solidity artifacts with ABI and bytecode from JSON output - `deployCompiledContract`: Handles contract deployment with support for legacy, EIP-1559, and EIP-2930 transaction types, including gas parameter configuration **Solidity Test Fixtures** - `BlockVariables.sol`: Exposes `block.gaslimit`, `block.chainid`, `block.number` via view functions for runtime validation - `Fibonacci.sol`: Provides `fib2(n)` pure function for computational gas testing - `MultiplyBy7.sol`: Minimal pure function contract for basic deployment testing **Enhanced Runtime Constants** - Added `EXTRINSIC_GAS_LIMIT` (52M), `BLOCK_WEIGHT_LIMIT` (2T), `MAX_POV_SIZE` (10MB) for all three environments - Constants derived from and documented against Rust runtime configuration in `operator/runtime/*/src/lib.rs` ### Dependencies - Added `solc@0.8.30` and supporting packages (`yargs`, `command-exists`, `js-sha3`, `memorystream`) for local Solidity compilation
2025-10-06 07:47:35 +00:00
"@types/yargs": "^17.0.33",
ci: 👷 Add CI to check PAPI metadata (#107) Add CI check for Polkadot-API metadata freshness This PR adds a new CI workflow that ensures the Polkadot-API metadata file (`test/.papi/metadata/datahaven.scale`) is kept up-to-date when runtime changes are made. Changes: - Added task-check-metadata.yml workflow that: - Reuses the WASM artifact from the build-operator job (no duplicate compilation) - Runs `bun x papi add` to regenerate metadata - Fails if the metadata file has uncommitted changes - Integrated the check into `CI.yml` as a second-tier job alongside `docker-build` Why: - Prevents TypeScript type definitions from becoming out of sync with the runtime - Reminds developers to run `bun generate:types:fast` when making runtime changes - Ensures consistent type safety across the codebase The check provides clear error messages with instructions when metadata is outdated.
2025-06-19 22:12:04 +00:00
"@typescript/native-preview": "^7.0.0-dev.20250618.1"
test: ⚙️ Add Kurtosis Launch Script (#34) This PR adds a script one-liner to spin up an ethereum network, with attached explorers and deploys our `/contracts` folder into there. --------- Co-authored-by: Facundo Farall <37149322+ffarall@users.noreply.github.com>
2025-04-08 15:42:45 +00:00
},
"peerDependencies": {
test: 🧪 Initial Relay Work (#52) Small changes to help progress us closer to relayer support --------- Co-authored-by: Facundo Farall <37149322+ffarall@users.noreply.github.com>
2025-04-23 15:28:19 +00:00
"typescript": "^5.8.3"
test: ⚙️ Add Kurtosis Launch Script (#34) This PR adds a script one-liner to spin up an ethereum network, with attached explorers and deploys our `/contracts` folder into there. --------- Co-authored-by: Facundo Farall <37149322+ffarall@users.noreply.github.com>
2025-04-08 15:42:45 +00:00
},
"dependencies": {
ci: 👷 Add CI to check PAPI metadata (#107) Add CI check for Polkadot-API metadata freshness This PR adds a new CI workflow that ensures the Polkadot-API metadata file (`test/.papi/metadata/datahaven.scale`) is kept up-to-date when runtime changes are made. Changes: - Added task-check-metadata.yml workflow that: - Reuses the WASM artifact from the build-operator job (no duplicate compilation) - Runs `bun x papi add` to regenerate metadata - Fails if the metadata file has uncommitted changes - Integrated the check into `CI.yml` as a second-tier job alongside `docker-build` Why: - Prevents TypeScript type definitions from becoming out of sync with the runtime - Reminds developers to run `bun generate:types:fast` when making runtime changes - Ensures consistent type safety across the codebase The check provides clear error messages with instructions when metadata is outdated.
2025-06-19 22:12:04 +00:00
"@biomejs/biome": "^2.0.0",
test: 🕹️ Upgrade CLI to commander (#53) This PR replaces our E2E script 😢 with Commander. This allows us to extend the CLI with new options, in a fully typesafe way without having to rely on manual definitions and nested logic. This also opens the path to adding new commands too if we want to do additional functionality through here. This may include, additional scripting, running tests, uploading artefacts etc.
2025-04-25 13:44:17 +00:00
"@commander-js/extra-typings": "^13.1.0",
ci: 👷 Add CI to check PAPI metadata (#107) Add CI check for Polkadot-API metadata freshness This PR adds a new CI workflow that ensures the Polkadot-API metadata file (`test/.papi/metadata/datahaven.scale`) is kept up-to-date when runtime changes are made. Changes: - Added task-check-metadata.yml workflow that: - Reuses the WASM artifact from the build-operator job (no duplicate compilation) - Runs `bun x papi add` to regenerate metadata - Fails if the metadata file has uncommitted changes - Integrated the check into `CI.yml` as a second-tier job alongside `docker-build` Why: - Prevents TypeScript type definitions from becoming out of sync with the runtime - Reminds developers to run `bun generate:types:fast` when making runtime changes - Ensures consistent type safety across the codebase The check provides clear error messages with instructions when metadata is outdated.
2025-06-19 22:12:04 +00:00
"@dotenvx/dotenvx": "^1.44.2",
"@inquirer/prompts": "^7.5.3",
test: Integrate moonwall (#185) ### Description This PR introduces the **Moonwall** end-to-end (E2E) testing framework. The primary motivation for this is to enable the porting of existing Mobeam tests into the `DataHaven` repository. ### Key Changes * **Node Manual Sealing:** * Introduced a `--sealing=manual` flag for the `datahaven-node`. When enabled, blocks are only produced on demand via an RPC call. This is the core mechanism that allows for deterministic tests. * **Moonwall Framework Integration:** * Added `@moonwall/cli` and `@moonwall/util` dependencies to the `test/package.json`. * A new `test/moonwall.config.json` file configures the test environment, defining how Moonwall should launch the `datahaven-node` with the manual sealing flag. * Added a `moonwall:test` script to `package.json` for running the tests. * **CI Workflow:** * A new reusable workflow, `.github/workflows/task-moonwall-tests.yml`, has been created to handle the setup, execution, and reporting of Moonwall tests. * The main `CI.yml` now includes a `moonwall-tests` job that runs after the `build-operator` job, ensuring it always tests the correct, freshly-built binary. * **Example Test Suite:** * A new test suite, `test/datahaven/suites/dev/test-block.ts`, had been copied from moonbeam. ### How to Run Locally 1. Navigate to the `test` directory. 2. Install dependencies: `bun install` 3. Run the tests: `bun run moonwall:test` --------- Co-authored-by: undercover-cactus <lola@moonsonglabs.com>
2025-09-30 14:47:39 +00:00
"@moonwall/cli": "^5.15.0",
"@moonwall/util": "^5.15.0",
ci: 👷 Add CI to check PAPI metadata (#107) Add CI check for Polkadot-API metadata freshness This PR adds a new CI workflow that ensures the Polkadot-API metadata file (`test/.papi/metadata/datahaven.scale`) is kept up-to-date when runtime changes are made. Changes: - Added task-check-metadata.yml workflow that: - Reuses the WASM artifact from the build-operator job (no duplicate compilation) - Runs `bun x papi add` to regenerate metadata - Fails if the metadata file has uncommitted changes - Integrated the check into `CI.yml` as a second-tier job alongside `docker-build` Why: - Prevents TypeScript type definitions from becoming out of sync with the runtime - Reminds developers to run `bun generate:types:fast` when making runtime changes - Ensures consistent type safety across the codebase The check provides clear error messages with instructions when metadata is outdated.
2025-06-19 22:12:04 +00:00
"@noble/curves": "^1.9.2",
feat: ✨ Add Polkadot API support to CLI and e2e testing infra (#68) In this PR: 1. Add [Polkadot API](https://papi.how/) support to `test/` directory (i.e. the e2e CLI and testing framework) to be able to interact with the Substrate chain. 1. This allows typed interactions with transactions, constants, storage, runtime APIs, and non-typed interaction with RPC methods. 2. Types are autogenerated when running `bun i`, from the `datahaven.scale` file that is part of this repo's version control. 2. Add new utilities file to `papi` related functionalities. For the time being, generating a new signer from a private key. 3. ~Add a new step to the CLI that sends a transaction to the DataHaven network. _*THIS SHOULD BE REMOVED SOON, IT'S JUST FOR TESTING PURPOSES*_~ 1. Both steps that send test transactions have been removed from the CLI, for convenience and being error prone. Their scripts remain usable for testing purposes if needed. 4. Removes the `apis.rs` files from the runtime definitions. Having them in a separate file meant that the runtime was not including the Runtime APIs in the metadata blob, preventing `papi` from creating types from it. This change can be reapplied after upgrading to `polkadot-sdk` `stable-2503`. More info [here](https://github.com/paritytech/polkadot-sdk/issues/6659). 5. Add script to re-generate types, and corresponding docs. 6. Makes logger synchronous to avoid prints happening before logs.
2025-05-13 03:03:21 +00:00
"@noble/hashes": "^1.8.0",
"@polkadot-api/descriptors": "file:.papi/descriptors",
ci: 👷 Add CI to check PAPI metadata (#107) Add CI check for Polkadot-API metadata freshness This PR adds a new CI workflow that ensures the Polkadot-API metadata file (`test/.papi/metadata/datahaven.scale`) is kept up-to-date when runtime changes are made. Changes: - Added task-check-metadata.yml workflow that: - Reuses the WASM artifact from the build-operator job (no duplicate compilation) - Runs `bun x papi add` to regenerate metadata - Fails if the metadata file has uncommitted changes - Integrated the check into `CI.yml` as a second-tier job alongside `docker-build` Why: - Prevents TypeScript type definitions from becoming out of sync with the runtime - Reminds developers to run `bun generate:types:fast` when making runtime changes - Ensures consistent type safety across the codebase The check provides clear error messages with instructions when metadata is outdated.
2025-06-19 22:12:04 +00:00
"@types/dockerode": "^3.3.41",
"@types/node": "^22.15.32",
test: ✨ New CLI functions (#84) ## Changes - New option: `--kurtosis-enclave-name` to allow you to specify a new ethereum network with a different enclave name. Neccesary step for setting up local testing with multiple enclaves running at once - Refactor: all single dash options (e.g. `-i`) have been renamed to have double dashes `--` for consistency others - sad times: CLI now must be invoked `bun cli launch` since we have multiple fns now - Package Update - New Function: `stop` which allows you to kill all components (eth, dh, relayers) or only a single part - Gonza's mac target build fix - Misc fixes to ci like caching and rate limits ## Additional Comment The CLI needs multiple commands and this PR adds the first new one `stop`. This syntax is faily extensible so @ffarall 's k8 command can follow suite. Originally we were going to have an `exec` command to expose internal fns to be callable by command line, i.e. generate beacon checkpoint - however the need for it has since been superceded by the deploy to k8 command that's going to be added. I've left the code in place though so when another usecase comes up we can just plug that in. --------- Co-authored-by: Facundo Farall <37149322+ffarall@users.noreply.github.com>
2025-05-22 20:02:12 +00:00
"@wagmi/cli": "^2.3.1",
ci: 👷 Add CI to check PAPI metadata (#107) Add CI check for Polkadot-API metadata freshness This PR adds a new CI workflow that ensures the Polkadot-API metadata file (`test/.papi/metadata/datahaven.scale`) is kept up-to-date when runtime changes are made. Changes: - Added task-check-metadata.yml workflow that: - Reuses the WASM artifact from the build-operator job (no duplicate compilation) - Runs `bun x papi add` to regenerate metadata - Fails if the metadata file has uncommitted changes - Integrated the check into `CI.yml` as a second-tier job alongside `docker-build` Why: - Prevents TypeScript type definitions from becoming out of sync with the runtime - Reminds developers to run `bun generate:types:fast` when making runtime changes - Ensures consistent type safety across the codebase The check provides clear error messages with instructions when metadata is outdated.
2025-06-19 22:12:04 +00:00
"@wagmi/core": "^2.17.3",
feat: 🧑‍💻 Turn scripts into an interactive CLI (#41) WARNING: This PR changes the kurtosis package to use the one from upstream, not our fork, as it was not working at the moment. This should be changed when fixed. In this PR: 1. Turn `launch-kurtosis` script into a CLI, which parses parameters and can interactively run multiple steps. 2. Separate steps of such CLI into their own scripts. 1. New script created `launch-kurtosis`, which detects if an enclave is running and prompts to relaunch it if it is. 2. New script created `deploy-contracts` to deploy all contracts. It can optionally verify them as well. 3. Each step can be interactively opted in/out, or choose whether to run it via CLI params. 4. The CLI offers a help command as well. 5. Cleanup logs of CLI. Logs of internal commands ran can be printed with LOG_LEVEL=debug. In case of error, they are always printed out.
2025-04-15 14:01:24 +00:00
"chalk": "^5.4.1",
test: 🕹️ Upgrade CLI to commander (#53) This PR replaces our E2E script 😢 with Commander. This allows us to extend the CLI with new options, in a fully typesafe way without having to rely on manual definitions and nested logic. This also opens the path to adding new commands too if we want to do additional functionality through here. This may include, additional scripting, running tests, uploading artefacts etc.
2025-04-25 13:44:17 +00:00
"commander": "^13.1.0",
ci: 👷 Add CI to check PAPI metadata (#107) Add CI check for Polkadot-API metadata freshness This PR adds a new CI workflow that ensures the Polkadot-API metadata file (`test/.papi/metadata/datahaven.scale`) is kept up-to-date when runtime changes are made. Changes: - Added task-check-metadata.yml workflow that: - Reuses the WASM artifact from the build-operator job (no duplicate compilation) - Runs `bun x papi add` to regenerate metadata - Fails if the metadata file has uncommitted changes - Integrated the check into `CI.yml` as a second-tier job alongside `docker-build` Why: - Prevents TypeScript type definitions from becoming out of sync with the runtime - Reminds developers to run `bun generate:types:fast` when making runtime changes - Ensures consistent type safety across the codebase The check provides clear error messages with instructions when metadata is outdated.
2025-06-19 22:12:04 +00:00
"dockerode": "^4.0.7",
feat(contracts): ✨ add set up validator script and execute it when starting integration tests (#47) This PR adds the `setup-validators` Typescript script that, given an already started up network, sets up a new validator set and sends it through Snowbridge's Gateway to the solochain. To accomplish that purpose, this PR: - Modifies the `DeployLocal` script to save in the `anvil.json` file not only the deployed strategies' addresses but also the owner of each strategy's underlying token. This owner is used as the source of funds to transfer tokens to other validators so they can register under that strategy. - Adds an `OPERATOR_SOLOCHAIN_ADDRESS` to the `Accounts` utility script contract. This address is the one used as the Solochain address when registering a new Operator. - Updates the `SignUpOperator` (which I believe is now deprecated since we have multiple Operator Sets) and `SignUpOperatorBase` scripts to adapt to both aforementioned changes. - Updates the `ELScriptStorage` script to save the new extra information of each deployed strategy (the creator of the underlying token) in storage. - Adds a `validator-set.json` file which contains the validators that should be registered in the AVS and sent to the Solochain network through the Snowbridge Gateway when starting any integration test. This is currently hardcoded but could be generated in any other way, giving us flexibility for testing. - Adds both a Markdown file and a Excalidraw diagram showcasing both how the setup of integration tests work and possible integration tests that will be added in a future PR. This list is not exahustive as there are many more scenarios we will want to test using integration tests. - Updates the `e2e-cli.ts` script to execute the validator setup when bootstrapping the network used for integration testing. --------- Co-authored-by: Facundo Farall <37149322+ffarall@users.noreply.github.com>
2025-04-22 19:49:51 +00:00
"dotenv": "^16.5.0",
test: ✨ New CLI functions (#84) ## Changes - New option: `--kurtosis-enclave-name` to allow you to specify a new ethereum network with a different enclave name. Neccesary step for setting up local testing with multiple enclaves running at once - Refactor: all single dash options (e.g. `-i`) have been renamed to have double dashes `--` for consistency others - sad times: CLI now must be invoked `bun cli launch` since we have multiple fns now - Package Update - New Function: `stop` which allows you to kill all components (eth, dh, relayers) or only a single part - Gonza's mac target build fix - Misc fixes to ci like caching and rate limits ## Additional Comment The CLI needs multiple commands and this PR adds the first new one `stop`. This syntax is faily extensible so @ffarall 's k8 command can follow suite. Originally we were going to have an `exec` command to expose internal fns to be callable by command line, i.e. generate beacon checkpoint - however the need for it has since been superceded by the deploy to k8 command that's going to be added. I've left the code in place though so when another usecase comes up we can just plug that in. --------- Co-authored-by: Facundo Farall <37149322+ffarall@users.noreply.github.com>
2025-05-22 20:02:12 +00:00
"octokit": "^4.1.4",
test: ⚙️ Parse & Generate Relayer Configs (#54) ## Human Written Description This PR adds the following to the E2E CLI: - Relayer config generation for: `beacon-relay` `beefy-relay` - The other two relayer types to be added later - Relayers don't actually work yet - By default turned off, this requires a binary to be present in: `<repo_root>/operator/target/release` dir - Datahaven network launching - DH network is using default `local` network chain spec - Launched with 5 nodes since our authority set is 6 large (and you need 2/3 + 1 of set size > [!NOTE] > Both the relayer and the DH node binaries are being run as local processes TEMPORARILY. This means that logging is done in a very rudimentary way (we pipe to a file whilst the CLI is running). > > This means that when the CLI finishes **the log files will no longer be written to**. > This is temporary since spawning binaries is a stop gap solution until docker images available. --- > [!IMPORTANT] > The following is AI generated slop describing this PR's changes: **Key Changes:** * **CLI Enhancements (`test/cli/index.ts`):** * Added options `--datahaven` and `--datahaven-bin-path` to enable launching local DataHaven nodes. * Added options `--relayer` and `--relayer-bin-path` to enable launching Snowbridge relayers (Beefy and Beacon). * Added negation flags (`--no-fund-validators`, `--no-setup-validators`, `--no-update-validator-set`) for more granular control over validator setup steps. * Added `--skip-cleaning` option to preserve Kurtosis state between runs. * Added a pre-action hook (`launchPreActionHook`) to validate flag combinations (e.g., `--verified` requires `--blockscout`). * **New CLI Handlers (`test/cli/handlers/launch/`):** * `datahaven.ts`: Logic for spawning DataHaven node processes using the specified binary. Manages ports and process cleanup. * `relayer.ts`: Logic for configuring and spawning Snowbridge relayer processes (Beefy and Beacon). Reads contract deployment addresses, updates relayer config templates, and uses specified private keys. Manages log files and process cleanup. * `summary.ts`: Generates and displays the table of running services (including dynamically launched DataHaven nodes) and their endpoints. * `validator.ts`: Extracted validator funding, setup, and set update logic into its own handler. * `index.ts`: Orchestrates the launch sequence based on CLI options, calling the appropriate handlers. Includes a `LaunchedNetwork` class to track spawned processes, file descriptors, and node ports for cleanup. * **Updated `package.json` Scripts:** * Added `start:e2e:minrelayer` script for a minimal setup including relayers and DataHaven nodes. * Modified `stop:e2e` to include `pkill datahaven` for proper cleanup. * Added `stop:e2e:quick` to only stop the Kurtosis enclave without full cleaning. * **Updated `launch-kurtosis.ts`:** Modified to use new Kurtosis utility functions and added a `skipCleaning` option. * **New Utility Functions:** * `test/utils/kurtosis.ts`: Functions to inspect Kurtosis services (`getServiceFromKurtosis`, `getPortFromKurtosis`, `getServicesFromKurtosis`). * `test/utils/parser.ts`: Zod schemas and parsing functions for Snowbridge relayer configurations. * **Constants & Minor Updates:** * Added `SUBSTRATE_FUNDED_ACCOUNTS` to `test/utils/constants.ts`. * Updated `tsconfig.json` include paths. * Refactored `test/utils/docker.ts` (though now largely superseded by Kurtosis utils). * Updated logging in `test/scripts/send-txn.ts`. **Reasoning:** This PR significantly expands the E2E testing capabilities by allowing developers to easily launch and integrate local DataHaven nodes and Snowbridge relayers into the test network, facilitating more comprehensive integration testing. The CLI refactoring makes managing these complex setups more robust and user-friendly.
2025-04-29 12:24:00 +00:00
"ora": "^8.2.0",
test: ✨ New CLI functions (#84) ## Changes - New option: `--kurtosis-enclave-name` to allow you to specify a new ethereum network with a different enclave name. Neccesary step for setting up local testing with multiple enclaves running at once - Refactor: all single dash options (e.g. `-i`) have been renamed to have double dashes `--` for consistency others - sad times: CLI now must be invoked `bun cli launch` since we have multiple fns now - Package Update - New Function: `stop` which allows you to kill all components (eth, dh, relayers) or only a single part - Gonza's mac target build fix - Misc fixes to ci like caching and rate limits ## Additional Comment The CLI needs multiple commands and this PR adds the first new one `stop`. This syntax is faily extensible so @ffarall 's k8 command can follow suite. Originally we were going to have an `exec` command to expose internal fns to be callable by command line, i.e. generate beacon checkpoint - however the need for it has since been superceded by the deploy to k8 command that's going to be added. I've left the code in place though so when another usecase comes up we can just plug that in. --------- Co-authored-by: Facundo Farall <37149322+ffarall@users.noreply.github.com>
2025-05-22 20:02:12 +00:00
"pino": "^9.7.0",
test: :white_check_mark: Add E2E Tests (#36) This PR adds the basic framework for E2E and the greater typescript testing framework for the repo. ### Contents - CI workflow for running E2E tests on push to main, PRs and manual - Test suite for E2E - Currently only read-only tests - Using `bun:test` for time being but we can always change in future - We are using the logging library `pino` so we can help with debugging - set environment variable `LOG_LEVEL` to get extra information on helpers - Local `utils` namespace for all our test suites for easy importing - Has helpers which interact with both the docker driver and also blockscout backend - Allows us to fetch deployed smart contracts by their name or address and interact with them --------- Co-authored-by: Facundo Farall <37149322+ffarall@users.noreply.github.com>
2025-04-14 19:22:43 +00:00
"pino-pretty": "^13.0.0",
Bum Polkadot api version to 1.15.1 (#125) Co-authored-by: Steve Degosserie <723552+stiiifff@users.noreply.github.com>
2025-07-31 07:40:58 +00:00
"polkadot-api": "^1.15.1",
test: port Moonwall block validation suite (#201) This PR restructures and port the block validation test suite from Moonbeam and add the necessary infrastructure for contract-based testing. ### Test Suites Added **Block Validation Suite 1** (`test-block-1.ts`) - *Refactored from original `test-block.ts`* - T01: Validates block number increments correctly after manual block creation - T02: Checks block timestamps are valid and within expected bounds - T03: Verifies complete block structure including gasLimit, difficulty, receiptsRoot, transactionsRoot, logsBloom, and other Ethereum-compatible fields - T04: Confirms blocks are retrievable by hash - T05: Confirms blocks are retrievable by number **Block Validation Suite 2** (`test-block-2.ts`) - *New* - T01: Verifies block number persistence across test cases - T02: Validates parent-child block hash linkage in the chain **Block Gas Limits Suite** (`test-block-gas.ts`) - *New* - T01-T06: Tests all three transaction types (legacy, eip1559, eip2930) can deploy contracts at max extrinsic gas limit, and correctly reject transactions exceeding that limit - T07: Deploys `BlockVariables` contract and verifies runtime gas limit is accessible from within contract execution **Block Genesis Suite** (`test-block-genesis.ts`) - *New* - T01: Validates genesis block (block 0) contains correct Ethereum-compatible structure and empty transaction/uncle lists - T02: Confirms genesis block is retrievable by hash ### Infrastructure Additions **Contract Deployment Helpers** - `fetchCompiledContract`: Loads compiled Solidity artifacts with ABI and bytecode from JSON output - `deployCompiledContract`: Handles contract deployment with support for legacy, EIP-1559, and EIP-2930 transaction types, including gas parameter configuration **Solidity Test Fixtures** - `BlockVariables.sol`: Exposes `block.gaslimit`, `block.chainid`, `block.number` via view functions for runtime validation - `Fibonacci.sol`: Provides `fib2(n)` pure function for computational gas testing - `MultiplyBy7.sol`: Minimal pure function contract for basic deployment testing **Enhanced Runtime Constants** - Added `EXTRINSIC_GAS_LIMIT` (52M), `BLOCK_WEIGHT_LIMIT` (2T), `MAX_POV_SIZE` (10MB) for all three environments - Constants derived from and documented against Rust runtime configuration in `operator/runtime/*/src/lib.rs` ### Dependencies - Added `solc@0.8.30` and supporting packages (`yargs`, `command-exists`, `js-sha3`, `memorystream`) for local Solidity compilation
2025-10-06 07:47:35 +00:00
"solc": "^0.8.30",
test: ⚙️ Add Kurtosis Launch Script (#34) This PR adds a script one-liner to spin up an ethereum network, with attached explorers and deploys our `/contracts` folder into there. --------- Co-authored-by: Facundo Farall <37149322+ffarall@users.noreply.github.com>
2025-04-08 15:42:45 +00:00
"tiny-invariant": "^1.3.3",
ci: 👷 Add CI to check PAPI metadata (#107) Add CI check for Polkadot-API metadata freshness This PR adds a new CI workflow that ensures the Polkadot-API metadata file (`test/.papi/metadata/datahaven.scale`) is kept up-to-date when runtime changes are made. Changes: - Added task-check-metadata.yml workflow that: - Reuses the WASM artifact from the build-operator job (no duplicate compilation) - Runs `bun x papi add` to regenerate metadata - Fails if the metadata file has uncommitted changes - Integrated the check into `CI.yml` as a second-tier job alongside `docker-build` Why: - Prevents TypeScript type definitions from becoming out of sync with the runtime - Reminds developers to run `bun generate:types:fast` when making runtime changes - Ensures consistent type safety across the codebase The check provides clear error messages with instructions when metadata is outdated.
2025-06-19 22:12:04 +00:00
"viem": "^2.31.3",
"wagmi": "^2.15.6",
test: ✨ New CLI functions (#84) ## Changes - New option: `--kurtosis-enclave-name` to allow you to specify a new ethereum network with a different enclave name. Neccesary step for setting up local testing with multiple enclaves running at once - Refactor: all single dash options (e.g. `-i`) have been renamed to have double dashes `--` for consistency others - sad times: CLI now must be invoked `bun cli launch` since we have multiple fns now - Package Update - New Function: `stop` which allows you to kill all components (eth, dh, relayers) or only a single part - Gonza's mac target build fix - Misc fixes to ci like caching and rate limits ## Additional Comment The CLI needs multiple commands and this PR adds the first new one `stop`. This syntax is faily extensible so @ffarall 's k8 command can follow suite. Originally we were going to have an `exec` command to expose internal fns to be callable by command line, i.e. generate beacon checkpoint - however the need for it has since been superceded by the deploy to k8 command that's going to be added. I've left the code in place though so when another usecase comes up we can just plug that in. --------- Co-authored-by: Facundo Farall <37149322+ffarall@users.noreply.github.com>
2025-05-22 20:02:12 +00:00
"yaml": "^2.8.0",
test: port Moonwall block validation suite (#201) This PR restructures and port the block validation test suite from Moonbeam and add the necessary infrastructure for contract-based testing. ### Test Suites Added **Block Validation Suite 1** (`test-block-1.ts`) - *Refactored from original `test-block.ts`* - T01: Validates block number increments correctly after manual block creation - T02: Checks block timestamps are valid and within expected bounds - T03: Verifies complete block structure including gasLimit, difficulty, receiptsRoot, transactionsRoot, logsBloom, and other Ethereum-compatible fields - T04: Confirms blocks are retrievable by hash - T05: Confirms blocks are retrievable by number **Block Validation Suite 2** (`test-block-2.ts`) - *New* - T01: Verifies block number persistence across test cases - T02: Validates parent-child block hash linkage in the chain **Block Gas Limits Suite** (`test-block-gas.ts`) - *New* - T01-T06: Tests all three transaction types (legacy, eip1559, eip2930) can deploy contracts at max extrinsic gas limit, and correctly reject transactions exceeding that limit - T07: Deploys `BlockVariables` contract and verifies runtime gas limit is accessible from within contract execution **Block Genesis Suite** (`test-block-genesis.ts`) - *New* - T01: Validates genesis block (block 0) contains correct Ethereum-compatible structure and empty transaction/uncle lists - T02: Confirms genesis block is retrievable by hash ### Infrastructure Additions **Contract Deployment Helpers** - `fetchCompiledContract`: Loads compiled Solidity artifacts with ABI and bytecode from JSON output - `deployCompiledContract`: Handles contract deployment with support for legacy, EIP-1559, and EIP-2930 transaction types, including gas parameter configuration **Solidity Test Fixtures** - `BlockVariables.sol`: Exposes `block.gaslimit`, `block.chainid`, `block.number` via view functions for runtime validation - `Fibonacci.sol`: Provides `fib2(n)` pure function for computational gas testing - `MultiplyBy7.sol`: Minimal pure function contract for basic deployment testing **Enhanced Runtime Constants** - Added `EXTRINSIC_GAS_LIMIT` (52M), `BLOCK_WEIGHT_LIMIT` (2T), `MAX_POV_SIZE` (10MB) for all three environments - Constants derived from and documented against Rust runtime configuration in `operator/runtime/*/src/lib.rs` ### Dependencies - Added `solc@0.8.30` and supporting packages (`yargs`, `command-exists`, `js-sha3`, `memorystream`) for local Solidity compilation
2025-10-06 07:47:35 +00:00
"yargs": "^18.0.0",
ci: 👷 Add CI to check PAPI metadata (#107) Add CI check for Polkadot-API metadata freshness This PR adds a new CI workflow that ensures the Polkadot-API metadata file (`test/.papi/metadata/datahaven.scale`) is kept up-to-date when runtime changes are made. Changes: - Added task-check-metadata.yml workflow that: - Reuses the WASM artifact from the build-operator job (no duplicate compilation) - Runs `bun x papi add` to regenerate metadata - Fails if the metadata file has uncommitted changes - Integrated the check into `CI.yml` as a second-tier job alongside `docker-build` Why: - Prevents TypeScript type definitions from becoming out of sync with the runtime - Reminds developers to run `bun generate:types:fast` when making runtime changes - Ensures consistent type safety across the codebase The check provides clear error messages with instructions when metadata is outdated.
2025-06-19 22:12:04 +00:00
"zod": "^3.25.67"
test: 🧪 Initial Relay Work (#52) Small changes to help progress us closer to relayer support --------- Co-authored-by: Facundo Farall <37149322+ffarall@users.noreply.github.com>
2025-04-23 15:28:19 +00:00
},
"trustedDependencies": [
"@biomejs/biome",
test: 🧙 Generate Type Bindings for Contracts (#58) ## Summary This PR adds statically typed bindings for contracts. This allows you to write E2E tests with full completions in TS. ## Additions - `ts-build.yml` New CI, this will make sure that if there's changes made to the contracts that the contract-bindings are up to date. - `package.json` script changes - `start:e2e:ci` - Designed to be run with all options specified since CIs are famously bad with iteractive CLI prompts - `test:e2e` - added timeout - `generate:wagmi` - This generates the smart contract bindings for our tests - New Function Helpers: - `generateRandomAccount()` Returns a viem account type object for a random account. Useful for tests where you want idempotency on a long lived network since the state is probabilistically fresh - `getContractInstance()` Returns a viem contract instance that allows you to read/write to the deployed contract. You should get full type inference here for the methods available and parameters required. ### Example ```ts it("avs() can be read from contract instance", async () => { const value = await instance.read.avs(); expect(isAddress(value), "AVS getter should return an address").toBeTrue(); }); ``` --------- Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> Co-authored-by: Facundo Farall <37149322+ffarall@users.noreply.github.com>
2025-05-01 10:14:19 +00:00
"bufferutil",
"cpu-features",
"esbuild",
"keccak",
"protobufjs",
"ssh2",
"utf-8-validate"
test: 🧪 Initial Relay Work (#52) Small changes to help progress us closer to relayer support --------- Co-authored-by: Facundo Farall <37149322+ffarall@users.noreply.github.com>
2025-04-23 15:28:19 +00:00
]
test: Native token transfer e2e tests (#120) ### Summary - **Add** `test/suites/native-token-transfer.test.ts` focused on the HAVE native token lifecycle via Snowbridge v2. - **Validate** registration, DataHaven → Ethereum mints, Ethereum → DataHaven unlocks, event emission, and 1:1 backing invariant. ### Tests added - should register DataHaven native token on Ethereum - should transfer tokens from DataHaven to Ethereum - should maintain 1:1 backing ratio - should emit transfer events - should transfer tokens from Ethereum to DataHaven (Snowbridge v2) ### What the suite covers - **Registration**: Sudo-registers the native token; confirms `ForeignTokenRegistered` on the Gateway; verifies ERC-20 metadata (`HAVE`/`wHAVE`, 18 decimals). - **DataHaven → Ethereum**: Executes `transfer_to_ethereum`; asserts Substrate events (`TokensLocked`, `TokensTransferredToEthereum`); observes Ethereum `Transfer` mint (from zero address); validates sender balance delta, sovereign account increase, and ERC-20 recipient credit. - **Backing invariant**: Ensures sovereign account balance ≥ ERC-20 total supply. - **Event emission**: Confirms key Substrate events without polling delays. - **Ethereum → DataHaven**: Approves and calls `Gateway.sendToken`; if unsupported locally, the test skips; otherwise asserts burn on Ethereum and unlock on DataHaven with corresponding balance deltas. --------- Co-authored-by: Steve Degosserie <723552+stiiifff@users.noreply.github.com>
2025-08-22 16:27:14 +00:00
}
Reference in a new issue Copy permalink