datahaven/test/utils/input.ts

99 lines
2.2 KiB
TypeScript
Raw Permalink Normal View History

test: ⚙️ Parse & Generate Relayer Configs (#54) ## Human Written Description This PR adds the following to the E2E CLI: - Relayer config generation for: `beacon-relay` `beefy-relay` - The other two relayer types to be added later - Relayers don't actually work yet - By default turned off, this requires a binary to be present in: `<repo_root>/operator/target/release` dir - Datahaven network launching - DH network is using default `local` network chain spec - Launched with 5 nodes since our authority set is 6 large (and you need 2/3 + 1 of set size > [!NOTE] > Both the relayer and the DH node binaries are being run as local processes TEMPORARILY. This means that logging is done in a very rudimentary way (we pipe to a file whilst the CLI is running). > > This means that when the CLI finishes **the log files will no longer be written to**. > This is temporary since spawning binaries is a stop gap solution until docker images available. --- > [!IMPORTANT] > The following is AI generated slop describing this PR's changes: **Key Changes:** * **CLI Enhancements (`test/cli/index.ts`):** * Added options `--datahaven` and `--datahaven-bin-path` to enable launching local DataHaven nodes. * Added options `--relayer` and `--relayer-bin-path` to enable launching Snowbridge relayers (Beefy and Beacon). * Added negation flags (`--no-fund-validators`, `--no-setup-validators`, `--no-update-validator-set`) for more granular control over validator setup steps. * Added `--skip-cleaning` option to preserve Kurtosis state between runs. * Added a pre-action hook (`launchPreActionHook`) to validate flag combinations (e.g., `--verified` requires `--blockscout`). * **New CLI Handlers (`test/cli/handlers/launch/`):** * `datahaven.ts`: Logic for spawning DataHaven node processes using the specified binary. Manages ports and process cleanup. * `relayer.ts`: Logic for configuring and spawning Snowbridge relayer processes (Beefy and Beacon). Reads contract deployment addresses, updates relayer config templates, and uses specified private keys. Manages log files and process cleanup. * `summary.ts`: Generates and displays the table of running services (including dynamically launched DataHaven nodes) and their endpoints. * `validator.ts`: Extracted validator funding, setup, and set update logic into its own handler. * `index.ts`: Orchestrates the launch sequence based on CLI options, calling the appropriate handlers. Includes a `LaunchedNetwork` class to track spawned processes, file descriptors, and node ports for cleanup. * **Updated `package.json` Scripts:** * Added `start:e2e:minrelayer` script for a minimal setup including relayers and DataHaven nodes. * Modified `stop:e2e` to include `pkill datahaven` for proper cleanup. * Added `stop:e2e:quick` to only stop the Kurtosis enclave without full cleaning. * **Updated `launch-kurtosis.ts`:** Modified to use new Kurtosis utility functions and added a `skipCleaning` option. * **New Utility Functions:** * `test/utils/kurtosis.ts`: Functions to inspect Kurtosis services (`getServiceFromKurtosis`, `getPortFromKurtosis`, `getServicesFromKurtosis`). * `test/utils/parser.ts`: Zod schemas and parsing functions for Snowbridge relayer configurations. * **Constants & Minor Updates:** * Added `SUBSTRATE_FUNDED_ACCOUNTS` to `test/utils/constants.ts`. * Updated `tsconfig.json` include paths. * Refactored `test/utils/docker.ts` (though now largely superseded by Kurtosis utils). * Updated logging in `test/scripts/send-txn.ts`. **Reasoning:** This PR significantly expands the E2E testing capabilities by allowing developers to easily launch and integrate local DataHaven nodes and Snowbridge relayers into the test network, facilitating more comprehensive integration testing. The CLI refactoring makes managing these complex setups more robust and user-friendly.
2025-04-29 12:24:00 +00:00
import {
createPrompt,
isEnterKey,
makeTheme,
type Status,
type Theme,
test: ⚙️ Parse & Generate Relayer Configs (#54) ## Human Written Description This PR adds the following to the E2E CLI: - Relayer config generation for: `beacon-relay` `beefy-relay` - The other two relayer types to be added later - Relayers don't actually work yet - By default turned off, this requires a binary to be present in: `<repo_root>/operator/target/release` dir - Datahaven network launching - DH network is using default `local` network chain spec - Launched with 5 nodes since our authority set is 6 large (and you need 2/3 + 1 of set size > [!NOTE] > Both the relayer and the DH node binaries are being run as local processes TEMPORARILY. This means that logging is done in a very rudimentary way (we pipe to a file whilst the CLI is running). > > This means that when the CLI finishes **the log files will no longer be written to**. > This is temporary since spawning binaries is a stop gap solution until docker images available. --- > [!IMPORTANT] > The following is AI generated slop describing this PR's changes: **Key Changes:** * **CLI Enhancements (`test/cli/index.ts`):** * Added options `--datahaven` and `--datahaven-bin-path` to enable launching local DataHaven nodes. * Added options `--relayer` and `--relayer-bin-path` to enable launching Snowbridge relayers (Beefy and Beacon). * Added negation flags (`--no-fund-validators`, `--no-setup-validators`, `--no-update-validator-set`) for more granular control over validator setup steps. * Added `--skip-cleaning` option to preserve Kurtosis state between runs. * Added a pre-action hook (`launchPreActionHook`) to validate flag combinations (e.g., `--verified` requires `--blockscout`). * **New CLI Handlers (`test/cli/handlers/launch/`):** * `datahaven.ts`: Logic for spawning DataHaven node processes using the specified binary. Manages ports and process cleanup. * `relayer.ts`: Logic for configuring and spawning Snowbridge relayer processes (Beefy and Beacon). Reads contract deployment addresses, updates relayer config templates, and uses specified private keys. Manages log files and process cleanup. * `summary.ts`: Generates and displays the table of running services (including dynamically launched DataHaven nodes) and their endpoints. * `validator.ts`: Extracted validator funding, setup, and set update logic into its own handler. * `index.ts`: Orchestrates the launch sequence based on CLI options, calling the appropriate handlers. Includes a `LaunchedNetwork` class to track spawned processes, file descriptors, and node ports for cleanup. * **Updated `package.json` Scripts:** * Added `start:e2e:minrelayer` script for a minimal setup including relayers and DataHaven nodes. * Modified `stop:e2e` to include `pkill datahaven` for proper cleanup. * Added `stop:e2e:quick` to only stop the Kurtosis enclave without full cleaning. * **Updated `launch-kurtosis.ts`:** Modified to use new Kurtosis utility functions and added a `skipCleaning` option. * **New Utility Functions:** * `test/utils/kurtosis.ts`: Functions to inspect Kurtosis services (`getServiceFromKurtosis`, `getPortFromKurtosis`, `getServicesFromKurtosis`). * `test/utils/parser.ts`: Zod schemas and parsing functions for Snowbridge relayer configurations. * **Constants & Minor Updates:** * Added `SUBSTRATE_FUNDED_ACCOUNTS` to `test/utils/constants.ts`. * Updated `tsconfig.json` include paths. * Refactored `test/utils/docker.ts` (though now largely superseded by Kurtosis utils). * Updated logging in `test/scripts/send-txn.ts`. **Reasoning:** This PR significantly expands the E2E testing capabilities by allowing developers to easily launch and integrate local DataHaven nodes and Snowbridge relayers into the test network, facilitating more comprehensive integration testing. The CLI refactoring makes managing these complex setups more robust and user-friendly.
2025-04-29 12:24:00 +00:00
useEffect,
useKeypress,
usePrefix,
useState
} from "@inquirer/core";
import type { PartialDeep } from "@inquirer/type";
import chalk from "chalk";
test: ⚙️ Parse & Generate Relayer Configs (#54) ## Human Written Description This PR adds the following to the E2E CLI: - Relayer config generation for: `beacon-relay` `beefy-relay` - The other two relayer types to be added later - Relayers don't actually work yet - By default turned off, this requires a binary to be present in: `<repo_root>/operator/target/release` dir - Datahaven network launching - DH network is using default `local` network chain spec - Launched with 5 nodes since our authority set is 6 large (and you need 2/3 + 1 of set size > [!NOTE] > Both the relayer and the DH node binaries are being run as local processes TEMPORARILY. This means that logging is done in a very rudimentary way (we pipe to a file whilst the CLI is running). > > This means that when the CLI finishes **the log files will no longer be written to**. > This is temporary since spawning binaries is a stop gap solution until docker images available. --- > [!IMPORTANT] > The following is AI generated slop describing this PR's changes: **Key Changes:** * **CLI Enhancements (`test/cli/index.ts`):** * Added options `--datahaven` and `--datahaven-bin-path` to enable launching local DataHaven nodes. * Added options `--relayer` and `--relayer-bin-path` to enable launching Snowbridge relayers (Beefy and Beacon). * Added negation flags (`--no-fund-validators`, `--no-setup-validators`, `--no-update-validator-set`) for more granular control over validator setup steps. * Added `--skip-cleaning` option to preserve Kurtosis state between runs. * Added a pre-action hook (`launchPreActionHook`) to validate flag combinations (e.g., `--verified` requires `--blockscout`). * **New CLI Handlers (`test/cli/handlers/launch/`):** * `datahaven.ts`: Logic for spawning DataHaven node processes using the specified binary. Manages ports and process cleanup. * `relayer.ts`: Logic for configuring and spawning Snowbridge relayer processes (Beefy and Beacon). Reads contract deployment addresses, updates relayer config templates, and uses specified private keys. Manages log files and process cleanup. * `summary.ts`: Generates and displays the table of running services (including dynamically launched DataHaven nodes) and their endpoints. * `validator.ts`: Extracted validator funding, setup, and set update logic into its own handler. * `index.ts`: Orchestrates the launch sequence based on CLI options, calling the appropriate handlers. Includes a `LaunchedNetwork` class to track spawned processes, file descriptors, and node ports for cleanup. * **Updated `package.json` Scripts:** * Added `start:e2e:minrelayer` script for a minimal setup including relayers and DataHaven nodes. * Modified `stop:e2e` to include `pkill datahaven` for proper cleanup. * Added `stop:e2e:quick` to only stop the Kurtosis enclave without full cleaning. * **Updated `launch-kurtosis.ts`:** Modified to use new Kurtosis utility functions and added a `skipCleaning` option. * **New Utility Functions:** * `test/utils/kurtosis.ts`: Functions to inspect Kurtosis services (`getServiceFromKurtosis`, `getPortFromKurtosis`, `getServicesFromKurtosis`). * `test/utils/parser.ts`: Zod schemas and parsing functions for Snowbridge relayer configurations. * **Constants & Minor Updates:** * Added `SUBSTRATE_FUNDED_ACCOUNTS` to `test/utils/constants.ts`. * Updated `tsconfig.json` include paths. * Refactored `test/utils/docker.ts` (though now largely superseded by Kurtosis utils). * Updated logging in `test/scripts/send-txn.ts`. **Reasoning:** This PR significantly expands the E2E testing capabilities by allowing developers to easily launch and integrate local DataHaven nodes and Snowbridge relayers into the test network, facilitating more comprehensive integration testing. The CLI refactoring makes managing these complex setups more robust and user-friendly.
2025-04-29 12:24:00 +00:00
type TimeoutConfirmConfig = {
message: string;
default?: boolean;
timeoutMs: number;
theme?: PartialDeep<Theme>;
};
export const timeoutConfirm = createPrompt<boolean, TimeoutConfirmConfig>((cfg, done) => {
const [status, setStatus] = useState<Status>("loading");
const [input, setInput] = useState("");
const [left, setLeft] = useState(cfg.timeoutMs);
const theme = makeTheme(cfg.theme);
const prefix = usePrefix({ status, theme });
useEffect(() => {
const startTime = Date.now();
const id = setInterval(() => {
const elapsed = Date.now() - startTime;
const newLeft = Math.max(0, cfg.timeoutMs - elapsed);
setLeft(newLeft);
if (newLeft <= 0) {
setStatus("done");
clearInterval(id);
done(cfg.default ?? true);
}
test: ⚡️ CI Refactor (#59) Eventually our CI will be required to run two private blockchains locally plus associated relayers. This PR is to prepare for this fate by improving run times and refactoring our existing CIs so they are a bit easier to reason about. ### Refactors - **_We now run ALL CIs on every PR!_** This is so that we decomplexify the logic around conditional builds and fetching built binaries from another source. This reduces the surface area of code we have to maintain at the cost of execution time - This penalty is ameliorated by a layered caching system. At best, it will be less than a minute to complete a build since everything will be cached. On GH runners this is about 6 minutes sadly. - We will no longer be at risk of important CIs being skipped erroneously which hide true failures. - Caching is a low-risk approach because at worst it has to build from scratch. A bad cache hit will never imply the wrong thing gets build since cargo is smart enough to just throw away any inappropriate build artefacts. - `setup-rust` action created so we have a unified way of setting up runner and unifying our approach to caching - Use a unique caching key for different activities and it will fallback to shared cache if no matches - we are using `mainnet` kurtosis config so that it works with relayer assumptions ### Additions - We can specify the ethereum block time via a new cli arg `--slot-time <seconds>` - We can specify arbitrary network_param args which get passed into the generated yaml - e.g. giving `bun cli --kurtosis-network-args="pet=cat food=fish" will add: ```yml network_params: # existing params... pet: cat food: fish ``` - We now have the ability to programmatically modify the yaml - This means we are back down to a single `minimal.yml` kurtosis config so we dont have to maintain changes between them - Flow is: `add new cli arg` -> `add if() block which mutates yaml` -> `profit` --------- Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> Co-authored-by: Facundo Farall <37149322+ffarall@users.noreply.github.com>
2025-05-06 20:20:02 +00:00
}, 500);
test: ⚙️ Parse & Generate Relayer Configs (#54) ## Human Written Description This PR adds the following to the E2E CLI: - Relayer config generation for: `beacon-relay` `beefy-relay` - The other two relayer types to be added later - Relayers don't actually work yet - By default turned off, this requires a binary to be present in: `<repo_root>/operator/target/release` dir - Datahaven network launching - DH network is using default `local` network chain spec - Launched with 5 nodes since our authority set is 6 large (and you need 2/3 + 1 of set size > [!NOTE] > Both the relayer and the DH node binaries are being run as local processes TEMPORARILY. This means that logging is done in a very rudimentary way (we pipe to a file whilst the CLI is running). > > This means that when the CLI finishes **the log files will no longer be written to**. > This is temporary since spawning binaries is a stop gap solution until docker images available. --- > [!IMPORTANT] > The following is AI generated slop describing this PR's changes: **Key Changes:** * **CLI Enhancements (`test/cli/index.ts`):** * Added options `--datahaven` and `--datahaven-bin-path` to enable launching local DataHaven nodes. * Added options `--relayer` and `--relayer-bin-path` to enable launching Snowbridge relayers (Beefy and Beacon). * Added negation flags (`--no-fund-validators`, `--no-setup-validators`, `--no-update-validator-set`) for more granular control over validator setup steps. * Added `--skip-cleaning` option to preserve Kurtosis state between runs. * Added a pre-action hook (`launchPreActionHook`) to validate flag combinations (e.g., `--verified` requires `--blockscout`). * **New CLI Handlers (`test/cli/handlers/launch/`):** * `datahaven.ts`: Logic for spawning DataHaven node processes using the specified binary. Manages ports and process cleanup. * `relayer.ts`: Logic for configuring and spawning Snowbridge relayer processes (Beefy and Beacon). Reads contract deployment addresses, updates relayer config templates, and uses specified private keys. Manages log files and process cleanup. * `summary.ts`: Generates and displays the table of running services (including dynamically launched DataHaven nodes) and their endpoints. * `validator.ts`: Extracted validator funding, setup, and set update logic into its own handler. * `index.ts`: Orchestrates the launch sequence based on CLI options, calling the appropriate handlers. Includes a `LaunchedNetwork` class to track spawned processes, file descriptors, and node ports for cleanup. * **Updated `package.json` Scripts:** * Added `start:e2e:minrelayer` script for a minimal setup including relayers and DataHaven nodes. * Modified `stop:e2e` to include `pkill datahaven` for proper cleanup. * Added `stop:e2e:quick` to only stop the Kurtosis enclave without full cleaning. * **Updated `launch-kurtosis.ts`:** Modified to use new Kurtosis utility functions and added a `skipCleaning` option. * **New Utility Functions:** * `test/utils/kurtosis.ts`: Functions to inspect Kurtosis services (`getServiceFromKurtosis`, `getPortFromKurtosis`, `getServicesFromKurtosis`). * `test/utils/parser.ts`: Zod schemas and parsing functions for Snowbridge relayer configurations. * **Constants & Minor Updates:** * Added `SUBSTRATE_FUNDED_ACCOUNTS` to `test/utils/constants.ts`. * Updated `tsconfig.json` include paths. * Refactored `test/utils/docker.ts` (though now largely superseded by Kurtosis utils). * Updated logging in `test/scripts/send-txn.ts`. **Reasoning:** This PR significantly expands the E2E testing capabilities by allowing developers to easily launch and integrate local DataHaven nodes and Snowbridge relayers into the test network, facilitating more comprehensive integration testing. The CLI refactoring makes managing these complex setups more robust and user-friendly.
2025-04-29 12:24:00 +00:00
return () => clearInterval(id);
}, []);
const finish = () => {
const val = /^(y|yes)$/i.test(input)
? true
: /^(n|no)$/i.test(input)
? false
: (cfg.default ?? true);
setStatus("done");
done(val);
};
useKeypress((key, rl) => {
if (isEnterKey(key)) finish();
else setInput(rl.line);
});
const defaultBadge = theme.style.defaultAnswer(cfg.default === false ? "y/N" : "Y/n");
const main = `${prefix} ${theme.style.message(cfg.message, status)} \
${defaultBadge} ${input}`;
test: ⚡️ CI Refactor (#59) Eventually our CI will be required to run two private blockchains locally plus associated relayers. This PR is to prepare for this fate by improving run times and refactoring our existing CIs so they are a bit easier to reason about. ### Refactors - **_We now run ALL CIs on every PR!_** This is so that we decomplexify the logic around conditional builds and fetching built binaries from another source. This reduces the surface area of code we have to maintain at the cost of execution time - This penalty is ameliorated by a layered caching system. At best, it will be less than a minute to complete a build since everything will be cached. On GH runners this is about 6 minutes sadly. - We will no longer be at risk of important CIs being skipped erroneously which hide true failures. - Caching is a low-risk approach because at worst it has to build from scratch. A bad cache hit will never imply the wrong thing gets build since cargo is smart enough to just throw away any inappropriate build artefacts. - `setup-rust` action created so we have a unified way of setting up runner and unifying our approach to caching - Use a unique caching key for different activities and it will fallback to shared cache if no matches - we are using `mainnet` kurtosis config so that it works with relayer assumptions ### Additions - We can specify the ethereum block time via a new cli arg `--slot-time <seconds>` - We can specify arbitrary network_param args which get passed into the generated yaml - e.g. giving `bun cli --kurtosis-network-args="pet=cat food=fish" will add: ```yml network_params: # existing params... pet: cat food: fish ``` - We now have the ability to programmatically modify the yaml - This means we are back down to a single `minimal.yml` kurtosis config so we dont have to maintain changes between them - Flow is: `add new cli arg` -> `add if() block which mutates yaml` -> `profit` --------- Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> Co-authored-by: Facundo Farall <37149322+ffarall@users.noreply.github.com>
2025-05-06 20:20:02 +00:00
const border = chalk.yellow("=".repeat(80));
test: ⚙️ Parse & Generate Relayer Configs (#54) ## Human Written Description This PR adds the following to the E2E CLI: - Relayer config generation for: `beacon-relay` `beefy-relay` - The other two relayer types to be added later - Relayers don't actually work yet - By default turned off, this requires a binary to be present in: `<repo_root>/operator/target/release` dir - Datahaven network launching - DH network is using default `local` network chain spec - Launched with 5 nodes since our authority set is 6 large (and you need 2/3 + 1 of set size > [!NOTE] > Both the relayer and the DH node binaries are being run as local processes TEMPORARILY. This means that logging is done in a very rudimentary way (we pipe to a file whilst the CLI is running). > > This means that when the CLI finishes **the log files will no longer be written to**. > This is temporary since spawning binaries is a stop gap solution until docker images available. --- > [!IMPORTANT] > The following is AI generated slop describing this PR's changes: **Key Changes:** * **CLI Enhancements (`test/cli/index.ts`):** * Added options `--datahaven` and `--datahaven-bin-path` to enable launching local DataHaven nodes. * Added options `--relayer` and `--relayer-bin-path` to enable launching Snowbridge relayers (Beefy and Beacon). * Added negation flags (`--no-fund-validators`, `--no-setup-validators`, `--no-update-validator-set`) for more granular control over validator setup steps. * Added `--skip-cleaning` option to preserve Kurtosis state between runs. * Added a pre-action hook (`launchPreActionHook`) to validate flag combinations (e.g., `--verified` requires `--blockscout`). * **New CLI Handlers (`test/cli/handlers/launch/`):** * `datahaven.ts`: Logic for spawning DataHaven node processes using the specified binary. Manages ports and process cleanup. * `relayer.ts`: Logic for configuring and spawning Snowbridge relayer processes (Beefy and Beacon). Reads contract deployment addresses, updates relayer config templates, and uses specified private keys. Manages log files and process cleanup. * `summary.ts`: Generates and displays the table of running services (including dynamically launched DataHaven nodes) and their endpoints. * `validator.ts`: Extracted validator funding, setup, and set update logic into its own handler. * `index.ts`: Orchestrates the launch sequence based on CLI options, calling the appropriate handlers. Includes a `LaunchedNetwork` class to track spawned processes, file descriptors, and node ports for cleanup. * **Updated `package.json` Scripts:** * Added `start:e2e:minrelayer` script for a minimal setup including relayers and DataHaven nodes. * Modified `stop:e2e` to include `pkill datahaven` for proper cleanup. * Added `stop:e2e:quick` to only stop the Kurtosis enclave without full cleaning. * **Updated `launch-kurtosis.ts`:** Modified to use new Kurtosis utility functions and added a `skipCleaning` option. * **New Utility Functions:** * `test/utils/kurtosis.ts`: Functions to inspect Kurtosis services (`getServiceFromKurtosis`, `getPortFromKurtosis`, `getServicesFromKurtosis`). * `test/utils/parser.ts`: Zod schemas and parsing functions for Snowbridge relayer configurations. * **Constants & Minor Updates:** * Added `SUBSTRATE_FUNDED_ACCOUNTS` to `test/utils/constants.ts`. * Updated `tsconfig.json` include paths. * Refactored `test/utils/docker.ts` (though now largely superseded by Kurtosis utils). * Updated logging in `test/scripts/send-txn.ts`. **Reasoning:** This PR significantly expands the E2E testing capabilities by allowing developers to easily launch and integrate local DataHaven nodes and Snowbridge relayers into the test network, facilitating more comprehensive integration testing. The CLI refactoring makes managing these complex setups more robust and user-friendly.
2025-04-29 12:24:00 +00:00
const hint = theme.style.help(
chalk.magenta(
`⏱ Will default to ${chalk.bold(cfg.default ? "YES" : "NO")} in ${chalk.bold((left / 1000).toFixed(0))}s`
)
);
return `${border}
${hint}
${main}
${border}`;
test: ⚙️ Parse & Generate Relayer Configs (#54) ## Human Written Description This PR adds the following to the E2E CLI: - Relayer config generation for: `beacon-relay` `beefy-relay` - The other two relayer types to be added later - Relayers don't actually work yet - By default turned off, this requires a binary to be present in: `<repo_root>/operator/target/release` dir - Datahaven network launching - DH network is using default `local` network chain spec - Launched with 5 nodes since our authority set is 6 large (and you need 2/3 + 1 of set size > [!NOTE] > Both the relayer and the DH node binaries are being run as local processes TEMPORARILY. This means that logging is done in a very rudimentary way (we pipe to a file whilst the CLI is running). > > This means that when the CLI finishes **the log files will no longer be written to**. > This is temporary since spawning binaries is a stop gap solution until docker images available. --- > [!IMPORTANT] > The following is AI generated slop describing this PR's changes: **Key Changes:** * **CLI Enhancements (`test/cli/index.ts`):** * Added options `--datahaven` and `--datahaven-bin-path` to enable launching local DataHaven nodes. * Added options `--relayer` and `--relayer-bin-path` to enable launching Snowbridge relayers (Beefy and Beacon). * Added negation flags (`--no-fund-validators`, `--no-setup-validators`, `--no-update-validator-set`) for more granular control over validator setup steps. * Added `--skip-cleaning` option to preserve Kurtosis state between runs. * Added a pre-action hook (`launchPreActionHook`) to validate flag combinations (e.g., `--verified` requires `--blockscout`). * **New CLI Handlers (`test/cli/handlers/launch/`):** * `datahaven.ts`: Logic for spawning DataHaven node processes using the specified binary. Manages ports and process cleanup. * `relayer.ts`: Logic for configuring and spawning Snowbridge relayer processes (Beefy and Beacon). Reads contract deployment addresses, updates relayer config templates, and uses specified private keys. Manages log files and process cleanup. * `summary.ts`: Generates and displays the table of running services (including dynamically launched DataHaven nodes) and their endpoints. * `validator.ts`: Extracted validator funding, setup, and set update logic into its own handler. * `index.ts`: Orchestrates the launch sequence based on CLI options, calling the appropriate handlers. Includes a `LaunchedNetwork` class to track spawned processes, file descriptors, and node ports for cleanup. * **Updated `package.json` Scripts:** * Added `start:e2e:minrelayer` script for a minimal setup including relayers and DataHaven nodes. * Modified `stop:e2e` to include `pkill datahaven` for proper cleanup. * Added `stop:e2e:quick` to only stop the Kurtosis enclave without full cleaning. * **Updated `launch-kurtosis.ts`:** Modified to use new Kurtosis utility functions and added a `skipCleaning` option. * **New Utility Functions:** * `test/utils/kurtosis.ts`: Functions to inspect Kurtosis services (`getServiceFromKurtosis`, `getPortFromKurtosis`, `getServicesFromKurtosis`). * `test/utils/parser.ts`: Zod schemas and parsing functions for Snowbridge relayer configurations. * **Constants & Minor Updates:** * Added `SUBSTRATE_FUNDED_ACCOUNTS` to `test/utils/constants.ts`. * Updated `tsconfig.json` include paths. * Refactored `test/utils/docker.ts` (though now largely superseded by Kurtosis utils). * Updated logging in `test/scripts/send-txn.ts`. **Reasoning:** This PR significantly expands the E2E testing capabilities by allowing developers to easily launch and integrate local DataHaven nodes and Snowbridge relayers into the test network, facilitating more comprehensive integration testing. The CLI refactoring makes managing these complex setups more robust and user-friendly.
2025-04-29 12:24:00 +00:00
});
test: ⚡️ CI Refactor (#59) Eventually our CI will be required to run two private blockchains locally plus associated relayers. This PR is to prepare for this fate by improving run times and refactoring our existing CIs so they are a bit easier to reason about. ### Refactors - **_We now run ALL CIs on every PR!_** This is so that we decomplexify the logic around conditional builds and fetching built binaries from another source. This reduces the surface area of code we have to maintain at the cost of execution time - This penalty is ameliorated by a layered caching system. At best, it will be less than a minute to complete a build since everything will be cached. On GH runners this is about 6 minutes sadly. - We will no longer be at risk of important CIs being skipped erroneously which hide true failures. - Caching is a low-risk approach because at worst it has to build from scratch. A bad cache hit will never imply the wrong thing gets build since cargo is smart enough to just throw away any inappropriate build artefacts. - `setup-rust` action created so we have a unified way of setting up runner and unifying our approach to caching - Use a unique caching key for different activities and it will fallback to shared cache if no matches - we are using `mainnet` kurtosis config so that it works with relayer assumptions ### Additions - We can specify the ethereum block time via a new cli arg `--slot-time <seconds>` - We can specify arbitrary network_param args which get passed into the generated yaml - e.g. giving `bun cli --kurtosis-network-args="pet=cat food=fish" will add: ```yml network_params: # existing params... pet: cat food: fish ``` - We now have the ability to programmatically modify the yaml - This means we are back down to a single `minimal.yml` kurtosis config so we dont have to maintain changes between them - Flow is: `add new cli arg` -> `add if() block which mutates yaml` -> `profit` --------- Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> Co-authored-by: Facundo Farall <37149322+ffarall@users.noreply.github.com>
2025-05-06 20:20:02 +00:00
export const confirmWithTimeout = async (
question: string,
defaultValue: boolean,
timeoutSeconds: number
test: ⚡️ CI Refactor (#59) Eventually our CI will be required to run two private blockchains locally plus associated relayers. This PR is to prepare for this fate by improving run times and refactoring our existing CIs so they are a bit easier to reason about. ### Refactors - **_We now run ALL CIs on every PR!_** This is so that we decomplexify the logic around conditional builds and fetching built binaries from another source. This reduces the surface area of code we have to maintain at the cost of execution time - This penalty is ameliorated by a layered caching system. At best, it will be less than a minute to complete a build since everything will be cached. On GH runners this is about 6 minutes sadly. - We will no longer be at risk of important CIs being skipped erroneously which hide true failures. - Caching is a low-risk approach because at worst it has to build from scratch. A bad cache hit will never imply the wrong thing gets build since cargo is smart enough to just throw away any inappropriate build artefacts. - `setup-rust` action created so we have a unified way of setting up runner and unifying our approach to caching - Use a unique caching key for different activities and it will fallback to shared cache if no matches - we are using `mainnet` kurtosis config so that it works with relayer assumptions ### Additions - We can specify the ethereum block time via a new cli arg `--slot-time <seconds>` - We can specify arbitrary network_param args which get passed into the generated yaml - e.g. giving `bun cli --kurtosis-network-args="pet=cat food=fish" will add: ```yml network_params: # existing params... pet: cat food: fish ``` - We now have the ability to programmatically modify the yaml - This means we are back down to a single `minimal.yml` kurtosis config so we dont have to maintain changes between them - Flow is: `add new cli arg` -> `add if() block which mutates yaml` -> `profit` --------- Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> Co-authored-by: Facundo Farall <37149322+ffarall@users.noreply.github.com>
2025-05-06 20:20:02 +00:00
) => {
await Bun.sleep(50); //debounce
return timeoutConfirm({
test: ⚙️ Parse & Generate Relayer Configs (#54) ## Human Written Description This PR adds the following to the E2E CLI: - Relayer config generation for: `beacon-relay` `beefy-relay` - The other two relayer types to be added later - Relayers don't actually work yet - By default turned off, this requires a binary to be present in: `<repo_root>/operator/target/release` dir - Datahaven network launching - DH network is using default `local` network chain spec - Launched with 5 nodes since our authority set is 6 large (and you need 2/3 + 1 of set size > [!NOTE] > Both the relayer and the DH node binaries are being run as local processes TEMPORARILY. This means that logging is done in a very rudimentary way (we pipe to a file whilst the CLI is running). > > This means that when the CLI finishes **the log files will no longer be written to**. > This is temporary since spawning binaries is a stop gap solution until docker images available. --- > [!IMPORTANT] > The following is AI generated slop describing this PR's changes: **Key Changes:** * **CLI Enhancements (`test/cli/index.ts`):** * Added options `--datahaven` and `--datahaven-bin-path` to enable launching local DataHaven nodes. * Added options `--relayer` and `--relayer-bin-path` to enable launching Snowbridge relayers (Beefy and Beacon). * Added negation flags (`--no-fund-validators`, `--no-setup-validators`, `--no-update-validator-set`) for more granular control over validator setup steps. * Added `--skip-cleaning` option to preserve Kurtosis state between runs. * Added a pre-action hook (`launchPreActionHook`) to validate flag combinations (e.g., `--verified` requires `--blockscout`). * **New CLI Handlers (`test/cli/handlers/launch/`):** * `datahaven.ts`: Logic for spawning DataHaven node processes using the specified binary. Manages ports and process cleanup. * `relayer.ts`: Logic for configuring and spawning Snowbridge relayer processes (Beefy and Beacon). Reads contract deployment addresses, updates relayer config templates, and uses specified private keys. Manages log files and process cleanup. * `summary.ts`: Generates and displays the table of running services (including dynamically launched DataHaven nodes) and their endpoints. * `validator.ts`: Extracted validator funding, setup, and set update logic into its own handler. * `index.ts`: Orchestrates the launch sequence based on CLI options, calling the appropriate handlers. Includes a `LaunchedNetwork` class to track spawned processes, file descriptors, and node ports for cleanup. * **Updated `package.json` Scripts:** * Added `start:e2e:minrelayer` script for a minimal setup including relayers and DataHaven nodes. * Modified `stop:e2e` to include `pkill datahaven` for proper cleanup. * Added `stop:e2e:quick` to only stop the Kurtosis enclave without full cleaning. * **Updated `launch-kurtosis.ts`:** Modified to use new Kurtosis utility functions and added a `skipCleaning` option. * **New Utility Functions:** * `test/utils/kurtosis.ts`: Functions to inspect Kurtosis services (`getServiceFromKurtosis`, `getPortFromKurtosis`, `getServicesFromKurtosis`). * `test/utils/parser.ts`: Zod schemas and parsing functions for Snowbridge relayer configurations. * **Constants & Minor Updates:** * Added `SUBSTRATE_FUNDED_ACCOUNTS` to `test/utils/constants.ts`. * Updated `tsconfig.json` include paths. * Refactored `test/utils/docker.ts` (though now largely superseded by Kurtosis utils). * Updated logging in `test/scripts/send-txn.ts`. **Reasoning:** This PR significantly expands the E2E testing capabilities by allowing developers to easily launch and integrate local DataHaven nodes and Snowbridge relayers into the test network, facilitating more comprehensive integration testing. The CLI refactoring makes managing these complex setups more robust and user-friendly.
2025-04-29 12:24:00 +00:00
message: question,
default: defaultValue,
timeoutMs: timeoutSeconds * 1000,
theme: {
style: {
message: (text: string) => chalk.cyan(text),
answer: (text: string) => chalk.green(text)
}
test: ⚙️ Parse & Generate Relayer Configs (#54) ## Human Written Description This PR adds the following to the E2E CLI: - Relayer config generation for: `beacon-relay` `beefy-relay` - The other two relayer types to be added later - Relayers don't actually work yet - By default turned off, this requires a binary to be present in: `<repo_root>/operator/target/release` dir - Datahaven network launching - DH network is using default `local` network chain spec - Launched with 5 nodes since our authority set is 6 large (and you need 2/3 + 1 of set size > [!NOTE] > Both the relayer and the DH node binaries are being run as local processes TEMPORARILY. This means that logging is done in a very rudimentary way (we pipe to a file whilst the CLI is running). > > This means that when the CLI finishes **the log files will no longer be written to**. > This is temporary since spawning binaries is a stop gap solution until docker images available. --- > [!IMPORTANT] > The following is AI generated slop describing this PR's changes: **Key Changes:** * **CLI Enhancements (`test/cli/index.ts`):** * Added options `--datahaven` and `--datahaven-bin-path` to enable launching local DataHaven nodes. * Added options `--relayer` and `--relayer-bin-path` to enable launching Snowbridge relayers (Beefy and Beacon). * Added negation flags (`--no-fund-validators`, `--no-setup-validators`, `--no-update-validator-set`) for more granular control over validator setup steps. * Added `--skip-cleaning` option to preserve Kurtosis state between runs. * Added a pre-action hook (`launchPreActionHook`) to validate flag combinations (e.g., `--verified` requires `--blockscout`). * **New CLI Handlers (`test/cli/handlers/launch/`):** * `datahaven.ts`: Logic for spawning DataHaven node processes using the specified binary. Manages ports and process cleanup. * `relayer.ts`: Logic for configuring and spawning Snowbridge relayer processes (Beefy and Beacon). Reads contract deployment addresses, updates relayer config templates, and uses specified private keys. Manages log files and process cleanup. * `summary.ts`: Generates and displays the table of running services (including dynamically launched DataHaven nodes) and their endpoints. * `validator.ts`: Extracted validator funding, setup, and set update logic into its own handler. * `index.ts`: Orchestrates the launch sequence based on CLI options, calling the appropriate handlers. Includes a `LaunchedNetwork` class to track spawned processes, file descriptors, and node ports for cleanup. * **Updated `package.json` Scripts:** * Added `start:e2e:minrelayer` script for a minimal setup including relayers and DataHaven nodes. * Modified `stop:e2e` to include `pkill datahaven` for proper cleanup. * Added `stop:e2e:quick` to only stop the Kurtosis enclave without full cleaning. * **Updated `launch-kurtosis.ts`:** Modified to use new Kurtosis utility functions and added a `skipCleaning` option. * **New Utility Functions:** * `test/utils/kurtosis.ts`: Functions to inspect Kurtosis services (`getServiceFromKurtosis`, `getPortFromKurtosis`, `getServicesFromKurtosis`). * `test/utils/parser.ts`: Zod schemas and parsing functions for Snowbridge relayer configurations. * **Constants & Minor Updates:** * Added `SUBSTRATE_FUNDED_ACCOUNTS` to `test/utils/constants.ts`. * Updated `tsconfig.json` include paths. * Refactored `test/utils/docker.ts` (though now largely superseded by Kurtosis utils). * Updated logging in `test/scripts/send-txn.ts`. **Reasoning:** This PR significantly expands the E2E testing capabilities by allowing developers to easily launch and integrate local DataHaven nodes and Snowbridge relayers into the test network, facilitating more comprehensive integration testing. The CLI refactoring makes managing these complex setups more robust and user-friendly.
2025-04-29 12:24:00 +00:00
}
});
test: ⚡️ CI Refactor (#59) Eventually our CI will be required to run two private blockchains locally plus associated relayers. This PR is to prepare for this fate by improving run times and refactoring our existing CIs so they are a bit easier to reason about. ### Refactors - **_We now run ALL CIs on every PR!_** This is so that we decomplexify the logic around conditional builds and fetching built binaries from another source. This reduces the surface area of code we have to maintain at the cost of execution time - This penalty is ameliorated by a layered caching system. At best, it will be less than a minute to complete a build since everything will be cached. On GH runners this is about 6 minutes sadly. - We will no longer be at risk of important CIs being skipped erroneously which hide true failures. - Caching is a low-risk approach because at worst it has to build from scratch. A bad cache hit will never imply the wrong thing gets build since cargo is smart enough to just throw away any inappropriate build artefacts. - `setup-rust` action created so we have a unified way of setting up runner and unifying our approach to caching - Use a unique caching key for different activities and it will fallback to shared cache if no matches - we are using `mainnet` kurtosis config so that it works with relayer assumptions ### Additions - We can specify the ethereum block time via a new cli arg `--slot-time <seconds>` - We can specify arbitrary network_param args which get passed into the generated yaml - e.g. giving `bun cli --kurtosis-network-args="pet=cat food=fish" will add: ```yml network_params: # existing params... pet: cat food: fish ``` - We now have the ability to programmatically modify the yaml - This means we are back down to a single `minimal.yml` kurtosis config so we dont have to maintain changes between them - Flow is: `add new cli arg` -> `add if() block which mutates yaml` -> `profit` --------- Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> Co-authored-by: Facundo Farall <37149322+ffarall@users.noreply.github.com>
2025-05-06 20:20:02 +00:00
};