datahaven/test/scripts/set-datahaven-parameters.ts

103 lines
3 KiB
TypeScript
Raw Permalink Normal View History

refactor: 🐳 Improve docker caching (again) (#86) ## Changes - New CI file for making Docker Prod images - Changed E2E tests use an image built from a local dockerfile - Some cargo build options to make it quicker - Fix the cache hit rate - added `tsgo` preview to the project 😎 - Can be invoked with `bun tsgo` to typecheck - Install in IDE [VSCode](https://code.visualstudio.com/docs/configure/extensions/extension-marketplace) & [Zed](https://github.com/zed-extensions/tsgo) for super-fast inline typechecking (as you type basically) ## Context This PR attempts to make the frankly unacceptable CI times better. This achieves that aim by making a crappy image for day-to-day usage and let the prod issue take ages since that will be infrequently used. The reason why the original design didn't work for us is because: 1) we are using the free GH runners 2) when we goto baremetal runners we'll lose our rapid caching abilities which make using docker cheap. Also, we add `tsgo` support to improve devex. The improvement is astounding. ```sh hyperfine -n tsc "bun tsc --incremental false --extendedDiagnostics" -n tsgo "bun tsgo --incremental false --extendedDiagnostics" Benchmark 1: tsc Time (mean ± σ): 5.500 s ± 0.221 s [User: 8.939 s, System: 0.400 s] Range (min … max): 5.196 s … 5.845 s 10 runs Benchmark 2: tsgo Time (mean ± σ): 99.1 ms ± 8.4 ms [User: 392.8 ms, System: 54.1 ms] Range (min … max): 88.3 ms … 116.0 ms 29 runs Summary tsgo ran 55.48 ± 5.22 times faster than tsc ```
2025-05-27 16:14:15 +00:00
import { parseArgs } from "node:util";
import { datahaven } from "@polkadot-api/descriptors";
import { createClient } from "polkadot-api";
import { withPolkadotSdkCompat } from "polkadot-api/polkadot-sdk-compat";
test: refactor e2e tests (#365) This PR significantly refactors and improves the end-to-end testing framework and infrastructure. The primary focus was on simplifying the test suites, improving reliability through better resource management, and hardening the relayer infrastructure. All E2E tests are now passing on the CI and demonstrate consistent reliability when run locally. ### Key Changes #### 1. E2E Test Suite Refactor & Cleanup * **Simplified Test Logic**: Heavily refactored the core test suites (`native-token-transfer.test.ts`, `rewards-message.test.ts`, and `validator-set-update.test.ts`). The new implementation is much cleaner, utilizing shared helpers to reduce boilerplate. * **Utility Consolidation**: Removed redundant utility files (`storage.ts`, `rewards-helpers.ts`) and simplified `events.ts`. Event waiting now uses `rxjs` for Substrate and native `viem` watchers for Ethereum, which is more robust and easier to maintain. * **Better Connector Management**: Unified the creation and cleanup of test clients in `ConnectorFactory`. It now handles the lifecycle of WebSocket connections more gracefully, including clearing the `socketClientCache` to prevent reconnection noise during teardown. #### 2. Infrastructure & Stability * **Relayer Relaunch Policy**: Added a restart policy for Snowbridge relayer containers. They are now configured with `--restart on-failure:5`, ensuring that relayers automatically relaunch if they crash during the sensitive initialization phase. * **WebSocket Integration**: * Updated the `ConnectorFactory` to prefer **WebSockets** for the Ethereum public client, which is essential for efficient, event-heavy E2E testing. * Enhanced `launchKurtosisNetwork` to correctly identify and register the Execution Layer's WebSocket endpoint from Kurtosis. * **Disabled Contract Injection**: This PR temporarily disables the automatic injection of contracts into the genesis state by default. * *Reason*: I encountered issues generating a valid `state-diff.json` for the latest contract versions. Even after applying several workarounds, the injected state remained unstable. As a result, I've reverted to manual contract deployment during the launch sequence for better reliability for now. #### 3. Documentation & Maintenance * Removed obsolete documentation (`event-utilities-guide.md`) that no longer reflects the simplified event-handling API. * Cleaned up `test/launcher/validators.ts` and moved logic into more appropriate helpers. --------- Co-authored-by: Steve Degosserie <723552+stiiifff@users.noreply.github.com>
2025-12-24 12:31:40 +00:00
import { getWsProvider } from "polkadot-api/ws-provider/node";
feat: 🚀 Add `deploy` command to CLI (#87) ### New Features 1. Add the `deploy` command to our CLI. 1. Conditionally deploys kurtosis eth network if we're in `stagenet` environment. 2. Deploys DH nodes. 3. Deploys contracts (all of them). In `mainnet` and `testnet` it shouldn't deploy EL contracts, but for now that's not implemented. 4. Configures parameters, validators and relayers in the same way as `launch`. 5. Currently, it only deploys `beefy` and `beacon` relayers, `execution` and `solochain` relayers are pending for a subsequent PR. 2. Add `waitFor` utility function that receives a lambda. ### Refactors 1. Several common functionalities used both by the `launch` and `deploy` command have been moved to the `cli/handlers/common` directory, from where both commands use them. These include 1. Checks for installed dependencies. 2. Common constants. 3. The `LaunchedNetwork` class has been moved to this directory. 4. DataHaven nodes common functions. 5. Kurtosis common functions. 6. Relayer common functions. 7. Kubernetes functions (although only used by `deploy`, it seemed fitting to have it here still). 8. Remove CLI questions and separator prints from `deploy-contracts.ts` and `set-datahaven-parameters.ts` scripts. These things should be in the `cli/launch` folder, which consumes the functions in these scripts. 9. Remove `setParametersFromCollection` from `utils` folder and put it in `cli`. 10. Create base snowbridge relayer configs for `local` and `stagenet` as two separate directories. ### Fixes 1. Sets the default time of the `deploy` command to 6s as Lodestar is slower than Lighthouse. 2. In `runShellCommandWithLogger` only print `stderr` if the command fails. ### Additional Minor Changes 1. K8s secret key names changed from `dh-beefy-relay-eth-key` to `dh-beefy-relay-ethereum-key` and `dh-beacon-relay-sub-key` to `dh-beacon-relay-substrate-key`, for simplicity in the deployment script. 11. Update suggested configs for `.vscode` configs. --------- Co-authored-by: Steve Degosserie <723552+stiiifff@users.noreply.github.com>
2025-06-12 08:24:03 +00:00
import { getEvmEcdsaSigner, logger, SUBSTRATE_FUNDED_ACCOUNTS } from "utils";
import { parseJsonToParameters } from "utils/types";
/**
* Sets DataHaven runtime parameters on the specified RPC URL from a JSON file.
*/
export const setDataHavenParameters = async (
rpcUrl: string,
parametersFilePath: string
): Promise<boolean> => {
const parametersJson = await Bun.file(parametersFilePath).json();
const parameters = parseJsonToParameters(parametersJson).filter((p) => p.value !== undefined);
if (parameters.length === 0) {
logger.warn("⚠️ No parameters to set.");
return false;
}
const client = createClient(withPolkadotSdkCompat(getWsProvider(rpcUrl)));
try {
const dhApi = client.getTypedApi(datahaven);
const signer = getEvmEcdsaSigner(SUBSTRATE_FUNDED_ACCOUNTS.ALITH.privateKey);
// Log parameters being set
for (const p of parameters) {
logger.debug(`🔧 Setting ${p.name} = ${p.value!.asHex()}`);
}
// Build parameter calls
const calls = parameters.map(
(p) =>
dhApi.tx.Parameters.set_parameter({
key_value: {
type: "RuntimeConfig",
value: { type: p.name, value: [p.value] }
}
}).decodedCall
);
// Batch all calls and wrap in sudo
const tx = dhApi.tx.Sudo.sudo({
call: dhApi.tx.Utility.batch_all({ calls }).decodedCall
});
const result = await tx.signAndSubmit(signer);
feat: automated validator set submission with era targeting (#433) ## Era-targeted validator set submission with dedicated submitter role > **Note:** This PR includes a detailed specification at [`specs/validator-set-submission/validator-set-submission.md`](https://github.com/datahaven-xyz/datahaven/blob/feat/validator-set-submitter/specs/validator-set-submission/validator-set-submission.md) that covers the design rationale, submission lifecycle, era-targeting rules, and failure modes. Reading the spec first will make the contract, pallet, and daemon changes easier to follow. ### Summary - Introduce a dedicated `validatorSetSubmitter` role on `DataHavenServiceManager`, separating validator set submission authority from the contract owner - Replace the unscoped `sendNewValidatorSet` with `sendNewValidatorSetForEra`, which encodes a `targetEra` into the Snowbridge message payload - Add server-side era validation in the `external-validators` pallet to reject stale, duplicate, or out-of-range submissions - Add a long-running TypeScript daemon that watches session changes and automatically submits each era's validator set at the right time ### Contract changes (`contracts/`) - **New `validatorSetSubmitter` storage slot** — set during `initialize` and rotatable via `setValidatorSetSubmitter` (owner-only). The storage gap is decremented accordingly. - **`sendNewValidatorSet` → `sendNewValidatorSetForEra`** — accepts a `uint64 targetEra` parameter and is restricted to `onlyValidatorSetSubmitter` instead of `onlyOwner`. - **`buildNewValidatorSetMessageForEra`** — the `NewValidatorSetPayload.externalIndex` is now caller-supplied instead of hardcoded to `0`. - **New events** — `ValidatorSetSubmitterUpdated`, `ValidatorSetMessageSubmitted`. - **New error** — `OnlyValidatorSetSubmitter`. - **New test suite** — `ValidatorSetSubmitter.t.sol` covering submitter set/rotate, access control, era encoding, and legacy function removal. ### Pallet changes (`operator/`) - **`validate_target_era`** in `external-validators` — enforces `activeEra < targetEra <= activeEra + 1` and `targetEra > ExternalIndex` (dedup guard). - **New errors** — `TargetEraTooOld`, `TargetEraTooNew`, `DuplicateOrStaleTargetEra`. - **Tests** — five new test cases for era boundary conditions (next-era acceptance, old-era rejection, too-new rejection, duplicate rejection, genesis behavior). Existing `era_hooks_with_external_index` test updated to use valid target eras. - **Runtime test fixes** — `external_index: 0` → `1` in mainnet/stagenet/testnet EigenLayer message processor tests to satisfy the new validation. ### Validator set submitter daemon (`test/tools/validator-set-submitter/`) - Event-driven service that subscribes to finalized `Session.CurrentIndex` via Polkadot-API `watchValue`. - Submits once per era during the last session, targeting `ActiveEra + 1`. - Tracks submitted eras to avoid duplicates; skips if `ExternalIndex` already covers the target. - Startup self-checks: Ethereum connectivity, DataHaven connectivity, on-chain submitter authorization. - Supports `--dry-run` mode and YAML configuration. - Graceful shutdown on `SIGINT`/`SIGTERM`. ### Test & tooling updates - **E2E test** (`validator-set-update.test.ts`) — calls `sendNewValidatorSetForEra` with a computed `targetEra` and filters the substrate event by `external_index`. - **`update-validator-set.ts` script** — accepts `--target-era` flag; defaults to era 1 for fresh networks. - **CLI launch** — wires validator set update as an interactive step after relayer launch. - **`package.json`** — new `submitter` and `submitter:dry-run` scripts. - Regenerated contract bindings, PAPI metadata, state-diff, and storage layout snapshots. ### Test plan - [x] `forge test` — passes, including new `ValidatorSetSubmitter.t.sol` - [x] `cargo test` — passes, including new era-validation tests in `external-validators` - [x] `bun test:e2e` — validator-set-update suite passes with era-targeted flow - [x] Manual: run submitter daemon against local network (`bun submitter`), verify it submits once per era at the correct session ## ⚠️ Breaking Changes ⚠️ - **`sendNewValidatorSet` removed** — replaced by `sendNewValidatorSetForEra(uint64 targetEra, ...)`. Callers must now supply a `targetEra` parameter. - **Access control changed** — validator set submission is now restricted to the `validatorSetSubmitter` role instead of the contract `owner`. The submitter address is set during `initialize` and rotatable via `setValidatorSetSubmitter` (owner-only). - **`external-validators` pallet now validates `targetEra`** — messages with a stale, duplicate, or out-of-range `external_index` are rejected on-chain. Existing integrations sending `external_index: 0` will fail validation. --------- Co-authored-by: Cursor <cursoragent@cursor.com>
2026-02-20 09:31:44 +00:00
// sudo always returns Ok at the extrinsic level — check the Sudid event
// for the inner call result
const sudidEvent = result.events.find(
(e: any) => e.type === "Sudo" && e.value?.type === "Sudid"
);
if (!sudidEvent) {
logger.error("❌ Sudo.Sudid event not found in transaction events");
return false;
}
const sudoResult = (sudidEvent.value as any).value.sudo_result;
if (sudoResult.type === "Err") {
logger.error(`❌ Sudo inner call failed: ${JSON.stringify(sudoResult)}`);
return false;
}
logger.success("Runtime parameters set successfully");
return true;
} catch (error) {
logger.error(`${error instanceof Error ? error.message : error}`);
return false;
} finally {
client.destroy();
}
};
// CLI entry point
if (import.meta.main) {
const { values } = parseArgs({
args: process.argv,
options: {
rpcUrl: { type: "string", short: "r" },
parametersFile: { type: "string", short: "f" }
},
strict: true
});
if (!values.rpcUrl || !values.parametersFile) {
console.error("Usage: --rpc-url <url> --parameters-file <path>");
process.exit(1);
}
setDataHavenParameters(values.rpcUrl, values.parametersFile)
.then((ok) => process.exit(ok ? 0 : 1))
.catch((e) => {
console.error(e);
process.exit(1);
});
}