datahaven/test/scripts/set-datahaven-parameters.ts

149 lines
4.7 KiB
TypeScript
Raw Permalink Normal View History

refactor: 🐳 Improve docker caching (again) (#86) ## Changes - New CI file for making Docker Prod images - Changed E2E tests use an image built from a local dockerfile - Some cargo build options to make it quicker - Fix the cache hit rate - added `tsgo` preview to the project 😎 - Can be invoked with `bun tsgo` to typecheck - Install in IDE [VSCode](https://code.visualstudio.com/docs/configure/extensions/extension-marketplace) & [Zed](https://github.com/zed-extensions/tsgo) for super-fast inline typechecking (as you type basically) ## Context This PR attempts to make the frankly unacceptable CI times better. This achieves that aim by making a crappy image for day-to-day usage and let the prod issue take ages since that will be infrequently used. The reason why the original design didn't work for us is because: 1) we are using the free GH runners 2) when we goto baremetal runners we'll lose our rapid caching abilities which make using docker cheap. Also, we add `tsgo` support to improve devex. The improvement is astounding. ```sh hyperfine -n tsc "bun tsc --incremental false --extendedDiagnostics" -n tsgo "bun tsgo --incremental false --extendedDiagnostics" Benchmark 1: tsc Time (mean ± σ): 5.500 s ± 0.221 s [User: 8.939 s, System: 0.400 s] Range (min … max): 5.196 s … 5.845 s 10 runs Benchmark 2: tsgo Time (mean ± σ): 99.1 ms ± 8.4 ms [User: 392.8 ms, System: 54.1 ms] Range (min … max): 88.3 ms … 116.0 ms 29 runs Summary tsgo ran 55.48 ± 5.22 times faster than tsc ```
2025-05-27 16:14:15 +00:00
import { parseArgs } from "node:util";
import { datahaven } from "@polkadot-api/descriptors";
import { createClient } from "polkadot-api";
import { withPolkadotSdkCompat } from "polkadot-api/polkadot-sdk-compat";
import { getWsProvider } from "polkadot-api/ws-provider/web";
feat: 🚀 Add `deploy` command to CLI (#87) ### New Features 1. Add the `deploy` command to our CLI. 1. Conditionally deploys kurtosis eth network if we're in `stagenet` environment. 2. Deploys DH nodes. 3. Deploys contracts (all of them). In `mainnet` and `testnet` it shouldn't deploy EL contracts, but for now that's not implemented. 4. Configures parameters, validators and relayers in the same way as `launch`. 5. Currently, it only deploys `beefy` and `beacon` relayers, `execution` and `solochain` relayers are pending for a subsequent PR. 2. Add `waitFor` utility function that receives a lambda. ### Refactors 1. Several common functionalities used both by the `launch` and `deploy` command have been moved to the `cli/handlers/common` directory, from where both commands use them. These include 1. Checks for installed dependencies. 2. Common constants. 3. The `LaunchedNetwork` class has been moved to this directory. 4. DataHaven nodes common functions. 5. Kurtosis common functions. 6. Relayer common functions. 7. Kubernetes functions (although only used by `deploy`, it seemed fitting to have it here still). 8. Remove CLI questions and separator prints from `deploy-contracts.ts` and `set-datahaven-parameters.ts` scripts. These things should be in the `cli/launch` folder, which consumes the functions in these scripts. 9. Remove `setParametersFromCollection` from `utils` folder and put it in `cli`. 10. Create base snowbridge relayer configs for `local` and `stagenet` as two separate directories. ### Fixes 1. Sets the default time of the `deploy` command to 6s as Lodestar is slower than Lighthouse. 2. In `runShellCommandWithLogger` only print `stderr` if the command fails. ### Additional Minor Changes 1. K8s secret key names changed from `dh-beefy-relay-eth-key` to `dh-beefy-relay-ethereum-key` and `dh-beacon-relay-sub-key` to `dh-beacon-relay-substrate-key`, for simplicity in the deployment script. 11. Update suggested configs for `.vscode` configs. --------- Co-authored-by: Steve Degosserie <723552+stiiifff@users.noreply.github.com>
2025-06-12 08:24:03 +00:00
import { getEvmEcdsaSigner, logger, SUBSTRATE_FUNDED_ACCOUNTS } from "utils";
import { type ParsedDataHavenParameter, parseJsonToParameters } from "utils/types";
// Interface for the options object of setDataHavenParameters
interface SetDataHavenParametersOptions {
rpcUrl: string;
parametersFilePath: string;
}
/**
* Sets DataHaven runtime parameters on the specified RPC URL from a JSON file.
*
* @param options - Configuration options for setting parameters
* @param options.rpcUrl - The RPC URL of the DataHaven node
* @param options.parametersFilePath - Path to the JSON file containing an array of parameters to set
* @returns Promise resolving to true if parameters were set successfully, false if skipped
*/
export const setDataHavenParameters = async (
options: SetDataHavenParametersOptions
): Promise<boolean> => {
feat: 🚀 Add `deploy` command to CLI (#87) ### New Features 1. Add the `deploy` command to our CLI. 1. Conditionally deploys kurtosis eth network if we're in `stagenet` environment. 2. Deploys DH nodes. 3. Deploys contracts (all of them). In `mainnet` and `testnet` it shouldn't deploy EL contracts, but for now that's not implemented. 4. Configures parameters, validators and relayers in the same way as `launch`. 5. Currently, it only deploys `beefy` and `beacon` relayers, `execution` and `solochain` relayers are pending for a subsequent PR. 2. Add `waitFor` utility function that receives a lambda. ### Refactors 1. Several common functionalities used both by the `launch` and `deploy` command have been moved to the `cli/handlers/common` directory, from where both commands use them. These include 1. Checks for installed dependencies. 2. Common constants. 3. The `LaunchedNetwork` class has been moved to this directory. 4. DataHaven nodes common functions. 5. Kurtosis common functions. 6. Relayer common functions. 7. Kubernetes functions (although only used by `deploy`, it seemed fitting to have it here still). 8. Remove CLI questions and separator prints from `deploy-contracts.ts` and `set-datahaven-parameters.ts` scripts. These things should be in the `cli/launch` folder, which consumes the functions in these scripts. 9. Remove `setParametersFromCollection` from `utils` folder and put it in `cli`. 10. Create base snowbridge relayer configs for `local` and `stagenet` as two separate directories. ### Fixes 1. Sets the default time of the `deploy` command to 6s as Lodestar is slower than Lighthouse. 2. In `runShellCommandWithLogger` only print `stderr` if the command fails. ### Additional Minor Changes 1. K8s secret key names changed from `dh-beefy-relay-eth-key` to `dh-beefy-relay-ethereum-key` and `dh-beacon-relay-sub-key` to `dh-beacon-relay-substrate-key`, for simplicity in the deployment script. 11. Update suggested configs for `.vscode` configs. --------- Co-authored-by: Steve Degosserie <723552+stiiifff@users.noreply.github.com>
2025-06-12 08:24:03 +00:00
const { rpcUrl, parametersFilePath } = options;
// Load parameters from the JSON file
let parameters: ParsedDataHavenParameter[];
try {
const parametersFile = Bun.file(parametersFilePath);
const parametersJson = await parametersFile.text();
// Parse and convert the parameters using our utility
parameters = parseJsonToParameters(JSON.parse(parametersJson));
if (parameters.length === 0) {
logger.warn("⚠️ The parameters file is empty. No parameters to set.");
return false;
}
} catch (error: any) {
logger.error(
`❌ Error reading or parsing parameters file at '${parametersFilePath}': ${error.message}`
);
throw error;
}
const client = createClient(withPolkadotSdkCompat(getWsProvider(rpcUrl)));
const dhApi = client.getTypedApi(datahaven);
logger.trace("Substrate client created");
const signer = getEvmEcdsaSigner(SUBSTRATE_FUNDED_ACCOUNTS.ALITH.privateKey);
logger.trace("Signer created for SUDO (ALITH)");
let allSuccessful = true;
try {
for (const param of parameters) {
// TODO: Add a graceful way to print the value of the parameter, since it won't always be representable as a hex string
feat: 🚀 Add `deploy` command to CLI (#87) ### New Features 1. Add the `deploy` command to our CLI. 1. Conditionally deploys kurtosis eth network if we're in `stagenet` environment. 2. Deploys DH nodes. 3. Deploys contracts (all of them). In `mainnet` and `testnet` it shouldn't deploy EL contracts, but for now that's not implemented. 4. Configures parameters, validators and relayers in the same way as `launch`. 5. Currently, it only deploys `beefy` and `beacon` relayers, `execution` and `solochain` relayers are pending for a subsequent PR. 2. Add `waitFor` utility function that receives a lambda. ### Refactors 1. Several common functionalities used both by the `launch` and `deploy` command have been moved to the `cli/handlers/common` directory, from where both commands use them. These include 1. Checks for installed dependencies. 2. Common constants. 3. The `LaunchedNetwork` class has been moved to this directory. 4. DataHaven nodes common functions. 5. Kurtosis common functions. 6. Relayer common functions. 7. Kubernetes functions (although only used by `deploy`, it seemed fitting to have it here still). 8. Remove CLI questions and separator prints from `deploy-contracts.ts` and `set-datahaven-parameters.ts` scripts. These things should be in the `cli/launch` folder, which consumes the functions in these scripts. 9. Remove `setParametersFromCollection` from `utils` folder and put it in `cli`. 10. Create base snowbridge relayer configs for `local` and `stagenet` as two separate directories. ### Fixes 1. Sets the default time of the `deploy` command to 6s as Lodestar is slower than Lighthouse. 2. In `runShellCommandWithLogger` only print `stderr` if the command fails. ### Additional Minor Changes 1. K8s secret key names changed from `dh-beefy-relay-eth-key` to `dh-beefy-relay-ethereum-key` and `dh-beacon-relay-sub-key` to `dh-beacon-relay-substrate-key`, for simplicity in the deployment script. 11. Update suggested configs for `.vscode` configs. --------- Co-authored-by: Steve Degosserie <723552+stiiifff@users.noreply.github.com>
2025-06-12 08:24:03 +00:00
logger.info(
`🔧 Attempting to set parameter: ${param.name.toString()} = ${param.value.asHex()}`
);
const setParameterArgs: any = {
key_value: {
type: "RuntimeConfig" as const,
value: {
type: param.name,
value: [param.value]
}
}
};
try {
const setParameterCall = dhApi.tx.Parameters.set_parameter(setParameterArgs);
logger.debug("Parameter set call:");
logger.debug(setParameterCall.decodedCall);
const sudoCall = dhApi.tx.Sudo.sudo({
call: setParameterCall.decodedCall
});
logger.debug(`Submitting transaction to set ${String(param.name)}...`);
const txFinalisedPayload = await sudoCall.signAndSubmit(signer);
if (!txFinalisedPayload.ok) {
logger.error(
`❌ Transaction to set parameter ${String(param.name)} failed. Block: ${txFinalisedPayload.block.hash}, Tx Hash: ${txFinalisedPayload.txHash}`
);
logger.error(`Events: ${JSON.stringify(txFinalisedPayload.events)}`);
allSuccessful = false;
}
} catch (txError: any) {
logger.error(
`❌ Error submitting transaction for parameter ${String(param.name)}: ${txError.message || txError}`
);
allSuccessful = false;
}
}
} finally {
client.destroy();
logger.trace("Substrate client destroyed");
}
if (allSuccessful) {
logger.success("All specified DataHaven parameters processed successfully.");
} else {
logger.warn("Some DataHaven parameters could not be set. Please check logs.");
}
return allSuccessful;
};
// Allow script to be run directly with CLI arguments
if (import.meta.main) {
const { values } = parseArgs({
args: process.argv,
options: {
rpcUrl: {
type: "string",
short: "r"
},
parametersFile: {
type: "string",
short: "f"
}
},
strict: true
});
if (!values.rpcUrl) {
console.error("Error: --rpc-url parameter is required");
process.exit(1);
}
if (!values.parametersFile) {
console.error("Error: --parameters-file <path_to_json_file> parameter is required.");
process.exit(1);
}
setDataHavenParameters({
rpcUrl: values.rpcUrl,
feat: 🚀 Add `deploy` command to CLI (#87) ### New Features 1. Add the `deploy` command to our CLI. 1. Conditionally deploys kurtosis eth network if we're in `stagenet` environment. 2. Deploys DH nodes. 3. Deploys contracts (all of them). In `mainnet` and `testnet` it shouldn't deploy EL contracts, but for now that's not implemented. 4. Configures parameters, validators and relayers in the same way as `launch`. 5. Currently, it only deploys `beefy` and `beacon` relayers, `execution` and `solochain` relayers are pending for a subsequent PR. 2. Add `waitFor` utility function that receives a lambda. ### Refactors 1. Several common functionalities used both by the `launch` and `deploy` command have been moved to the `cli/handlers/common` directory, from where both commands use them. These include 1. Checks for installed dependencies. 2. Common constants. 3. The `LaunchedNetwork` class has been moved to this directory. 4. DataHaven nodes common functions. 5. Kurtosis common functions. 6. Relayer common functions. 7. Kubernetes functions (although only used by `deploy`, it seemed fitting to have it here still). 8. Remove CLI questions and separator prints from `deploy-contracts.ts` and `set-datahaven-parameters.ts` scripts. These things should be in the `cli/launch` folder, which consumes the functions in these scripts. 9. Remove `setParametersFromCollection` from `utils` folder and put it in `cli`. 10. Create base snowbridge relayer configs for `local` and `stagenet` as two separate directories. ### Fixes 1. Sets the default time of the `deploy` command to 6s as Lodestar is slower than Lighthouse. 2. In `runShellCommandWithLogger` only print `stderr` if the command fails. ### Additional Minor Changes 1. K8s secret key names changed from `dh-beefy-relay-eth-key` to `dh-beefy-relay-ethereum-key` and `dh-beacon-relay-sub-key` to `dh-beacon-relay-substrate-key`, for simplicity in the deployment script. 11. Update suggested configs for `.vscode` configs. --------- Co-authored-by: Steve Degosserie <723552+stiiifff@users.noreply.github.com>
2025-06-12 08:24:03 +00:00
parametersFilePath: values.parametersFile
}).catch((error: Error) => {
console.error("Setting DataHaven parameters failed:", error.message || error);
process.exit(1);
});
}