Upgrade to StorageHub version 0.2.10. This is a minor release, including
breaking changes.
## ⚠️ Breaking Changes ⚠️
The changes applied in this PR are according to the suggested changes in
StorageHub's [v0.2.10
release](https://github.com/Moonsong-Labs/storage-hub/releases/tag/v0.2.10)
## Summary
- Feature-gate `frame-benchmarking-cli` behind `runtime-benchmarks`
feature, making it an optional dependency
- Remove unused `cumulus-client-service` workspace dependency
- Remove unused `storage-hub-runtime` workspace dependency
- Add `#[cfg(feature = "runtime-benchmarks")]` guards to
benchmark-related code
## Motivation
The `frame-benchmarking-cli` crate pulls in
`cumulus-client-parachain-inherent` and other cumulus dependencies
transitively. Since DataHaven is a solochain (not a parachain), these
dependencies are unnecessary for regular builds.
By making the benchmarking CLI optional and only compiling it when the
`runtime-benchmarks` feature is enabled, we reduce:
- Compile time for regular development builds
- Final binary size (when not benchmarking)
- Dependency tree complexity
Co-authored-by: Claude <noreply@anthropic.com>
## ⚠️ Breaking Changes ⚠️
Upgrades to SH version 0.2.0. Breaking changes for this version are
outlined in the corresponding
[release](https://github.com/Moonsong-Labs/storage-hub/releases/tag/v0.2.0).
Particularly, in this PR, the following breaking changes are implemented
for DH node operators:
### Breaking CLI changes vs `main`
- **Fisherman vs provider role**
- `--fisherman` now has `conflicts_with = "provider"`
(`FishermanConfigurations::fisherman`).
- Any existing scripts that started a node with both `--provider` and
`--fisherman` will now fail clap validation.
- **Removed / replaced fisherman tuning flags**
- The following flags no longer exist and will cause errors if still
used:
- `--fisherman-incomplete-sync-max` (field
`fisherman_incomplete_sync_max`)
- `--fisherman-incomplete-sync-page-size` (field
`fisherman_incomplete_sync_page_size`)
- `--fisherman-sync-mode-min-blocks-behind` (field
`fisherman_sync_mode_min_blocks_behind`)
- They are replaced by:
- `--fisherman-batch-interval-seconds`
(`fisherman_batch_interval_seconds`, default `60`)
- `--fisherman-batch-deletion-limit` (`fisherman_batch_deletion_limit`,
default `1000`)
- **MSP DB wiring no longer piggybacks on the indexer DB**
- Previously, enabling the indexer (`IndexerConfigurations`) also wired
its DB pool into the MSP move‑bucket path via
`with_indexer_db_pool(maybe_indexer_db_pool)`.
- Now, MSP DB access is **only** configured if you pass the new
`--msp-database-url` provider flag; the indexer’s `--indexer` /
`--indexer-database-url` no longer implicitly provide DB access to MSP
logic. This will change behaviour for MSP nodes that relied on just the
indexer flags.
### New / additive CLI flags (non‑breaking but behaviourally relevant)
- **Provider flags**
- `--pending-db-url` (`pending_db_url`, env `SH_PENDING_DB_URL`) for
persisting pending extrinsics.
- `--internal-buffer-size` (`internal_buffer_size`, default `1024`) for
DB chunk batching during file transfer.
- **Reordered but unchanged**
- `--msp-distribute-files` still exists (bool flag), just moved within
`ProviderConfigurations`; name and type are unchanged, but now also
explicitly toggles `enable_msp_distribute_files` only when
`provider_type == msp`.
Update the 3 DataHaven environments' chain IDs & native token ticker as
follows:
* **Mainnet**
* **Chain ID**: 55930
* **Ticker**: HAVE
* **TestNet**
* **Chain ID**: 55931
* **Ticker**: MOCK
* **Stagenet**
* **Chain ID**: 55932
* **Ticker**: STAGE
The PR includes a storage migration for the Stagenet & Testnet
environments, that are already live, to update the EVM Chain ID stored
in the `pallet-evm-chain-id` pallet.
Note: the token symbol will only be updated with the genesis config
presets or newly generated chain specs. For already live networks, the
existing chain spec must be updated (i.e. the tokenSymbol property
changed) and used by all nodes in the network. This change in the chain
spec will not alter the chain genesis so it safe to do (in the very
early stages of the chain obviously).
---------
Co-authored-by: Claude <noreply@anthropic.com>
Upgrade to SH release
[v0.1.1](https://github.com/Moonsong-Labs/storage-hub/releases/tag/v0.1.1)
---------
Co-authored-by: Ahmad Kaouk <56095276+ahmadkaouk@users.noreply.github.com>
Co-authored-by: Ahmad Kaouk <ahmadkaouk.93@gmail.com>
Co-authored-by: Steve Degosserie <723552+stiiifff@users.noreply.github.com>
### Description
This PR introduces the **Moonwall** end-to-end (E2E) testing framework.
The primary motivation for this is to enable the porting of existing
Mobeam tests into the `DataHaven` repository.
### Key Changes
* **Node Manual Sealing:**
* Introduced a `--sealing=manual` flag for the `datahaven-node`. When
enabled, blocks are only produced on demand via an RPC call. This is the
core mechanism that allows for deterministic tests.
* **Moonwall Framework Integration:**
* Added `@moonwall/cli` and `@moonwall/util` dependencies to the
`test/package.json`.
* A new `test/moonwall.config.json` file configures the test
environment, defining how Moonwall should launch the `datahaven-node`
with the manual sealing flag.
* Added a `moonwall:test` script to `package.json` for running the
tests.
* **CI Workflow:**
* A new reusable workflow, `.github/workflows/task-moonwall-tests.yml`,
has been created to handle the setup, execution, and reporting of
Moonwall tests.
* The main `CI.yml` now includes a `moonwall-tests` job that runs after
the `build-operator` job, ensuring it always tests the correct,
freshly-built binary.
* **Example Test Suite:**
* A new test suite, `test/datahaven/suites/dev/test-block.ts`, had been
copied from moonbeam.
### How to Run Locally
1. Navigate to the `test` directory.
2. Install dependencies: `bun install`
3. Run the tests: `bun run moonwall:test`
---------
Co-authored-by: undercover-cactus <lola@moonsonglabs.com>
In this PR we update StorageHub to its latest tage `v0.0.4-alpha`. It
includes so minor changes on the FilesystemAPI.
Its also force to specify the `maintenance_mode` options for fisherman.
For now we are not allowing this mode so we just skip entirely.
---------
Co-authored-by: Steve Degosserie <723552+stiiifff@users.noreply.github.com>
Added missing CLI option for the indexer configuration for Storage Hub.
Indexer is needed for MSP user.
---------
Co-authored-by: Steve Degosserie <723552+stiiifff@users.noreply.github.com>
## Summary
- Upgraded Rust toolchain from 1.87 to 1.88 for latest compiler
improvements
- Updated Docker build environment with latest paritytech base image and
build tools
- Fixed node branding and support URL to reflect DataHaven project
## Changes
### Rust Toolchain Update (`rust-toolchain.toml`)
- Upgraded from Rust 1.87 to 1.88
### Docker Build Environment (`Dockerfile`)
- Updated base image from `paritytech/ci-unified:bullseye-1.85.0` to
`bullseye-1.88.0`
- Upgraded mold linker from v2.39.0 to v2.40.4 for faster builds
- Added protoc v21.12 for protobuf compilation support
- Added libpq-dev dependency for PostgreSQL integration
- Updated cargo-chef from 0.1.71 to 0.1.72
---------
Co-authored-by: Claude <noreply@anthropic.com>
This PR updates the project dependencies as follows:
- **Polkadot SDK**: from the `stable2412` (moving) branch to the
`polkadot-stable2412-6` (fixed) tag.
- **StorageHub**: to revision
`f8281283b6003a3009a32431ed0f3cd628561d6b`, which also depends on
Polkadot SDK `polkadot-stable2412-6`.
- **Frontier**: revision `75329a2df49e2cc7981485392c31160929d1bd48n`
which, likewise, depends on Polkadot SDK `polkadot-stable2412-6`.
In this PR we add StorageHub pallets to the runtime. It will only be
added if the `storage-hub` feature is activated.
---------
Co-authored-by: Gonza Montiel <gonzamontiel@users.noreply.github.com>
In this PR:
1. Add [Polkadot API](https://papi.how/) support to `test/` directory
(i.e. the e2e CLI and testing framework) to be able to interact with the
Substrate chain.
1. This allows typed interactions with transactions, constants, storage,
runtime APIs, and non-typed interaction with RPC methods.
2. Types are autogenerated when running `bun i`, from the
`datahaven.scale` file that is part of this repo's version control.
2. Add new utilities file to `papi` related functionalities. For the
time being, generating a new signer from a private key.
3. ~Add a new step to the CLI that sends a transaction to the DataHaven
network. _*THIS SHOULD BE REMOVED SOON, IT'S JUST FOR TESTING
PURPOSES*_~
1. Both steps that send test transactions have been removed from the
CLI, for convenience and being error prone. Their scripts remain usable
for testing purposes if needed.
4. Removes the `apis.rs` files from the runtime definitions. Having them
in a separate file meant that the runtime was not including the Runtime
APIs in the metadata blob, preventing `papi` from creating types from
it. This change can be reapplied after upgrading to `polkadot-sdk`
`stable-2503`. More info
[here](https://github.com/paritytech/polkadot-sdk/issues/6659).
5. Add script to re-generate types, and corresponding docs.
6. Makes logger synchronous to avoid prints happening before logs.
Add support for multiple runtimes: `stagenet`, `testnet` & `mainnet`.
- Moved common types to `datahaven-runtime-common` crate.
- Made the node's command & service code generic over different
runtimes. Each runtime has a `dev` & `local` genesis config preset.
- More types / constants can be moved to the `datahaven-runtime-common`
crate ... this will be done in subsequent PRs.
---------
Co-authored-by: Gonza Montiel <gon.montiel@gmail.com>
Co-authored-by: Gonza Montiel <gonzamontiel@users.noreply.github.com>
Co-authored-by: Facundo Farall <37149322+ffarall@users.noreply.github.com>
This PR adds Ethereum RPC API support to the node, enabling the
interaction with the node using standard Ethereum tools.
1. Integration of Frontier's Ethereum RPC modules (eth, net, web3,
txpool)
2. Added RPC configurations necessary for Ethereum compatibility
3. Implemented a BABE consensus data provider for handling pending
blocks
4. Added required dependencies and configurations in the node's service
5. Created necessary filter pools, block data caches, and notification
systems
6. Configured the RPC system to handle Ethereum API calls
7. Changed the transaction pool implementation to be compatible with the
fork-aware transaction pool
(https://github.com/paritytech/polkadot-sdk/pull/4639)
EDIT:
The new `transaction_pool` in `polkadot-stable2412` that comes with a
`fork-aware` feature, allowing different views of the transaction pool,
is not compatible with the current state in Frontier.
In Frontier the only supported pool is the `BasicPool`, that maintains a
similar behaviour to the one in `polkadot-stable2409`. I used this pool
directly.
---------
Co-authored-by: Ahmad Kaouk <56095276+ahmadkaouk@users.noreply.github.com>
Co-authored-by: Facundo Farall <37149322+ffarall@users.noreply.github.com>
Based on https://github.com/paritytech/polkadot-sdk/pull/5868.
Note: this (somehow) breaks the ability to run a local network with the
zombienet config. It seems that the config generated by Zombienet that
automatically set the balance of validators is leading to an issue
(because of the address type).
---------
Co-authored-by: Facundo Farall <37149322+ffarall@users.noreply.github.com>
I noticed `cargo clippy` was failing in CI, I fixed it. I still think we
should tweak the lints.
For example, clippy complains at things like `1 * HOURS ` because it's
redundant to multiply by 1, buy maybe we want it explicitly that way, so
it's more evident it can be changed. In this PR I don't tweak the rule
but try to make clippy happy.
This PR remove the unneeded action that install rustfmt as it is now
already available. It remove a step in the CI.
The action removed is `dtolnay/rust-toolchain@stable`
---------
Co-authored-by: Facundo Farall <37149322+ffarall@users.noreply.github.com>
Adds the `Substrate` node and runtime, as well as configuration and test
files, from https://github.com/Moonsong-Labs/flamingo to the `operator`
folder in DataHaven
