Upgrade to StorageHub version 0.2.10. This is a minor release, including
breaking changes.
## ⚠️ Breaking Changes ⚠️
The changes applied in this PR are according to the suggested changes in
StorageHub's [v0.2.10
release](https://github.com/Moonsong-Labs/storage-hub/releases/tag/v0.2.10)
## ⚠️ Breaking Changes ⚠️
Upgrades to SH version 0.2.0. Breaking changes for this version are
outlined in the corresponding
[release](https://github.com/Moonsong-Labs/storage-hub/releases/tag/v0.2.0).
Particularly, in this PR, the following breaking changes are implemented
for DH node operators:
### Breaking CLI changes vs `main`
- **Fisherman vs provider role**
- `--fisherman` now has `conflicts_with = "provider"`
(`FishermanConfigurations::fisherman`).
- Any existing scripts that started a node with both `--provider` and
`--fisherman` will now fail clap validation.
- **Removed / replaced fisherman tuning flags**
- The following flags no longer exist and will cause errors if still
used:
- `--fisherman-incomplete-sync-max` (field
`fisherman_incomplete_sync_max`)
- `--fisherman-incomplete-sync-page-size` (field
`fisherman_incomplete_sync_page_size`)
- `--fisherman-sync-mode-min-blocks-behind` (field
`fisherman_sync_mode_min_blocks_behind`)
- They are replaced by:
- `--fisherman-batch-interval-seconds`
(`fisherman_batch_interval_seconds`, default `60`)
- `--fisherman-batch-deletion-limit` (`fisherman_batch_deletion_limit`,
default `1000`)
- **MSP DB wiring no longer piggybacks on the indexer DB**
- Previously, enabling the indexer (`IndexerConfigurations`) also wired
its DB pool into the MSP move‑bucket path via
`with_indexer_db_pool(maybe_indexer_db_pool)`.
- Now, MSP DB access is **only** configured if you pass the new
`--msp-database-url` provider flag; the indexer’s `--indexer` /
`--indexer-database-url` no longer implicitly provide DB access to MSP
logic. This will change behaviour for MSP nodes that relied on just the
indexer flags.
### New / additive CLI flags (non‑breaking but behaviourally relevant)
- **Provider flags**
- `--pending-db-url` (`pending_db_url`, env `SH_PENDING_DB_URL`) for
persisting pending extrinsics.
- `--internal-buffer-size` (`internal_buffer_size`, default `1024`) for
DB chunk batching during file transfer.
- **Reordered but unchanged**
- `--msp-distribute-files` still exists (bool flag), just moved within
`ProviderConfigurations`; name and type are unchanged, but now also
explicitly toggles `enable_msp_distribute_files` only when
`provider_type == msp`.
Upgrade to SH release
[v0.1.1](https://github.com/Moonsong-Labs/storage-hub/releases/tag/v0.1.1)
---------
Co-authored-by: Ahmad Kaouk <56095276+ahmadkaouk@users.noreply.github.com>
Co-authored-by: Ahmad Kaouk <ahmadkaouk.93@gmail.com>
Co-authored-by: Steve Degosserie <723552+stiiifff@users.noreply.github.com>
### Description
This PR introduces the **Moonwall** end-to-end (E2E) testing framework.
The primary motivation for this is to enable the porting of existing
Mobeam tests into the `DataHaven` repository.
### Key Changes
* **Node Manual Sealing:**
* Introduced a `--sealing=manual` flag for the `datahaven-node`. When
enabled, blocks are only produced on demand via an RPC call. This is the
core mechanism that allows for deterministic tests.
* **Moonwall Framework Integration:**
* Added `@moonwall/cli` and `@moonwall/util` dependencies to the
`test/package.json`.
* A new `test/moonwall.config.json` file configures the test
environment, defining how Moonwall should launch the `datahaven-node`
with the manual sealing flag.
* Added a `moonwall:test` script to `package.json` for running the
tests.
* **CI Workflow:**
* A new reusable workflow, `.github/workflows/task-moonwall-tests.yml`,
has been created to handle the setup, execution, and reporting of
Moonwall tests.
* The main `CI.yml` now includes a `moonwall-tests` job that runs after
the `build-operator` job, ensuring it always tests the correct,
freshly-built binary.
* **Example Test Suite:**
* A new test suite, `test/datahaven/suites/dev/test-block.ts`, had been
copied from moonbeam.
### How to Run Locally
1. Navigate to the `test` directory.
2. Install dependencies: `bun install`
3. Run the tests: `bun run moonwall:test`
---------
Co-authored-by: undercover-cactus <lola@moonsonglabs.com>
Added missing CLI option for the indexer configuration for Storage Hub.
Indexer is needed for MSP user.
---------
Co-authored-by: Steve Degosserie <723552+stiiifff@users.noreply.github.com>
In this PR:
1. Add [Polkadot API](https://papi.how/) support to `test/` directory
(i.e. the e2e CLI and testing framework) to be able to interact with the
Substrate chain.
1. This allows typed interactions with transactions, constants, storage,
runtime APIs, and non-typed interaction with RPC methods.
2. Types are autogenerated when running `bun i`, from the
`datahaven.scale` file that is part of this repo's version control.
2. Add new utilities file to `papi` related functionalities. For the
time being, generating a new signer from a private key.
3. ~Add a new step to the CLI that sends a transaction to the DataHaven
network. _*THIS SHOULD BE REMOVED SOON, IT'S JUST FOR TESTING
PURPOSES*_~
1. Both steps that send test transactions have been removed from the
CLI, for convenience and being error prone. Their scripts remain usable
for testing purposes if needed.
4. Removes the `apis.rs` files from the runtime definitions. Having them
in a separate file meant that the runtime was not including the Runtime
APIs in the metadata blob, preventing `papi` from creating types from
it. This change can be reapplied after upgrading to `polkadot-sdk`
`stable-2503`. More info
[here](https://github.com/paritytech/polkadot-sdk/issues/6659).
5. Add script to re-generate types, and corresponding docs.
6. Makes logger synchronous to avoid prints happening before logs.
Add support for multiple runtimes: `stagenet`, `testnet` & `mainnet`.
- Moved common types to `datahaven-runtime-common` crate.
- Made the node's command & service code generic over different
runtimes. Each runtime has a `dev` & `local` genesis config preset.
- More types / constants can be moved to the `datahaven-runtime-common`
crate ... this will be done in subsequent PRs.
---------
Co-authored-by: Gonza Montiel <gon.montiel@gmail.com>
Co-authored-by: Gonza Montiel <gonzamontiel@users.noreply.github.com>
Co-authored-by: Facundo Farall <37149322+ffarall@users.noreply.github.com>
This PR adds Ethereum RPC API support to the node, enabling the
interaction with the node using standard Ethereum tools.
1. Integration of Frontier's Ethereum RPC modules (eth, net, web3,
txpool)
2. Added RPC configurations necessary for Ethereum compatibility
3. Implemented a BABE consensus data provider for handling pending
blocks
4. Added required dependencies and configurations in the node's service
5. Created necessary filter pools, block data caches, and notification
systems
6. Configured the RPC system to handle Ethereum API calls
7. Changed the transaction pool implementation to be compatible with the
fork-aware transaction pool
(https://github.com/paritytech/polkadot-sdk/pull/4639)
EDIT:
The new `transaction_pool` in `polkadot-stable2412` that comes with a
`fork-aware` feature, allowing different views of the transaction pool,
is not compatible with the current state in Frontier.
In Frontier the only supported pool is the `BasicPool`, that maintains a
similar behaviour to the one in `polkadot-stable2409`. I used this pool
directly.
---------
Co-authored-by: Ahmad Kaouk <56095276+ahmadkaouk@users.noreply.github.com>
Co-authored-by: Facundo Farall <37149322+ffarall@users.noreply.github.com>
I noticed `cargo clippy` was failing in CI, I fixed it. I still think we
should tweak the lints.
For example, clippy complains at things like `1 * HOURS ` because it's
redundant to multiply by 1, buy maybe we want it explicitly that way, so
it's more evident it can be changed. In this PR I don't tweak the rule
but try to make clippy happy.
Adds the `Substrate` node and runtime, as well as configuration and test
files, from https://github.com/Moonsong-Labs/flamingo to the `operator`
folder in DataHaven
