Upgrade to SH release
[v0.1.1](https://github.com/Moonsong-Labs/storage-hub/releases/tag/v0.1.1)
---------
Co-authored-by: Ahmad Kaouk <56095276+ahmadkaouk@users.noreply.github.com>
Co-authored-by: Ahmad Kaouk <ahmadkaouk.93@gmail.com>
Co-authored-by: Steve Degosserie <723552+stiiifff@users.noreply.github.com>
## Summary
- Adds automated license compliance checking via GitHub Actions CI
workflow
- Implements a license verification script that validates all Rust
dependencies against approved licenses, authors, and packages
- Standardizes author metadata across Cargo manifests to "Moonsong Labs"
## Changes
**CI Workflow** (`.github/workflows/task-check-licenses.yml`)
- Triggers on pull requests and manual dispatch
- Installs Rust 1.88.0 toolchain and `cargo-license` tool
- Executes license verification script to enforce compliance
**License Verification Script** (`operator/scripts/verify-licenses.sh`)
- Uses `cargo-license` to extract dependency license information
- Maintains three allowlists:
- **Licenses**: Apache-2.0, MIT, BSD variants, GPL-3.0, MPL-2.0, and
compatible combinations
- **Authors**: PureStake, Parity Technologies, Moonsong Labs, Frontier
developers, StorageHub Team
- **Package Names**: Known safe packages like ring
- Fails the build if any dependency has unapproved license/author/name
combination
**Cargo Manifest Updates**
- `operator/Cargo.toml`: Standardized workspace author to "Moonsong
Labs"
- `operator/precompiles/precompile-registry/Cargo.toml`: Uses workspace
author field
- `operator/runtime/common/Cargo.toml`: Added workspace author field
## Benefits
- **Legal Compliance**: Ensures all dependencies use OSI-approved or
compatible licenses
- **Supply Chain Security**: Validates dependencies come from trusted
sources
- **Automated Enforcement**: Catches licensing issues during PR review
rather than at release time
- **Transparency**: Provides clear audit trail of approved licenses and
authors
### Add missing weights for BABE, GRANDPA, and Randomness
#### Summary
Adds generated weights and wires them into the runtime for the BABE,
GRANDPA, and Randomness pallets to replace defaults and ensure accurate
execution costs across networks.
#### What’s changed
- **New weights added** for `pallet_grandpa`, `pallet_babe` and
`pallet_randomness`
- **Runtime configs updated to use new weights**
- `operator/runtime/mainnet/src/configs/mod.rs`
- `operator/runtime/stagenet/src/configs/mod.rs`
- `operator/runtime/testnet/src/configs/mod.rs`
#### For follow-up PRS
- fix `pallet_identity` failure at running benchmarks
- fix `pallet_collective` benchmarking missmatch (related to
https://github.com/paritytech/polkadot-sdk/pull/6435)
- add `pallet_session_benchmarking` without including `pallet_staking`
(or some workaround)
- add StorageHub weights to our benchmarked pallets (`pallet_nfts`,
`pallet_storage_providers`, `pallet_payment_streams`,
`pallet_proofs_dealer`, `pallet_file_system`, `pallet_bucket_nfts`, etc)
## 🔨 Add Slashing Support for Runtime
This PR introduces the slashing functionality for the DataHaven runtime,
enabling punitive measures against misbehaving validators.
### Features
- Deferred slashing with configurable veto periods
- Cross-chain slashing message delivery trough Snowbridge
- Governance controls for slashing parameters and emergency cancellation
We introduced the `external-validator-slashes` pallet, which allows to
slash validators that misbehave. The slashing is triggered when an
offence is reported via the offence pallet (which is already
implemented). The message is sent through Snowbrige's outbound queue and
the real slashing happens in the contracts side, which will come in a
follow up PR.
There is a configurable window of time between the time the validator is
being reported, and the time the slash is triggered. This allows that in
case of an error we are still able to cancel the slashing, using a sudo
account.
For convenience, we also have extrinsics for corner cases:
- **`force_inject_slash`**: Root-only function to manually inject
slashes for specific validators with custom percentages. Useful for
emergency situations or governance-directed slashing outside normal
offence detection
- **`cancel_deferred_slash`**: Allows governance to cancel pending
slashes during the defer period by specifying era and slash indices.
Provides safety mechanism against false positives or malicious slash
reports
- **`set_slashing_mode`**: Configurable slashing behavior with three
modes - `Enabled` (normal operation), `LogOnly` (track offences without
applying slashes), and `Disabled` (completely halt slashing). Critical
for emergency response and testing
---------
Co-authored-by: Gonza Montiel <gon.montiel@gmail.com>
Co-authored-by: Gonza Montiel <gonzamontiel@users.noreply.github.com>
# Add Referenda Precompile
This PR introduces the referenda precompile from Moonbeam to enable
governance functionality through EVM calls.
## Changes Made
- Added `operator/precompiles/referenda/` with complete implementation
- Updated all runtime configs (mainnet, stagenet, testnet) to include
the referenda precompile
- Adapted track processing logic to work with DataHaven's runtime
configuration
- Adapted tests and mock according to our runtime
---------
Co-authored-by: Steve Degosserie <723552+stiiifff@users.noreply.github.com>
This PR upgrades the StorageHub dependencies to tag
[v0.0.6-alpha](https://github.com/Moonsong-Labs/storage-hub/releases/tag/v0.0.6-alpha).
This includes the fix to connect through TLS to a Postgres DB (allowing
connecting to an AWS hosted DB for instance), and a fix for a missing
indexer DB migration.
Additionally, it adds a new runtime API.
EDIT (previously breaking changes):
As of the new version, the name of a column in the indexer DB has
changed. This can affect the functionality of nodes running a Postgres
DB with the old schema. A
[migration](05d269a26d)
is included in the new
[tag](https://github.com/Moonsong-Labs/storage-hub/commits/v0.0.6-alpha/),
so no need to mark it as breaking.
---------
Co-authored-by: Ahmad Kaouk <56095276+ahmadkaouk@users.noreply.github.com>
Co-authored-by: Gonza Montiel <gonzamontiel@users.noreply.github.com>
Co-authored-by: Steve Degosserie <723552+stiiifff@users.noreply.github.com>
## Add Preimage Precompile
This PR integrates the Preimage precompile from Moonbeam into the
DataHaven runtime across all three environments (mainnet, stagenet,
testnet).
**Key Changes:**
- Added Preimage precompile implementation at address `2067` in all
runtime configurations
- Updated precompile sets in mainnet, stagenet, and testnet runtimes
- Updated `is_governance_precompile()` for the Preimage precompile
## Add Conviction Voting Precompile
This PR introduces a new EVM precompile for conviction voting
functionality, enabling smart contracts to interact with the Substrate
conviction voting pallet.
### Key Changes
- **New Precompile**: Added `ConvictionVotingPrecompile` at address
`0x0000000000000000000000000000000000000812` (2066)
- **Solidity Interface**: Complete Solidity interface
(`ConvictionVoting.sol`) with all conviction voting operations
- **Runtime Integration**: Integrated precompile across all runtime
configurations (mainnet, testnet, stagenet)
- **Comprehensive Testing**: Inherits test suite from Moonbeam
### Features
- Vote casting with different conviction levels (None, Locked1x-6x)
- Vote delegation and undelegation
- Poll management and tallying
- Support for all conviction voting pallet operations
---------
Co-authored-by: Ahmad Kaouk <56095276+ahmadkaouk@users.noreply.github.com>
Co-authored-by: Steve Degosserie <723552+stiiifff@users.noreply.github.com>
## Add Collective Precompile
Adds the pallet collective precompile to `mainnet`, `stagenet`, and
`testnet` according to Moonbeam's configuration.
### Changes:
- Added `pallet-evm-precompile-collective` dependency to workspace
- Configured collective precompile at address `2064` using
`TreasuryCouncilInstanc`
- Configured collective precompile at address `2068` using
`TechnicalCommitteeInstance`
The precompile provides EVM access to collective governance
functionality including proposal execution, voting, and membership
management.
---------
Co-authored-by: Steve Degosserie <723552+stiiifff@users.noreply.github.com>
## Add Identity Precompile
Adds the pallet Identity precompile to `mainnet`, `stagenet`, and
`testnet` according to Moonbeam's configuration.
### Changes:
- Added `pallet-evm-precompile-identity` dependency to workspace
- Added and configured at address `2072` for all runtimes
The precompile provides a Solidity interface access to the Substrate
Identity pallet functionality.
---------
Co-authored-by: Ahmad Kaouk <56095276+ahmadkaouk@users.noreply.github.com>
Co-authored-by: Steve Degosserie <723552+stiiifff@users.noreply.github.com>
In this PR we update StorageHub to its latest tage `v0.0.4-alpha`. It
includes so minor changes on the FilesystemAPI.
Its also force to specify the `maintenance_mode` options for fisherman.
For now we are not allowing this mode so we just skip entirely.
---------
Co-authored-by: Steve Degosserie <723552+stiiifff@users.noreply.github.com>
## Overview
This PR integrates Substrate's `pallet-migrations` into all DataHaven
runtimes (mainnet, stagenet, testnet) to enable robust multi-block
migration capabilities. This infrastructure allows complex runtime
upgrades to be executed across multiple blocks while maintaining chain
stability and providing governance controls.
## What Changed
### Core Integration
- **Added `pallet-migrations` dependency** across all runtime
configurations
- **Integrated migration pallet** as pallet index 39 in all runtimes
- **Created shared migration configuration** in
`datahaven-runtime-common`
### Runtime Configuration
- **Mainnet, Stagenet, and Testnet** now include identical migration
configurations
- **MaxServiceWeight** parameter set to 75% of max block weight for safe
migration execution
- **Migration cursor limits** configured (65KB max cursor, 256B max
identifier)
- **Failure handling** configured to freeze the chain on migration
failures (similar to Moonbeam's maintenance mode)
## Future Work
- [ ] Add custom failure handler (safe mode) to replace chain freeze
- [ ] Generate DataHaven-specific benchmarks for migration weights
---------
Co-authored-by: undercover-cactus <lola@moonsonglabs.com>
Incorporate `pallet-evm-precompile-file-system` into `stagenet`,
`testnet` and `mainnet`.
---------
Co-authored-by: Steve Degosserie <723552+stiiifff@users.noreply.github.com>
Co-authored-by: Ahmad Kaouk <56095276+ahmadkaouk@users.noreply.github.com>
Added missing CLI option for the indexer configuration for Storage Hub.
Indexer is needed for MSP user.
---------
Co-authored-by: Steve Degosserie <723552+stiiifff@users.noreply.github.com>
## Summary
This PR adds Moonbeam's Batch precompile to DataHaven, enabling
efficient batching of multiple EVM transactions in a single call. This
implementation follows Moonbeam's exact architecture and provides
significant gas savings for batch operations.
## Key Features
### Batch Precompile Functions
- **`batchSome(address[], uint256[], bytes[], uint64[])`**: Execute
multiple calls, continuing on failures
- **`batchSomeUntilFailure(address[], uint256[], bytes[], uint64[])`**:
Execute calls until first failure
- **`batchAll(address[], uint256[], bytes[], uint64[])`**: Execute all
calls, reverting if any fail
### Technical Implementation
- **Address**: `0x0808` (2056 in decimal)
- **Access Control**: Restricted nesting with `SubcallWithMaxNesting<2>`
- **Self-recursion**: Only the Batch precompile can call itself
(`OnlyFrom<AddressU64<2056>>`)
- **Gas Management**: Proper gas estimation and refund handling
- **Error Handling**: Comprehensive revert reasons and event logging
Depends on #137
---------
Co-authored-by: Ahmad Kaouk <ahmadkaouk.93@gmail.com>
## Summary
This PR implements a comprehensive EVM precompile registry system for
DataHaven, following Moonbeam's exact architecture and patterns. The
implementation includes:
- **Registry Precompile**: A new precompile at address `0x0815` (2069)
that manages and queries available precompiles
- **Core Ethereum Precompiles**: Standard Ethereum precompiles
(ECRecover, SHA256, RIPEMD160, Identity, ModExp, BN128Add, BN128Mul,
BN128Pairing, Blake2F, SHA3FIPS)
- **Modular Architecture**: Clean separation following Moonbeam's
structure with dedicated precompile modules per runtime
## Key Features
### Registry Precompile Functions
- `isPrecompile(address)`: Check if an address corresponds to any
precompile (active or inactive)
- `isActivePrecompile(address)`: Check if a precompile is currently
active in the runtime
- `updateAccountCode(address)`: Insert dummy EVM bytecode for Solidity
compatibility
### Runtime Integration
- Integrated across all three runtimes (testnet, stagenet, mainnet)
- Uses Moonbeam's `PrecompileSetBuilder` pattern for composable
precompile management
- Proper gas accounting with database read/write operations
- Access control through `CallableByContract` and `CallableByPrecompile`
traits
---------
Co-authored-by: undercover-cactus <lola@moonsonglabs.com>
This PR updates the project dependencies as follows:
- **Polkadot SDK**: from the `stable2412` (moving) branch to the
`polkadot-stable2412-6` (fixed) tag.
- **StorageHub**: to revision
`f8281283b6003a3009a32431ed0f3cd628561d6b`, which also depends on
Polkadot SDK `polkadot-stable2412-6`.
- **Frontier**: revision `75329a2df49e2cc7981485392c31160929d1bd48n`
which, likewise, depends on Polkadot SDK `polkadot-stable2412-6`.
## Summary
This PR integrates the Substrate Proxy pallet into DataHaven runtimes
(testnet, stagenet, mainnet) with comprehensive test coverage. The proxy
pallet enables account delegation functionality, allowing accounts to
authorize other accounts to execute calls on their behalf with
configurable permissions.
## Changes
### Proxy Pallet Integration
- **Added proxy pallet** to all three DataHaven runtimes (testnet,
stagenet, mainnet)
- **Configured custom ProxyType enum** with DataHaven-specific proxy
types:
- `Any` - Unrestricted proxy access
- `NonTransfer` - All calls except balance transfers
- `Governance` - Governance and utility calls only
- `Staking` - Staking operations (placeholder)
- `CancelProxy` - Proxy announcement cancellation
- `Balances` - Balance transfer operations only
- `IdentityJudgement` - Identity judgement operations
- `SudoOnly` - Privileged sudo operations only
### Runtime Configuration
- **Integrated pallet-proxy** into runtime construction macros
- **Configured proxy deposits** (base + per-proxy fees)
- **Set proxy limits** (maximum proxies per account)
- **Implemented InstanceFilter** for call filtering per proxy type
- **Added proxy pallet to runtime APIs** and metadata
### Comprehensive Test Suite
- `operator/runtime/testnet/tests/proxy.rs` - 24 comprehensive proxy
tests
- `operator/runtime/stagenet/tests/proxy.rs` - 24 comprehensive proxy
tests
- `operator/runtime/mainnet/tests/proxy.rs` - 24 comprehensive proxy
tests
- Updated `lib.rs` files in all three runtimes to include proxy test
modules
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- New Features
- Enable account proxies across mainnet, stagenet, and testnet with
configurable types, delays, and per-type call permissions.
- Support anonymous (pure) proxies, proxy announcements with delays, and
deposit/limit parameters for proxy management.
- Tests
- Add comprehensive integration tests covering proxy lifecycle,
filtering, pure proxies, announcements, batching, chaining, multisig,
identity, and sudo paths.
- Test builder now supports optional sudo setup.
- Chores
- Add benchmarking, weights, and try-runtime support for proxies.
- Update internal package metadata version.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
---------
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Ahmad Kaouk <ahmadkaouk.93@gmail.com>
Co-authored-by: Ahmad Kaouk <56095276+ahmadkaouk@users.noreply.github.com>
In this PR we add StorageHub pallets to the runtime. It will only be
added if the `storage-hub` feature is activated.
---------
Co-authored-by: Gonza Montiel <gonzamontiel@users.noreply.github.com>
This PR integrates the Substrate FRAME Treasury pallet across all three
DataHaven runtime environments (`testnet`, `mainnet`, `stagenet`) with a
custom fee allocation mechanism and comprehensive test coverage.
### Key Changes
#### Treasury Pallet Integration:
- Added Treasury pallet to all three runtimes (testnet, mainnet,
stagenet) with 20% fee allocation and 80% burn mechanism.
- Implemented dynamic fee proportion control via
`FeesTreasuryProportion` runtime parameter.
- Integrated treasury with custom fee handlers:
`DealWithEthereumBaseFees`, `DealWithEthereumPriorityFees`, and
`DealWithSubstrateFeesAndTip`.
#### Comprehensive Testing Infrastructure:
- Created robust test architecture with session management and validator
setup across all runtimes
- Added block author management utilities for treasury testing requiring
pallet_authorship integration
- Implemented 15 total tests (5 tests × 3 runtimes) covering:
- EVM transaction fee allocation with/without priority fees
- Substrate fee and tip handling validation
- Treasury spending functionality via sudo
- Complete fee flow verification with actual network base fee
calculations
#### Technnical Implementation
Fee Allocation Logic:
- Base fees: 20% to treasury, 80% burned
- Priority fees: 100% to block author
- Substrate tips: 100% to block author
The implementation is based on
https://github.com/moonbeam-foundation/moonbeam/pull/3120 in
[Moonbeam](https://github.com/moonbeam-foundation/moonbeam), and
assisted by Claude Code.
### Summary
- Implement native token transfers from DataHaven to Ethereum using
Snowbridge infrastructure
- Add comprehensive datahaven-native-transfer pallet with lock-and-mint
mechanism for cross-chain token representation
### Key Features
**New Pallet: datahaven-native-transfer**
- Cross-chain transfers: Transfer DataHaven native tokens to Ethereum
addresses via `transfer_to_ethereum extrinsic`
- Token locking mechanism: Secure token storage in deterministic
Ethereum sovereign account during transfers
- Fee management: Required fees for all transfers to compensate relayers
and prevent spam
- Emergency controls: Pause/unpause functionality
- Token registration: Integration with Snowbridge's token registration
for native token identification
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **New Features**
- Introduced the DataHaven Native Transfer pallet enabling secure
cross-chain transfers of DataHaven native tokens to and from Ethereum
via Snowbridge.
- Added token locking on DataHaven, minting on Ethereum, and mandatory
fee collection to cover gas costs and incentivize relayers.
- Implemented pause and unpause controls for emergency management of
token transfers.
- **Configuration**
- Integrated the new pallet into mainnet, stagenet, and testnet runtimes
with updated network, account, and treasury settings.
- Added Ethereum sovereign account constants and native token ID support
for optimized cross-chain operations.
- **Documentation**
- Added comprehensive README and inline documentation detailing pallet
features, extrinsics, events, errors, and security considerations.
- **Tests**
- Added extensive unit and integration tests covering token transfers,
native token registration, fee handling, pause functionality, and event
validation across all supported networks.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
---------
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Steve Degosserie <723552+stiiifff@users.noreply.github.com>
This PR introduces and integrates the new
`pallet-external-validators-rewards` into the operator runtimes. This
pallet is responsible for managing and distributing rewards to external
validators. The pallet was originally introduced by Tanssi. It was
heavily modified to abstract the message building and sending with
Snowbridge.
### Key Changes Related to `pallet-external-validators-rewards`:
* **Pallet and Runtime API Addition:**
* The `pallet-external-validators-rewards` has been added to `mainnet`,
`stagenet`, and `testnet` runtimes
* **Runtime Configuration
(`operator/runtime/{mainnet,stagenet,testnet}/src/configs/mod.rs`):**
* **`RewardsRegistryAddress` Parameter:** A new dynamic runtime
parameter `RewardsRegistryAddress` (type `H160`) has been added to
`runtime_params.rs`. This will hold the Ethereum address of the Rewards
Registry contract.
* **Author Rewards with `RewardsPoints`:**
* A new struct `RewardsPoints` implements
`pallet_authorship::EventHandler`.
* Its `note_author` function rewards block authors (if they are not
whitelisted validators) by calling
`ExternalValidatorsRewards::reward_by_ids`.
* The `EventHandler` in `pallet_authorship::Config` is updated to
`(RewardsPoints, ImOnline)`.
* **Integration with `pallet-external-validators`:**
* The `OnEraStart` and `OnEraEnd` hooks in
`pallet_external_validators::Config` are now set to
`ExternalValidatorsRewards`, allowing the rewards pallet to react to era
changes.
* **`RewardsSendAdapter` for Ethereum Communication:**
* A new struct `RewardsSendAdapter` implements
`pallet_external_validators_rewards::types::SendMessage`.
* `build()`: Constructs an `OutboundMessage` to call the
`updateRewardsMerkleRoot(bytes32)` function on the
`RewardsRegistryAddress` on Ethereum. The calldata includes the selector
(hashed from `UpdateRewardsMerkleRootSignature`) and the
`rewards_merkle_root`.
* `validate()` and `deliver()`: Utilize `OutboundQueueV2` for message
validation and delivery.
* A constant `UpdateRewardsMerkleRootSignature` (
`b"updateRewardsMerkleRoot(bytes32)"`) is defined for this purpose.
* **Runtime API
(`operator/runtime/{mainnet,stagenet,testnet}/src/lib.rs`):**
* The runtime implements
`pallet_external_validators_rewards_runtime_api::ExternalValidatorsRewardsApi`.
* This exposes two functions:
* `generate_rewards_merkle_proof(account_id: AccountId, era_index:
EraIndex) -> Option<MerkleProof>`
* `verify_rewards_merkle_proof(merkle_proof: MerkleProof) -> bool`
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **New Features**
- Introduced a rewards system for external validators with era-based
tracking, Merkle proof generation, and verification.
- Added runtime APIs to generate and verify validator rewards Merkle
proofs.
- Enabled rewards distribution via outbound messaging to an external
registry contract.
- Configured runtime parameters for the rewards registry address.
- Added fixed author reward points and integrated rewards logic into
block authorship events.
- Provided a mock runtime environment and benchmarking support for the
rewards pallet.
- **Tests**
- Added comprehensive tests and benchmarks for validator rewards
distribution and Merkle proof functionality.
- **Chores**
- Integrated new pallets and runtime APIs into mainnet, stagenet, and
testnet configurations.
- Updated dependencies and feature groups to include new rewards-related
pallets.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
---------
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
Co-authored-by: Facundo Farall <37149322+ffarall@users.noreply.github.com>
In this PR:
1. Add new `datahaven-node-local.dockerfile` for building a local image
with the locally built DataHaven Node binary. This severely improves
iteration speed on running `bun cli` if there are changes in the DH
node. Previously, it relied on the published dockerfile, which builds
the Cargo project inside of it, taking +20m even with no changes.
2. Building this local dockerfile is integrated to the CLI, which now
also asks if the user wants to rebuild the local docker image of the DH
node.
3. A new script `cargo-crossbuild` is added, to be able to build the
DataHaven node Cargo project both from Mac and Linux, with the target
being `x86_64-unknown-linux-gnu`. For building from Mac, it uses `cargo
zigbuild`, so `zig` is now a dependency. Building for this target is
needed because the docker image is an Ubuntu image, so it will need to
run a linux binary.
4. Added `zig` as dependency in docs.
5. CI still uses the docker image built by the CI itself, which builds
the Cargo project inside of it. The CI can take advantage of caching for
this.
This PR replaces `pallet-validator-set` with a new
`pallet-external-validators` pallet from Tanssi.
## Key Changes
- **New ExternalValidators Pallet**
- Supports whitelisted validators (set by governance, not rewarded)
- Manages external validators (can be enabled/disabled)
- Implements era-based rotation (eras change after configurable
sessions)
- **Bridge Integration**
- Updated `EigenLayerMessageProcessor` in
`operator/primitives/bridge/src/lib.rs`
- Replaced the old `SetValidators` command with
```rust
ReceiveValidators { validators, external_index }
```
Add support for multiple runtimes: `stagenet`, `testnet` & `mainnet`.
- Moved common types to `datahaven-runtime-common` crate.
- Made the node's command & service code generic over different
runtimes. Each runtime has a `dev` & `local` genesis config preset.
- More types / constants can be moved to the `datahaven-runtime-common`
crate ... this will be done in subsequent PRs.
---------
Co-authored-by: Gonza Montiel <gon.montiel@gmail.com>
Co-authored-by: Gonza Montiel <gonzamontiel@users.noreply.github.com>
Co-authored-by: Facundo Farall <37149322+ffarall@users.noreply.github.com>
Eventually our CI will be required to run two private blockchains
locally plus associated relayers.
This PR is to prepare for this fate by improving run times and
refactoring our existing CIs so they are a bit easier to reason about.
### Refactors
- **_We now run ALL CIs on every PR!_** This is so that we decomplexify
the logic around conditional builds and fetching built binaries from
another source. This reduces the surface area of code we have to
maintain at the cost of execution time
- This penalty is ameliorated by a layered caching system. At best, it
will be less than a minute to complete a build since everything will be
cached. On GH runners this is about 6 minutes sadly.
- We will no longer be at risk of important CIs being skipped
erroneously which hide true failures.
- Caching is a low-risk approach because at worst it has to build from
scratch. A bad cache hit will never imply the wrong thing gets build
since cargo is smart enough to just throw away any inappropriate build
artefacts.
- `setup-rust` action created so we have a unified way of setting up
runner and unifying our approach to caching
- Use a unique caching key for different activities and it will fallback
to shared cache if no matches
- we are using `mainnet` kurtosis config so that it works with relayer
assumptions
### Additions
- We can specify the ethereum block time via a new cli arg `--slot-time
<seconds>`
- We can specify arbitrary network_param args which get passed into the
generated yaml
- e.g. giving `bun cli --kurtosis-network-args="pet=cat food=fish" will
add:
```yml
network_params:
# existing params...
pet: cat
food: fish
```
- We now have the ability to programmatically modify the yaml
- This means we are back down to a single `minimal.yml` kurtosis config
so we dont have to maintain changes between them
- Flow is: `add new cli arg` -> `add if() block which mutates yaml` ->
`profit`
---------
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Facundo Farall <37149322+ffarall@users.noreply.github.com>
This PR introduces a new script in `./operator/scripts` that allows us
to sort cargo dependencies alphabetically, it is based in my
[gist](https://gist.github.com/gonzamontiel/b4594c62685175f99760442ad2e2dd98).
I modified it so we can run it both in `--fix` and `--check` modes.
To sort a single cargo file, you can do:
```
./scripts/sort-cargo-deps.sh /path/to/Cargo.toml
```
Btw, make sure you are in the operator folder and you have exec
permissions:
```
cd operator
chmod +x ./scripts/sort-cargo-deps.sh
```
But what I recommend is that you format every Cargo.toml just in case,
like this:
```
find . -name "Cargo.toml" -exec ./scripts/sort-cargo-deps.sh {} \;
```
The CI will run
```
find . -name "Cargo.toml" -exec ./scripts/sort-cargo-deps.sh {} check \;
```
---------
Co-authored-by: Facundo Farall <37149322+ffarall@users.noreply.github.com>
This PR implements functionality to store outbound message commitments
in the BEEFY MMR leaf extra field for cross-chain verification. A new
`pallet-outbound-commitment-store` has been introduced to facilitate
this process.
## Changes
- Added a new `pallet-outbound-commitment-store` pallet to capture and
store outbound message commitments
- Implemented the `CommitmentHandler` to receive commitments from the
outbound queue and store them in the new pallet
- Updated the `LeafExtraDataProvider` to include these commitments in
the BEEFY MMR leaf extra field
## Implementation Details
The process works as follows:
1. The outbound queue generates a commitment hash for messages
2. The commitment is stored in the `pallet-outbound-commitment-store`
via the `CommitmentHandler`
3. The `LeafExtraDataProvider` retrieves the latest commitment and
includes it in the BEEFY MMR leaf extra field
4. This commitment can then be verified by other chains using the BEEFY
light client
The new pallet provides the necessary functions to store and retrieve
these commitments.
This PR introduces the Snowbridge `system-v2` pallet and associated
runtime components
**Key Changes:**
* **Added `system-v2` Pallet:** Integrated the
`snowbridge-pallet-system-v2` pallet, providing functionalities for the
Ethereum side of the bridge.
* **Runtime API Integration:**
* Implemented the `ControlV2Api` trait in the runtime
(`operator/runtime/src/apis.rs`) to allow looking up the `AgentId`
associated with a `VersionedLocation`.
* **System V1 Compatibility:** Added the `system-v1` pallet
(`snowbridge-pallet-system`) and related configuration/code references
in various locations.
**Important:** This `system-v1` is included *solely* because the
`system-v2` pallet requires it for compilation and compatibility. It is
**not functionally used** in this runtime.
---------
Co-authored-by: Facundo Farall <37149322+ffarall@users.noreply.github.com>
This PR adds Ethereum RPC API support to the node, enabling the
interaction with the node using standard Ethereum tools.
1. Integration of Frontier's Ethereum RPC modules (eth, net, web3,
txpool)
2. Added RPC configurations necessary for Ethereum compatibility
3. Implemented a BABE consensus data provider for handling pending
blocks
4. Added required dependencies and configurations in the node's service
5. Created necessary filter pools, block data caches, and notification
systems
6. Configured the RPC system to handle Ethereum API calls
7. Changed the transaction pool implementation to be compatible with the
fork-aware transaction pool
(https://github.com/paritytech/polkadot-sdk/pull/4639)
EDIT:
The new `transaction_pool` in `polkadot-stable2412` that comes with a
`fork-aware` feature, allowing different views of the transaction pool,
is not compatible with the current state in Frontier.
In Frontier the only supported pool is the `BasicPool`, that maintains a
similar behaviour to the one in `polkadot-stable2409`. I used this pool
directly.
---------
Co-authored-by: Ahmad Kaouk <56095276+ahmadkaouk@users.noreply.github.com>
Co-authored-by: Facundo Farall <37149322+ffarall@users.noreply.github.com>
This pull request integrates the snowbridge-pallet-outbound-queue-v2 and
its dependencies into the DataHaven runtime. This pallet is responsible
for handling the submission, processing, and commitment of outbound
messages destined for the Ethereum network via the Snowbridge bridge
(v2).
### Key Changes
#### Added New Pallets
- **snowbridge-pallet-outbound-queue-v2**: The core pallet for managing
the outbound message lifecycle.
- **pallet-message-queue**: Introduced to handle the queuing and
processing of messages.
#### Added Crates to Datahaven codebase
- snowbridge-merkle-tree
- bridge-hub-common
- snowbridge-outbound-queue-v2-runtime-api
---------
Co-authored-by: Facundo Farall <37149322+ffarall@users.noreply.github.com>
This PR adds the [pallet
parameters](https://docs.rs/pallet-parameters/0.11.0/pallet_parameters/index.html)
to the DataHaven runtime.
This allows config parameters for pallets to be dynamically modified,
needed, for instance, to set `EthereumfGatewayAddress` after the
deployment of that smart contract on Ethereum. StorageHub also uses this
pallet extensively.
Additionally, refactors the pallet configuration files to have them more
visually organised.
This PR adds the Snowbridge Inbound Queue V2 pallet to the runtime. It
also defines bridge primitives, including the message structure for
EigenLayer, and introduces the EigenLayerMessageProcessor to handle
messages coming from EigenLayer.
---------
Co-authored-by: Facundo Farall <37149322+ffarall@users.noreply.github.com>
