This pull request updates the identity pallet configuration to ensure
that slashed funds are now sent to the treasury, rather than being
discarded. This change is a step towards proper fund management within
the runtime.
Identity pallet configuration update:
* Changed the `Slashed` associated type implementation in the
`pallet_identity::Config` for `Runtime` to use `Treasury`, so that
slashed funds are now sent to the treasury instead of being ignored.
* Add `run_benchmarks.sh` script to run runtime benchmarks
* Sets up benchmark configs and directory structure to store weights
(`operator/runtime/<RUNTIME>/weights`)
* (naive) fixes to some benchmarks:
* `pallet_datahaven_native_transfer`:
* use a mock for `NativeTokenId`
* look at the balance difference of the treasury instead of the total
(this makes the benchmark agnostic to genesis setup)
* `snowbridge_pallet_system` / `snowbridge_pallet_system_v2` use native
token xcm location vs relay chain one. Add missing benchmark methods and
update fixture with valid data.
* `snowbridge_pallet_ethereum_client`: update fixtures with valid data
* `snowbrige_pallet_inbound_queue_v2`: set EthereumGatewayAddress when
initializing storage on benchmark and use a mock message processor ( as
fixture has `CreateAsset` payload which is not supported in the
`EigenLayerMessageProcessor`)
* `snowbridge_pallet_outbound_queue_v2`: add missing
`submit_delivery_receipt` benchmark which required a dedicated fixture
(all copied from the upstream pallet)
* `pallet_treasury`: Use an `ExistentialDeposit` of `1` on benchmark,
else payout fails.
* `pallet_transaction_payment`: Use a custom `WeightToFee` that makes
the Fee small, else account in benchmark cannot pay for fees (It is
funded a multiplier of `ExistentialDeposit` and is expected for that to
be enough, but it's not in our particular setup).
* comment out `pallet_identity` and `pallet_im_online` due to
incompatibilities (to be addressed later)
* Basic benchmark run to set `WeightInfo` from `weights` in configs
(real run should be done later using target hardware)
---------
Co-authored-by: Ahmad Kaouk <ahmadkaouk.93@gmail.com>
Co-authored-by: Tobi Demeco <50408393+TDemeco@users.noreply.github.com>
Co-authored-by: undercover-cactus <lola@moonsonglabs.com>
Co-authored-by: TDemeco <tdemeco@itba.edu.ar>
## Summary of changes
- We decided to remove the topics and nonce from the massage encoding
since we don't use them (original commit:
ee2a3f2fd4).
- Besides, we already have a nonce at the Snowbridge message level
f4ab5c2b2e/operator/primitives/snowbridge/inbound-queue/src/v2/message.rs (L105)
- I had to recreate the static test for _encoding_ (happens in
[DataHavenSnowbridgeMessages.sol](d12d40634f/contracts/src/libraries/DataHavenSnowbridgeMessages.sol)
) / _decoding_ (happens in
[operator/primitives/bridge/src/lib.rs)](f9f9cc65fe/operator/primitives/bridge/src/lib.rs).
Now it matches the current structure. The idea is that now we can test
that we don't break the decoding in followup refactoring.
- Fixes a problem with EigenLayer validator addresses. In all our
contracts we were using `bytes32` to refer to a Solochain validator
address. But on our Substrate change we actually expect AccountId20, so
only 20 bytes. This was causing the decoding to fail.
- I opted for the minimal change that would be to take the right-most 20
bytes to send that to our chain. But we might want aswell to limit our
EigenLayer contracts to be only 20 bytes long. @ahmadkaouk showcase this
[here](92a34c273c)
- Adds a bash script to run the static test. The test will compile the
contracts, run the encoding test, compile the operator, and run the
decoding test. This saves a huge amount of time since we don't need to
run the full e2e setup. The way of running it is the following:
```bash
cd operator/test/scripts
./test_message_encoding.sh
```
- As a consequence of this PR, the execution relayer now works properly.
EDIT:
> [!IMPORTANT]
**We decided to use 20-byte addresses in our contracts**. So what is
stated above is not valid anymore.
The change implies that the mapping from Ethereum addresses to bytes32
addresses now it's a mapping as follows:
dd3ba99ac0/contracts/src/DataHavenServiceManager.sol (L51-L52)
I've updated helper functions, tests, etc to be compliant with this
change. The execution relayer and beefy relayer look stable now.
---------
Co-authored-by: Ahmad Kaouk <ahmadkaouk.93@gmail.com>
Co-authored-by: Ahmad Kaouk <56095276+ahmadkaouk@users.noreply.github.com>
## Overview
This PR implements the **return path** for DataHaven's native token
bridge, enabling tokens that were previously transferred from DataHaven
to Ethereum to flow back to DataHaven.
**Context**: DataHaven native tokens can be transferred to Ethereum
where they exist as wrapped ERC20 tokens. This PR completes the
bidirectional bridge by implementing the mechanism to transfer these
tokens back from Ethereum to DataHaven, effectively "unwrapping" them
and restoring them to their native form.
## Key Changes
- **Added `NativeTokenTransferMessageProcessor`** , a new message
processor specifically designed to handle native token return transfers
from Ethereum, and extended `InboundQueueV2` processor tuple to include
native token transfer handling alongside existing EigenLayer message
processing
## How It Works
### Detailed Flow: Ethereum → DataHaven
#### 1. **Ethereum Initiation**
- **User Transaction**: User calls `v2_sendMessage` on Snowbridge
Gateway contract:
```solidity
v2_sendMessage(
assets: [{
kind: ForeignToken,
assetId: <registered_native_token_id>, // Must match pre-registered
token ID
amount: <transfer_amount>
}],
claimer: <datahaven_recipient_as_h160>, // 20-byte recipient address
(SCALE-encoded)
message: [], // Empty for native transfers
// ... standard Snowbridge fees
)
```
- **Token Burning**: Gateway burns the wrapped ERC20 tokens on Ethereum
(removing them from circulation)
- **Message Encoding**: Gateway encodes transfer details into
standardized Snowbridge message format
- **Event Emission**: `MessageAccepted` event logged on Ethereum for
relayers to pick up
#### 2. **Cross-Chain Delivery Infrastructure**
- **Relayer Network**: Snowbridge relayers monitor Ethereum events and
submit messages to DataHaven
- **Consensus Verification**: DataHaven's `EthereumBeaconClient` pallet
verifies Ethereum beacon chain consensus
- **Cryptographic Proofs**: Messages include Merkle proofs and execution
receipts for tamper-proof verification
- **Direct Processing**: Relayers submit messages via
`InboundQueueV2::submit()` which processes them immediately
#### 3. **DataHaven Message Processing**
**Immediate Processing Flow:**
When a relayer calls `InboundQueueV2::submit()`, the message is
processed immediately using a tuple-based processor system:
```rust
type MessageProcessor = (
EigenLayerMessageProcessor<Runtime>, // Handles validator operations
NativeTokenTransferMessageProcessor<Runtime>, // Handles native token returns
);
```
**Processing Steps:**
1. **Message Verification**: Cryptographic verification and nonce
checking (prevents replay attacks)
2. **Processor Selection**: Evaluation of processors (First processor
returning `true` handles the message):
- `EigenLayerMessageProcessor::can_process_message()` - returns `false`
for token transfers
- `NativeTokenTransferMessageProcessor::can_process_message()` -
validates:
- Native token is registered via `SnowbridgeSystemV2::register_token()`
(`T::NativeTokenId::get().is_some()`)
- All assets are `ForeignTokenERC20` with matching native token ID
- Assets array is not empty
**Token Unlocking Process:**
1. **Recipient Extraction**: Decodes H160 address from `claimer` field →
converts to DataHaven AccountId (must contain valid SCALE-encoded H160
address)
2. **Sovereign Account Transfer**: Moves tokens from Ethereum sovereign
account to recipient (requires sufficient balance from previous outbound
transfers)
3. **Event Emission**: `TokensUnlocked` event confirms successful
transfer completion
---------
Co-authored-by: Tobi Demeco <50408393+TDemeco@users.noreply.github.com>
Co-authored-by: Claude <noreply@anthropic.com>
In this PR we add StorageHub pallets to the runtime. It will only be
added if the `storage-hub` feature is activated.
---------
Co-authored-by: Gonza Montiel <gonzamontiel@users.noreply.github.com>
This PR integrates the Substrate FRAME Treasury pallet across all three
DataHaven runtime environments (`testnet`, `mainnet`, `stagenet`) with a
custom fee allocation mechanism and comprehensive test coverage.
### Key Changes
#### Treasury Pallet Integration:
- Added Treasury pallet to all three runtimes (testnet, mainnet,
stagenet) with 20% fee allocation and 80% burn mechanism.
- Implemented dynamic fee proportion control via
`FeesTreasuryProportion` runtime parameter.
- Integrated treasury with custom fee handlers:
`DealWithEthereumBaseFees`, `DealWithEthereumPriorityFees`, and
`DealWithSubstrateFeesAndTip`.
#### Comprehensive Testing Infrastructure:
- Created robust test architecture with session management and validator
setup across all runtimes
- Added block author management utilities for treasury testing requiring
pallet_authorship integration
- Implemented 15 total tests (5 tests × 3 runtimes) covering:
- EVM transaction fee allocation with/without priority fees
- Substrate fee and tip handling validation
- Treasury spending functionality via sudo
- Complete fee flow verification with actual network base fee
calculations
#### Technnical Implementation
Fee Allocation Logic:
- Base fees: 20% to treasury, 80% burned
- Priority fees: 100% to block author
- Substrate tips: 100% to block author
The implementation is based on
https://github.com/moonbeam-foundation/moonbeam/pull/3120 in
[Moonbeam](https://github.com/moonbeam-foundation/moonbeam), and
assisted by Claude Code.
This PR improves the CLI to get from the deployments the
`RewardsRegistryAddress` (address of the RewardsRegistry contract
deployed), `RewardsAgentOrigin` (origin used for the agent in charge of
updating the rewards merkle root in the RewardsRegistry contract) and
`RewardsUpdateSelector` (function selector of the function that the
agent must execute to do the aforementioned update) and then set these
values in the `parameters` pallet of the runtime.
After these changes the rewards merkle root is being updated on the
Ethereum side. 🎉
### Summary
- Implement native token transfers from DataHaven to Ethereum using
Snowbridge infrastructure
- Add comprehensive datahaven-native-transfer pallet with lock-and-mint
mechanism for cross-chain token representation
### Key Features
**New Pallet: datahaven-native-transfer**
- Cross-chain transfers: Transfer DataHaven native tokens to Ethereum
addresses via `transfer_to_ethereum extrinsic`
- Token locking mechanism: Secure token storage in deterministic
Ethereum sovereign account during transfers
- Fee management: Required fees for all transfers to compensate relayers
and prevent spam
- Emergency controls: Pause/unpause functionality
- Token registration: Integration with Snowbridge's token registration
for native token identification
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **New Features**
- Introduced the DataHaven Native Transfer pallet enabling secure
cross-chain transfers of DataHaven native tokens to and from Ethereum
via Snowbridge.
- Added token locking on DataHaven, minting on Ethereum, and mandatory
fee collection to cover gas costs and incentivize relayers.
- Implemented pause and unpause controls for emergency management of
token transfers.
- **Configuration**
- Integrated the new pallet into mainnet, stagenet, and testnet runtimes
with updated network, account, and treasury settings.
- Added Ethereum sovereign account constants and native token ID support
for optimized cross-chain operations.
- **Documentation**
- Added comprehensive README and inline documentation detailing pallet
features, extrinsics, events, errors, and security considerations.
- **Tests**
- Added extensive unit and integration tests covering token transfers,
native token registration, fee handling, pause functionality, and event
validation across all supported networks.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
---------
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Steve Degosserie <723552+stiiifff@users.noreply.github.com>
This PR fixes a critical issue in which the messages enqueued in the
`MessageQueue` pallet weren't being correctly commited by the
`SnowbridgeOutboundQueueV2` pallet because of a misconfiguration in our
runtimes:
- Before this PR, the `MessageQueue` pallet had a lower index than the
`SnowbridgeOutboundQueueV2` pallet. This means that its `on_initialize`
hook was executed before the `on_initialize` hook of the latter.
- The `MessageQueue` pallets processes messages in its `on_initialize`
hook. In our case, this processing meant calling the
`do_process_message` function of the `SnowbridgeOutboundQueueV2` pallet,
which pushes the message into both its `Messages` and `MessageLeaves`
storage for further processing afterwards, sets a pending order for it
for the relayer and increments the nonce for the next message.
- Then, the `SnowbridgeOutboundQueueV2`'s `on_initialize` hook gets
called, and it completely clears both the `Messages` and `MessageLeaves`
storage to start fresh for the new block. This means it would clear up
the previously processed messages.
- Finally, in the `on_finalize` hook, the messages processed in this
block would get commited from the `MessageLeaves` storage, but since it
would always be empty nothing got commited, which means the Solochain
relayer never found anything to relay and messages weren't getting to
Ethereum.
## This PR includes:
- Running the execution relayer on the CLI
- Modifying the Payload generation in the `DataHavenServiceManager.sol`
- Modified the `EigenLayerMessageProcessor` to work with the
ValidatorSet update message, but for this change we are loosing the
generic message type (it was the only way to make it work so far).
- Adds a `--no-wait` argument to the cli launch and stop commands to
bootstrap faster.
### Testing the Snowbridge message encoding / decoding
- Added`MessageEncoding.t.sol` is documented and explains how to
generate bin data to use in the rust test.
- Added a Rust unit test to `EigenLayerMessageProcessor` to compare the
message encoding/decoding taking the bytes from a file (previously
generated with some mock data).
Specifically, we want that:
3cbca0db6d/contracts/src/libraries/DataHavenSnowbridgeMessages.sol (L78-L85)
Generates the right bytes encoding for
0e2c9cd518/operator/primitives/bridge/src/lib.rs (L51)
If the test passes, it's very likely that the CLI will also pass, if
not, then we might wanna check something else is missing.
### Breaking change ⚠️
For compatibility reasons with Snowbridge contracts (they call specific
extrinsics of specific pallets), I had to rename:
- `InboundQueueV2` -> `EthereumInboundQueueV2`
- `OutboundQueueV2` -> `EthereumOutboundQueueV2`
## For follow up PRs:
- Add an automated way of generating the Solidity bytes fo testing, so
we don't need to maintain the MessageEncoding.t.sol and generate the
binary data manually.
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **New Features**
- Added support for the "execution" relayer type in relay configuration,
parsing, and CLI launch utilities.
- Introduced a Solidity test contract for encoding and logging validator
set messages.
- Added a comprehensive "start:all" script to streamline launching and
setup processes.
- **Enhancements**
- Improved message encoding for validator set updates, aligning with new
struct field names and message formats.
- Updated relay configuration schema and validation to support execution
relayers and OFAC settings.
- Increased beacon datastore capacity and adjusted relay scheduling
parameters in configuration files.
- **Refactor**
- Renamed runtime type aliases for inbound/outbound queues to more
descriptive names across mainnet, stagenet, and testnet.
- Centralized and streamlined validator set update logic in CLI
utilities.
- Centralized message decoding logic and improved visibility of message
fields in Rust components.
- **Bug Fixes**
- Improved error handling and decoding logic for message processing in
Rust components.
- **Tests**
- Added Rust and Solidity tests for message encoding and processing
validation.
- **Chores**
- Updated dependencies and feature flags in Rust project configuration.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
---------
Co-authored-by: Facundo Farall <37149322+ffarall@users.noreply.github.com>
This PR introduces and integrates the new
`pallet-external-validators-rewards` into the operator runtimes. This
pallet is responsible for managing and distributing rewards to external
validators. The pallet was originally introduced by Tanssi. It was
heavily modified to abstract the message building and sending with
Snowbridge.
### Key Changes Related to `pallet-external-validators-rewards`:
* **Pallet and Runtime API Addition:**
* The `pallet-external-validators-rewards` has been added to `mainnet`,
`stagenet`, and `testnet` runtimes
* **Runtime Configuration
(`operator/runtime/{mainnet,stagenet,testnet}/src/configs/mod.rs`):**
* **`RewardsRegistryAddress` Parameter:** A new dynamic runtime
parameter `RewardsRegistryAddress` (type `H160`) has been added to
`runtime_params.rs`. This will hold the Ethereum address of the Rewards
Registry contract.
* **Author Rewards with `RewardsPoints`:**
* A new struct `RewardsPoints` implements
`pallet_authorship::EventHandler`.
* Its `note_author` function rewards block authors (if they are not
whitelisted validators) by calling
`ExternalValidatorsRewards::reward_by_ids`.
* The `EventHandler` in `pallet_authorship::Config` is updated to
`(RewardsPoints, ImOnline)`.
* **Integration with `pallet-external-validators`:**
* The `OnEraStart` and `OnEraEnd` hooks in
`pallet_external_validators::Config` are now set to
`ExternalValidatorsRewards`, allowing the rewards pallet to react to era
changes.
* **`RewardsSendAdapter` for Ethereum Communication:**
* A new struct `RewardsSendAdapter` implements
`pallet_external_validators_rewards::types::SendMessage`.
* `build()`: Constructs an `OutboundMessage` to call the
`updateRewardsMerkleRoot(bytes32)` function on the
`RewardsRegistryAddress` on Ethereum. The calldata includes the selector
(hashed from `UpdateRewardsMerkleRootSignature`) and the
`rewards_merkle_root`.
* `validate()` and `deliver()`: Utilize `OutboundQueueV2` for message
validation and delivery.
* A constant `UpdateRewardsMerkleRootSignature` (
`b"updateRewardsMerkleRoot(bytes32)"`) is defined for this purpose.
* **Runtime API
(`operator/runtime/{mainnet,stagenet,testnet}/src/lib.rs`):**
* The runtime implements
`pallet_external_validators_rewards_runtime_api::ExternalValidatorsRewardsApi`.
* This exposes two functions:
* `generate_rewards_merkle_proof(account_id: AccountId, era_index:
EraIndex) -> Option<MerkleProof>`
* `verify_rewards_merkle_proof(merkle_proof: MerkleProof) -> bool`
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **New Features**
- Introduced a rewards system for external validators with era-based
tracking, Merkle proof generation, and verification.
- Added runtime APIs to generate and verify validator rewards Merkle
proofs.
- Enabled rewards distribution via outbound messaging to an external
registry contract.
- Configured runtime parameters for the rewards registry address.
- Added fixed author reward points and integrated rewards logic into
block authorship events.
- Provided a mock runtime environment and benchmarking support for the
rewards pallet.
- **Tests**
- Added comprehensive tests and benchmarks for validator rewards
distribution and Merkle proof functionality.
- **Chores**
- Integrated new pallets and runtime APIs into mainnet, stagenet, and
testnet configurations.
- Updated dependencies and feature groups to include new rewards-related
pallets.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
---------
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
Co-authored-by: Facundo Farall <37149322+ffarall@users.noreply.github.com>
This PR:
- Adds a check to make sure the BEEFY RPC endpoint is ready before
spinning up the BEEFY relayer, otherwise it would just fail and crash.
- Adds the `--enable-offchain-indexing=true` flag to the Datahaven nodes
run when starting up the E2E infra. This is needed because otherwise
nodes can't be queried by the relayer to generate the required proofs
since they would not store the MMR leafs/nodes/root, so the relayer
would just crash.
- Updates the way we were generating the merkle root of the validator
set.
- The BEEFY pallet (and as such, the relayer) generate the validator set
merkle tree by getting the validator list and treating it as an already
ordered set of merkle leafs, hashing each pair in succession without
caring what each leaf value is.
- Meanwhile, the OpenZeppelin crypto library (and as such, the
EigenLayer contracts) also gets the merkle leafs list but hashes each
pair of leafs in value order.
- This means that, for example, if the list of leafs is: ["0x124",
"0x123"]
- BEEFY would do `hash("0x124123")` to get the value of the parent node.
- OZ and EigenLayer would do `hash("0x123124")` to get the value of the
parent node.
- This created a mismatch between what the BEEFY relayer was expecting
and what was actually calculated in our script. A way to obtain a merkle
tree using the BEEFY way was added to solve this.
- Updates the authority set to not be a hardcoded array of keys, now the
BEEFY keys are obtained by directly querying the runtime before
deploying the BEEFY contracts.
- Renames a few files from `flamingo` to `datahaven`.
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **New Features**
- Added support for calculating and generating Merkle roots and proofs
without sorting, enabling new use cases for validator set management.
- Introduced dynamic fetching and configuration of Datahaven validator
authorities during network launch.
- Added readiness check for the BEEFY protocol before relayer startup,
improving reliability of relayer operations.
- **Bug Fixes**
- Fixed indentation issues in configuration files for improved
readability and consistency.
- **Chores**
- Updated validator lists and addresses in configuration files.
- Enhanced e2e test scripts and added new commands for relayer and
minimal test scenarios.
- Added new dependency to test package.
- Updated version strings in package metadata.
- **Refactor**
- Improved logging and configuration handling during network and relayer
launches.
- Simplified import statements and removed unused code.
- **Style**
- Reformatted configuration and TypeScript files for better readability.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
---------
Co-authored-by: Tim B <79199034+timbrinded@users.noreply.github.com>
Co-authored-by: Facundo Farall <37149322+ffarall@users.noreply.github.com>
This PR replaces `pallet-validator-set` with a new
`pallet-external-validators` pallet from Tanssi.
## Key Changes
- **New ExternalValidators Pallet**
- Supports whitelisted validators (set by governance, not rewarded)
- Manages external validators (can be enabled/disabled)
- Implements era-based rotation (eras change after configurable
sessions)
- **Bridge Integration**
- Updated `EigenLayerMessageProcessor` in
`operator/primitives/bridge/src/lib.rs`
- Replaced the old `SetValidators` command with
```rust
ReceiveValidators { validators, external_index }
```
In this PR:
1. Add [Polkadot API](https://papi.how/) support to `test/` directory
(i.e. the e2e CLI and testing framework) to be able to interact with the
Substrate chain.
1. This allows typed interactions with transactions, constants, storage,
runtime APIs, and non-typed interaction with RPC methods.
2. Types are autogenerated when running `bun i`, from the
`datahaven.scale` file that is part of this repo's version control.
2. Add new utilities file to `papi` related functionalities. For the
time being, generating a new signer from a private key.
3. ~Add a new step to the CLI that sends a transaction to the DataHaven
network. _*THIS SHOULD BE REMOVED SOON, IT'S JUST FOR TESTING
PURPOSES*_~
1. Both steps that send test transactions have been removed from the
CLI, for convenience and being error prone. Their scripts remain usable
for testing purposes if needed.
4. Removes the `apis.rs` files from the runtime definitions. Having them
in a separate file meant that the runtime was not including the Runtime
APIs in the metadata blob, preventing `papi` from creating types from
it. This change can be reapplied after upgrading to `polkadot-sdk`
`stable-2503`. More info
[here](https://github.com/paritytech/polkadot-sdk/issues/6659).
5. Add script to re-generate types, and corresponding docs.
6. Makes logger synchronous to avoid prints happening before logs.
Add support for multiple runtimes: `stagenet`, `testnet` & `mainnet`.
- Moved common types to `datahaven-runtime-common` crate.
- Made the node's command & service code generic over different
runtimes. Each runtime has a `dev` & `local` genesis config preset.
- More types / constants can be moved to the `datahaven-runtime-common`
crate ... this will be done in subsequent PRs.
---------
Co-authored-by: Gonza Montiel <gon.montiel@gmail.com>
Co-authored-by: Gonza Montiel <gonzamontiel@users.noreply.github.com>
Co-authored-by: Facundo Farall <37149322+ffarall@users.noreply.github.com>
