In this PR:
1. Add new `datahaven-node-local.dockerfile` for building a local image
with the locally built DataHaven Node binary. This severely improves
iteration speed on running `bun cli` if there are changes in the DH
node. Previously, it relied on the published dockerfile, which builds
the Cargo project inside of it, taking +20m even with no changes.
2. Building this local dockerfile is integrated to the CLI, which now
also asks if the user wants to rebuild the local docker image of the DH
node.
3. A new script `cargo-crossbuild` is added, to be able to build the
DataHaven node Cargo project both from Mac and Linux, with the target
being `x86_64-unknown-linux-gnu`. For building from Mac, it uses `cargo
zigbuild`, so `zig` is now a dependency. Building for this target is
needed because the docker image is an Ubuntu image, so it will need to
run a linux binary.
4. Added `zig` as dependency in docs.
5. CI still uses the docker image built by the CI itself, which builds
the Cargo project inside of it. The CI can take advantage of caching for
this.
This PR replaces `pallet-validator-set` with a new
`pallet-external-validators` pallet from Tanssi.
## Key Changes
- **New ExternalValidators Pallet**
- Supports whitelisted validators (set by governance, not rewarded)
- Manages external validators (can be enabled/disabled)
- Implements era-based rotation (eras change after configurable
sessions)
- **Bridge Integration**
- Updated `EigenLayerMessageProcessor` in
`operator/primitives/bridge/src/lib.rs`
- Replaced the old `SetValidators` command with
```rust
ReceiveValidators { validators, external_index }
```
In this PR:
1. Add [Polkadot API](https://papi.how/) support to `test/` directory
(i.e. the e2e CLI and testing framework) to be able to interact with the
Substrate chain.
1. This allows typed interactions with transactions, constants, storage,
runtime APIs, and non-typed interaction with RPC methods.
2. Types are autogenerated when running `bun i`, from the
`datahaven.scale` file that is part of this repo's version control.
2. Add new utilities file to `papi` related functionalities. For the
time being, generating a new signer from a private key.
3. ~Add a new step to the CLI that sends a transaction to the DataHaven
network. _*THIS SHOULD BE REMOVED SOON, IT'S JUST FOR TESTING
PURPOSES*_~
1. Both steps that send test transactions have been removed from the
CLI, for convenience and being error prone. Their scripts remain usable
for testing purposes if needed.
4. Removes the `apis.rs` files from the runtime definitions. Having them
in a separate file meant that the runtime was not including the Runtime
APIs in the metadata blob, preventing `papi` from creating types from
it. This change can be reapplied after upgrading to `polkadot-sdk`
`stable-2503`. More info
[here](https://github.com/paritytech/polkadot-sdk/issues/6659).
5. Add script to re-generate types, and corresponding docs.
6. Makes logger synchronous to avoid prints happening before logs.
Add support for multiple runtimes: `stagenet`, `testnet` & `mainnet`.
- Moved common types to `datahaven-runtime-common` crate.
- Made the node's command & service code generic over different
runtimes. Each runtime has a `dev` & `local` genesis config preset.
- More types / constants can be moved to the `datahaven-runtime-common`
crate ... this will be done in subsequent PRs.
---------
Co-authored-by: Gonza Montiel <gon.montiel@gmail.com>
Co-authored-by: Gonza Montiel <gonzamontiel@users.noreply.github.com>
Co-authored-by: Facundo Farall <37149322+ffarall@users.noreply.github.com>
This PR implements functionality to store outbound message commitments
in the BEEFY MMR leaf extra field for cross-chain verification. A new
`pallet-outbound-commitment-store` has been introduced to facilitate
this process.
## Changes
- Added a new `pallet-outbound-commitment-store` pallet to capture and
store outbound message commitments
- Implemented the `CommitmentHandler` to receive commitments from the
outbound queue and store them in the new pallet
- Updated the `LeafExtraDataProvider` to include these commitments in
the BEEFY MMR leaf extra field
## Implementation Details
The process works as follows:
1. The outbound queue generates a commitment hash for messages
2. The commitment is stored in the `pallet-outbound-commitment-store`
via the `CommitmentHandler`
3. The `LeafExtraDataProvider` retrieves the latest commitment and
includes it in the BEEFY MMR leaf extra field
4. This commitment can then be verified by other chains using the BEEFY
light client
The new pallet provides the necessary functions to store and retrieve
these commitments.
This PR introduces the Snowbridge `system-v2` pallet and associated
runtime components
**Key Changes:**
* **Added `system-v2` Pallet:** Integrated the
`snowbridge-pallet-system-v2` pallet, providing functionalities for the
Ethereum side of the bridge.
* **Runtime API Integration:**
* Implemented the `ControlV2Api` trait in the runtime
(`operator/runtime/src/apis.rs`) to allow looking up the `AgentId`
associated with a `VersionedLocation`.
* **System V1 Compatibility:** Added the `system-v1` pallet
(`snowbridge-pallet-system`) and related configuration/code references
in various locations.
**Important:** This `system-v1` is included *solely* because the
`system-v2` pallet requires it for compilation and compatibility. It is
**not functionally used** in this runtime.
---------
Co-authored-by: Facundo Farall <37149322+ffarall@users.noreply.github.com>
This PR adds Ethereum RPC API support to the node, enabling the
interaction with the node using standard Ethereum tools.
1. Integration of Frontier's Ethereum RPC modules (eth, net, web3,
txpool)
2. Added RPC configurations necessary for Ethereum compatibility
3. Implemented a BABE consensus data provider for handling pending
blocks
4. Added required dependencies and configurations in the node's service
5. Created necessary filter pools, block data caches, and notification
systems
6. Configured the RPC system to handle Ethereum API calls
7. Changed the transaction pool implementation to be compatible with the
fork-aware transaction pool
(https://github.com/paritytech/polkadot-sdk/pull/4639)
EDIT:
The new `transaction_pool` in `polkadot-stable2412` that comes with a
`fork-aware` feature, allowing different views of the transaction pool,
is not compatible with the current state in Frontier.
In Frontier the only supported pool is the `BasicPool`, that maintains a
similar behaviour to the one in `polkadot-stable2409`. I used this pool
directly.
---------
Co-authored-by: Ahmad Kaouk <56095276+ahmadkaouk@users.noreply.github.com>
Co-authored-by: Facundo Farall <37149322+ffarall@users.noreply.github.com>
This pull request integrates the snowbridge-pallet-outbound-queue-v2 and
its dependencies into the DataHaven runtime. This pallet is responsible
for handling the submission, processing, and commitment of outbound
messages destined for the Ethereum network via the Snowbridge bridge
(v2).
### Key Changes
#### Added New Pallets
- **snowbridge-pallet-outbound-queue-v2**: The core pallet for managing
the outbound message lifecycle.
- **pallet-message-queue**: Introduced to handle the queuing and
processing of messages.
#### Added Crates to Datahaven codebase
- snowbridge-merkle-tree
- bridge-hub-common
- snowbridge-outbound-queue-v2-runtime-api
---------
Co-authored-by: Facundo Farall <37149322+ffarall@users.noreply.github.com>
This PR adds the [pallet
parameters](https://docs.rs/pallet-parameters/0.11.0/pallet_parameters/index.html)
to the DataHaven runtime.
This allows config parameters for pallets to be dynamically modified,
needed, for instance, to set `EthereumfGatewayAddress` after the
deployment of that smart contract on Ethereum. StorageHub also uses this
pallet extensively.
Additionally, refactors the pallet configuration files to have them more
visually organised.
This PR adds the Snowbridge Inbound Queue V2 pallet to the runtime. It
also defines bridge primitives, including the message structure for
EigenLayer, and introduces the EigenLayerMessageProcessor to handle
messages coming from EigenLayer.
---------
Co-authored-by: Facundo Farall <37149322+ffarall@users.noreply.github.com>
Based on https://github.com/paritytech/polkadot-sdk/pull/5868.
Note: this (somehow) breaks the ability to run a local network with the
zombienet config. It seems that the config generated by Zombienet that
automatically set the balance of validators is leading to an issue
(because of the address type).
---------
Co-authored-by: Facundo Farall <37149322+ffarall@users.noreply.github.com>
The pallet_author_inherent was used for finding the author of the
current block, used in pallet_evm, among others, but it was tightly
coupled to the Nimbus protocol. Luckily, Babe also provides a
find_author that we can use for the same purpose.
