Skip to main content

Oracle Operator Manual

This document is intended for those who wish to participate in the Lido protocol as entity that runs - an entity who runs a daemons synchronizing state from Beacon Layer to Execution Layer of the protocol. Due to the lack of native communication between these two networks, Lido employs a network of oracles to synchronize the system at regular intervals.

TL;DR​

  1. Generate an Ethereum address.
  2. Launch and sync an archive (archive data for at least 2 weeks) Execution Layer node with JSON-RPC endpoint enabled.
  3. Launch and sync an archive Consensus Layer node with API endpoint enabled.
  4. Launch and sync a Keys API Service.
  5. Launch the accounting and ejector modules of Oracle.
  6. [Optional] Add alerts to Oracle's Prometheus metrics.
  7. In case of mainnet share your address and intention to join the Oracle set with public. You need to publish it on Twitter and also write a message with a twitter link under Onboarding post on the Research forum.
  8. Propose your Oracle's ethereum address to Lido Team to vote on your address being added to the Oracle Members.

Intro​

The Lido Oracle mechanism comprises three main components. The first component is the Oracle smart-contract suite, which receives update reports from the oracles and passes them on to the Lido contract to execute the necessary actions based on the reported changes. The second component is the off-chain oracle daemon, run by each oracle node and responsible for monitoring the protocol state and generating update reports. The third component is the network of computer nodes that run by oracle member, which collectively provide the necessary information to the Oracle smart contract to calculate the new state of the protocol.

Based on the update reports received from the oracles, the Lido smart contract performs state transitions such as updating user balances, processing withdrawal requests, and distributing rewards to node operators. Thus, the Lido Oracle mechanism acts as a synchronization device that bridges the protocol across the execution and consensus layers. It ensures that the protocol is updated in a timely and accurate manner and allows for smooth and efficient operation of the entire Lido system.

The two core contracts in the Lido Oracle suite are called AccountingOracle and ValidatorsExitBus. Together, these contracts collect information submitted by oracles about the state of validators and their balances, the amount of funds accumulated on protocol vaults, the number of withdrawal requests the protocol is able to process, and the validators are expected to be voluntary exited to finalize more withdrawal requests. This information is then used for these crucial processes:

  • rebasing user balances,
  • distributing node operator rewards,
  • processing withdrawal requests,
  • making decision which validators should initiate voluntary exit,
  • distributing stake,
  • putting the protocol into the bunker mode.

Oracle phases​

In order to send the report data by the oracle operator to both AccountingOracle and ValidatorsExitBusOracle, it is necessary that:

  • this operator participates in the oracle committee, and
  • a consensus for the corresponding report must be reached

A process of sending the report data can be divided into 3 major stages:

Phase 1. Submitting a report hash and reaching consensus​

At the first stage, the oracles operators collect a report for a certain refSlot and send the hash to the HashConsensus contract.

The diagram below shows: ReportProcessor - AccountingOracle or ValidatorsExitBusOracle contract. HashConsensus - a contract which manages oracle members committee and allows the members to reach consensus on the particular data hash for each reporting frame.

You can read more about HashConsensus here.

Phase 2. Submitting a report data​

When the consensus is reached, one of the oracles operators submits report data and triggers the core protocol state update (including the token rebase, distribution of node operator rewards, finalization of withdrawal requests, and deciding whether to go in the bunker mode) or emits ValidatorExitRequest events to inform node operators about new voluntary exit requests needed to perform.

Phase 3. Submitting a report extra data​

This step is required for AccountingOracle, involving reward distribution for staking modules on this phase.

Committee membership​

The current Oracle set consists of 9 participants:

The quorum is 5/9. This means that the report finalization can only occur when there are 5 identical reports from the 5 different oracle members.

See Expansion of Lido on Ethereum Oracle set post for more details.

Prerequisites​

Execution Client Node​

To prepare the report, Oracle fetches up to 10 days old events, makes historical requests for balance data and makes simulated reports on historical blocks. This requires an archive execution node. Oracle needs two weeks of archived data.

ClientTestedNotes
Geth--gcmode=archive
--syncmode=snap

OR

--gcmode=archive
--syncmode=full
NethermindNot tested yet
BesuUse
--rpc-max-logs-range=100000
--sync-mode=FULL
--data-storage-format="FOREST"
--pruning-enabled
--pruning-blocks-retained=100000
params
ErigonUse
--prune=htc
--prune.h.before=100000
--prune.t.before=100000
--prune.c.before=100000
params

Consensus Client Node​

To calculate some metrics for bunker mode Oracle needs archive consensus node.

ClientTestedNotes
LighthouseUse --reconstruct-historic-states param
LodestarNot tested yet
NimbusNot tested yet
PrysmUse
--grpc-max-msg-size=104857600
--enable-historical-state-representation=true
--slots-per-archive-point=1024
params
TekuUse
--data-storage-mode=archive
--data-storage-archive-frequency=1024
--reconstruct-historic-states=true
params

Keys API Service​

This is a separate service that uses Execution Client to fetch all lido keys. It stores the latest state of lido keys in database.

Lido Keys API repository.

The oracle daemon​

The Oracle daemon is a Python application that contains two modules:

  • Accounting module
  • Ejector module

The oracle source code is available at https://github.com/lidofinance/lido-oracle.

Modules fetch the reportable slot, and if this slot is finalized, calculate and send the report to AccountingOracle and ExitBusOracle smart contracts.

Environment variables​

The oracle daemon requires the following environment variables:

Required

  • EXECUTION_CLIENT_URI - list of Execution Client uris separated with comma. The second and next uris will be used as fallback.
  • CONSENSUS_CLIENT_URI - list of Consensus Client uris separated with comma. The second and next uris will be used as fallback.
  • KEYS_API_URI - list of Key API client uris separated with comma. The second and next uris will be used as fallback.
  • LIDO_LOCATOR_ADDRESS - Lido Locator smart contract address.

Optional

One of:

  • MEMBER_PRIV_KEY - Private key of the Oracle member account.
  • MEMBER_PRIV_KEY_FILE - A path to the file contained the private key of the Oracle member account.

Full list could be found here.

Lido Locator address​

Mainnet 0xC1d0b3DE6792Bf6b4b37EccdcC24e45978Cfd2Eb

Holešky 0x28FAB2059C713A7F9D8c86Db49f9bb0e96Af1ef8

Running the daemon​

Startup accounting module

docker run -d --name lido-oracle-accounting \
  --env "EXECUTION_CLIENT_URI=$EXECUTION_CLIENT_URI" \
  --env "CONSENSUS_CLIENT_URI=$CONSENSUS_CLIENT_URI" \
  --env "KEYS_API_URI=$KEYS_API_URI" \
  --env "LIDO_LOCATOR_ADDRESS=$LOCATOR_ADDRESS" \
  --env "MEMBER_PRIV_KEY=$MEMBER_PRIV_KEY" \
  lidofinance/oracle@<image-hash> accounting

Startup ejector module

docker run -d --name lido-oracle-ejector \
  --env "EXECUTION_CLIENT_URI=$EXECUTION_CLIENT_URI" \
  --env "CONSENSUS_CLIENT_URI=$CONSENSUS_CLIENT_URI" \
  --env "KEYS_API_URI=$KEYS_API_URI" \
  --env "LIDO_LOCATOR_ADDRESS=$LOCATOR_ADDRESS" \
  --env "MEMBER_PRIV_KEY=$MEMBER_PRIV_KEY" \
  lidofinance/oracle@<image-hash> ejector

Latest image hash https://docs.lido.fi/guides/tooling/#oracle

This will start the oracle in daemon mode. You can also run it in a one-off mode, for example if you’d prefer to trigger oracle execution as a cron job. In this case, set the DAEMON environment variable to 0.

Metrics and Alerts​

How to set up alerts and details about metrics could be found here.