Skip to content

Dedicated proof lower-bound pipeline for op-reth - #357

Merged
tonatoz merged 7 commits into
mainfrom
feat/proofs-sync-status
Sep 8, 2026
Merged

tonatoz merged 7 commits into
mainfrom
feat/proofs-sync-status

Conversation

@tonatoz

@tonatoz tonatoz commented Sep 2, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Proofs get their own lower-bound detector. EvmProofLowerBoundDetector asks, in order and on every cycle: debug_proofsSyncStatus (op-reth historical proof store), then eth_capabilities, then the eth_getProof binary search. It also sets the label historical_proofs=true. One sync-status call replaces the binary search on an op-reth node.

The separate pipeline exists because eth_capabilities is unreliable for proofs. op-reth on op-sepolia reports stateproofs.oldestBlock == head.number while --proofs-history serves a window 129600 blocks deep; Ethereum geth reports a sane value. The proof detector therefore trusts stateproofs.oldestBlock only when it is below the head.number of the same report, and goes to the search otherwise. ProofBound is gone from the generic EvmLowerBoundDetector entirely.

The window's upper edge is not published. Measurements on live nodes (op-reth v2.4.2 on optimism and op-sepolia, reth v2.5.2 on base-sepolia) show latest == head +- 2 blocks with a sliding window of 129600 blocks, and earliest == head - 129600. The upper edge therefore follows from the head that nodecore already tracks, and a separate bound type would only duplicate it. This PR carried such a type in earlier revisions; it is gone from nodecore, and drpcorg/public removed LOWER_BOUND_PROOF_UPPER in v1.3.1 with enum 11 reserved and unused.

Changes

  • deps: github.com/drpcorg/public v1.2.1 -> v1.3.1, which ships the debug_proofsSyncStatus spec entry with cacheable: false.
  • evm bounds: new EvmProofLowerBoundDetector in proof_bound.go with the three-stage pipeline and no cached verdicts. Each cycle is request -> parse -> result or error: a rejected or malformed sync status simply falls through to the next stage, and an upstream without the method pays one rejected request per 3 minutes. That replaces the mutex, the unsupported flag and the hourly re-probe timer of the deleted EvmProofsSyncStatus (review comment on proofs_sync_status.go:37).
  • evm bounds: ProofBound = earliest (earliest = 0 is coerced to 1, because 0 reads as "unknown" to routing). An empty window (latest = 0, or earliest > latest while the store initialises) is not a bound and falls through.
  • evm bounds: EvmLowerBoundDetector loses the proof stage and the proofsSyncStatus field; its order is now eth_capabilities -> gold bound -> search. The shared JSON-RPC helper (call, fetchLatestHeight) moved into an embedded evmRpcClient value both detector types use.
  • evm bounds: the eth_capabilities snapshot now carries the reported head.number (0 when absent or unparseable). A report without a head cannot be validated, so its stateproofs value is unusable for the proof bound.
  • evm bounds: parseEvmBlockNumber accepts both shapes a block number arrives in - a quoted hex string ("0x1a") and a bare decimal number (26). Live nodes answer decimal, which the hex parser would read as 0x26.
  • wiring: the proof detector is attached whenever the chain spec has eth_getProof. The hasMethod("debug_proofsSyncStatus") gate is dropped for it, because the detector asks the method every cycle and treats a rejection as "next stage". The historical_proofs label detector now uses the same eth_getProof gate: the label states what backs that method, and debug_proofsSyncStatus sits in the base eth-json-rpc spec every EVM chain inherits, so its gate excluded nothing while chains that disable eth_getProof (viction, hyperliquid) still ran the detector.
  • labels: EthHistoricalProofsLabelsDetector publishes historical_proofs=true when the method answers with a window (even an empty one - the store exists), false when the upstream definitely lacks the method, and nothing on transient or malformed answers, so the last verdict stands. Unchanged by the refactor.
  • methods: debug_proofsSyncStatus joins probedMethods, so a node that lists the debug module but is not op-reth stops advertising it.

No protocol, flow, chain-aggregation or emerald mapping changes: ProofBound behaves like every other lower bound.

Verification

go build ./... is clean, go vet ./internal/upstreams/... is clean, gofmt -l internal is empty, and go test ./internal/upstreams/lower_bounds/... ./internal/upstreams/chains_specific/... ./internal/upstreams/labels/... passes.

Eight tests in proof_bound_test.go cover the pipeline: a sync-status window yields the bound with one upstream call; decimal input with earliest = 0 yields 1; a rejected sync status (-32601, textual "Method not found", unclassified error) falls to eth_capabilities and is asked again on the next cycle (2 requests over 2 cycles, no memoized verdict); malformed and empty windows fall to capabilities; stateproofs.oldestBlock == head.number and a report without a head both fall to the search; disabled: true publishes nothing; a detector without capabilities searches directly.

Runtime check against a fake op-reth on chain optimism (head 5000000, window 129600):

  • Sync-status mode: upstream 'fake-opreth' lower bound of type PROOF is 4870400 - exactly head - 129600. eth_getProof call count on the fake node: 0.
  • op-sepolia shape (sync status answered with -32601, eth_capabilities with stateproofs.oldestBlock == head.number): the log shows upstream 'fake-opreth' eth_capabilities reports proofs from 5000000 with head 5000000, ignoring it for the proof bound, then lower bound of type PROOF is 4000000 - the height the eth_getProof search finds on the fake node, never the head.
  • After two 3-minute cycles against that node, debug_proofsSyncStatus was requested twice, so nothing is memoized in the production wiring.
  • upstream 'fake-opreth' label of historical_proofs is true in sync-status mode.
  • debug_proofsSyncStatus through the HTTP entrypoint returned {"earliest": 4870400, "latest": 5000000}, and each call raised the upstream's call counter, so the response is not cached.

Notes for reviewers

  • The implementation plan is deleted and replaced by docs/superpowers/specs/2026-09-07-proof-lower-bound-design.md, which describes the shipped three-stage detector. The plan specified LOWER_BOUND_PROOF_UPPER, a memoizing sync-status source and a gRPC mapping, none of which shipped.
  • The plan asked for a mirrored proto edit in emerald-grpc/proto/blockchain.proto. That submodule was removed in Method specs and dshackle proto from public package #353, nothing imports github.com/drpcorg/emerald-grpc, and the authoritative proto now comes from drpcorg/public, so the mirror was skipped.
  • Nothing downstream needs a new enum: LOWER_BOUND_PROOF already exists in every client.

@tonatoz
tonatoz requested a review from KirillPamPam September 2, 2026 15:51
@tonatoz
tonatoz force-pushed the feat/proofs-sync-status branch from 55e7f12 to d3e1980 Compare September 7, 2026 10:33
@tonatoz tonatoz changed the title Support op-reth debug_proofsSyncStatus and the upper proof bound Read the proof lower bound from op-reth debug_proofsSyncStatus Sep 7, 2026
@tonatoz
tonatoz force-pushed the feat/proofs-sync-status branch from d3e1980 to 126a756 Compare September 7, 2026 10:48
connector connectors.ApiConnector
reprobeInterval time.Duration

mu sync.Mutex

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's omit these fields. It could be come down to the simple logic: request -> request -> parsing -> error/result

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done: the memoizing fields are gone. proofs_sync_status.go is deleted and the proof stage now lives in EvmProofLowerBoundDetector.detectFromProofsSyncStatus as plain request -> parse -> result/error, with no mutex, no unsupported verdict and no re-probe timer. An upstream without the method rejects one call per 3-minute cycle and the detector moves to the next stage.

While removing the cache I also split proofs off the generic detector: eth_capabilities is wrong for them on op-reth (op-sepolia reports stateproofs.oldestBlock == head.number while serving a 129600-block window), so the new pipeline is debug_proofsSyncStatus -> eth_capabilities trusted only below the reported head.number -> eth_getProof search. Updated description and tests in 148c2ce.

@tonatoz tonatoz changed the title Read the proof lower bound from op-reth debug_proofsSyncStatus Dedicated proof lower-bound pipeline for op-reth Sep 7, 2026
@tonatoz
tonatoz requested a review from KirillPamPam September 7, 2026 14:58
v1.3.1 ships the debug_proofsSyncStatus spec entry for eth-json-rpc chains and
drops the unused LOWER_BOUND_PROOF_UPPER bound type, leaving enum 11 reserved.
op-reth exposes the block window its historical proof store serves. One call
replaces the eth_getProof binary search, so the source runs before
eth_capabilities and before probing.

Only the window's earliest block is published, as ProofBound. Measured op-reth
and reth nodes keep latest at the head (+-2 blocks on optimism, op-sepolia and
base-sepolia over a 129600-block window), so the upper edge follows from the
head and needs no bound of its own.

An upstream that rejects the method as unknown, or answers with an unparseable
body, is remembered as unsupported and re-asked hourly. Transient failures and
an empty window leave the verdict alone and fall back to the other sources.

Live nodes report earliest and latest as decimal numbers, so
parseEvmBlockNumber accepts both a quoted hex string and a bare decimal.
An upstream that answers debug_proofsSyncStatus with a usable window gets the
label historical_proofs=true, so clients can select proof-serving nodes.

The chain specifics attach both the label detector and the sync-status bound
source only when the upstream advertises the method, and the method probe asks
for debug_proofsSyncStatus during capability detection.
eth_capabilities is unreliable for proofs: op-reth on op-sepolia reports
stateproofs.oldestBlock at head.number while --proofs-history serves a
window 129600 blocks deep. Give proofs their own detector instead of
putting debug_proofsSyncStatus in front of the generic chain.

EvmProofLowerBoundDetector asks, every cycle and with no cached verdict:
debug_proofsSyncStatus, then eth_capabilities (trusted only when
stateproofs.oldestBlock is below the head of the same report), then the
eth_getProof binary search. A rejected sync-status call costs one request
per 3 minutes and replaces the mutex, the unsupported flag and the
re-probe timer of the removed EvmProofsSyncStatus.

ProofBound leaves EvmLowerBoundDetector entirely; the JSON-RPC helper
both detectors need moves into the embedded evmRpcClient. The
eth_capabilities snapshot now carries the reported head. The
historical_proofs label detector is unchanged and keeps its spec gate.
The label states what backs eth_getProof, so it must follow that method.
debug_proofsSyncStatus lives in the base eth-json-rpc spec every EVM
chain inherits, so the old gate never excluded anything, while chains
that disable eth_getProof (viction, hyperliquid) kept running a detector
whose verdict describes a method they do not serve.
…plan with a spec

The window type, response type and parse function existed to hand back a
single earliest block. Inline the parse, drop the null check the call
helper already performs, and fix a %w of a nil error.

Delete the implementation plan: it specified LOWER_BOUND_PROOF_UPPER, a
memoizing sync-status source and a gRPC mapping, none of which shipped.
The new spec describes the three-stage detector as built.
@tonatoz
tonatoz force-pushed the feat/proofs-sync-status branch from 9b75c84 to 4d42695 Compare September 8, 2026 09:48
@tonatoz
tonatoz merged commit 81c3639 into main Sep 8, 2026
5 checks passed
@tonatoz
tonatoz deleted the feat/proofs-sync-status branch September 8, 2026 09:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants