fix(core): queue all future blocks of a batch instead of failing - #2550
fix(core): queue all future blocks of a batch instead of failing#2550gzliudan wants to merge 5 commits into
Conversation
|
Important Review skippedAuto reviews are disabled on base/target branches other than the default branch. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Team Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
There was a problem hiding this comment.
🟡 Changes recommended
A verification error following queued future blocks is still discarded by the final nil-error return.
Once you've addressed the issues Copilot identified, you can request another Copilot review.
Pull request overview
Queues entire future-block batches to prevent downloader peer eviction near the chain tip.
Changes:
- Accepts
ErrFutureBlockwhile queuing descendants. - Adds tests for first/mid-batch queuing and resumed import.
File summaries
| File | Description |
|---|---|
core/blockchain.go |
Expands future-block queue handling. |
core/blockchain_futureblocks_test.go |
Tests queuing and later processing. |
Review details
- Files reviewed: 2/2 changed files
- Comments generated: 1
- Review effort level: Balanced
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
633d5cc to
8d97961
Compare
There was a problem hiding this comment.
🟡 Changes recommended
Two error paths can omit bad-block reporting or silently report an incomplete import as successful.
Once you've addressed the issues Copilot identified, you can request another Copilot review.
Review details
Suppressed comments (1)
core/blockchain.go:1959
- These exemptions currently suppress both the bad-block report and the return. If iteration stops on
ErrPrunedAncestororErrKnownBlock,blockis still unconsumed but execution falls through to the final nil return, recreating a silent partial import. Exempt legitimate states only fromreportBlock; every non-nil iterator error must still be returned.
if err != nil && !errors.Is(err, consensus.ErrFutureBlock) &&
!errors.Is(err, consensus.ErrPrunedAncestor) && !errors.Is(err, ErrKnownBlock) {
bc.reportBlock(block, nil, err)
return it.index, events, coalescedLogs, err
- Files reviewed: 2/2 changed files
- Comments generated: 1
- Review effort level: Balanced
15971f6 to
e960998
Compare
9823b56 to
380e6c9
Compare
bcc88c0 to
c6bb2ed
Compare
…chain.go Replace direct == / != comparisons against sentinel errors (consensus.ErrPrunedAncestor, consensus.ErrFutureBlock, consensus.ErrUnknownAncestor, ErrKnownBlock, ErrStopPreparingBlock) with errors.Is, which also matches wrapped errors.
stats is a function-local insertStats and stats.report is only called from the main import loop, so every stats write on a path that returns before reaching the loop is dead and never logged: - stats.queued += it.processed() on the first-block future path is not only unreachable for reporting but off by one: it.processed() returns it.index+1, which over-counts by one both when the whole batch is drained (it.index == len(chain) at exhaustion) and when the loop aborts on a non-exempt error (it.index points at the block that was never queued). Both queued counters date back to ff435e0. - stats.queued++ in the tail future loop (which also skipped the block that triggered the queueing) and stats.ignored += it.remaining() on both future paths are written then dropped on return. - stats.ignored += len(it.chain) in the first-block error abort has the same write-then-return shape. Remove all of them. Drop the insertStats.queued field and the "queued" log context in report() that could never fire once the counters are gone, matching upstream geth whose insertStats no longer carries a queued field, and remove the now-unused insertIterator helpers processed() and remaining(). The live stats.ignored accounting (the ErrKnownBlock skip loop that falls through to the import loop and the InsertReceiptChain stats) is untouched.
c6bb2ed to
5b4e565
Compare
Add regression tests asserting that header verification checks the timestamp first and answers ErrFutureBlock for blocks too far ahead, without requiring their parent to be resolvable. XDPoS v1/v2 (under full verification) order the checks this way; these tests pin the behavior the future-batch tail queueing (queueFutureTail) relies on: children of a future block surface as ErrFutureBlock, not ErrUnknownAncestor. No production code is changed.
1752135 to
ad6bab5
Compare
…canonical Problem importBlockResults fed the last fetched block to the proposed-block handler as long as the batch import returned nil, taking a nil error to mean the last block reached the canonical chain. A nil InsertChain does not imply canonicality: a fork batch writes its blocks as side-chain entries that a mere existence check cannot tell from canonicality, and processQC overwrites highestQuorumCert before its own existence check (GetHeaderByHash), so a master node can end up voting for a block that is not in the canonical chain. Changes - Gate the proposed-block handler on storage and canonicality: the batch tail must be present as a full block and be the canonical block at its own height, via a new GetCanonicalHash method on the downloader's BlockChain interface. The canonical hash is keyed by height, so it is immune to a concurrent import advancing the head past the tail between the insertion and this read (a CurrentBlock comparison would spuriously skip the handler in that window); only a reorg changes it, and a reorged-away block must not reach the handler. HasBlock covers the fast sync header phase, where the header chain marks a height canonical before its body is imported. - downloadTester gains storeBlock/GetCanonicalHash, a canonical number-to-hash table advanced the way HeaderChain.WriteHeader and insertChain advance theirs, plus parkTailOnce and extendTailAfterInsert hooks so the tests can present batches that never reached the chain or were later reorganized away. Both hooks are consumed by the single InsertChain call they apply to. GetCanonicalHash picks the head by total difficulty, the way the real chain resolves a reorg, instead of by insertion order. HasBlock now also requires the stored block to match the requested height, like the real chain. Tests - TestImportBlockResultsProposedBlockHandler covers six scenarios: a batch that never reached the chain must not trigger the handler, a fully imported batch triggers it exactly once on the batch tail, a stored fork batch re-delivered after the local chain grew past it must not trigger it again, a concurrent import advancing the head past the canonical tail must not skip the handler (height-keyed lookup), a heavier fork stays canonical once a lighter chain is stored after it, and a height that the fast-sync header phase made canonical without a body must not trigger it either.
Problem ------- XDPoS engines check the header timestamp before the parent lookup with zero tolerance, so the children of a future block also fail verification with ErrFutureBlock rather than ErrUnknownAncestor. The insertChain future-block loops only accepted ErrUnknownAncestor, so a batch whose first or middle block was in the future stopped at its second block and returned the error, which the downloader wraps into errInvalidChain and answers by dropping the peer on a valid delivery. Before XinFinOrg#2534 the same scenario silently dropped the tail and failed one batch later with a bogus bad-block report. Changes ------- * Accept ErrFutureBlock in both insertChain future-block loops so the whole tail enters the futureBlocks queue and the import reports success; the queued blocks are imported by procFutureBlocks once their timestamps are reached. Queueing is bounded by addFutureBlock's maxTimeFutureBlocks (30s) window: a tail whose timestamps span past it aborts the batch with the enqueue error (non-sentinel, so the downloader still drops the peer) — the same visible outcome as before — while the in-window prefix stays queued. * Gate HandleProposedBlock on canonicality instead of block existence (importBlockResults now requires the tail to be the current head). A nil InsertChain no longer implies the last block reached the chain, and a queued tail can sit at or below the head height (re-delivered history on a side chain); feeding such a block into processQC overwrites highestQuorumCert before its block-existence check, so a master node could end up voting for a block that is not in the chain. * In the fetcher, move the sign hook, the proposed-block handling and the final broadcast behind a "block exists in the chain" gate. * Evict parked blocks that fail to import: procFutureBlocks used to keep every failed block, so garbage delivered inside the future window could fill the queue permanently and be re-verified and re-reported as bad blocks on every futureBlocksLoop tick forever. A block is now evicted unless it is still in the future or its parent is itself parked; blocks are sorted by number, so evicting a failed parent cascades to its queued children within the same pass. * Collapse the duplicated enqueue loops of the two insertChain future paths into the shared queueFutureTail helper, and drop the dead insertStats accounting on the abort paths (stats.report is only called from the main import loop). Tests ----- Future-block regression tests live in core/blockchain_futureblocks_test.go.
ad6bab5 to
4862bd0
Compare
Summary
insertChaintreats a mid-batchErrFutureBlockas a fatal verification error: both future-block loops only acceptedErrUnknownAncestor, so a batch whose first or middle block was in the future stopped at its second block and returnedconsensus.ErrFutureBlock. The downloader wraps any non-nilInsertChainerror intoerrInvalidChain(eth/downloader/downloader.go) andSynchroniseanswers that by dropping the delivering peer, aborting the round and retrying with another peer. Near the chain tip this repeats and continuously evicts innocent peers.Root cause
The XDPoS engines check the header timestamp before the parent lookup and with zero tolerance (
header.Time > now), unlike upstream geth's 15-secondallowedFutureBlockTimeSeconds. Consequently the children of a future block fail the future check too — they surface asErrFutureBlock, never asErrUnknownAncestor, so the inner loops (which only acceptErrUnknownAncestor) stop at the second block and the rest of the batch is silently skipped.Before #2534 the same scenario silently dropped the tail and failed one batch later with a bogus bad-block report.
Fix
Accept
consensus.ErrFutureBlockin both future-block loops ofinsertChain(first-block path and tail path), so the whole tail enters thefutureBlocksqueue and the import reports success. No error is swallowed: the loops drain naturally to(nil, nil)and genuine verification errors still propagate. The queued blocks are imported byprocFutureBlocks(100 ms ticker) once their timestamps are reached, matching the upstream geth design intent.Blocks beyond the
now+30sfuture window still return an error (abort + peer drop), which is the documented behaviour for a local clock more than 30 seconds behind — an environment problem, not an invalid chain.A genuine verification error on a block after the queued future tail (e.g. a malformed validator field, which the XDPoS engines check before the timestamp) also surfaces: it is recorded as a bad block and returned, so only a fully drained tail reports success. Future, pruned-ancestor and known blocks remain exempt from the bad-block report — they are legitimate chain states. The same applies to a genuine error after the queued prefix when the batch starts with a future block: it is recorded and returned as well. When the known-block import PR lands, its final-return filter reports the same block again;
WriteBadBlockdeduplicates by (number, hash), so the only effect is one extra BAD BLOCK log line.Tests
New
core/blockchain_futureblocks_test.go(self-contained, ownfailRangeEnginewith anatomic.Uint64fail range because the chain's future-block loop callsVerifyHeadersconcurrently):TestInsertChainQueuesMidBatchFutureBlocks— a batch rejected as future mid-way returnsn == len(blocks), err nil, head stays at blocks[1], the whole tail is infutureBlocks, no bad-block records.TestInsertChainQueuesFutureBatchFromFirstBlock— same for a batch whose first block is already in the future (pre-existing scenario).TestInsertChainProcFutureBlocksResumesImport— once headers verify again, the queued tail imports and the head reaches the batch tip (head polled with a deadline becauseInsertChainusesTryLockand the background loop may be mid-import).All three are mutation-verified: reverting either loop condition alone makes the corresponding test fail.
go test ./core/andgo test ./eth/downloader/pass, also under-race;gofmt/go vetclean.Compatibility
No consensus-rule changes: block/receipt structure, state transition and wire protocol are untouched. Only the error handling of the import path changes. No migration or node-operator action required. Suggested label:
consensus.