Title
prune_account_codes re-scans all public accounts on every block apply.
This should be done after (or stack with) #2438.
Context
prune_history runs inside every block-apply transaction and prunes three tables: account_vault_assets, account_storage_map_values, and account_codes. With the validity-interval schema (valid_until intervals, migration 004), the vault and storage prunes are incremental: they range-scan a partial cleanup index for valid_until <= cutoff, and every row found is deleted in the same statement, so each block's scan only visits rows that expired since the previous prune. Cost is O(per-block write churn).
The codes prune is not incremental:
DELETE FROM account_codes
WHERE code_commitment NOT IN (
SELECT DISTINCT code_commitment
FROM accounts INDEXED BY idx_accounts_code_validity
WHERE code_commitment IS NOT NULL
AND valid_until > ?1 -- ?1 = cutoff (prune_tip − HISTORICAL_BLOCK_RETENTION)
);
The subquery recomputes the full "still referenced" set from scratch each block. Deleting nothing does not shrink the next scan: the valid_until > cutoff range permanently contains every open-ended (current) row — roughly one index entry per public account — plus recent churn.
Cost analysis
- The scan is index-only over the covering partial index
idx_accounts_code_validity (valid_until, code_commitment) WHERE code_commitment IS NOT NULL (verified via EXPLAIN QUERY PLAN: USING COVERING INDEX), and the outer NOT IN probes only account_codes, which is small (unique code blobs, heavily shared across accounts).
- Ballpark at ~100ns per index entry:
- 10k public accounts → ~1ms per block: negligible.
- 1M public accounts → tens of ms per block, inside the block-apply transaction, on the write path.
This is not a regression — the previous is_latest-based query scanned the same magnitude per block — but after the interval refactor it is the only prune whose cost scales with total state rather than churn.
Proposed fix
Make it churn-driven (structural).
The only codes that can newly become collectable at a given prune are those referenced by accounts rows whose valid_until crossed the cutoff since the previous prune. Sketch:
- Writer threads the previous prune cutoff through to
prune_history (prune_tip is monotone; initialize conservatively to 0 after restart, which degrades to one full-range pass).
- Collect candidate commitments from rows with
valid_until in (prev_cutoff, cutoff] — a tiny range, O(churn).
- For each candidate, delete its code iff no row with
valid_until > cutoff still references it — an indexed existence probe, which needs a new (code_commitment, valid_until) index.
This makes the codes prune O(churn) per block like the other two, at the cost of new state plumbing and an extra index on a large table.
Title
prune_account_codesre-scans all public accounts on every block apply.This should be done after (or stack with) #2438.
Context
prune_historyruns inside every block-apply transaction and prunes three tables:account_vault_assets,account_storage_map_values, andaccount_codes. With the validity-interval schema (valid_untilintervals, migration 004), the vault and storage prunes are incremental: they range-scan a partial cleanup index forvalid_until <= cutoff, and every row found is deleted in the same statement, so each block's scan only visits rows that expired since the previous prune. Cost is O(per-block write churn).The codes prune is not incremental:
The subquery recomputes the full "still referenced" set from scratch each block. Deleting nothing does not shrink the next scan: the
valid_until > cutoffrange permanently contains every open-ended (current) row — roughly one index entry per public account — plus recent churn.Cost analysis
idx_accounts_code_validity (valid_until, code_commitment) WHERE code_commitment IS NOT NULL(verified viaEXPLAIN QUERY PLAN:USING COVERING INDEX), and the outerNOT INprobes onlyaccount_codes, which is small (unique code blobs, heavily shared across accounts).This is not a regression — the previous
is_latest-based query scanned the same magnitude per block — but after the interval refactor it is the only prune whose cost scales with total state rather than churn.Proposed fix
Make it churn-driven (structural).
The only codes that can newly become collectable at a given prune are those referenced by
accountsrows whosevalid_untilcrossed the cutoff since the previous prune. Sketch:prune_history(prune_tipis monotone; initialize conservatively to 0 after restart, which degrades to one full-range pass).valid_untilin(prev_cutoff, cutoff]— a tiny range, O(churn).valid_until > cutoffstill references it — an indexed existence probe, which needs a new(code_commitment, valid_until)index.This makes the codes prune O(churn) per block like the other two, at the cost of new state plumbing and an extra index on a large table.