Skip to content

Improve account code pruning performance #2444

Description

@sergerad

Title

prune_account_codes re-scans all public accounts on every block apply.

This should be done after (or stack with) #2438.

Context

prune_history runs inside every block-apply transaction and prunes three tables: account_vault_assets, account_storage_map_values, and account_codes. With the validity-interval schema (valid_until intervals, migration 004), the vault and storage prunes are incremental: they range-scan a partial cleanup index for valid_until <= cutoff, and every row found is deleted in the same statement, so each block's scan only visits rows that expired since the previous prune. Cost is O(per-block write churn).

The codes prune is not incremental:

DELETE FROM account_codes
WHERE code_commitment NOT IN (
    SELECT DISTINCT code_commitment
    FROM accounts INDEXED BY idx_accounts_code_validity
    WHERE code_commitment IS NOT NULL
      AND valid_until > ?1   -- ?1 = cutoff (prune_tip − HISTORICAL_BLOCK_RETENTION)
);

The subquery recomputes the full "still referenced" set from scratch each block. Deleting nothing does not shrink the next scan: the valid_until > cutoff range permanently contains every open-ended (current) row — roughly one index entry per public account — plus recent churn.

Cost analysis

  • The scan is index-only over the covering partial index idx_accounts_code_validity (valid_until, code_commitment) WHERE code_commitment IS NOT NULL (verified via EXPLAIN QUERY PLAN: USING COVERING INDEX), and the outer NOT IN probes only account_codes, which is small (unique code blobs, heavily shared across accounts).
  • Ballpark at ~100ns per index entry:
    • 10k public accounts → ~1ms per block: negligible.
    • 1M public accounts → tens of ms per block, inside the block-apply transaction, on the write path.

This is not a regression — the previous is_latest-based query scanned the same magnitude per block — but after the interval refactor it is the only prune whose cost scales with total state rather than churn.

Proposed fix

Make it churn-driven (structural).
The only codes that can newly become collectable at a given prune are those referenced by accounts rows whose valid_until crossed the cutoff since the previous prune. Sketch:

  1. Writer threads the previous prune cutoff through to prune_history (prune_tip is monotone; initialize conservatively to 0 after restart, which degrades to one full-range pass).
  2. Collect candidate commitments from rows with valid_until in (prev_cutoff, cutoff] — a tiny range, O(churn).
  3. For each candidate, delete its code iff no row with valid_until > cutoff still references it — an indexed existence probe, which needs a new (code_commitment, valid_until) index.

This makes the codes prune O(churn) per block like the other two, at the cost of new state plumbing and an extra index on a large table.

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Fields

Priority

None yet

Projects

Status
In progress

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions