feat: improve deferral interface + add verify-stark example

[stephenh-axiom-xyz]

Resolves INT-8458.

Overview

This PR adds a reusable cached SDK proving-key flow, makes supported deferral configurations serializable, restructures SDK deferral proving into clearer layers, and adds an end-to-end verify-stark example that reuses cached keys to wrap existing OpenVM STARK proofs. It also updates docs and CI to exercise the new example.

Suggested review order:

1. Deferral config/prover model in crates/sdk-config, crates/sdk, crates/continuations, and guest-libs/verify-stark/circuit.
2. Cached SDK proving-key construction/reconstruction in crates/sdk.
3. The new examples/verify-stark host/guest flow and docs.
4. Smaller memory-config, recursion helper, and test updates.

SDK Proving Key Caching

Files:

• crates/sdk/src/keygen/mod.rs
• crates/sdk/src/lib.rs
• crates/sdk/src/builder.rs
• crates/sdk/src/types.rs

Changes:

• Adds SdkCachedProvingKey<VC>, containing app and aggregation proving keys plus optional deferral, deferral-aggregation, and root proving keys.
• Adds Sdk::cached_proving_key() to extract already-generated or seeded proving keys without triggering additional keygen. Halo2 proving keys remain excluded and still use the Halo2-specific file helpers.
• Adds Sdk::from_cached_proving_key(...) for non-deferral SDKs.
• Adds Sdk::from_deferral_cached_proving_key(...) for SdkVmConfig SDKs with supported deferral proving keys.
• Moves CommitBytes hex serde support into openvm-continuations, so AppExecutionCommit, VkCommitJson, and VerificationBaselineJson no longer need local serializer modules.

Reviewer notes:

• The generic cached-key constructor rejects cached deferral keys because it cannot reconstruct arbitrary deferral prover types.
• The deferral-aware constructor reconstructs supported deferrals from the cached app VM config and errors on missing/incomplete deferral key pairs.
• Custom deferral circuits still need manual prover setup through the builder APIs.

Serializable Deferral Configuration

Files:

• crates/sdk-config/src/deferral.rs
• crates/sdk-config/src/lib.rs
• crates/continuations/src/commit_bytes.rs

Changes:

• Introduces DeferralConfig, DeferralCircuitConfig, and SupportedDeferral.
• Changes SdkVmConfig.deferral from Option<DeferralExtension> to Option<DeferralConfig>.
• Converts DeferralConfig into the lower-level DeferralExtension when building transpiler/execution/prover components.
• Adds SdkVmConfig::to_toml() so programmatically generated deferral commits can be written back to openvm.toml.
• Adds CommitBytes::to_field_le_bytes() for converting serialized deferral commits back to the byte format expected by DeferralExtension.

Reviewer notes:

• SupportedDeferral::VerifyStark is fully reconstructable. SupportedDeferral::Other preserves config shape through serde but installs a panic placeholder unless callers replace it with the real DeferralFn.
• SdkVmConfig::optimize() now keeps the deferral address space when deferrals are configured and zeros it when they are not.

Deferral Prover Restructure

Files:

• crates/sdk/src/prover/deferral/agg.rs
• crates/sdk/src/prover/deferral/multi_circuit.rs
• crates/sdk/src/prover/deferral/single_circuit.rs
• crates/sdk/src/prover/deferral/mod.rs
• crates/sdk/src/prover/stark.rs
• crates/sdk/src/prover/root.rs
• crates/sdk/src/prover/evm.rs
• crates/sdk/src/keygen/dummy.rs

Changes:

• Splits deferral proving into three explicit layers:
  • SingleDeferralCircuitProver: proves one concrete deferral circuit and wraps those proofs into leaf/internal-for-leaf proofs.
  • MultiDeferralCircuitProver: owns the ordered set of deferral circuits and emits hook proofs or absent deferral public values.
  • DeferralAggProver: aggregates hook proofs into the deferral proof consumed by the main SDK prover.
• Replaces the older DeferralProver / DeferralPathProver builder surface with multi_deferral_circuit_prover(...) / deferral_agg_prover(...).
• Moves verify-stark-specific deferral aggregation setup into DeferralAggProver::verify_stark(...).
• Pads per-circuit deferral proof outputs to a power-of-two length with absent proofs.
• Keeps the existing deferral Merkle-proof logic separate in merkle.rs.

Reviewer notes:

• Check that deferral circuit ordering remains index-stable across DeferralConfig.circuits, guest DEF_IDX, MultiDeferralCircuitProver, and cached proving keys.
• Check the fixed-point verify-stark setup in DeferralAggProver::verify_stark(...), especially the dummy circuit use and def_hook_commit derivation.

Deferral Circuit Proving Keys

Files:

• crates/continuations/src/prover/deferral/mod.rs
• guest-libs/verify-stark/circuit/src/prover/mod.rs
• guest-libs/verify-stark/circuit/Cargo.toml

Changes:

• Extends DeferralCircuitProver with get_pk() and from_pk(...).
• Adds DeferralCircuitProverKey, which stores a base STARK proving key plus encoded auxiliary data.
• Implements key extraction/reconstruction for the verify-stark deferral prover.
• Stores verify-stark auxiliary proving data with bitcode, including child VK, cached commit, memory dimensions, public-value count, optional deferral hook commit, and deferral index.
• Changes verify-stark prover constructors to take Option<CommitBytes> for def_hook_commit.

Reviewer notes:

• The aux payload is type-specific. The SDK can reconstruct verify-stark because it knows that prover type; custom deferral provers still need their own reconstruction path.

VM, Recursion, and Memory Config

Files:

• crates/vm/src/arch/config.rs
• crates/vm/src/arch/testing/cpu.rs
• crates/recursion/src/batch_constraint/mod.rs
• crates/recursion/src/system/mod.rs
• extensions/deferral/*

Changes:

• Adds DEFAULT_DEFERRAL_ADDR_SPACE_CELLS = 1 << 14 and reserves the deferral address space in the default MemoryConfig.
• Removes ad hoc deferral address-space overrides from several tests.
• Updates deferral tests to use the shared default deferral address-space size.
• Factors child VK commitment into a generic commit_child_vk(...) helper usable by CPU and GPU paths.

Reviewer notes:

• The default memory config now includes deferral capacity, but SdkVmConfig::optimize() removes it for non-deferral SDK configs.

Verify-Stark Example

Files:

• examples/verify-stark/guest/*
• examples/verify-stark/host/*
• examples/verify-stark/verify-book-examples.sh

Changes:

• Adds a guest program that reads an input commit, calls verify_stark_unchecked::<0>(), and prints the verified app execution output.
• Adds a standalone host crate with keygen, build, and prove commands.
• keygen builds the verify-stark deferral prover, writes a generated openvm.toml, writes the cached SDK proving key, and writes the example aggregation VK.
• build reconstructs the SDK from cached proving keys, builds the guest, writes the VM executable, and writes its verification baseline.
• prove reconstructs the SDK from cached proving keys, prepares guest stdin and deferral input from a child STARK proof, and writes the wrapped verify-stark STARK proof.
• Adds a script that proves each book example, wraps the proof through the verify-stark example, and verifies the wrapped proof with cargo openvm verify stark.

Reviewer notes:

• The host crate declares its own [workspace], so it is not built as part of the root workspace by default.
• The example is CUDA-oriented in CI through VERIFY_STARK_HOST_FEATURES defaulting to cuda.

Docs and CI

Files:

• .github/workflows/cli.yml
• docs/vocs/docs/pages/book/acceleration-using-extensions/deferral.mdx
• docs/vocs/docs/pages/book/advanced-usage/sdk.mdx
• docs/vocs/docs/pages/book/guest-libraries/verify-stark.mdx
• docs/repo/layout.md

Changes:

• Adds a GPU CI job that builds cargo-openvm with CUDA and runs examples/verify-stark/verify-book-examples.sh.
• Updates deferral docs from DeferralExtension-centric setup to DeferralConfig and the new SDK builder methods.
• Documents SDK proving-key caching and the deferral-aware cached-key constructor.
• Adds the verify-stark end-to-end example to the guest-library docs.

Tests Updated

Files:

• crates/sdk/src/tests.rs
• crates/continuations/src/tests/e2e.rs
• extensions/deferral/circuit/src/*/tests.rs
• extensions/deferral/tests/src/lib.rs

Changes:

• Updates SDK deferral tests to use DeferralConfig, SupportedDeferral::VerifyStark, MultiDeferralCircuitProver, and DeferralAggProver.
• Adjusts nested verify-stark tests for the CommitBytes def_hook_commit API.
• Updates deferral circuit tests for the new default deferral address-space behavior.

[socket-security]

Review the following changes in direct dependencies. Learn more about Socket for GitHub.

Diff	Package	Supply Chain Security	Vulnerability	Quality	Maintenance	License
	cargo/​proc-macro2@​1.0.101 ⏵ 1.0.106
	cargo/​quote@​1.0.40 ⏵ 1.0.45	-18
	cargo/​clap@​4.5.45 ⏵ 4.6.1
	cargo/​syn@​2.0.106 ⏵ 2.0.118

View full report

[github-actions]

Code review

No issues found. Checked for bugs and CLAUDE.md compliance.

[github-actions]

group	app.proof_time_ms	app.cycles	leaf.proof_time_ms
fibonacci	3,086	12,000,265	676
keccak	16,298	18,655,329	3,017
sha2_bench	9,097	14,793,960	1,110
regex	1,166	4,137,067	350
ecrecover	605	123,583	286
pairing	937	1,745,757	307
kitchen_sink	4,104	2,579,903	879

Note: cells_used metrics omitted because CUDA tracegen does not expose unpadded trace heights.

Commit: 3120e05

Benchmark Workflow