Skip to content

verus: parser idempotence (parse ∘ print ∘ parse == parse), three phases - #17

Open
kiranandcode wants to merge 5 commits into
kg/parser-roundtrip-verifyfrom
kg/parser-idempotence
Open

verus: parser idempotence (parse ∘ print ∘ parse == parse), three phases#17
kiranandcode wants to merge 5 commits into
kg/parser-roundtrip-verifyfrom
kg/parser-idempotence

Conversation

@kiranandcode

Copy link
Copy Markdown
Collaborator

Stacked on #15. This proves parser idempotence at the token level: re-printing what you parsed and parsing that again lands exactly where the first parse did.

The roundtrip work in #15 fixes one direction, parse(print(a)) == a for a printable AST a. This branch adds the complementary, input-driven property:

for every token sequence toks,
if parse(toks) = Some(a) then parse(print(a)) = parse(toks).

i.e. parse ∘ print ∘ parse == parse, over both the expression grammar and the full statement grammar, axiom-free.

Three phases

The proof composes three phases, all in src/sql/parser/verified_idempotence.rs:

  • Phase 1, the parse image is printable. A successful parse yields a printable AST. This is the new work: bottom-up image lemmas over every sub-parser (columns, the FROM join tree, assignments, select items, rows, order lists, the SELECT clauses), each recovering the parser's own structural guarantees (is_stable right children, CROSS/predicate coupling, list len >= 1) from the parse itself. Embedded expressions reuse the expression-level image lemma.
  • Phase 2, the printer right-inverts. printable(a) ==> parse(print(a)) = (Some(a), []). This is the existing roundtrip lemma, reused verbatim.
  • Phase 3, idempotence. Compose the two.

The finite-float fragment

Phase 1 is not universally true, and the reason is worth stating plainly. The headlines carry a guard, finite_floats / finite_floats_stmt, and it is the tightest honest precondition rather than a proof convenience. Two things carve it out, and they are different in kind.

Overflow floats are a genuine non-printability. A number token like 1e400 parses to Float(inf), and literal_views(Float(inf)) is None. An infinite float has no canonical print, so there is nothing to reparse.

Aliased * is a domain guard, not a printer limitation. SELECT * AS a parses to (All, Some("a")), which printable_select_item rejects. But * AS a prints and reparses back to itself, so it roundtrips. The clause sits in the printable predicate because the canonical select grammar does not alias *, as SQL does not, and it is load-bearing here only because Phase 3 routes through printable_stmt.

On the finite-float fragment, which is all real SQL, the guard is discharged and idempotence holds unconditionally.

Result

lemma_parse_idempotent_expr_finite and lemma_parse_idempotent_stmt_finite state the headline: parse(print(a)) == parse(toks) with the natural finite_floats guard in place of the opaque printable. verify.sh stays green across all 19 modules; the idempotence module is 53 verified, 0 errors. No assume, admit, or external_body; the trust surface is unchanged.

🤖 Generated with Claude Code

kiranandcode and others added 5 commits August 28, 2026 14:39
New module verified_idempotence.rs states token-level parse idempotence,
`parse ∘ print ∘ parse == parse`, in three phases:

- Phase 1 (parse image is printable): `parse(toks) = Some(a) ==> printable(a)`.
  Stated as the obligation that makes the headlines unconditional; the one new
  structural induction (literal leaf aside). Not yet discharged.
- Phase 2 (printer right-inverts): the existing roundtrip lemmas
  `lemma_sparse_sprint` / `lemma_sparse_stmt_sprint`, reused verbatim.
- Phase 3 (idempotence): compose 1 and 2.

Phases 2 and 3 are proved here for both the expression grammar (SExpr) and the
full statement grammar (SStmt): four headlines, a full-consume exact-equality
form (`parse(print(a)) == parse(toks)`) and a tail-agnostic AST-component form,
each currently carrying `printable(a)` as the Phase-1 hypothesis. Axiom-free —
the headlines only compose already-proved lemmas.

verify.sh: 561 verified, 0 errors (was 557; +4 lemmas). Module opted into
verify.sh; cargo build/clippy/fmt clean.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…nite guard)

Phase 1 (parse image is printable) is true structurally and on integer/keyword/
string literals, but NOT universally: an overflow number token like `1e400`
parses to Literal::Float(inf), which printable_literal rejects (demands
is_finite) because Rust prints inf as "inf" and that re-lexes as an identifier.
So the parser can emit a non-printable AST that fails to roundtrip. The
`printable` hypothesis on the Phase-3 headlines is therefore necessary, not a
stopgap — it carves out exactly the finite-float fragment the printer can
reproduce, the same load-bearing finite guard the float-trust decision rests on.

Doc-only; verify.sh 561 verified, 0 errors.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…tability

Adds the finite-float fragment predicate `finite_floats(e)` (constrains float
leaves to finite/non-negative, matching printable_literal's float clause;
integer leaves left free) and proves Phase 1 at the expression level:

  parse(toks) = Some(e) && finite_floats(e) ==> printable_se(e)

via three mutually-recursive image lemmas (`lemma_sparse_printable`,
`lemma_sparse_operator_printable`, `lemma_sparse_args_printable`) matching
sparse's (fuel, k) measure. The integer leaf is discharged by `parse_i64_nonneg`
(parse_i64_spec returns `value as i64` for u64 value <= I64_MAX, hence >= 0); the
float leaf follows directly from finite_floats; every other case is structural.

New headline `lemma_parse_idempotent_expr_finite`: unconditional expression
idempotence on the finite-float fragment (all real SQL number literals) —
`parse(print(e)) == parse(toks)` with `finite_floats(e)` in place of the opaque
`printable_se(e)`. Axiom-free.

verify.sh: 568 verified, 0 errors (was 561). Statement-level Phase 1 (the
larger sparse_stmt induction) remains as follow-up.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…printability

Lifts parser-idempotence Phase 1 from expressions to the full statement grammar.
Adds finite_floats_stmt (and a finite_floats_* family for columns, FROM trees,
assignments, select items, rows, order lists) and proves, axiom-free:

  sparse_stmt(toks, fuel) = (Some(s), rest) && finite_floats_stmt(s)
      ==> printable_stmt(s)

via bottom-up image lemmas for every sub-parser (columns, FROM join tree with
fold-reassembly, from-list, assign/set-list, expr-list, order-list, select
item/list, rows, the three SELECT clause helpers), each recovering the parser's
structural guarantees (is_stable right children, CROSS/predicate coupling, list
len >= 1) from the parser image. The dispatcher lemma_sparse_stmt_printable
covers all 10 kinds (Explain recursive); embedded expressions reuse the
expression-level lemma_sparse_printable.

New headline lemma_parse_idempotent_stmt_finite: unconditional statement
idempotence on the finite-float fragment — parse(print(s)) == parse(toks) with
finite_floats_stmt(s) in place of printable_stmt(s).

Two load-bearing guards, both genuine (a non-printable AST the parser can emit):
float finiteness (overflow number tokens -> inf), and `All ==> no alias`
(`SELECT * AS a` parses to an aliased `*` the printer cannot reproduce). Every
other printability constraint is discharged from the parser image.

verify.sh: exit 0 (all 19 modules); module 53 verified, 0 errors. No
assume/admit/external_body; fmt/clippy/build clean. (+1146 lines, one file.)

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…rintable

My earlier doc claimed SELECT * AS a "cannot be reproduced by the printer". That
is wrong: it prints as `* AS a` and the mirror parser reparses it to
(All, Some("a")), so it roundtrips. The `All => no alias` clause is a domain
guard matching the canonical/production select grammar (SQL does not alias `*`),
load-bearing only because Phase 3 routes through printable_stmt. The genuine
non-printability is the infinite-float case (literal_views(Float(inf)) == None).

Doc-only.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant