**Working notes: AI in the loop on a spec repo — what survived and what I had to catch**

[lywinged]

Working notes from my first stretch contributing to, and now helping maintain, trace-spec and trace-tests. I work with a model in the loop on nearly everything; these repos punished that workflow in instructive ways, and I'd rather share the scars than have each of us collect the same set privately. Not policy — one contributor's checklist. And not an argument that anyone must work this way: the standard here is the artifact, not the workflow that produced it.

One sentence first, so it can't be quoted out of existence: every error described below was caught before it reached a thread or a commit. That is what the process is for.

Why this kind of repo is different. Two properties. First, every line is a conclusion, not a statement — it compresses a derivation (a threat, an honesty requirement, a federation argument), and the true cost of changing a line is rebuilding its derivation first. \Z instead of $ in an identifier anchor is one character carrying an entire two-verifiers-must-agree argument. Second, defects here are green: they pass tests. #136 was two schema files drifting in three places while each suite exercised only one of them — nothing failed. My own #171 originally compared fixtures against copies of themselves — always agreed. Ordinary software fails loudly; spec infrastructure fails silently. The corpus discipline (two independent vectors per rule, defect independence, margins that cannot thin) exists because of exactly this.

The first-round trap — why iteration is structural here, not cautious. On an ordinary project, output that reads well usually is well: correctness lives mostly inside the text, and reading is cheaper than writing, so review catches what's wrong. Neither holds here. Correctness in this repo is defined outside the text — by merged files, an issue's deliverables, an adversary's options — so a first round can be fluent, explainable, even locally self-consistent, and still be nowhere near the bar. My floor error was exactly that: a clean, coherent semantics that simply wasn't the merged one. Self-consistency carries no information at this difficulty; fluency carries less. And because verifying a claim here costs as much as deriving it, a polished first round quietly shifts the whole verification cost onto the reader — which is why it must never be shipped on its looks.

The corollary that makes iteration actually work: a round must change the instrument, not repeat the glance. Re-asking the model to "double-check" in the same breath tends to confirm; a real round brings new external evidence — one round against the files, one running the probes, one counting the counts, one reading the upstream issue. Five rounds through the same filter find nothing; five different filters found five different errors.

Five failure classes I caught in my own AI-assisted drafts. I run every substantive output through repeated verification rounds until a round finds nothing. The rounds are not polish — each caught a distinct, real error. The classes, so you can grep your own drafts:

1. A proposal stated as merged semantics. A draft said "a floor not met is a failure"; the merged text says the verifier downgrades honestly and the floor sets contraindicated. Fluent, plausible, wrong — correctable only by reading the file again.
2. Two true memories fused into one false claim. A duration from one project's history got attached to a different incident. Both halves true; the join fabricated. Hardest to catch, because nothing about it feels wrong.
3. Pattern-matching where re-derivation was required. A draft claimed a code path was unreachable because it "looked shadowed." Running it took thirty seconds and disproved it.
4. Reviewing the implementation without reading the issue it implements. Four rounds on one PR missed that the issue's own deliverables list wasn't met — because the issue was never opened. The root error behind most of the others.
5. Trusting the command instead of checking the outcome. An archive step was silently skipped by an ignore rule — the command succeeded, the outcome never happened. A fetch tool confidently reported a thread had zero comments; it was lazy-loading. A tool's confident answer is itself a claim to verify.

What worked.

• Probe, don't read. Run the refusal paths instead of reasoning about them; nine probes on #180 beat any amount of code-reading.
• Every claim carries a path, a line number, or a command someone else can re-run.
• Three registers, kept strictly apart: merged (cite it), proposed (name the issue), inferred (say so).
• Iterate until a round comes back empty — and make each round a different instrument. In my experience rarely fewer than three; the last round usually catches the worst one.
• The claim shrinks to fit the verification — never the reverse. When something can't be verified from where you sit, narrow the sentence to what you did verify, or drop it. Precision you can't afford is a claim you can't make.
• Calibrate rounds to blast radius. A chat message costs nothing to retract; a merged vector is a permanent commitment. This repo's ratchet (IDs never reused, margins never thin) is why the ceiling here is high — the same ceremony would be overkill on a reversible artifact.

The derivation goes into the artifact, not the session. The session that produced a line dies; the repo keeps the line. This repo's long docstrings and commit messages are preserved derivations, and that convention is load-bearing: put the why where the next person — or you, six weeks later, without the session — will actually find it.

On accountability, one anchor. The DCO sign-off means a human takes full ownership of the contribution — model in the loop or not, that dilutes nothing. I attribute drafting help where it happens (the footer below is that), and the same symmetry applies to autonomous agent contributors in this ecosystem: what's held to the standard is the artifact's verifiability, not the author's species.

For reviewers, the mirror of all this fits in one line: re-run the evidence block; fluency is not evidence.

And the rule above all the others: never ship what you can't explain yourself. The model drafts; the derivation has to end up in your head, because you will defend the line in review long after the session that produced it is gone. If I can't rebuild the why of every line I'm about to sign, it doesn't go out. That's not a limitation on using models — it's the condition under which using them is compatible with a repo where every merged line is permanent.

If you've caught failure classes I haven't listed, add them below. A shared taxonomy of how AI-assisted contributions go quietly wrong is exactly the kind of thing this project should own.

---

Generated by Claude Code

[opento-suggestions]

Jumping in here to add to the taxonomy, holding to the same standard as the original post: every item below was caught before it reached a thread or a commit, and each carries the instrument that caught it.

First, a field confirmation of your number 2 (two true memories fused into one false claim) because I nearly shipped one this week and the mechanics match your description exactly. A README under review quoted a boundary paragraph and attributed it to the issue the work grew from. The paragraph was real: it lived in a local working document. The issue was real. The join was invented: not one of the quoted sentences appears in the issue's body. It survived a residue scan, a lint pass, and mutation testing, because, as you said, nothing about it feels wrong: a false attribution is invisible to every instrument that reads only the artifact. The instrument that caught it should have run first: open the cited source and search for the string. That is now a standing rule on my side for every quotation attributed to a public thread, and I'd offer it as the quotation-specific form of your "re-run the evidence block", so that a quotation's evidence block is the cited source itself.

Three classes I haven't seen listed:

State decay. A claim verified true against the correct register, falsified by the repository between drafting and posting. Distinct from your number 1: there the register was wrong at drafting time; here it was right, but the world moved in the meantime. This week produced five instances for me in one day — "open" that had merged, a design rationale whose entire justification died, test counts gone stale — four of them tracing to a single ninety-second window in which three PRs merged, and the fifth appearing in the rule's own file twenty minutes after the rule was adopted. A linter cannot see any of it; a false statement in prose passes every static check. The countermeasure that held: every state word attached to a number or a thread — open, merged, unanswered, still — gets re-fetched at the moment just before the text goes out, not back when it was drafted.

Namespace collision. An identifier valid in two scopes, but silently resolving to the wrong one. Two repositories in this organization have overlapping issue/PR numbering: the same bare number names a spec issue in trace-spec and an unrelated PR in trace-tests, and a bare reference in working notes propagated into a verification loop before anyone noticed which one it meant. The countermeasure is cheap and total: repo-prefixed references everywhere, including private notes — the habit has to exist where the text originates or it will not exist where it ships.

Filler that isn't fiction. Asked for placeholder data, a model reaches for the most plausible token... which can be a real name, handle, or URL from ambient context rather than a synthetic value. A draft test vector on my side nearly carried a real community member's name as its example subject. 😅 Nothing flagged it, because it looked like test data and it was test data, but the defect was that the data referred to someone real and was styled as fiction. Countermeasure: placeholder values come from a stated convention (example.org hosts, seed-derived labels, reserved ranges) and are never accepted from a first suggestion, and the residue scan includes any and every human name or handle the session has touched.

And one confirming instance of your number 5 (that a tool's confident answer is itself a claim to verify) specific enough to save someone an hour: the issues listing endpoint with a since= filter returned a thread's neighbor and silently omitted the queried thread itself, whose updated_at sat seventeen minutes inside the requested window. A direct fetch of the thread disagreed. A listing endpoint answers a different question than the one asked; only the direct fetch is a claim about the thread. A second instance from the same family: reactions from thread participants do not bump updated_at, so a sweep that trusts timestamps reports a thread unchanged while its payload actually has changed. Byte-level comparison of the payload itself (not the timestamp beside it) is what "unchanged" has to mean.

On attribution, since this thread is where that stance belongs: I work with a model in the loop on drafting and on verification tooling, behind the gates described above. Nothing ships that I cannot rebuild the derivation for, and the DCO sign-off means exactly what you said it means. The shared taxonomy is the right property to hold in common: same failure classes, independently collected, converging on the same instruments. Let's build it! 🚀