Skip to content

docs: CLAUDE.md — PR prose closes issues too, via undocumented keyword inflections - #313

Closed
rubenhensen wants to merge 2 commits into
mainfrom
docs/closing-keyword-inflections
Closed

docs: CLAUDE.md — PR prose closes issues too, via undocumented keyword inflections#313
rubenhensen wants to merge 2 commits into
mainfrom
docs/closing-keyword-inflections

Conversation

@rubenhensen

Copy link
Copy Markdown
Contributor

Found while working the map ticket that archived cryptify; part of #247.

The import-closes bullet already covers two vectors, both needing an imported history. There is a third that needs nothing at all. PR #311's body opened with the words "Closing note for" in front of a bare issue reference, meaning it as a pointer. Merging it shut that issue one second later, commit_id: null — the PR-body attribution the same bullet documents.

closing appears in none of GitHub's three documented keyword lists (close/closes/closed, fix/fixes/fixed, resolve/resolves/resolved) and behaved like one regardless. Two consequences worth writing down:

  • The documented list is a floor, not the set. Keep every inflection away from a bare #N in a PR body unless you mean it.
  • The audit grep the bullet already prescribes, (close[sd]?|fix(e[sd])?|resolve[sd]?), does not match participles — it would have passed this body clean. The bullet now carries a wider pattern for PR bodies.

No harm done in this instance: the close was wanted, and an audit of everything closed in that window confirmed nothing else went with it — but it was luck, not the phrasing.

…d inflections

"Closing note for #294" at the top of #311's body shut #294 one second
after the merge. `closing` is not in GitHub's documented keyword list but
behaves like one, and the grep the import-audit bullet prescribes does not
match participles, so it would not have caught this.
@dobby-coder

dobby-coder Bot commented Aug 9, 2026

Copy link
Copy Markdown
Contributor

Got it — I'm on this. I'll review PR #313 with the third close vector in mind (the bare #N reference read as a closing pointer via PR-body attribution, plus the note that the audit grep needs widening to catch participles like "closing") and follow up with a consolidated review.

@dobby-coder
dobby-coder Bot requested a review from leonbotros August 9, 2026 12:58

@dobby-coder dobby-coder Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Docs-only, +1/-1 on CLAUDE.md. Title is conventional-commit compliant (docs:; pr-title.yml does run here). Prose is clean against the writing rules. part of #247 in the body correctly avoids auto-closing the map ticket.

The blocker is the new bullet's evidence. The third vector rests on closing having behaved as an undocumented closing keyword, and PR #311's body carries a literal Closes #294 trailer on its own line — the ordinary documented keyword, present and sufficient. One sub-claim is verifiably false, and the remedy the bullet prescribes would not catch the phrasing it says it fixes. Detail inline.

This is the reproduction the same bullet asks for, two sentences earlier: "Confirm it on a throwaway repo and record the answer here before relying on it." A scratch repo, a PR whose body says only Closing note for #1 and carries no trailer, would settle it either way. If it reproduces, the bullet is right and needs a wider grep than the one it currently proposes. If it does not, the honest bullet is the opposite one — a deliberate Closes trailer closed the issue as designed.

Two smaller notes, neither blocking:

  • If the behaviour does reproduce, its natural home is the shared pr-close-issue-keywords rule rather than this repo's CLAUDE.md alone. "GitHub's keyword matcher accepts inflections the docs do not list" fires in every repo in both orgs, and single-repo-lessons-belong-in-that-repos-claude-md puts the general core in rules/ with only the postguard specifics here. Worth moving in the same PR that confirms it.
  • Pre-existing, not introduced here — recording it so the next review does not rediscover it. cargo test --workspace --all-features cannot compile on a host target: error: "web" feature should only be enabled on wasm32 targets at pg-core/src/client/web/mod.rs:19. Plain cargo test --workspace passes clean. Since this file already documents an --all-features invocation for the api_gate suite, a short note that --all-features is wasm32-only workspace-wide would save the next person the failed run.

Comment thread CLAUDE.md Outdated
- `pg-pkg/api-description.yaml` is the pinned v2 HTTP contract and is additive-only (see `COMPATIBILITY.md`). Its breaking-change gate is `.github/workflows/api-diff.yml`, which runs on every PR as a **two-spec matrix** — `pg-pkg/api-description.yaml` (versioned `/v2` routes) and `cryptify/api-description.yaml` (unversioned) — so the check names carry the spec path. The matrix is not cosmetic: `pg-pkg/tests/api_gate.rs` and cryptify's in-crate `mod api_gate_tests` both grep this file for exactly **one** `fail-on:` and one `include-checks:` line, so two separate oasdiff steps would give two of each and fail both suites. Vary only the spec paths; moving `fail-on` into the matrix removes the literal both tests look for. `pg-pkg/tests/api_gate.rs` is the executable record of what the gate stops on the pg-pkg side: it mutates the spec 19 ways and asserts each verdict. The verdict test skips when `oasdiff` is not on `PATH`, which is the case in CI; the other two run everywhere, one checking that every mutation still finds its anchor and one reading `fail-on`/`include-checks`/the pinned action ref back out of the workflow YAML, so editing the step and the constants apart fails the suite instead of quietly weakening the gate. The action ref is pinned because it is what selects the engine version (`v0.1.10` is `FROM tufin/oasdiff:v1.26.1`), so bumping the action silently re-measures every verdict. Those anchors are multi-line raw strings matched against the spec read off disk, which is why root `.gitattributes` marks `pg-pkg/api-description.yaml -text`: rustc normalises CRLF inside raw strings but `read_to_string` does not, so without it a Windows checkout fails `every_mutation_still_applies` on a clean tree. Run it (and read it) before touching the gate's settings: `go install github.com/oasdiff/oasdiff@v1.26.1 && cargo test --manifest-path pg-pkg/Cargo.toml --all-features --test api_gate`.
- The oasdiff gate's settings are **not** self-evident and `--fail-on ERR` alone fails open. `fail-on: WARN` is deliberate: oasdiff rates removing or renaming an *optional* response property, and removing a request parameter, as WARN, and this spec marks only `status` as `required`, so at ERR the gate silently passed a removed `key` (the IBE user secret key `/v2/request/key` exists to return), a renamed `proofStatus` and a dropped `timestamp` parameter. Two more, a changed non-success status (401 to 403) and a dropped response enum value, rate ERR but are **opt-in**, so they only run when named in `include-checks`. Of the 31 WARN checks, `response-property-enum-value-added` is the only one that fires on something `COMPATIBILITY.md` does not already forbid, which is why that document now names a new response enum value as non-additive too. Reproduce a verdict with the exact flags the action's entrypoint builds (`oasdiff/oasdiff-action/breaking@v0.1.10` is `FROM tufin/oasdiff:v1.26.1`, so the pinned tag is what makes a local run authoritative): `git show origin/main:pg-pkg/api-description.yaml > /tmp/base.yaml && oasdiff breaking /tmp/base.yaml pg-pkg/api-description.yaml --allow-external-refs=false --composed=false --fail-on WARN --include-checks response-non-success-status-removed,response-property-enum-value-removed`. Two traps: `oasdiff --version` prints `oasdiff version main` after a `go install` of a tag because the version comes from release ldflags (the code is still the tag), and `--fail-on` takes `ERR`/`WARN` while `oasdiff checks --severity` takes `error`/`warn`/`info`. `--severity ERR` is a usage error, so `oasdiff checks --severity ERR | wc -l` counts the help text instead and badly undercounts the tier (it is 213 error checks, 31 warn, 265 info). The spec has no external `$ref`s, so `allow-external-refs` stays at its safe (SSRF-guarding) default. The gate only sees paths the spec documents, and the spec documents canonical paths only, so dropping the `/v2/irma/...` alias handlers (#257) passes it.
- **Importing another repo's history here silently closes issues here. Two vectors fire it, each sufficient on its own, so rule out neither.** Imported commits carry their original messages verbatim, closing keywords included, and GitHub resolves those against the **destination** repo's numbering. **Vector 1, the squash body.** This repo's `squash_merge_commit_message` is `COMMIT_MESSAGES` (the same setting the release-plz bullet above turns on its head — there it is why a `BREAKING CHANGE:` footer in a PR body never reaches the commit), so squash-merging the import PR concatenates every imported commit message into the merge commit's body. `ba380a1`'s body is 1678 lines and carries all 17 refs (`Closes #38 #45 #52 #54 #123 #125 #134 #142 #146 #153 #155 #157 #159 #167 #186 #191 #194`); 16 pointed at numbers already closed here, and one — #146, a live unimplemented feature request — was closed four seconds after it landed, with nothing warning, and stood four days before being found and reopened. Note what this means: a plain squash merge is enough by itself, *because* squashing concatenates the messages even as it throws the history away. **Vector 2, the history itself.** Here GitHub never reads the merge commit's message — it attributes the closes to the **imported commits**, once they become reachable from the default branch. This vector went untested in this repo rather than disproven: `9887e1a` carries no keywords of its own, and by the time it landed thirteen minutes later all 17 targets were already closed, and GitHub does not re-close a closed issue. It is measured in encryption4all/postguard-js#139, which is also the guard: import PR #137 merged with a real two-parent merge commit (`b1bb2ee`, a three-line message, no keywords, no squash body anywhere), and js#128/#129 closed six seconds later attributed to imported commits `a0ce27f`/`2acf42f` — single-parent, committed 2026-06-04 — then reopened fifteen minutes later. **So the audit is the only step that covers both.** Before merging *and* after, from the import branch: `git log origin/main..HEAD --pretty=%B | grep -oiE '(close[sd]?|fix(e[sd])?|resolve[sd]?) +#[0-9]+' | sort -u`, then check each number here. Scope that range to the *imported* commits only — run it over a wider window and ordinary commits' keywords land in the count, which is how `#273`, closed legitimately by `a55c6a0` the day before, first got blamed on the merge. Editing the squash body in the merge dialog defuses vector 1 and only vector 1; on vector 2 there is no body to edit, and rewriting the keywords out of the imported commits costs every imported SHA (`git filter-repo`). Do **not** reach for `commit_message` on `PUT /pulls/N/merge` as the scripted version of that: the REST reference words it "Extra detail to append to automatic commit message" and documents nothing about `merge_method=squash`, so whether it replaces the concatenated body or appends to it is unverified — and if it appends, the defusal silently no-ops and every keyword still fires. Confirm it on a throwaway repo and record the answer here before relying on it. Reading close *state* cannot tell you which happened, so read the close **event** — and read it correctly, because the obvious rule is wrong. A keyword in a *commit message* is attributed to the commit and carries a `commit_id`; a keyword in the *PR body* is attributed to the PR and carries `commit_id: null`. #146 shows `ba380a1464…` because PR #277's body said only `Closes #255`, while #273 shows `null` because PR #274's body said `Closes #273`. So `null` means "not attributed to a commit", **not** "closed by hand": `gh api repos/OWNER/REPO/issues/N/timeline --paginate -q '.[] | select(.event=="closed") | .created_at+" "+(.commit_id//"-")'`. Same failure class as the unapplied-workflow half below (#272).
- **Importing another repo's history here silently closes issues here. Two vectors fire it, each sufficient on its own, so rule out neither.** Imported commits carry their original messages verbatim, closing keywords included, and GitHub resolves those against the **destination** repo's numbering. **Vector 1, the squash body.** This repo's `squash_merge_commit_message` is `COMMIT_MESSAGES` (the same setting the release-plz bullet above turns on its head — there it is why a `BREAKING CHANGE:` footer in a PR body never reaches the commit), so squash-merging the import PR concatenates every imported commit message into the merge commit's body. `ba380a1`'s body is 1678 lines and carries all 17 refs (`Closes #38 #45 #52 #54 #123 #125 #134 #142 #146 #153 #155 #157 #159 #167 #186 #191 #194`); 16 pointed at numbers already closed here, and one — #146, a live unimplemented feature request — was closed four seconds after it landed, with nothing warning, and stood four days before being found and reopened. Note what this means: a plain squash merge is enough by itself, *because* squashing concatenates the messages even as it throws the history away. **Vector 2, the history itself.** Here GitHub never reads the merge commit's message — it attributes the closes to the **imported commits**, once they become reachable from the default branch. This vector went untested in this repo rather than disproven: `9887e1a` carries no keywords of its own, and by the time it landed thirteen minutes later all 17 targets were already closed, and GitHub does not re-close a closed issue. It is measured in encryption4all/postguard-js#139, which is also the guard: import PR #137 merged with a real two-parent merge commit (`b1bb2ee`, a three-line message, no keywords, no squash body anywhere), and js#128/#129 closed six seconds later attributed to imported commits `a0ce27f`/`2acf42f` — single-parent, committed 2026-06-04 — then reopened fifteen minutes later. **So the audit is the only step that covers both.** Before merging *and* after, from the import branch: `git log origin/main..HEAD --pretty=%B | grep -oiE '(close[sd]?|fix(e[sd])?|resolve[sd]?) +#[0-9]+' | sort -u`, then check each number here. Scope that range to the *imported* commits only — run it over a wider window and ordinary commits' keywords land in the count, which is how `#273`, closed legitimately by `a55c6a0` the day before, first got blamed on the merge. Editing the squash body in the merge dialog defuses vector 1 and only vector 1; on vector 2 there is no body to edit, and rewriting the keywords out of the imported commits costs every imported SHA (`git filter-repo`). Do **not** reach for `commit_message` on `PUT /pulls/N/merge` as the scripted version of that: the REST reference words it "Extra detail to append to automatic commit message" and documents nothing about `merge_method=squash`, so whether it replaces the concatenated body or appends to it is unverified — and if it appends, the defusal silently no-ops and every keyword still fires. Confirm it on a throwaway repo and record the answer here before relying on it. Reading close *state* cannot tell you which happened, so read the close **event** — and read it correctly, because the obvious rule is wrong. A keyword in a *commit message* is attributed to the commit and carries a `commit_id`; a keyword in the *PR body* is attributed to the PR and carries `commit_id: null`. #146 shows `ba380a1464…` because PR #277's body said only `Closes #255`, while #273 shows `null` because PR #274's body said `Closes #273`. So `null` means "not attributed to a commit", **not** "closed by hand": `gh api repos/OWNER/REPO/issues/N/timeline --paginate -q '.[] | select(.event=="closed") | .created_at+" "+(.commit_id//"-")'`. Same failure class as the unapplied-workflow half below (#272). **A third vector needs no import at all: ordinary PR prose, because the keyword matcher accepts inflections the docs do not list.** PR #311's body opened "Closing note for #294" — meant as a pointer, not a directive — and merging it shut #294 one second later, `commit_id: null`, the PR-body attribution above. GitHub documents `close`/`closes`/`closed`, `fix`/`fixes`/`fixed`, `resolve`/`resolves`/`resolved`; `closing` is in none of those lists and behaved like one anyway, so treat the documented list as a floor rather than the set. In a PR body, keep **every** inflection of those three verbs away from a bare `#N` unless you mean it — write "context for #N", "part of #N", "follow-up to #N" — and grep the body before merging, with the participles included: `grep -oiE '(clos(e[sd]?|ing)|fix(e[sd]|ing)?|resolv(e[sd]?|ing)) +#[0-9]+'`. The narrower `(close[sd]?|fix(e[sd])?|resolve[sd]?)` pattern the import audit above uses does not catch this one.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Three problems with the new third-vector text, in descending order of weight.

1. The central claim is unsupported — the documented keyword is present and sufficient.

PR #311's body carries a literal Closes #294 trailer on its own line (body line 10 of 11):

gh pr view 311 --repo encryption4all/postguard --json body -q .body | sed -n '10p'
# -> Closes #294

That alone explains the close. GitHub agrees it was a body-attributed closing reference:

gh api graphql -f query='{repository(owner:"encryption4all",name:"postguard"){pullRequest(number:311){mergedAt closingIssuesReferences(first:10){nodes{number}}}}}'
# -> mergedAt 2026-08-09T12:47:04Z, closingIssuesReferences: [294]

and #294 has exactly one close event, 2026-08-09T12:47:05Z with commit_id: null — one second after the merge, precisely the PR-body attribution this bullet already documents. Because the trailer and the Closing note for opener point at the same issue, GitHub emits one close event either way, so this incident cannot distinguish them. The documented cause is present; the undocumented one is not evidenced.

There is also positive evidence against the participle theory, from our own measurements. pr-close-issue-keywords records that the keyword must immediately precede the reference, measured on privacybydesign/irmago#592: Closes the investigation in irmamobile#579 did not auto-close — humans closed it by hand, commit_id: null. Closing note for #294 has note for sitting between the verb and the reference, so under the adjacency behaviour we have already measured it would not fire even if closing were a keyword. Yet the bullet states it as measured fact and generalises to "the documented list is a floor, not the set".

2. "The narrower pattern ... does not catch this one" is verifiably false.

It catches it on the first try — the Closes #294 trailer is a match:

gh pr view 311 --repo encryption4all/postguard --json body -q .body > /tmp/311.body
grep -oiE '(close[sd]?|fix(e[sd])?|resolve[sd]?) +#[0-9]+' /tmp/311.body
# -> Closes #294   (exit 0)

The PR description's "it would have passed this body clean" is the same error. Since this is the stated motivation for widening the pattern, the justification does not hold as written.

3. The quote is inaccurate, and the new pattern would not catch the quoted form either.

The body's first line actually reads Closing note for [#294](https://github.com/encryption4all/postguard/issues/294) — a markdown-linked reference, not the bare Closing note for #294 quoted here. The newly prescribed pattern returns nothing against it:

sed -n '1p' /tmp/311.body | grep -oiE '(clos(e[sd]?|ing)|fix(e[sd]|ing)?|resolv(e[sd]?|ing)) +#[0-9]+'
# -> no output (exit 1)

# and against the whole body with the Closes trailer removed:
sed '10d' /tmp/311.body | grep -oiE '(clos(e[sd]?|ing)|fix(e[sd]|ing)?|resolv(e[sd]?|ing)) +#[0-9]+'
# -> no output (exit 1)

It fails on two independent counts: the intervening words note for , and the [#294](...) link form rather than a bare #294. So the audit command this bullet prescribes would still have passed this very body clean — the exact failure it claims to fix. Advising readers to keep inflections away from a bare #N compounds it, since it steers them away from the one form that was not used while leaving the linked form (the form actually in the body) unmentioned.

If the participle behaviour does reproduce on a throwaway repo, the pattern needs to tolerate both intervening words and [#N](...) links to be worth prescribing.

No suggestion block here on purpose — the right replacement text depends on which way that scratch-repo test comes out.

@dobby-coder

dobby-coder Bot commented Aug 9, 2026

Copy link
Copy Markdown
Contributor

Thanks for writing this up — but I had to request changes on postguard#313, because the evidence for the third vector doesn't hold up under checking.

PR #311's body actually carries a literal Closes #294 trailer, so the documented keyword was already there and sufficient — GitHub's own closingIssuesReferences and the close event both point at that same attribution, with nothing left for "closing" to explain. Our own pr-close-issue-keywords rule cuts the other way too: it requires the keyword to immediately precede the reference, and here the text is "Closing note for #294" — a markdown link with words in between, which wouldn't fire that rule even if "closing" counted. And the claim that the narrower grep pattern "would have passed this body clean" isn't true either — it matches the Closes #294 trailer on the first try, so the stated motivation for widening the pattern doesn't survive.

Left one inline comment on CLAUDE.md:20 with the details, and pointed back to the bullet's own ask: confirm this on a throwaway repo (a PR body with only "Closing note for #1", no trailer) before it goes in as fact — happy to take another look once that's settled. Also flagged a pre-existing, unrelated cargo test --workspace --all-features compile issue for the next reviewer, non-blocking.

Full review: #313 (review)

dobby's review on #313 showed PR #311's body carries a literal `Closes
#294` trailer, which fully explains the close (commit_id: null matches
the existing PR-body-attribution note) and which the existing narrow
grep already matches — so the "third vector" evidence didn't hold up.

Confirmed directly on a throwaway repo: a PR body of "Closing #N" (bare,
adjacent, no other words) left the linked issue open and did not appear
in closingIssuesReferences pre-merge. "closing" is not a closing
keyword at all, in any form tested. Reverting to the pre-#313 bullet.
@rubenhensen

Copy link
Copy Markdown
Contributor Author

Confirmed — the third-vector claim doesn't hold up, on all three points raised.

Checked directly:

Went further and tested the underlying hypothesis directly on a throwaway repo, since the review flagged it as unverified: opened a PR with body Closing #N (bare, adjacent, no intervening words, no trailer) against an open issue and merged it. closingIssuesReferences was empty pre-merge and the issue stayed open post-merge. So closing isn't a closing keyword in any form tested — not just miscounted in this one incident.

Reverted CLAUDE.md to the pre-#313 bullet (639c924); the branch now diffs to nothing against main. Given there's no surviving claim to add, I'll leave it to a maintainer whether to close this PR outright or repurpose it — happy to do either.

@rubenhensen

Copy link
Copy Markdown
Contributor Author

Closing — the underlying claim didn't survive review + a direct throwaway-repo test (see comments above). Reverted to a no-op diff against main, nothing to merge.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant