From 5637838f740cef86a149b587033c593d9d65f56a Mon Sep 17 00:00:00 2001 From: "Jonathan D.A. Jewell" <6759885+hyperpolymath@users.noreply.github.com> Date: Tue, 21 Jul 2026 06:38:01 +0100 Subject: [PATCH 1/5] fix(ci): estate-rescan retried a 403 five times, hiding a frozen fact-store MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The push step retried ANY push failure with backoff (2/4/8/16s) on the assumption of writer contention. But `HYPATIA_DISPATCH_PAT` lacks Contents:write on hyperpolymath/verisimdb-data, so every attempt returned: remote: Permission to hyperpolymath/verisimdb-data.git denied to hyperpolymath fatal: ... The requested URL returned error: 403 A permission error can never clear by retrying. The loop turned a hard, actionable auth failure into five rounds of flaky-looking noise, and the estate fact-store silently went stale: verisimdb-data pushed_at was stuck at 2026-07-06 and was still stale on 2026-07-21 (15 days) while the neurosymbolic pipeline kept reasoning over the frozen snapshot. Retry remains correct for genuine contention, so this narrows rather than removes it: classify stderr, fail fast with an actionable ::error on auth/permission, keep the backoff for everything else. NOTE: this makes the failure legible; it does not grant the permission. The PAT still needs Contents:write on verisimdb-data (read already works — the clone step succeeds — so the token is valid but under-scoped). Verified with a stubbed `git` on PATH, all four paths: 403 denied -> 1 attempt, exit 78 (was 5 attempts, exit 128) contention reject -> 5 attempts, exit 1 (unchanged) immediate success -> 1 attempt, exit 0 (unchanged) transient x2 then ok -> 3 attempts, exit 0 (unchanged, still self-heals) shellcheck clean; workflow YAML re-parsed. Co-Authored-By: Claude Opus 4.8 --- .github/workflows/estate-rescan.yml | 22 +++++++++++++++++++--- 1 file changed, 19 insertions(+), 3 deletions(-) diff --git a/.github/workflows/estate-rescan.yml b/.github/workflows/estate-rescan.yml index 44b618a3..e8394b5e 100644 --- a/.github/workflows/estate-rescan.yml +++ b/.github/workflows/estate-rescan.yml @@ -225,15 +225,31 @@ jobs: # (scans, then index) on top; index regen does not need rerunning # because a concurrent scans push is rare and the freshness gate # would catch any genuine drift. + # A permission failure is NOT contention and will never clear by + # retrying — but the old loop retried it 5x with backoff, so a hard + # 403 presented as flaky-looking noise and the fact-store silently + # went stale (frozen 2026-07-06 .. 2026-07-21 before this was found). + # Retry only on contention; fail fast and loudly on auth/permission. + ERR="$RUNNER_TEMP/push-err.txt" + push_once() { + local rc=0 + git push origin HEAD:main 2>"$ERR" || rc=$? + cat "$ERR" >&2 + if [ "$rc" -ne 0 ] && grep -qiE '403|permission to .+ denied|authentication failed' "$ERR"; then + echo "::error title=verisimdb-data push denied::HYPATIA_DISPATCH_PAT cannot write to hyperpolymath/verisimdb-data. Read works (the clone succeeded), so the token is valid but lacks Contents:write on that repo. Grant it and re-run — retrying cannot fix a permission error." + exit 78 + fi + return "$rc" + } for delay in 2 4 8 16; do - if git push origin HEAD:main; then + if push_once; then exit 0 fi - echo "push failed; rebasing and retrying in ${delay}s" + echo "push failed (treating as contention); rebasing and retrying in ${delay}s" sleep "$delay" git pull --rebase origin main || true done - git push origin HEAD:main + push_once - name: Job summary env: From 47f572116515376bda1a93d7ad20524482acc2e6 Mon Sep 17 00:00:00 2001 From: "Jonathan D.A. Jewell" <6759885+hyperpolymath@users.noreply.github.com> Date: Tue, 21 Jul 2026 15:53:39 +0100 Subject: [PATCH 2/5] =?UTF-8?q?fix(ci):=20the=20verisimdb-data=20403=20is?= =?UTF-8?q?=20a=20RULESET,=20not=20the=20token=20=E2=80=94=20publish=20via?= =?UTF-8?q?=20PR?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Corrects the diagnosis in the previous commit on this branch. That commit made the 403 fail fast instead of retrying it five times, which was right, but it named the wrong cause and told the operator to grant the PAT Contents:write. That would not have worked. ## Actual cause `hyperpolymath/verisimdb-data` carries an active `Base` ruleset on ~DEFAULT_BRANCH whose rules include `pull_request` and `required_signatures`, plus `deletion`, `non_fast_forward` and `required_linear_history`. Its only bypass actor is RepositoryRole(id=5) in mode `pull_request`. So `git push origin HEAD:main` is refused for everyone, admin included, whatever the token can do. Timeline, which is what identified it: Base ruleset created ............ 2026-04-12 bot pushed fine (16 commits) .... through 2026-06-25 17:40 Base ruleset UPDATED ............ 2026-06-26 08:57 bot pushes since ................ none The ruleset was updated ~15h after the last successful bot push. ## Two measurement traps this exposed - The repo's `pushed_at` read 2026-07-21 and looked healthy, because humans kept committing. Bot-data freshness must be judged by the last commit AUTHORED BY THE BOT — which was 2026-06-25, i.e. 26 days stale, longer than the "frozen 2026-07-06" previously recorded. - A hard permission wall retried five times with backoff is indistinguishable from flakiness in the log. ## Fix Push a feature branch (unrestricted — the ruleset targets only the default branch), open a PR, and merge it. That satisfies `pull_request`, and satisfies `required_signatures` because GitHub signs the commits it creates server-side, which a runner's raw `git push` can never do. No ruleset needs weakening and no new token is needed. `--rebase`, not `--squash`: squashing would collapse the scans commit and the index commit into one, breaking the invariant regen-index.sh relies on (its `last_updated` comes from the newest commit touching scans/, so the index must land as a separate commit that does not touch scans/). Rebase also satisfies `required_linear_history`. Adds a postcondition asserting the PR actually reached MERGED. A refused merge that exits 0 is precisely the silent-failure class that let this store go stale for 26 days. The retained 403 handler now covers the branch push only, where a denial genuinely *is* a token problem, and says so. Verified: YAML parses; zero live `push origin HEAD:main` lines remain; shellcheck -S warning on the step body returns 0 with no findings. Co-Authored-By: Claude Opus 4.8 --- .github/workflows/estate-rescan.yml | 95 +++++++++++++++++++++-------- 1 file changed, 68 insertions(+), 27 deletions(-) diff --git a/.github/workflows/estate-rescan.yml b/.github/workflows/estate-rescan.yml index e8394b5e..13dc0ee7 100644 --- a/.github/workflows/estate-rescan.yml +++ b/.github/workflows/estate-rescan.yml @@ -212,44 +212,85 @@ jobs: git commit -m "index: regen after estate rescan" fi - - name: Push to verisimdb-data + - name: Publish to verisimdb-data (branch -> PR -> rebase-merge) if: inputs.dry_run != 'true' && steps.commit_scans.outputs.changed == 'true' env: GH_TOKEN: ${{ secrets.HYPATIA_DISPATCH_PAT }} + RUN_ID: ${{ github.run_id }} run: | set -euo pipefail DATA="$RUNNER_TEMP/verisimdb-data" + REPO="hyperpolymath/verisimdb-data" cd "$DATA" - # Retry with rebase: another writer (ingest.yml, manual push) may - # land between our clone and push. --rebase keeps our two commits - # (scans, then index) on top; index regen does not need rerunning - # because a concurrent scans push is rare and the freshness gate - # would catch any genuine drift. - # A permission failure is NOT contention and will never clear by - # retrying — but the old loop retried it 5x with backoff, so a hard - # 403 presented as flaky-looking noise and the fact-store silently - # went stale (frozen 2026-07-06 .. 2026-07-21 before this was found). - # Retry only on contention; fail fast and loudly on auth/permission. + + # WHY THIS IS NO LONGER A DIRECT PUSH TO main. + # + # `git push origin HEAD:main` CANNOT succeed against this repo, and + # no token change will make it succeed. verisimdb-data carries an + # active `Base` ruleset on ~DEFAULT_BRANCH whose rules include + # `pull_request` and `required_signatures` (plus deletion, + # non_fast_forward, required_linear_history). Its only bypass actor + # is RepositoryRole(id=5) in mode `pull_request` — i.e. nobody may + # push straight to main, admin included. + # + # Measured timeline, which is what identified this: + # Base ruleset created ....... 2026-04-12 + # bot pushed fine (16 commits) through 2026-06-25 17:40 + # Base ruleset UPDATED ....... 2026-06-26 08:57 + # bot pushes since ........... none + # The fact-store then sat 26 days stale while the old loop retried + # the resulting 403 five times, making a hard permission wall look + # like flakiness. + # + # NB the repo's `pushed_at` still looked recent throughout, because + # humans kept committing. Judge bot-data freshness by the last + # commit AUTHORED BY THE BOT, never by pushed_at. + # + # A feature branch is unrestricted (the ruleset targets only the + # default branch), and merging a PR satisfies both `pull_request` + # and `required_signatures` — GitHub signs the commits it creates + # server-side, which a runner's raw `git push` can never do. + + BRANCH="estate-rescan/${RUN_ID}" ERR="$RUNNER_TEMP/push-err.txt" - push_once() { - local rc=0 - git push origin HEAD:main 2>"$ERR" || rc=$? + + if ! git push origin "HEAD:refs/heads/${BRANCH}" 2>"$ERR"; then cat "$ERR" >&2 - if [ "$rc" -ne 0 ] && grep -qiE '403|permission to .+ denied|authentication failed' "$ERR"; then - echo "::error title=verisimdb-data push denied::HYPATIA_DISPATCH_PAT cannot write to hyperpolymath/verisimdb-data. Read works (the clone succeeded), so the token is valid but lacks Contents:write on that repo. Grant it and re-run — retrying cannot fix a permission error." + if grep -qiE '403|permission to .+ denied|authentication failed' "$ERR"; then + echo "::error title=verisimdb-data branch push denied::HYPATIA_DISPATCH_PAT could not create a branch on ${REPO}. The clone succeeded, so the token is valid but lacks Contents:write. Unlike a push to main, this one IS a token problem — the Base ruleset does not restrict non-default branches." exit 78 fi - return "$rc" - } - for delay in 2 4 8 16; do - if push_once; then - exit 0 - fi - echo "push failed (treating as contention); rebasing and retrying in ${delay}s" - sleep "$delay" - git pull --rebase origin main || true - done - push_once + exit 1 + fi + echo "pushed ${BRANCH}" + + PR_URL="$(gh pr create --repo "$REPO" --base main --head "${BRANCH}" \ + --title "scan: estate rescan (run ${RUN_ID})" \ + --body "Automated estate rescan from hyperpolymath/hypatia run ${RUN_ID}. Two commits, deliberately kept separate: a scans-only commit, then an index regen that does not touch scans/ so index-freshness stays green.")" + PR_NUM="${PR_URL##*/}" + echo "opened ${PR_URL}" + + # --rebase, NOT --squash. Squash would collapse the scans commit and + # the index commit into one, which breaks the invariant that + # regen-index.sh depends on (see the index step above): its + # `last_updated` is derived from the newest commit touching scans/, + # so the index must land as a separate commit that does not. + # Rebase also satisfies `required_linear_history`. + if ! gh pr merge "$PR_NUM" --repo "$REPO" --rebase --delete-branch 2>"$ERR"; then + cat "$ERR" >&2 + echo "::error title=verisimdb-data merge refused::Branch ${BRANCH} pushed and PR #${PR_NUM} opened, but the merge was refused. Check the Base ruleset on ${REPO} and that the token may merge pull requests." + exit 78 + fi + + # POSTCONDITION — never assume the merge landed. A refused merge that + # exits 0 is exactly the class of silent failure that let this repo + # go stale for 26 days in the first place. + STATE="$(gh pr view "$PR_NUM" --repo "$REPO" --json state --jq .state)" + if [ "$STATE" != "MERGED" ]; then + echo "::error title=verisimdb-data not updated::PR #${PR_NUM} is ${STATE}, not MERGED. The fact-store did NOT receive this rescan." + exit 1 + fi + echo "verisimdb-data updated via ${PR_URL}" - name: Job summary env: From c3dd802fc49243abeb096c850a4d88715caf7e27 Mon Sep 17 00:00:00 2001 From: "Jonathan D.A. Jewell" <6759885+hyperpolymath@users.noreply.github.com> Date: Tue, 21 Jul 2026 16:15:56 +0100 Subject: [PATCH 3/5] fix(ci): preflight the verisimdb-data write before spending 20 minutes scanning MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The write to verisimdb-data was the LAST step of this job, after a ~300-repo panic-attack scan. Every time the token was wrong, the run burned the whole scan and then died at the final step. That is a large part of why a hard permission wall went unnoticed for 26 days: reaching the failure was expensive, and it presented as a flaky tail-end push rather than as a categorical "this cannot work". Adds a first step that creates a ref on verisimdb-data and immediately deletes it. No content, no commit, about a second. It is the same differential test that finally identified the fault — the one thing that distinguishes a token problem from a ruleset problem, since a ruleset cannot discriminate by token. Skipped when dry_run is set, since a dry run never publishes. The error message names the specific grants required (Contents: Read and write AND Pull requests: Read and write) and notes that a fine-grained PAT must list the repository explicitly rather than inheriting account scope — which is the trap that made the earlier "the token is valid, the clone worked" reasoning misleading. Verified: YAML parses, preflight is step 0, shellcheck -S warning returns 0 with no findings. Co-Authored-By: Claude Opus 4.8 --- .github/workflows/estate-rescan.yml | 29 +++++++++++++++++++++++++++++ 1 file changed, 29 insertions(+) diff --git a/.github/workflows/estate-rescan.yml b/.github/workflows/estate-rescan.yml index 13dc0ee7..2993bcb6 100644 --- a/.github/workflows/estate-rescan.yml +++ b/.github/workflows/estate-rescan.yml @@ -68,6 +68,35 @@ jobs: timeout-minutes: 300 steps: + # Fail in seconds, not in twenty minutes. + # + # The write to verisimdb-data is the LAST thing this job does, after a + # ~300-repo scan. So every time the token was wrong, the run burned the + # entire scan and then died at the final step. That is how a hard + # permission wall stayed unnoticed for 26 days: the failure was + # expensive to reach and looked like a flaky tail-end push. + # + # This probe is the same differential test that finally identified the + # fault: create a ref and delete it again. It touches no content, makes + # no commit, and takes about a second. + - name: Preflight — can we actually write to verisimdb-data? + if: inputs.dry_run != 'true' + env: + GH_TOKEN: ${{ secrets.HYPATIA_DISPATCH_PAT }} + run: | + set -euo pipefail + R="hyperpolymath/verisimdb-data" + PROBE="preflight/${GITHUB_RUN_ID}" + SHA="$(gh api "repos/${R}/git/ref/heads/main" --jq .object.sha)" + if ! gh api -X POST "repos/${R}/git/refs" \ + -f "ref=refs/heads/${PROBE}" -f "sha=${SHA}" --jq .ref >/dev/null 2>"$RUNNER_TEMP/probe-err.txt"; then + cat "$RUNNER_TEMP/probe-err.txt" >&2 + echo "::error title=verisimdb-data is not writable::HYPATIA_DISPATCH_PAT cannot create a branch on ${R}, so this rescan would discard ~20 minutes of scanning at the final step. Grant the token repository access to ${R} with Contents: Read and write AND Pull requests: Read and write. Note a fine-grained PAT must list the repo explicitly; org/user-wide scope is not inherited." + exit 78 + fi + gh api -X DELETE "repos/${R}/git/refs/heads/${PROBE}" >/dev/null + echo "verisimdb-data is writable — proceeding with the scan." + - name: Resolve panic-attack version id: pa_ref run: echo "sha=$(git ls-remote https://github.com/hyperpolymath/panic-attack.git HEAD | cut -f1)" >> "$GITHUB_OUTPUT" From 9c9214b7aa31c70f7348001c5e82473c7e2544bf Mon Sep 17 00:00:00 2001 From: "Jonathan D.A. Jewell" <6759885+hyperpolymath@users.noreply.github.com> Date: Tue, 21 Jul 2026 16:25:41 +0100 Subject: [PATCH 4/5] =?UTF-8?q?fix(ci):=20squash-merge=20in=20two=20PRs=20?= =?UTF-8?q?=E2=80=94=20rebase=20merges=20cannot=20be=20signed?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The branch+PR route reached the merge and was refused. GitHub said why, verbatim: GraphQL: Base branch requires signed commits. Rebase merges cannot be automatically signed by GitHub (mergePullRequest) So my previous commit's claim — that a rebase merge would satisfy `required_signatures` — was wrong. Measured, not inferred, this time. ## The merge method is fully determined verisimdb-data's `Base` ruleset requires `pull_request`, `required_signatures` and `required_linear_history` on main: direct push -> refused (pull_request required) merge commit -> refused (required_linear_history) rebase merge -> refused (cannot be signed) <- measured above squash merge -> GitHub authors one new commit server-side and signs it Squash is the only method that satisfies all three. ## Why that forces TWO PRs A single squash of "scans + index" breaks the index contract. scripts/regen-index.sh line 44: last_updated="$(git log -1 --format=%cI -- "$SCAN_DIR")" i.e. the commit time of the newest commit touching scans/. Squashing both changes together produces ONE commit touching scans/ *and* index.json, while index.json still carries a timestamp computed before that commit existed — so index-freshness.yml, which reruns the same script at HEAD and diffs, would fail. Publishing as two squashes keeps them separate: PR 1 lands exactly one commit touching scans/; PR 2 lands exactly one commit that does not. The index is regenerated AGAINST MERGED main (fetch + reset --hard FETCH_HEAD after PR 1 lands), so the timestamp it records is the real post-merge scans commit. Both PRs assert they reached MERGED before the step succeeds. Verified: YAML parses; shellcheck -S warning rc 0, no findings; and the live-code ordering (comments excluded) is publish scans -> reset to merged main -> regen index -> publish index. Co-Authored-By: Claude Opus 4.8 --- .github/workflows/estate-rescan.yml | 138 ++++++++++++---------------- 1 file changed, 60 insertions(+), 78 deletions(-) diff --git a/.github/workflows/estate-rescan.yml b/.github/workflows/estate-rescan.yml index 2993bcb6..149cd250 100644 --- a/.github/workflows/estate-rescan.yml +++ b/.github/workflows/estate-rescan.yml @@ -222,26 +222,7 @@ jobs: echo "changed=true" >> "$GITHUB_OUTPUT" git commit -m "scan: estate rescan ($SCANNED repos, panic-attack refresh)" - - name: Regenerate index.json (canonical generator) - if: inputs.dry_run != 'true' && steps.commit_scans.outputs.changed == 'true' - run: | - set -euo pipefail - DATA="$RUNNER_TEMP/verisimdb-data" - cd "$DATA" - # Use the repo's own deterministic generator rather than a bespoke - # schema. regen-index.sh derives `last_updated` from the commit - # time of the latest commit touching scans/ — which is the - # scans-only commit just made above. Committing the index as a - # SEPARATE commit (next step) that does NOT touch scans/ keeps that - # timestamp stable, so the index-freshness.yml gate (which reruns - # this same script at HEAD and diffs) stays green. - bash scripts/regen-index.sh - git add index.json - if ! git diff --cached --quiet; then - git commit -m "index: regen after estate rescan" - fi - - - name: Publish to verisimdb-data (branch -> PR -> rebase-merge) + - name: Publish to verisimdb-data (two signed, squash-merged PRs) if: inputs.dry_run != 'true' && steps.commit_scans.outputs.changed == 'true' env: GH_TOKEN: ${{ secrets.HYPATIA_DISPATCH_PAT }} @@ -252,74 +233,75 @@ jobs: REPO="hyperpolymath/verisimdb-data" cd "$DATA" - # WHY THIS IS NO LONGER A DIRECT PUSH TO main. + # WHY TWO PRs, AND WHY SQUASH. # - # `git push origin HEAD:main` CANNOT succeed against this repo, and - # no token change will make it succeed. verisimdb-data carries an - # active `Base` ruleset on ~DEFAULT_BRANCH whose rules include - # `pull_request` and `required_signatures` (plus deletion, - # non_fast_forward, required_linear_history). Its only bypass actor - # is RepositoryRole(id=5) in mode `pull_request` — i.e. nobody may - # push straight to main, admin included. + # verisimdb-data's `Base` ruleset requires `pull_request` AND + # `required_signatures` on main, plus `required_linear_history`. + # That eliminates every other option, as GitHub itself told us: # - # Measured timeline, which is what identified this: - # Base ruleset created ....... 2026-04-12 - # bot pushed fine (16 commits) through 2026-06-25 17:40 - # Base ruleset UPDATED ....... 2026-06-26 08:57 - # bot pushes since ........... none - # The fact-store then sat 26 days stale while the old loop retried - # the resulting 403 five times, making a hard permission wall look - # like flakiness. + # GraphQL: Base branch requires signed commits. + # Rebase merges cannot be automatically signed by GitHub # - # NB the repo's `pushed_at` still looked recent throughout, because - # humans kept committing. Judge bot-data freshness by the last - # commit AUTHORED BY THE BOT, never by pushed_at. + # direct push -> refused, `pull_request` required + # merge commit -> refused, `required_linear_history` + # rebase merge -> refused, cannot be signed (measured, above) + # squash merge -> GitHub authors ONE new commit server-side and + # signs it. The only method that satisfies all three. # - # A feature branch is unrestricted (the ruleset targets only the - # default branch), and merging a PR satisfies both `pull_request` - # and `required_signatures` — GitHub signs the commits it creates - # server-side, which a runner's raw `git push` can never do. - - BRANCH="estate-rescan/${RUN_ID}" - ERR="$RUNNER_TEMP/push-err.txt" + # But a single squash of "scans + index" would break the index + # contract. scripts/regen-index.sh line 44 derives the timestamp as: + # + # last_updated="$(git log -1 --format=%cI -- "$SCAN_DIR")" + # + # i.e. the commit time of the newest commit touching scans/. Squashing + # both changes together makes that ONE commit touch scans/ as well as + # index.json, while index.json still carries a timestamp computed + # before that commit existed — so index-freshness.yml, which reruns + # this same script at HEAD and diffs, would fail. + # + # Publishing as two squashes keeps them separate: PR 1 lands exactly + # one commit touching scans/, PR 2 lands exactly one commit that does + # not. The index is regenerated AGAINST MERGED main, so the timestamp + # it records is the real post-merge scans commit. - if ! git push origin "HEAD:refs/heads/${BRANCH}" 2>"$ERR"; then - cat "$ERR" >&2 - if grep -qiE '403|permission to .+ denied|authentication failed' "$ERR"; then - echo "::error title=verisimdb-data branch push denied::HYPATIA_DISPATCH_PAT could not create a branch on ${REPO}. The clone succeeded, so the token is valid but lacks Contents:write. Unlike a push to main, this one IS a token problem — the Base ruleset does not restrict non-default branches." - exit 78 + publish() { + local suffix="$1" title="$2" + local branch pr_url pr_num state + branch="estate-rescan/${RUN_ID}-${suffix}" + git push origin "HEAD:refs/heads/${branch}" + pr_url="$(gh pr create --repo "$REPO" --base main --head "$branch" \ + --title "$title" \ + --body "Automated estate rescan from hyperpolymath/hypatia run ${RUN_ID}. Squash-merged: it is the only merge method that satisfies required_signatures + required_linear_history on this repo.")" + pr_num="${pr_url##*/}" + echo "opened ${pr_url}" + if ! gh pr merge "$pr_num" --repo "$REPO" --squash --delete-branch; then + echo "::error title=verisimdb-data merge refused::PR #${pr_num} (${suffix}) opened but would not merge. Check the Base ruleset and that the token has Pull requests: Read and write." + return 1 fi - exit 1 - fi - echo "pushed ${BRANCH}" + state="$(gh pr view "$pr_num" --repo "$REPO" --json state --jq .state)" + if [ "$state" != "MERGED" ]; then + echo "::error title=verisimdb-data not updated::PR #${pr_num} (${suffix}) is ${state}, not MERGED. The fact-store did NOT receive this rescan." + return 1 + fi + echo "merged ${pr_url}" + } - PR_URL="$(gh pr create --repo "$REPO" --base main --head "${BRANCH}" \ - --title "scan: estate rescan (run ${RUN_ID})" \ - --body "Automated estate rescan from hyperpolymath/hypatia run ${RUN_ID}. Two commits, deliberately kept separate: a scans-only commit, then an index regen that does not touch scans/ so index-freshness stays green.")" - PR_NUM="${PR_URL##*/}" - echo "opened ${PR_URL}" + # ---- PR 1: the scans commit made in the previous step ------------- + publish "scans" "scan: estate rescan (run ${RUN_ID})" - # --rebase, NOT --squash. Squash would collapse the scans commit and - # the index commit into one, which breaks the invariant that - # regen-index.sh depends on (see the index step above): its - # `last_updated` is derived from the newest commit touching scans/, - # so the index must land as a separate commit that does not. - # Rebase also satisfies `required_linear_history`. - if ! gh pr merge "$PR_NUM" --repo "$REPO" --rebase --delete-branch 2>"$ERR"; then - cat "$ERR" >&2 - echo "::error title=verisimdb-data merge refused::Branch ${BRANCH} pushed and PR #${PR_NUM} opened, but the merge was refused. Check the Base ruleset on ${REPO} and that the token may merge pull requests." - exit 78 + # ---- regenerate the index against MERGED main --------------------- + git fetch --depth=50 origin main + git reset --hard FETCH_HEAD + bash scripts/regen-index.sh + git add index.json + if git diff --cached --quiet; then + echo "index already current after the scans merge — nothing further to publish." + exit 0 fi + git commit -m "index: regen after estate rescan (run ${RUN_ID})" - # POSTCONDITION — never assume the merge landed. A refused merge that - # exits 0 is exactly the class of silent failure that let this repo - # go stale for 26 days in the first place. - STATE="$(gh pr view "$PR_NUM" --repo "$REPO" --json state --jq .state)" - if [ "$STATE" != "MERGED" ]; then - echo "::error title=verisimdb-data not updated::PR #${PR_NUM} is ${STATE}, not MERGED. The fact-store did NOT receive this rescan." - exit 1 - fi - echo "verisimdb-data updated via ${PR_URL}" + # ---- PR 2: index only, touching no file under scans/ -------------- + publish "index" "index: regen after estate rescan (run ${RUN_ID})" - name: Job summary env: From 89ce653c9c6e4b35651571439e48f95c9af5f9f6 Mon Sep 17 00:00:00 2001 From: "Jonathan D.A. Jewell" <6759885+hyperpolymath@users.noreply.github.com> Date: Tue, 21 Jul 2026 16:43:56 +0100 Subject: [PATCH 5/5] =?UTF-8?q?fix(ci):=20use=20--auto=20and=20poll=20?= =?UTF-8?q?=E2=80=94=20the=20merge=20was=20landing=20AFTER=20we=20gave=20u?= =?UTF-8?q?p?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Run 29843933209 reported the publish step as failed. It had not failed: verisimdb-data#54 merged at 15:37:35Z with a verified signature, and the 363-repo rescan is on main. `gh pr merge --squash` is evaluated the instant the PR is created, before the base-branch policy has settled, and returns not mergeable: the base branch policy prohibits the merge ...add the `--auto` flag then the merge lands ~30s later regardless. So the step exited 1 on a success, and — because it exited before opening its second PR — left index.json stale on main while scans/ had moved. That is the exact "reports failure, actually succeeded" inverse of the fake-gate problem. Now: --auto queues the merge, and the step POLLS for state==MERGED rather than inferring it from the exit code of the command that requested it. Also proven while fixing the stale index by hand (verisimdb-data#55): an index-only PR passes index-freshness cleanly under squash, because regen-index.sh's `git log -1 -- scans/` is unaffected by a squash that touches no file under scans/. So the two-PR split is sound and the "squash vs regen-index" contradiction I reported does NOT exist. Verified: YAML parses; shellcheck rc 0; verisimdb-data main is now d81aec63 with last_updated=2026-07-21T16:37:34 and 498 scans, and the index-freshness gate is green on main. Co-Authored-By: Claude Opus 4.8 --- .github/workflows/estate-rescan.yml | 24 ++++++++++++++++++++---- 1 file changed, 20 insertions(+), 4 deletions(-) diff --git a/.github/workflows/estate-rescan.yml b/.github/workflows/estate-rescan.yml index 149cd250..61d0fe3d 100644 --- a/.github/workflows/estate-rescan.yml +++ b/.github/workflows/estate-rescan.yml @@ -274,13 +274,29 @@ jobs: --body "Automated estate rescan from hyperpolymath/hypatia run ${RUN_ID}. Squash-merged: it is the only merge method that satisfies required_signatures + required_linear_history on this repo.")" pr_num="${pr_url##*/}" echo "opened ${pr_url}" - if ! gh pr merge "$pr_num" --repo "$REPO" --squash --delete-branch; then - echo "::error title=verisimdb-data merge refused::PR #${pr_num} (${suffix}) opened but would not merge. Check the Base ruleset and that the token has Pull requests: Read and write." + # --auto, not a bare merge. A plain `gh pr merge --squash` is + # evaluated the instant the PR is created, before the base-branch + # policy has settled, and returns + # "not mergeable: the base branch policy prohibits the merge" + # — while the merge then lands ~30s later anyway. That exact + # false negative made run 29843933209 report failure for a rescan + # that HAD in fact merged (verisimdb-data#54, merged 15:37:35Z). + # --auto queues it and lets GitHub merge when the policy allows. + if ! gh pr merge "$pr_num" --repo "$REPO" --squash --auto --delete-branch; then + echo "::error title=verisimdb-data merge could not be queued::PR #${pr_num} (${suffix}) opened but auto-merge was refused. Check the Base ruleset and that the token has Pull requests: Read and write." return 1 fi - state="$(gh pr view "$pr_num" --repo "$REPO" --json state --jq .state)" + # Poll for the END STATE. Never infer the merge from the exit code + # of the command that requested it. + state="" + for _ in $(seq 1 40); do + state="$(gh pr view "$pr_num" --repo "$REPO" --json state --jq .state)" + [ "$state" = "MERGED" ] && break + [ "$state" = "CLOSED" ] && break + sleep 15 + done if [ "$state" != "MERGED" ]; then - echo "::error title=verisimdb-data not updated::PR #${pr_num} (${suffix}) is ${state}, not MERGED. The fact-store did NOT receive this rescan." + echo "::error title=verisimdb-data not updated::PR #${pr_num} (${suffix}) is ${state:-UNKNOWN} after 10 minutes, not MERGED. The fact-store did NOT receive this rescan." return 1 fi echo "merged ${pr_url}"