Skip to content

fix(ci): stop the release audit calling a healthy publish broken - #567

Merged
us merged 2 commits into
mainfrom
fix/release-verify-npm-propagation
Sep 19, 2026
Merged

us merged 2 commits into
mainfrom
fix/release-verify-npm-propagation

Conversation

@us

@us us commented Sep 19, 2026

Copy link
Copy Markdown
Owner

The v0.36.0 release audit went red on three findings. One was real. Two were the
verification checks reading a registry too eagerly and calling a healthy publish broken.
That mix is the problem: a release that is red for reasons nobody trusts is a release
whose real failure nobody sees, and today the real one was only found by hand.

What was wrong

verify_npm_sdk.sh polled npm for 60 seconds. a03d2b0 already measured that as too
short on the 0.34.0 release and widened verify_npm.sh to five minutes, but the change was
never applied to this script. So it false-failed both v0.35.1 and v0.36.0 in exactly the
same way: npm logged + crw-sdk@<version>, warned that the package "may take a few minutes
to become available", and the check gave up at 60 s while the release itself was fine.
crw-sdk@0.36.0 is on npm and installs correctly.

verify_docker.sh read each tag once. The audit called ghcr.io/us/crw:latest missing
while :0.36.0 and :0.36, pushed in the same operation, both passed. latest now
resolves to sha256:8c74c910..., byte for byte the 0.36.0 digest, so the tag was simply
not readable yet when the check ran.

What changed

  • verify_npm_sdk.sh comes to parity with its sibling: the same 30 x 10 s existence poll
    and the same install-smoke retry.
  • verify_docker.sh polls each tag, with the deadline shared across tags rather than
    per tag, so a genuinely failed release is still reported in five minutes and not in
    fifteen.

Measured, against the live registries, both directions

verify_npm_sdk.sh 0.36.0    -> exit 0 in 2 s
verify_npm_sdk.sh 99.99.99  -> exit 1 in 325 s  ("not on npm after retries (got: MISSING)")
verify_docker.sh  0.36.0    -> exit 0, three tags, both arches
verify_docker.sh  99.99.99  -> exit 1 in 318 s

Both still fail for a genuinely missing artifact, which is the point: the window moved, the
verdict did not.

The real finding is not masked by this

crw-mcp-linux-x64@0.36.0 was genuinely absent from npm even though the publish step logged
+ crw-mcp-linux-x64@0.36.0. verify_npm.sh already polls five minutes and reported it
correctly, so that check needed no change. It has since published on a workflow re-run, and
a linux/x64 install now resolves the platform package:

npm install crw-mcp@0.36.0 --os=linux --cpu=x64
  -> crw-mcp, crw-mcp-linux-x64

Before the re-run that install produced the wrapper alone, and the launcher's
GitHub-download fallback would have covered it at the cost of a 10 MB download on first run.

Cost

A genuinely failed release now takes up to five minutes per check to report instead of
failing fast. That is the trade this PR is making on purpose.

us added 2 commits September 19, 2026 04:48
Two of the release verification checks read a registry once, or for only a
minute, and report a release that shipped correctly as failed. The v0.36.0 run
went red on both while every artifact was fine, which is the state that trains
everyone to ignore a red release.

- verify_npm_sdk.sh polled npm for 60 s. a03d2b0 already measured that as too
  short and widened verify_npm.sh to five minutes, but never applied it here,
  so this script false-failed v0.35.1 and v0.36.0: npm logged
  `+ crw-sdk@<version>` and warned the package "may take a few minutes to
  become available", then the check gave up at 60 s. Brought to parity with its
  sibling, existence poll and install-smoke retry both.
- verify_docker.sh read each tag once. The v0.36.0 audit called
  ghcr.io/us/crw:latest missing while :0.36.0 and :0.36, pushed in the same
  operation, passed; latest resolved to the same digest as 0.36.0 minutes
  later. Now polls, with the deadline shared across tags so a genuinely failed
  release is still reported in five minutes rather than fifteen.

Measured against the live registries, both directions:

  verify_npm_sdk.sh 0.36.0   -> exit 0 in 2 s
  verify_npm_sdk.sh 99.99.99 -> exit 1 in 325 s
  verify_docker.sh  0.36.0   -> exit 0, all three tags, both arches
  verify_docker.sh  99.99.99 -> exit 1 in 318 s

The v0.36.0 audit's third finding, crw-mcp-linux-x64@0.36.0 missing from npm,
was a true failure and is not masked by this: that package published on a
re-run and a linux/x64 install now resolves it.
@us
us merged commit 43b11f7 into main Sep 19, 2026
10 checks passed
@github-actions github-actions Bot locked and limited conversation to collaborators Sep 19, 2026
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant