fix: harden tap bump against layout changes and bad downloads - #233
Merged
Conversation
This workflow rewrote each checksum by line position:
gsed -i "/ARTIFACT\"/!b;n;c\ sha256 \"${sha}\""
It matched the url line, advanced one line, and replaced whatever it
landed on. Correct only while sha256 sits directly beneath url inside an
on_macos or on_linux block, which is how the three bssh formulae happen
to be written today. Nine substitutions depend on that.
lablup/mlxcel ran the same construct, copied from the same source. When
its formula moved the url to the top level with a `version` stanza in
between, the next bump overwrote `version` with the new checksum and left
the previous release's sha256 below it. The formula ended up with no
version and two sha256 lines, the second of which wins, so `brew install
mlxcel` failed on checksum. Nothing validated the result and it sat broken
in the tap for three days (lablup/homebrew-tap 9969ec4).
bssh is not broken today, but it is one formula edit away from the same
outcome, with three times the exposure.
Stanzas are now located by name. `set_artifact` finds the url stanza whose
value contains a given artifact, rewrites it, then rewrites the first
sha256 stanza below it, preserving indentation. Anything in between is
skipped rather than clobbered. Guards abort the run when a formula does not
hold exactly one matching url, or when no sha256 follows it, instead of
editing the wrong line.
Two other faults surfaced while fixing this:
Downloads used `curl -Ls` with no `-f`, so a missing or renamed asset
produced a zero-byte file and its checksum went to the tap unnoticed. They
now use `-f --retry 3`, reject empty files, and verify each archive is
readable with `unzip -t` or `tar -tzf`, so a truncated download cannot
become a valid-looking sha256 either.
`git commit` ran unconditionally, so re-running for a version already in
the tap failed on an empty index. It now exits cleanly.
A validation step runs `ruby -c` and `brew style` on each formula, checks
the version stanza, and requires the exact set of checksums downloaded this
run, which catches a substitution that silently no-ops. `ruby -c` alone is
not enough: the corrupted mlxcel formula was valid Ruby.
Verified against the current tap formulae. Bumping all three to a synthetic
version puts every url and checksum in the right place and `brew style`
reports no offenses. Replaying the layout that broke mlxcel (url, then
`version`, then sha256) now updates url and sha256 and leaves `version`
untouched. The corrupted-file state trips the guards (version 0, sha256 2),
and a missing artifact trips the url guard. The workflow YAML parses.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
update_homebrew_formula.ymlrewrote each checksum by line position:gsed -i "/ARTIFACT\"/!b;n;c\ sha256 \"${sha}\""It matched the url line, advanced one line (
n), and replaced whatever it landed on (c\). That is correct only while sha256 sits directly beneath url inside anon_macos/on_linuxblock, which is how the three bssh formulae happen to be written today. Nine substitutions depend on it.lablup/mlxcel ran the same construct, copied from the same source. When its formula moved the url to the top level with a
versionstanza in between, the next bump overwroteversionwith the new checksum and left the previous release's sha256 below it:No version stanza, two sha256 lines,
brew installfailing on checksum. Nothing validated the result and it sat broken in the tap for three days (lablup/homebrew-tap9969ec4).bssh is not broken today, but it is one formula edit away from the same outcome, with three times the exposure.
Fix
Stanzas are located by name.
set_artifactfinds the url stanza whose value contains a given artifact, rewrites it, then rewrites the first sha256 stanza below it, preserving indentation. Anything in between is skipped rather than clobbered. Guards abort when a formula does not hold exactly one matching url, or when no sha256 follows it.Two other faults surfaced while fixing this:
curl -Lswith no-f. A missing or renamed asset produced a zero-byte file whose checksum went to the tap unnoticed. Now-f --retry 3, plus an empty-file check and an archive integrity check (unzip -t/tar -tzf), so a truncated download cannot become a valid-looking sha256 either.git commit. Re-running for a version already in the tap failed on an empty index. It now exits cleanly.A validation step runs
ruby -candbrew styleon each formula, checks the version stanza, and requires the exact set of checksums downloaded this run, which catches a substitution that silently no-ops.Verification
brew styleon the resultsversion, sha256)versionuntouched0, sha2562)ruby -calone is not sufficient: the corrupted mlxcel formula was valid Ruby and passes a syntax check. That is whybrew styleis in the validation step.Related
10facb2repairs the damage in the tap.lablup/all-smihas the identical positional construct and is not covered here.