Skip to content

Validate fork PRs without executing the fork's validator - #70

Open
Queen-Mo wants to merge 1 commit into
block:mainfrom
Queen-Mo:fix/validate-skills-fork-code-execution
Open

Validate fork PRs without executing the fork's validator#70
Queen-Mo wants to merge 1 commit into
block:mainfrom
Queen-Mo:fix/validate-skills-fork-code-execution

Conversation

@Queen-Mo

@Queen-Mo Queen-Mo commented Aug 8, 2026

Copy link
Copy Markdown

The "Validate changed skills only (fork PRs via ./pr)" step in validate-skills.yml runs under pull_request_target with working-directory: pr, so node scripts/validate-skills.mjs resolves to the fork's copy of the validator rather than the base repo's.

This runs the trusted validator from the base checkout and passes the untrusted tree in as data instead, matching what skills-security-scan.yml already does with --repo-root pr:

  • validate-skills.yml: drop working-directory: pr, add --repo-root pr
  • validate-skills.mjs: accept --repo-root, defaulting to cwd so the same-repo and push paths are unchanged; error out rather than falling back to cwd when the flag is passed without a value, since in the fork job cwd is the base checkout and a silent fallback would validate the trusted tree instead of the fork's

Coverage is unchanged: --skills and --report were already accepted and ignored, and validation still walks every skill directory under the root.

Verified locally: validating a separate tree via --repo-root passes and exits 0; passing --repo-root with no value exits 2 with an explicit error.

Worth doing in a follow-up: both actions/checkout@v4 references should be pinned to SHAs chosen deliberately. Pinning to a pre-fix SHA would reintroduce this.

The "Validate changed skills only (fork PRs via ./pr)" step ran under
pull_request_target with `working-directory: pr`, so
`node scripts/validate-skills.mjs` resolved to the *fork's* copy of the
validator. A fork author could edit that file and have it run in the base
repo's runner with the workflow's GITHUB_TOKEN (contents: read,
pull-requests: write). The same substitution also lets the PR report its
own validation as passing.

Run the trusted validator from the base checkout and pass the untrusted
tree as data instead, matching what skills-security-scan.yml already does
with `--repo-root pr`:

- validate-skills.yml: drop `working-directory: pr`, add `--repo-root pr`
- validate-skills.mjs: accept `--repo-root`, defaulting to cwd so the
  same-repo/push path is unchanged; error out rather than falling back to
  cwd if the flag is passed without a value, since in the fork job cwd is
  the base checkout and that would silently validate the trusted tree

Coverage is unchanged: `--skills` and `--report` were already accepted and
ignored, and validation still walks every skill directory under the root.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@Queen-Mo

Queen-Mo commented Aug 8, 2026

Copy link
Copy Markdown
Author

🤖 Posted by Yinka Amosun's agent (Fizz) for Block Vulnerability Management.

Cross-reference so these two don't get merged independently: #68 (open since 2026-07-30) addresses the same validator-resolution issue in this workflow, by invoking ../scripts/validate-skills.mjs from the base checkout while keeping working-directory: pr. That works — the validator comes from the trusted tree and process.cwd() still points at the PR tree being validated.

I opened this before reading #68 in full, so treat them as alternatives rather than complements:

  • This PR is the narrower change — one workflow, one script, no behaviour change on the same-repo and push paths.
  • Fix secure validation for fork pull requests #68 is broader: it applies the same principle to skills-security-scan.yml as well, and additionally pins the fetched PR head to github.event.pull_request.head.sha rather than resolving refs/pull/N/head at checkout time, which closes a race this PR does not address.

Maintainers should pick one. If @block/goose-devrel would rather take #68 as the more complete change, I'm happy to close this — the SHA-pinning in particular is worth keeping either way, and could land on top of this if you'd prefer the smaller diff first.

@Queen-Mo

Queen-Mo commented Aug 8, 2026

Copy link
Copy Markdown
Author

🤖 Posted by Yinka Amosun's agent (Fizz) for Block Vulnerability Management.

validate and security_scan are red on this PR, but not because of this change — they fail identically on #68 and #69, and have since at least 2026-07-30. Because pull_request_target runs the workflow file from main, none of these PRs' own changes are exercised by their own checks.

The failing step is Checkout PR head into ./pr (untrusted):

Refusing to check out fork pull request code from a pull_request_target workflow. […] To opt in, review the risks at https://gh.io/securely-using-pull_request_target and set allow-unsafe-pr-checkout: true on the actions/checkout step.

actions/checkout now declines this by default. Everything downstream is skipped, so skill validation and the security scan currently do not run on any fork PR — the jobs fail before they reach the validator rather than validating anything.

Two things follow, and the second is the reason I'm flagging it here:

  1. This is a real coverage gap on its own — fork contributions have been going unvalidated for over a week.
  2. The obvious way to make CI green again is allow-unsafe-pr-checkout: true, and that is the one change I'd ask nobody to make on top of the workflow as it stands on main today. That flag restores the fork tree to disk while main still runs the validator with working-directory: pr — which is precisely the arrangement this PR (and Fix secure validation for fork pull requests #68) exist to remove. Landing the resolution fix first makes the opt-in safe; the reverse order does not.

#68's approach has an advantage worth noting in this light: by fetching the PR ref directly and using git worktree instead of actions/checkout, it restores fork validation without needing the unsafe opt-in at all. This PR does not address the checkout failure.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant