ci: report test quality and exercise native Bun runners

[mldangelo-oai]

Summary

Make test results easier to inspect and measure safer Bun runner modes before changing required CI.

Part 4 of 4 in the test-quality stack. Depends on #507; merge the earlier stack layers first.

Changes

• Upload JUnit and LCOV from the required Ubuntu Node 22 lane.
• Run static checks once and remove duplicate in-tree smoke checks while retaining installed-package validation on every supported runtime lane.
• Add a separate weekly, manual, and workflow-change runner trial for baseline, isolated, parallel, randomized, and seven-way native Windows sharding.
• Compare exact JUnit test identities and outcomes, reject ambiguous parameterized names, and report every mode's timing before failing on mismatches.
• Await Python report subprocesses and replay seed 1 on pull requests; scheduled and manual runs vary the seed.
• Preserve the 120-second Windows test allowance in the experimental runners.
• Run the scoped mutation trial and retain its diagnostic report.
• Document test commands, fixture isolation, property replay, mutation review, and the Windows promotion criteria.

Testing

• Full SDK suite with JUnit and LCOV on this commit: 1,344 passed, 11 skipped, 0 failed.
• LCOV reports 93.02% of loaded JavaScript/TypeScript lines; Python helpers and child processes are outside that measurement.
• JUnit identity/outcome self-check: passed with no duplicate identities.
• Full isolated/randomized seed-1 suite: 1,344 passed, 11 skipped, with identical JUnit inventory and outcomes.
• Focused workflow and report-comparison tests: 14 passed.
• Final-head packed-package inspection and strict installed-consumer smoke test: passed.
• Red/green regression: an early comparison failure still records all seven results and leaves the job failed.
• Type checking, formatting, actionlint, and git diff --check: passed.
• Three fresh native Codex reviews and a separate verification on the refreshed head: no findings.

Risk and rollout

The three required Node 22 check names and release workflows are unchanged. Diagnostic report-upload failures do not fail required tests. The experimental workflow is not a required check or a release trigger. Keep the current file-balanced Windows runner and serial machine-policy test until repeated hosted results show matching inventories and acceptable timings.

Public disclosure review

• No customer, partner, prospect, or user identities, data, or identifying details are included.
• No credentials, personal data, private source, scan findings, or nonpublic links or tickets are included.
• I reviewed the branch name, title, description, commits, changes, comments, logs, screenshots, attachments, and links for public disclosure.

[mldangelo-oai]

@codex review

Please review the current head 28a5562cb0667564e55cabf5461ee5866c506504.

[chatgpt-codex-connector]

Security review completed. No security issues were found in this pull request.

Reviewed commit: 28a5562cb0

ℹ️ About Codex security reviews in GitHub

This is an experimental Codex feature. Security reviews are triggered when:

• You comment "@codex security review"
• A regular code review gets triggered (for example, "@codex review" or when a PR is opened), and you’re opted in so security review runs alongside code review

Once complete, Codex will leave suggestions, or a comment if no findings are found.

[chatgpt-codex-connector]

Codex Review: Didn't find any major issues. What shall we delve into next?

Reviewed commit: 28a5562cb0

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[mldangelo-oai]

@codex review

Please review the current head 51630f10514a045747105e69938052a8f0977394.

[mldangelo-oai]

Validation for 51630f10514a045747105e69938052a8f0977394:

• Required node-ci: passed, including the Ubuntu, macOS, and Windows Node 22 checks.
• Experimental test-quality: passed. All isolated, parallel, randomized seed-1, and native-shard inventories match their platform baseline. The Linux seed-1 case that failed twice at 28a5562c now passes with awaited report subprocesses.
• Hosted mutation trial: 487 killed, 17 survived, and 1 timed out. The local trial had 487 killed and 18 survived, so the timeout is not counted as stronger assertion coverage.

The native Windows runner is not ready to replace the balanced runner. On this head, the slowest native test step took 321 seconds versus 146 seconds for the required balanced runner. The two earlier measurements were 189 versus 126 seconds and 198 versus 126 seconds. The required runner and serial machine-policy test remain unchanged.

Local verification: the normal coverage run and full isolated seed-1 run each passed 1,338 tests with 11 skips and identical inventories. Types, formatting, actionlint, and three fresh native reviews plus independent verification passed.

[chatgpt-codex-connector]

Security review completed. No security issues were found in this pull request.

Reviewed commit: 51630f1051

ℹ️ About Codex security reviews in GitHub

This is an experimental Codex feature. Security reviews are triggered when:

• You comment "@codex security review"
• A regular code review gets triggered (for example, "@codex review" or when a PR is opened), and you’re opted in so security review runs alongside code review

Once complete, Codex will leave suggestions, or a comment if no findings are found.

[chatgpt-codex-connector]

Codex Review: Didn't find any major issues. Delightful!

Reviewed commit: 51630f1051

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[mldangelo-oai]

@codex review

Please review the refreshed head e43f5554f507dd97b39206b2b2224014e05bf40c. This additive restack preserves the upstream 120-second Windows timeout, carries the installed-declaration correction, and gives the new Boolean CLI cases distinct report names. The baseline and isolated/randomized seed-1 runs both passed all 1,344 tests with 11 expected skips and identical JUnit inventories. Package checks and the fresh native review gate also passed.

[chatgpt-codex-connector]

Codex Review: Didn't find any major issues. Hooray!

Reviewed commit: e43f5554f5

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[chatgpt-codex-connector]

Security review completed. No security issues were found in this pull request.

Reviewed commit: e43f5554f5

ℹ️ About Codex security reviews in GitHub

This is an experimental Codex feature. Security reviews are triggered when:

• You comment "@codex security review"
• A regular code review gets triggered (for example, "@codex review" or when a PR is opened), and you’re opted in so security review runs alongside code review

Once complete, Codex will leave suggestions, or a comment if no findings are found.

[mldangelo-oai]

Refreshed stack verification:

PR	Head	node-ci
#504	dbe0392de8b66feae1f5028932af94bb88804a1c	Passed
#506	40b23460a1012505d4c41176a336b7074839f90b	Passed
#507	b178a833ca2edcf5093bfecff54c268cffe7610a	Passed
#508	e43f5554f507dd97b39206b2b2224014e05bf40c	Passed

All three protected Node 22 check names are green on each head. Hosted Codex code and security reviews are clean on these exact commits.

The separate test-quality trial passed on attempt 2. Attempt 1 exposed two intermittent failures in unchanged Windows tests: credential-file inspection during parallel imports, and Python process startup. One same-head failed-job retry passed. All seven JUnit inventory/outcome comparisons now match: 1,355 cases on Ubuntu and 1,352 on Windows.

The mutation report records 487 killed, 17 survived, and 1 timed out. The timeout is not counted as stronger assertion coverage. In this run, the slowest native Windows shard step was 176 seconds versus 140 seconds for the required file-balanced runner. Native sharding and mutation scoring remain diagnostic; the required runner and check names are unchanged.