Stop filing issues for routine find-ui corpus drift

[Jaylyn Barbee (Jaylyn-Barbee)]

Description

The find-ui corpus drift job filed an issue for a condition that is the normal steady state. Upstream gains samples between releases, so #807 sat open permanently and collected a weekly "still present" comment nobody could act on. It shared the find-ui-corpus label with the genuine alerts, so the noise devalued the label that also carries the real signal.

Routine drift now goes to the workflow run summary. Issues are reserved for the two actionable cases. Because a run summary is not read at the moment it matters, the release bake now prints the same deltas, where an operator can still act on them.

What files an issue

Condition	Before	Now
Bake fetched nothing / produced no files	Issue	Issue (unchanged)
A source lost >25% of its scenarios	Issue titled "drifted from upstream"	Issue, own title and investigation steps
A committed source vanished from a successful bake	Not detected	Issue
Growth, small deltas, new sources, content edits	Issue + weekly "still present" comment	Run summary only

The >25% case is the one nothing else catches. SnapshotBaker.BakeAsync only fails a source that returns zero scenarios, so a fetcher that survives an upstream restructure while matching a fraction of what it used to bakes green and would ship a gutted source.

Usage Example

build-cli.ps1 on -Stable (or -Bake) now reports what moved:

[BAKE] Corpus refreshed.
[BAKE] Scenario counts vs the previously committed corpus:
[BAKE]   gallery      321 ->   323  (+2)
[BAKE]   reactor       93 ->    95  (+2)
[BAKE]   toolkit      120 ->   120  (no change)

And when a fetcher breaks without failing the bake:

WARNING: [BAKE]   gallery      321 ->    40  (-281) -- lost 88% of its scenarios. Check the fetcher for 'gallery' before shipping this corpus.

It warns rather than fails. A legitimate upstream purge should not block a release, it should make someone look.

What replaces the #807 issue, in the run summary:

## find-ui corpus

### Routine drift

No action required, re-baked automatically on the next `-Stable` release.

- `snapshot-gallery.json.br` — 321 → 323 scenarios (+2)
- `snapshot-reactor.json.br` — 93 → 95 scenarios (+2)

Related Issue

Closes #807

Type of Change

• 🔧 Config/build

Checklist

• Tested locally on Windows

Additional Notes

Verification

Both changes were tested by extracting the shipped code (the workflow's embedded PowerShell, and the delta block from build-cli.ps1) and running it against synthetic corpora, so the tests exercise the real text rather than a copy.

Drift workflow, 10 cases:

Case	Before → after	Status	Files issue
Identical corpus	321/93 → 321/93	clean	no
Routine growth (the #807 case)	321/93 → 323/95	routine	no
Small loss	100 → 95	routine	no
Boundary, exactly -25%	100 → 75	routine	no
Boundary, -26%	100 → 74	regression	yes
Fetcher gutted	321 → 40	regression	yes
Source vanished	reactor 93 → absent	regression	yes
New source	toolkit absent → 50	routine	no
Bake exited nonzero	n/a	bake-failed	yes
Bake produced no files	n/a	bake-failed	yes

The -25% boundary is exclusive: a quarter lost is not a regression, more than a quarter is.

Bake delta reporting, 7 cases: routine growth, gutted source, both -25% boundaries, new source added, source dropped from the manifest, and a first bake with no previous manifest. Output matches the samples above.

Also syntax-checked the workflow's embedded PowerShell and JS (wrapped as github-script wraps it) and re-parsed build-cli.ps1.

Incidental fix

$committed in the drift workflow was a relative path, but Read-Snapshot reads through System.IO, which resolves relative paths against the process working directory rather than PowerShell's location. The two agree in Actions so this was not a live bug, but it let a local run silently compare against a different corpus than Get-ChildItem enumerated. It surfaced while testing. Now resolved to absolute via $PWD.Path.

Follow-up not included here

The find-ui-corpus label description still reads "drift / bake failure". The workflow creates it as "regression / bake failure", but createLabel returns 422 on an existing label and does not update it, so this needs a manual edit in repo settings.

[github-actions]

Build Metrics Report

Binary Sizes

Artifact	Baseline	Current	Delta
CLI (ARM64)	46.12 MB	46.12 MB	📈 +1.5 KB (+0.00%)
CLI (x64)	46.03 MB	46.03 MB	📈 +1.5 KB (+0.00%)
MSIX (ARM64)	19.04 MB	19.04 MB	📈 +3.4 KB (+0.02%)
MSIX (x64)	20.18 MB	20.19 MB	📈 +1.4 KB (+0.01%)
NPM Package	39.61 MB	39.62 MB	📈 +5.0 KB (+0.01%)
NuGet Package	39.75 MB	39.75 MB	📈 +2.9 KB (+0.01%)

Test Results

✅ 5816 passed, 18 skipped out of 5834 tests in 944.2s (+7 tests, +99.9s vs. baseline)

Test Coverage

✅ 87.5% line coverage, 80.8% branch coverage · ✅ no change vs. baseline

CLI Startup Time

63ms median (x64, winapp --version) · ✅ no change vs. baseline

Try This Build

Installs the MSIX for your architecture, replacing any previously installed build. Needs the GitHub CLI — the command offers to install it and sign you in if it is missing.

& ([scriptblock]::Create((irm https://raw.githubusercontent.com/microsoft/winappCli/main/scripts/winapp-pr.ps1))) 825

Switching between builds often?

Put the tool on your PATH once:

& ([scriptblock]::Create((irm https://raw.githubusercontent.com/microsoft/winappCli/main/scripts/winapp-pr.ps1))) -AddToPath

Then this build is just:

winapp-pr 825

Run winapp-pr with no arguments to pick from a list of open PRs.

---

Updated 2026-09-15 21:12:34 UTC · commit cd94334 · workflow run

[Copilot]

🟡 Changes recommended

Boundary rounding and alert wording can inaccurately describe why a regression was reported.

Get a fresh assessment by requesting another Copilot review.

Pull request overview

Moves routine find-ui corpus drift from recurring issues into workflow summaries while preserving actionable regression alerts.

Changes:

• Detects >25% scenario losses and vanished sources.
• Reports release-bake count deltas and warnings.
• Reserves issues for bake failures and regressions.

File summaries

File	Description
scripts/build-cli.ps1	Reports corpus changes during release bakes.
.github/workflows/find-ui-corpus-drift.yml	Separates routine drift from actionable alerts.

Review details

Suppressed comments (1)

.github/workflows/find-ui-corpus-drift.yml:241

• The alert title says the source lost “most” of its scenarios, but this path triggers at any loss over 25%; a 100 → 74 drop retains most scenarios while opening this issue. Use the actual threshold (or “significant loss”) so the operator-facing title accurately describes why the alert fired.

              : 'find-ui corpus: a source lost most of its scenarios';

• Files reviewed: 2/2 changed files
• Comments generated: 2
• Review effort level: Balanced

---

💡 Configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[Copilot]

🟡 Changes recommended

Percentage rounding can still report an actionable loss as exactly 25% in both reporting paths.

Get a fresh assessment by requesting another Copilot review.

Review details

• Files reviewed: 2/2 changed files
• Comments generated: 2
• Review effort level: Balanced

[Copilot]

🟢 Approval recommended

The revised classification and reporting paths are correct and passed syntax and synthetic boundary-case validation.

Review details

• Files reviewed: 2/2 changed files
• Comments generated: 0 new
• Review effort level: Balanced