Diff the working tree when it is dirty, instead of scanning everything

[Ibrahimrahhal]

Why

Incremental scans are effectively not happening for CLI users. Over 14 days of production traffic (trigger_source = 'cli', engine = 'corgea-blast', non-prework):

gate reached	scans	scanned incrementally
no branch or sha	854	0
dirty flag absent (pre-1.11 CLI)	210	0
worktree dirty	3,466	0
reached the baseline gates	17	3

76% of CLI scans arrive flagged dirty, and before this change every one of them refused incremental unless the run passed --ignore-dirty-worktree. Only 17 scans in two weeks got as far as looking for a baseline. That is expected rather than surprising: CI dirties a worktree as a matter of course by installing dependencies or building before it scans, and the flag is opt-in.

What changed

resolve_incremental_plan no longer consults ignore_dirty_worktree. A dirty tree now diffs the working tree, which is what the flag already did.

The refusal was never needed for correctness. The flag did not paper over a dirty tree — it moved the far side of the diff from HEAD to the index and working tree, so files edited but not committed are named in the changed-file list and rescanned like any other change. The concern the gate existed for (a commit-to-commit diff leaves a modified file off the list, so its old findings are copied forward over content nothing analyzed) is exactly what the worktree diff avoids. Since the archive is the working tree, it is the accurate diff for a dirty upload rather than a concession.

Two things make it safe end to end:

• Gitignored files are excluded from the worktree diff and from the archive (packaging walks with WalkBuilder::standard_filters(true)), so no unnamed file is left to carry stale findings forward. Covered by a new test.
• The server already accepts this: _try_incremental_scan's dirty gate passes when the client diff claims to cover the worktree, and the CLI sends incremental_covers_worktree=true. The plan is built from the same reconciled RepoInfo the upload's dirty field comes from, so the two cannot disagree.

--ignore-dirty-worktree keeps its other, genuinely lossy meaning for --skip-if-commit-scanned-recently, where it reuses a prior scan that does not cover the uncommitted changes. Its help text now covers only that, and --disable-incremental is still the way to opt out of incremental entirely.

The flag now warns when it governs nothing

1.11.1 defined the flag with requires = "skip_if_commit_scanned_recently", so passing it alone was a hard argument error. 1.12.0 dropped that clause precisely so the flag could also put incremental back on the table for a dirty tree — which is no longer needed.

That leaves a flag a run can pass alone, where it now governs nothing. Pipelines do pass it that way, to get the incremental scan of a dirty tree it used to enable, and would otherwise get full scans with no indication why. Re-adding requires would break those pipelines on upgrade, so the second commit warns instead and says where the behavior went.

Notes for review

• The unused ignore_dirty_worktree_for_run parameter threaded through blast::run → start_new_scan existed only to feed this refusal, so it is gone. main.rs was passing the same value twice.
• blast_upload_plan encoded "a dirty tree never asks" for a baseline; it now always expects the lookup. narrowed_blast_upload_plan covers the one case that still skips it, --target/--exclude, where carrying findings forward for files the archive no longer holds would be wrong.
• a_dirty_worktree_skips_the_baseline_lookup_and_scans_everything asserted behavior that no longer exists and is redundant with the worktree-diff test plus a_project_with_no_baseline_scan_uploads_without_a_diff, so it is deleted.

Testing

./harness check — 814 tests pass, strict clippy and format clean.

The two existing tests for the worktree diff (a_worktree_diff_names_edited_and_untracked_files, a_worktree_diff_still_spans_the_commits_behind_it) already covered the path this promotes to the default. Added a_worktree_diff_leaves_gitignored_files_out for the gitignore claim above. The end-to-end a_dirty_worktree_diffs_the_working_tree_rather_than_scanning_everything now runs without the flag, asserting the upload carries dirty=true alongside incremental_base_sha, incremental_changed_files and incremental_covers_worktree=true; ignore_dirty_worktree_alone_warns_and_scans_incrementally_anyway shares that setup and asserts the same scan plus the warning.

What this does not fix

This clears the gate that 76% of CLI scans stop at. It does not, on its own, guarantee they go incremental, because a dirty scan can never become a baseline (base_filter requires worktree_dirty=False, correctly — a dirty scan's contents are not reproducible from its commit). A project whose every CLI scan is dirty has no baseline to diff against and needs one from a clean run, CI, or an SCM integration. Separately, baseline_branches only ever asks about origin/HEAD (usually absent in CI checkouts) plus main and master, so a project whose trunk is named anything else finds no baseline either. Both are being measured before any change is proposed.

The other two buckets are untouched: 854 scans with no branch or sha (detached HEAD, which is what PR pipelines check out) and 210 on a CLI too old to report the flag at all.