fix(e2e): stop the harness reporting checks it never made

[latekvo]

Found while running the full E2E harness against the packed 0.17.0 bundle for the release check. All of these are harness bugs, not product bugs.

The core problem

The harness reported results it had not actually measured.

assert_reject looked for zod issues on stdout, but the CLI prints them on stderr and leaves stdout empty on a rejection. Both arms of the check called pass(), so the miss was invisible: the zod-path argument every case passes in was decorative, and a CLI that simply exited 1 for every input would have scored the same 88/88. All 88 validation cases asserted nothing beyond "exit code was non-zero."

Once that check could fail, it exposed the rest:

• awk has no \b word boundary. /\bnumber\b/ matches nothing, so every flag kind resolved to unknown, model_number_flags was always empty, and the bad-type matrix never emitted a single case - while _build_args handed the string "x" to every required numeric/boolean/object field.
• The describe parser matched --[a-zA-Z0-9-]+, truncating every snake_case flag at the underscore (--device_id → device), and kept the -json display suffix (--selector-json) that is not part of the schema key. Both produced payloads carrying keys no schema knows.
• missing-required asserted invalid_type for enum flags, which zod reports as invalid_value.

Parsed kinds went from 270 unknown / 21 enum / 1 array to 131 string, 68 number, 28 boolean, 25 integer, 21 enum, 16 array, 3 object, 2 unknown (the last two are any, which is correct).

Teardown and the exit code

run-e2e.sh had no trap. An unbound variable inside a phase is fatal to the whole shell - run_phase || warn does not catch it - so an abort skipped the cleanup phase, the report, and rm -rf $E2E_WORK, leaking a ~500MB sandbox plus a tool-server, Electron, Metro and a fixture http server. Report generation, teardown and the exit code now live in an EXIT trap, with a completion flag: without it an aborted run exits 0 on "0 failures", which is verified below by mutation.

Teardown runs before the sandbox is removed - $E2E_HOME holds the detached tool-server's pid file and that server is started with no idle timeout, so deleting the sandbox first strands a process argent server stop can no longer find.

90-cleanup.sh gated stop-all-simulator-servers and stop-metro on ARGENT_TOOLS_URL, which the harness never sets (ensure_server deliberately does not pin it), so neither ever ran and every run left a simulator-server and Metro alive - while teardown-complete passed regardless.

Other findings fixed

• --android-avd could never work: boot-device was asked for 840s inside run_tool's 120s timeout, and the booted serial stayed in a local, so the RN tier blanket-skipped all 24 debugger/profiler tools for want of a device that was right there. Nothing shut the emulator down either - the tool-server only reaps devices Lens booted, not ones boot-device started.
• The validation tier was byte-identical green against a dead server. argent run does no client-side validation, so every call failed with a transport error that no one distinguished from a rejection. It refuses to run instead, and no longer counts timeout(1)'s 124-127 as a rejection.
• The post-uninstall restore hardcoded --omit=optional and the sandbox prefix - dropping electron on a full run (the chromium tier then skipped itself as "electron not installed"), and under --system restoring the machine's real global argent into the sandbox instead of putting it back.
• The Android teardown claimed to leave an injected device alone and then issued the same stop-simulator-server call as the self-booted branch, discarded the result, and recorded a skip saying it had not - tearing down the allocator's device service immediately before the RN tier drove that same serial.
• The chromium port-owner fallback matched :$port as a substring and took the first pid anywhere in ss -ltnp output, so cleaning up :1234 would kill a process listening on :12345.
• The RN tier's stop-metro sat after every early return (a failed debugger-connect left Metro running) and ran unconditionally, killing a developer's own Metro on :8081 that the harness never started.
• The chromium display gate accepted an installed xvfb-run as a substitute for DISPLAY, but nothing wraps the Electron spawn in it - so a headless box with it installed cleared the gate and failed boot-device, turning the gate red for exactly the environment it exists to skip.
• A phase file with a syntax error silently re-ran the previous phase under the new phase's name; an unknown --phase selected nothing and exited 0; -h printed 7 lines of source code; a value-taking flag with no value died on an unbound $2; two runs in the same second shared one results file.
• flow-add-step declares its own args property, which makes --args that field's value rather than the whole-payload escape hatch, so run_tool cannot address its fields. Skipped with that reason rather than failing on a payload the tool never received.

Verification

Against swmansion-argent-0.17.0.tgz, introspection,validation:

	before	after
totals	177 pass / 0 fail / 5 skip	221 pass / 0 fail / 6 skip
rejection cases naming the field they test	0 of 88	130 of 131
bad-type cases emitted	0	43

• assert_reject checked against right-field, wrong-field, wrong-code, no-path and unreachable-binary inputs. Every one of those cases passes on the un-fixed code.
• Abort path: injected an unbound variable into a phase - cleanup phase runs, report written, sandbox removed, exit 1, PARTIAL reported. With the completion flag mutated to always-set, the same abort exits 0, which is what pins it.
• awk \b, the -json suffix, the underscore truncation, the ss port collision and the timeout 124-127 contract were each reproduced standalone before being fixed.

Found by running against a live Android emulator

The android tier executed end-to-end for the first time: 23 pass / 1 fail / 10 skip, including the screenshot-diff pair that the original unbound-variable bug aborted on. Three further defects only surfaced by running:

• Nine pass/fail/skip calls passed their message as the third argument, which _record() takes as the case name — so the detail column was empty and the case string changed every run. The chromium boot failure was the visible cost: it recorded case=" " with no diagnostic whatsoever.
• A call killed by timeout(1) writes nothing, so RT_OUT is empty exactly when the operator most needs to know what happened. A hung boot-device was recorded as a failure with a blank detail. All nine ad-hoc detail formatters now share rt_detail(), which names the timeout (124) and a command timeout could not run (125-127).
• The chromium display gate tested that $DISPLAY was non-empty, not that it was usable. A display-manager session switch on this box left $DISPLAY pointing at a server the process holds no cookie for; Electron then bound its CDP port and never answered on it (verified: the TCP connection is accepted, the HTTP response never arrives), so the tier hard-failed instead of skipping.

Moving profiler-stack-query and profiler-combined-report into the native block also closed a hole their removal had opened: both were listed in RN_TOOLS and in the failure-arm skips but had no case on the success path, so a successful react-profiler-start left them with no pass, fail or skip at all.

Not covered here

• The postinstall assertion is left alone - fix(e2e): pin the absence of an npm postinstall hook #556 already fixes it, and this branch does not touch it. Until that merges the install phase still records that one failure.
• The RN tier still has not executed. The emulator itself is fine — it cold-boots in ~24-30s and ran the whole android tier — but it dies from CPU starvation (detected a hanging thread 'QEMU2 main loop') whenever this machine's load average climbs, which it does to 90-120 on 16 cores from concurrent work. Four attempts: it survived the android tier at load ~10, and was starved each time by a competing npm install or by Metro's own bundler startup. The four RN profiler fixes are verified against the tools' source contracts (the throw sites and their required params), not against a live run.
• The chromium tier still has not executed past boot-device, for the display reason above.
• Known remaining, deliberately left: assert_ok cannot fail for calls whose result is a constant true; the cookie/storage set→get pair never verifies the round-trip value; a tier-level skip writes one row so report.py's coverage matrix silently drops the tools that tier would have recorded; report.py renders an empty or partial log as a clean green report.