[W08] Rebuild paper failure figures from real artifacts (currently unlabeled synthetic demo)

[duckyquang]

Task metadata

Field	Value
Category	Writing
Priority	Medium
Complexity	Low
Est. days	1
Related	#68 (W07), PR #74

Description

The case-report PDF under paper/build/ uses failure-framework figures (taxonomy, complementarity) whose caption says "representative seed" — but the only artifacts that existed when it was built came from scripts/run_failure_analysis.py --demo, i.e. planted synthetic data (n=150). Nothing in the caption says synthetic, so the figures read as real-cohort results.

Since PR #74 the harness has a --manifest route and real artifacts exist:

python scripts/run_failure_analysis.py \
  --manifest data/processed/echo_hubert_manifest.parquet --out results/failure

Real numbers on the 245-row test frame (ridge harness, seed-fixed): echo wins per-example 245/0, LOO marginal value echo 107.3 vs ecg 6.5, drop_echo induces 188 critical failures at silent rate 1.0, drop_ecg 59 at 0.27. The real story is stronger than the synthetic one — dropping echo fails silently across the board.

The .tex source is not in this repo (only paper/build/ outputs), so this needs whoever holds the paper source.

Steps:

1. Regenerate the taxonomy / complementarity / dropout figures from results/failure (real) and swap them into the paper.
2. Update captions with real n and provenance; anywhere synthetic demo output remains, label it as harness validation on synthetic data.
3. Rebuild the PDF so paper/build/ stops carrying figures over unlabeled synthetic numbers.

Acceptance criteria:

• Paper figures trace to the real-manifest run, or are explicitly labeled synthetic.
• No results table or figure in the repo presents demo output as cohort results ([W07] Restate reported numbers from the canonical run #68 acceptance item).

[duckyquang]

The real figures exist now. I regenerated the paper's full figure set from the committed sanitized artifacts of the canonical run — no data access needed, everything traces to checkpoint 7cae4f92… / manifest 81694c9b…:

• taxonomy, complementarity matrix, and loud-vs-silent dropout profile from failure_report.fused.json — the fused checkpoint's own predictions, not the ridge harness, so the figures finally match Table 4's model
• degradation curve from missing_modality.metrics.json (with the bootstrap CIs)
• per-condition reliability diagrams from the committed calibration bins
• the fairness panel from fairness_metrics.json

All as PDF (fonttype 42, LaTeX-ready) + 300-dpi PNG.

The fused numbers behind the three views (245-row test fold): drop_echo induces 174 critical failures — 135 loud / 39 silent (silent rate 0.22) at +10.1 MAE; drop_ecg induces just 7, but 6 of the 7 are silent (rate 0.86) at +0.8 MAE. Per-example winners: echo 191 / ecg 54. Note this reads differently from the ridge-harness numbers quoted in this issue's description (188 critical, silent rate 1.0) — the fused story is the one the paper should carry, and it lines up cleanly with the below-chance ecg_dropped AUROC: few failures, no observable signal, inverted gate underneath.

@sebasmos the files are ready to swap into the .tex — I'll get them to you today (happy to attach here or push under version control if we'd rather have them tracked). Captions need: fused checkpoint, real test fold, n=245.