feat(spr,drift): exact directional EW scorer — spr_search default + drift/sector opt-in + findings#273
Merged
Merged
Conversation
Fresh rooted union-of-finals is NOT an exact EW-Fitch insertion cost, under any rooting or on binary data. Three-way empirical verdict over 6066 reconnection candidates vs TreeLength full-rescore ground truth: - Per-child edge set E(D)=combine(prelim(D),up(D)): EXACT (6066/6066). - Union of TRUE edge sets E(A)|E(D): sound LOWER bound (under-counts, never exact). - Union of the DEPLOYED simplified final_ (fitch_indirect_length): OVER-counts (unsafe as a lower bound), because uppass_node's simplified rule never unions so final_ is a strict subset of prelim. Fresh finals => formula limitation, not staleness. The code comment claiming it under-counts is wrong for the deployed path. => "root like TNT -> cheap exact union-of-finals" refactor is a mirage. Adds dev/profiling/exactness-gate.md (verdict) + exactness-gate.R (harness). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The DRIFT phase (ts_drift.cpp) and standalone spr_search (ts_search.cpp) still scored pure-EW reconnection candidates with the union-of-finals approximation (fitch_indirect_length_bounded, reading the incrementally-maintained tree.final_), never migrated to the exact directional edge set that main TBR adopted in 2b299e4. Severity (dev/profiling/drift-exactness-gate.md): no inexact score is ever leaked (every accept is drift_full_rescore'd / full_rescore-verified), but on the deployed incremental path the union error is TWO-SIDED (over- AND under-counts up to ~100 steps, not the strict over-count the exactness gate measured with fresh finals). argmin selection then suffers an optimizer's curse (picks moves the scan is optimistic about), so drift admits moves outside its AFD envelope (env_violation 52-62% on Zhu2013/Zanol2014) and spr_search converges prematurely (union 810 vs exact 639). Fix (opt-in, union stays default; NA/IW untouched, gate !has_na && !use_iw): - TS_DRIFT_EXACT routes drift's SPR + reroot EW scans to fitch_indirect_length_cached over a per-clip compute_insertion_edge_sets, mirroring tbr_search. - TS_SPR_EXACT does the same for spr_search. - TS_DRIFT_SCANCHK: two-sided scan-error + decision census (select_flip, env_violation, applied-move predicted-vs-full_rescore; cf. TS_IW_SCANCHK). Validation: exact path applied_mismatch == 0 across all applied moves incl. reroots; default (flags-off) path byte-identical (drift/SPR/char-ordering tests pass). Local matched-wall pilot: frontier CROSSOVER -- exact wins the auto/ wall-race regime on all 3, union wins the thorough/saturated regime on 2/3 -> land opt-in, do not flip the default. spr_search is an unambiguous reach win. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Dedicated build (lib-driftexact, does not clobber the shared cpp-search lib)
fetching claude/happy-jennings-e7a165, and a 3-task array (one per dataset)
running dev/profiling/drift-exactness-gate-bench.R at 30 seeds x nCycles
{8,16,32,64,128,256}, union vs TS_DRIFT_EXACT, to resolve the matched-wall
plateau-vs-climb crossover the local pilot showed.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
lib-driftexact holds only the branch's TreeSearch; point R_LIBS_USER at the shared TreeSearch/lib so Rcpp/TreeTools/ape resolve at build+run time while the harness loads our TreeSearch via lib.loc=GATE_LIB. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Hamilton array (17870087, 30 seeds x nCycles {8..256}, 3 datasets) confirms the
8-seed pilot's matched-wall CROSSOVER with clean monotone gaps: exact wins the
auto/wall-race regime on all three (~2.7-3x faster, no time-to-optimum
regression), union wins the thorough/saturated regime on 2/3 (Zhu2013, Zanol2014
- exact plateaus, union over-diversifies deeper); Dikow2009 exact dominates every
budget. best-of-30 equal on all three (reliability effect, not reachability).
Verdict: LAND OPT-IN (TS_DRIFT_EXACT), do not flip the drift default. Adds the
per-dataset raw CSVs and the frontier reanalysis script.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The nc<=256 gate compared a censored exact tail (exact ~3x faster, so frozen at
its nc256 score with unspent wall) against union's fuller curve; exact was still
descending nc128->256, not plateaued. So "union wins thorough on 2/3" is not
established. auto/wall-race win stands. Extension (exact+union nc {512,1024} on
Zhu2013+Zanol2014) running to settle at matched wall. Opt-in landing unaffected.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Extended exact+union to nc {512,1024} on Zhu2013/Zanol2014 (30 seeds) to remove
the nCycles-cap censoring. Exact does NOT plateau (Zhu 624.80->624.40) but
descends slower than union; at genuinely matched wall union stays ahead through
the saturated tail (Zhu floor 624.0 vs exact ~624.4; Zanol narrower ~0.13). So
the auto/thorough crossover is real, not an artifact: exact wins the wall-race
regime on all three, union wins the thorough regime on 2/3.
Verdict CLOSED: land TS_DRIFT_EXACT opt-in, do not flip the drift default. Adds
the extension CSVs and the combined-frontier reanalysis script.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…route B) Full "thorough" search with driftCycles=0 (main-tree drift off; verified gd=0 => 0 drift calls, gd=25 => all-sectoral) isolates the in-sector godrift scorer. Three arms none/union/exact, maxReplicates knob, matched-wall frontier. Array adds mbank_X30754 (180t) as the larger/harder case where sector drift is most active. Reuses lib-driftexact (no rebuild). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
spr_search is a pure hill-climb with no exact-TBR phase to launder union's over-count, so union converges catastrophically short (multistart best-of-25: Zhu2013 650 vs exact 625; Zanol2014 1292 vs 1265 — systematic, not variance), and exact is also faster per start. No crossover, unlike drift. Flip default to exact; TS_SPR_UNION restores the old scorer (kill switch). Tests green (SPR/spr-nni-opt/spr-state-restore/drift). Reach today = standalone ts_spr_search (sprFirst OFF in presets), but this makes an SPR warmup viable. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…atched wall) From a shared per-seed local optimum, compare where a wall-second is best spent: incumbent ratchet vs exact-drift vs union-drift (control). If exact-drift's anytime frontier beats ratchet's over some wall range and dominates union-drift, drift earns a place in wall-limited recipes (route 3) -> worth re-tuning. mbank_X30754 (180t) is the large/hard anchor. Reuses lib-driftexact. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…rpus recipe test arm A (+replicate): is a wall-second better on exact-drift than on the project's real incumbent lever, more RAS+TBR replicates? Same datasets as route3 for frontier comparison. arm B' (recipe): whole-search thorough on TRAINING-split MorphoBank (EW-recoded, one key/task, sector drift off), arms none/union/exact main-tree drift, to see if the drift scorer choice moves a wall-limited recipe. Both reuse lib-driftexact. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…edundant) Marginal value (from a shared local-opt): exact-drift is the best single continuation, beating ratchet + union-drift + replicate. But the whole-search recipe test on 8 training-split MorphoBank keys (EW, sector-drift off, sequestered) shows main-tree drift is REDUNDANT in the multistart+ratchet+ sectorial+TBR ensemble at both saturated (none ties + faster) and tight-budget ends (exact beats none only 1/4 keys). So the isolated win doesn't survive -> no recipe surgery; exact-drift stays opt-in (TS_DRIFT_EXACT), D preset-refactor not worth it. Adds companion note + raw CSVs + analysis. (spr_search exact default stands as the real landed win from this line.) Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
ms609
added a commit
that referenced
this pull request
Jul 16, 2026
Fail-fast discriminator for whether an EXACT SPR warmup (now that spr_search is an exact hill-climb, PR #273) beats the incumbent NNI warmup as a handoff to TBR. 4 arms (nni/spr_ex/spr_un-control/none) at wagnerStarts=1 (SPR's best case), pipeline stripped so the returned score IS the end-of-first-TBR handoff. Size-stratified training-split MorphoBank, weighted large (the only region a better SPR basin can survive TBR subsuming SPR). Reuses lib-driftexact, no rebuild. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Migrates the pure-EW spr_search hill-climb off the union-of-finals insertion-cost approximation onto the exact directional edge set (the fix
tbr_searchadopted in 2b299e4), and adds the same as an opt-in for the drift phase, plus the instrumentation and matched-wall evidence behind the decision.Landed behaviour change (the win)
spr_search→ exact directional scorer by DEFAULT (ts_search.cpp). Union is a pure hill-climb with no exact-TBR phase to recover, so its over-count trips thedominatedgate and skips improving moves → converges catastrophically short (multistart best-of-25: Zhu2013 650 vs exact 625; Zanol2014 1292 vs 1265), and exact is also faster per start. No crossover. Kill-switch:TS_SPR_UNION. Reaches the standalonets_spr_search(sprFirst is off in all presets today).Opt-in / instrumentation (default behaviour unchanged)
TS_DRIFT_EXACT— exact directional scorer for the pure-EW drift SPR + reroot scans (drift default stays union).TS_DRIFT_SCANCHK— two-sided scan-error + decision-flip census (cf.TS_IW_SCANCHK).!has_na && !use_iwgate).Findings (dev/profiling/)
applied_mismatch==0).auto/wall-race regime on all 3, union wins thethorough/saturated regime on 2/3 → drift default stays union (opt-in exact).CI
Relies on GHA (ASan / memcheck / Windows) to validate the C++ change — not runnable locally on this MinGW box.
🤖 Generated with Claude Code