Compare every baseline at the setting its documentation recommends - #53
Merged
Conversation
HTD ran at its stock invocation, a single min-fill pass that ends long before the ten seconds are up, while the other anytime solvers searched to the end of them. It has been rerun at `--opt width --iterations 0 --strategy challenge`, the setting its documentation recommends for the smallest width inside a fixed time limit, and the table holds that run. HTD gains on every column. Because "Exact best" is the smallest width seen among the displayed solvers, a stronger HTD lowers the exact-best count of every other row, goatd's from 8,745 to 8,205. The paragraph under the table now states how the solvers are run: at their recommended settings, searching to the time limit, with goatd stopping at its own hard cutoff just under it. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The README comparison table measured every solver over 9,413 selected graphs
at ten seconds, but HTD was the only entry running at its stock invocation:
a single min-fill pass that ends long before the ten seconds are up, while
the other anytime solvers searched to the end of them. That is not a
comparison of what HTD can do in ten seconds. It has been rerun at
--opt width --iterations 0 --strategy challenge, the setting its owndocumentation recommends for the smallest width inside a fixed time limit,
and this pull request replaces the table with that run. The paragraph under
the table gains three sentences: that each solver runs at its recommended
setting, naming HTD's; that the anytime solvers search to the limit and
report whatever they hold when the harness stops them; and that goatd stops
itself at its hard cutoff just under the limit. Nothing outside README.md
changes.
The rows that moved, exact-best count and its share of the 9,413:
HTD was rerun over both shards of the corpus under the same lane conditions
as the rest of the field: the same graphs, the same ten-second limit on one
pinned CPU with the same memory cap, the same seed, and the same validator on
the output. Its own numbers rise on every column, nontrivial decompositions
from 9,139 to 9,203 and exact best from 328 to 2,460, part width and part
coverage, since the challenge strategy banks a decomposition before it starts
searching instead of printing nothing until a min-fill pass completes. Every
other row falls, because "Exact best" counts the smallest width seen among
the displayed solvers and a stronger HTD supplies a smaller one on graphs
where it now wins. goatd's within +1 and within +4 columns move for the same
reason. No solver other than HTD was rerun, and no solver code changed.
Pull requests #50, #51 and #52 are open and each carries its own goatd row,
so this table needs one more refresh after whichever of them merges first.