Skip to content

Restate the verification report's preamble, and declare its charset #110

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
jonfroehlich merged 3 commits into from
Aug 15, 2026
Merged

Restate the verification report's preamble, and declare its charset #110

Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
f059c78
Restate the verification report's preamble, and declare its charset
jonfroehlich Aug 5, 2026
624056d
Neutralise the report's heading too
jonfroehlich Aug 5, 2026
e12059c
Give the report a doctype, and finish neutralising section 7 (#110 re…
jonfroehlich Aug 15, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 10 additions & 5 deletions docs/eval_protocol_verification.html
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,3 +1,5 @@
<!doctype html>
<meta charset="utf-8">
<title>RampNet eval protocol — verification report</title>
<style>
:root{
Expand Down Expand Up @@ -65,9 +67,12 @@

<header class="col">
<div class="eyebrow">RampNet · internal verification · 2026-07-02</div>
<h1>Did we really over-count? A re-verification of the published evaluation protocol</h1>
<p class="sub">Requested by Jon: “Are you absolutely certain there was an error in the original paper and our
experimental approach? Double- and triple-check, produce visual examples of what we double-counted, and explain in greater detail.”</p>
<h1>Re-verification of the published evaluation protocol</h1>
<p class="sub">A re-examination of the matching rule used to score RampNet's detections, prompted by a
discrepancy identified after publication. Its scope: to establish whether predictions were genuinely
over-counted, to confirm that finding by independent means rather than rest on a single line of
evidence, to document the over-count with worked examples drawn from the released model's own
output, and to quantify its effect on the published metrics.</p>
</header>

<section class="col">
Expand Down Expand Up @@ -277,8 +282,8 @@ <h2><span class="no">6</span>Aggregate impact, measured on Hyak (all 1,000 panos
</section>

<section class="col">
<h2><span class="no">7</span>Corrections to my own earlier claims</h2>
<p>You asked me to triple-check; the check cut both ways. Three things in the drafted issue #9 / README erratum
<h2><span class="no">7</span>Corrections to earlier claims in this investigation</h2>
<p>The re-check cut both ways. Three things in the drafted issue #9 / README erratum
need fixing before anything is posted:</p>
<div class="tblwrap"><table>
<tr><th>Earlier claim</th><th>Verdict on re-check</th></tr>
Expand Down
Loading