Review location precision against the STREET-LEVEL imagery Stage 1 actually consumes, not an aerial proxy (#96)

[jonfroehlich]

Why

#96 §5m established that supply is not the constraint — the ArcGIS Hub synonym sweep found 1,972,275 candidate records across 65 publishers (a floor; every query hit the page cap), against a 500k target. DVRPC's Philadelphia-region layer alone is 295,389, larger than the entire current training corpus.

So the binding constraint is assessment throughput. Today that costs, per city:

1. a municipal basemap hunt — which has failed in four distinct ways so far (§5e Esri grey tiles; §5h King County leaf-on canopy; §5h Charlotte 404ing ~25–35% of tiles that exist; and probe_basemap.py checking one downtown point while the sheet samples citywide),
2. a 60-chip sheet build,
3. an hour of the only reviewer we have,
4. …who then discarded 41.7% of Seattle to tree canopy (§5l).

At that rate the queue above is decades of work. The next investment belongs in the instrument.

The proposal (Jon's)

Review against the street-level imagery Stage 1 actually consumes: show the panorama, the projected crop Stage 1 would cut, and the government point's projected bearing — and have the reviewer judge whether the ramp falls inside that crop.

Why this is better than the aerial sheet, beyond convenience

1. It measures the actual gate quantity. §5g proved Stage 1 consumes the published coordinate only for its bearing, and cuts a ±18.37° strip around it. The aerial sheet measures a metric offset — a proxy — which then has to be pushed through a Monte Carlo to become a decision. This has the reviewer answer the real question directly.
2. It eliminates the basemap problem. No municipal basemap, so all four failure modes above go away at once. GSV coverage is uniform and global.
3. It looks under the canopy. Seattle's 41.7% unjudgeable was tree cover in plan view. Street level is beneath the trees.
4. Temporal matching becomes per-record. GSV has many panos per location, each with a capture date, so you select one captured after the ramp's install date. The confound that dominated §5i/§5l is eliminated, not mitigated.

What already exists — this is mostly assembly

The rendering path is the production path, not a reimplementation:

need	where
find panos near a lat/lng, with dates	stage_one/dataset_generation/search_panos.py
assemble the equirectangular pano	download_dataset.py::fetch_panorama(pano_id)
pano heading	download_dataset.py::heading_to_azimuth(get_pano_heading(...))
the exact crop Stage 1 cuts	equirectangular_to_perspective(equi, 90, azimuth, -30, 1024, 1024)[0:1024, 341:341+341]
the ±18.37° half-angle	scripts/analysis/stage1_offset_tolerance.py::crop_half_angle_deg() — import, never re-derive
frozen inventories + date parsing	data/inventories/, temporal_gap.py::parse_ym (carries the §5c sentinel handling)
sheet scaffold: sampling, strata, localStorage, export	scripts/analysis/inventory_review_sheet.py

Design decisions already taken

• Show the full 90° perspective with the strip boundaries overlaid, not the bare 341-px strip. The reviewer must be able to distinguish "ramp just outside the crop" from "no ramp here" — that distinction is the measurement.
• Record the verdict as an ANGULAR offset. This puts human review into the same units as §5j's automatic bearing residual (stage1_bearing_residual.py), whose null over 90,006 published records is |mean| ≤ 0.25°, |median| ~3.3°. §5j measures cities already in the corpus; this measures candidates; same units, so they cross-validate.
• Reuse the strata sampler (--strata-year-field), because §5l showed the dated/undated split is city-specific and must be earned per city.

Calibration plan — do this first

Pilot on Denver. It is the one city with a trusted aerial answer (§5f: median 0.29 m, 92.3% within 1 m, and §5g says its coordinate error costs Stage 1 0.21%). If the street-level tool independently returns "Denver is Good", the instrument is calibrated and can be trusted where aerial fails. If it disagrees, we learn that before betting the queue on it.

Second pilot: Seattle, where the aerial instrument demonstrably struggled (41.7% unjudgeable, MEDIUM confidence). If the street-level unjudgeable rate collapses there, that is the proof the tool is worth its build cost.

Risks, stated up front

• GSV endpoints are undocumented (search_panos.py and the tile fetch in download_dataset.py talk directly to them) and can break without notice. This is a real dependency risk that the aerial path does not have.
• ~1,900 tile requests per 60-chip sheet, and panorama assembly is slower than tile fetch — expect 10–20 min per build. Rate limiting is the dominant unknown, same as §7's Stage 1 generation concern.
• Angular offsets are not metric. This tool cannot produce a number comparable to Denver's 0.29 m or the paper's Table 1. That is acceptable — angular is what Stage 1 cares about — but it must be stated, and it means the aerial sheet is not fully retired.
• Different selection bias, not none: parked vans, poles and low sun replace tree canopy. Probably smaller, but it must be measured, not assumed.
• Sheet weight. 60 panoramas is a lot of pixels; the aerial sheet embeds tiles as data URIs and already reaches 3–5 MB. Will need downsampled context views plus full-res strips.

Lessons from the aerial sheet that must not be re-earned

• Two-path bugs. Twice now a fix landed on one of two paths that must agree: §5h's retry logic was inert because the cache stored absences as zero-byte files; §5l's stratum was added to the verdicts template but not the chip dict the export rebuilds from, so every exported stratum came back null. When adding a field, find the second path.
• Probe at the sample sites, not one point (probe_basemap_at_sites.py). The street-level analogue: check pano availability and date coverage at the actual sampled records before building.
• A drop count is a claim about the fetcher until checked against the sample. Charlotte's no_imagery_dropped: 59 was reported as "outside the basemap footprint" and was false.
• Null date sentinels — Boston's 18991230, the old 2000-01 — must go through parse_ym, never a fresh parser.
• State what the instrument cannot measure, next to the numbers (§5e's recall note is the template).

Scope for v1

Denver, 60 records, context view + strip overlay + projected crosshair, angular verdict, reusing the production projection path and the existing sheet scaffold. Everything else is follow-on.

🤖 Generated with Claude Code (claude-opus-5[1m])

[jonfroehlich]

Probe ran first — 58 of 59 Denver sites pick a panorama

probe_panos_at_sites.py against the aerial sheet's exact 59 records (committed: analysis_out/probe_panos_denver-co.json, c2ef43c):

pick rate	58/59 = 98.3%
chosen range	median 5.75 m (19 sites <5 m, 36 at 5–10 m, 3 beyond)
chosen capture	2016–2026, median ~2021; every pick postdates its record's CREATEDATE
failure	68791 — no_panos: a genuine GSV coverage gap, at a corner the aerial sheet measured cleanly (0.28 m)

Two findings worth their own lines:

1. The §5h failure class reproduced on the new endpoint, and was caught by the probe as designed. First pass: 15/59 sites failed with HTTP 502 from GetMetadata — rate limiting on pano-dense corners (one metadata POST per pano per search), every one recoverable. cached_search now retries transient statuses with backoff before believing them (cheap: the per-pano date lookup is lru-cached, so a retry only re-POSTs what the last attempt missed); schema drift still raises immediately. A drop count is a claim about the fetcher until checked — this one was 100% fetcher.
2. The pick rule samples the near field (nearest eligible pano ≈ the GSV car passing the corner), so §5o's criterion 1 was re-derived at the actual ranges before any review: expected outside-strip count ≈ 0.5 of ~55 measured (5× the naive pooled 0.21% figure — §5g's tolerance is tightest inside 5 m). Pass bar stays ≤1; 2 is investigated; ≥3 fails. Also: chosen captures are ~2021+, so the street sheet judges current reality where the 2016 aerial sheet measured digitising-time precision — an instrument disagreement on a ramp rebuilt in between is §5e's missing component finally being measured, not error. Both amendments are committed in §5o before the review.

Instrument status: rampnet/gsv.py lift + sheet + probe + summary are on feat/street-review-103 (stacked on #97's branch), 633 tests green. Next: build the 58-chip sheet (~2k tile requests, cache-resumable), smoke the export→summary round-trip, open the PR.

🤖 Generated with Claude Code (claude-fable-5)

[jonfroehlich]

The Denver sheet is built — 58 of 59 rendered, PR #105 open

Build 5035ec33, one pass: 58/59 sites rendered; the sole drop is 68791 (no_panos — the genuine GSV coverage gap the probe predicted, recorded as its own status, aerial had it clean at 0.28 m). The unfilled verdicts template — per-record pano choice, range, bearing, terminal status — is committed as the sheet's provenance (6d364d6). Export→summary round trip smoke-tested against the real sheet and the real aerial pairing.

A production find on the way: the tile endpoint now returns 403 PERMISSION_DENIED to python-requests' default User-Agent — i.e. the paper-era fetch_panorama fetches nothing today. Any explicit UA works (we send RampNet-sourcing/1.0). Fixed in rampnet/gsv.py, where Stage 1 and this instrument now share the code — the consolidation paying for itself on day one.

For the review hour (Jon)

1. Rebuild-or-open: analysis_out/review_denver-co-gsv/review_sheet.html (19 MB, file://; regenerates with the command in the script docstring — the cache makes it ~2 min). Check the header says build 5035ec33.
2. Per chip: click the ramp this record denotes (only the horizontal position measures; the amber dashed lines are the exact crop edges — they do NOT bound where you click), or p = no ramp (phantom), or u = unjudgeable plus a reason (1–7) — the page refuses to count a reason-less unjudgeable as done. Magenta dashed bearings are other published records, labelled with ground distance from this record. ? opens the full rubric.
3. Export (verdicts.json), replace analysis_out/review_denver-co-gsv/verdicts.json, then:
   python scripts/analysis/street_review_summary.py analysis_out/review_denver-co-gsv/verdicts.json --aerial-verdicts analysis_out/review_denver-co/verdicts.json

The pass/fail criteria are pre-registered in §5o (d5c5a89, amended c2ef43c with the probe-measured ranges — expectation ≈0.5 outside-strip of ~55 measured; ≤1 passes). If Denver fails, the instrument is wrong, not Denver.

🤖 Generated with Claude Code (claude-fable-5)