Mine hard positives from multi-view consensus: a training-label source that does not depend on city inventories (#96, #59)

[jonfroehlich]

#96 concludes that 500,000 ramps is not reachable from assessed city inventories — good plus every OK city lands at 470,513 — which makes inventory assessment the critical path. #59 asks the complementary question of whether 2–3M labels would even buy recall.

Multi-view fusion in the auto-labeler (ProjectSidewalk/sidewalk-auto-labeler#27) opens a label source that does not depend on city inventories at all, and that is targeted at the model's own failures rather than sampling more of what it already gets right.

The mechanism

A fused site with ≥3 corroborating operational views is almost certainly a real ramp — the ghost check puts other-pano support at 0.967 for verdict-true detections vs 0.636 for verdict-false. Now find the panos that stood close enough to see that ramp and produced no detection for it. Each one is a miss by the detector at a known world position, which projects back into that pano's pixels as a training target. No human review anywhere in the loop.

That the misses exist in quantity is already established: own-view (per-pano) recall is 0.72–0.83 while the union across views is 0.945–1.000, so for 12–28% of ramps some view saw it and the view we scored did not.

Measured yield

Sites with ≥3 operational views, against panos within radius that contributed no member (auto-labeler runs, five cities):

city	strong sites	mined ≤10 m	mined ≤15 m	≤15 m as ×(operational detections)
paterson	4,553	1,091	3,974	0.13×
gainesville	1,859	499	1,652	0.12×
sao_paulo	1,947	437	1,382	0.11×
richmond	945	1,210	3,011	0.32×
bend	9,709	1,281	5,448	0.11×

This is an upper bound. Proximity is standing in for visibility — occlusion is not modelled at all (see risks).

Be honest about the size: this is thousands per city, not millions. Against a 276,615-label corpus it is a supplement, not a route to 500k on its own. The argument for it is not volume, it is per-label value — these are examples the model demonstrably failed on, which is the population #84's auto-label-ceiling question is really about. It also scales linearly with cities processed, in cities that have no inventory and never will.

Risks, in the order they would bite

1. RampNet#101 is a hard prerequisite. The flat-ground range carries a ~11% range-proportional scale error. Mining bakes that straight into the training targets, and it is systematic, not random — the model would learn our geometry bug as ground truth, and every downstream position measurement would then agree with it for the wrong reason. Do not mine before that is anchored.
2. Occlusion is the accuracy ceiling on the yield above. A ramp behind a building, a parked van, or the capture vehicle's own body is a legitimate non-detection. Labelling it as a miss teaches the model to hallucinate ramps behind obstructions — actively worse than not training on it. A visibility test is not optional.
3. Self-training feedback. Consensus among correlated errors is still an error; the model would reinforce its own biases. The ghost-check separation is a handle on this, not a solution.
4. Points, not extent. Mined labels inherit the current point supervision, so this does nothing for Extend RampNet output from points to boxes/polygons (point-prompted SAM2; CenterNet-style size head; segmentation) #83 and cannot be a route to mensuration.

The first step is not the miner

Measure the precision of the mined set before building anything that consumes it. On a benchmark city, take the mined candidates that fall in the 125 judged panos and check them against the reviewer's missed-ramp marks: a mined candidate that coincides with a missed mark is a true mined positive; one in a pano the reviewer marked no_missed is a false one — occlusion, a fusion error, or a ghost site.

That is a cheap experiment against verdicts that already exist, it needs no new labelling, and it converts the yield table above from an upper bound into a real number. If mined precision is low, the idea dies for the price of an afternoon.

🤖 Generated with Claude Code (claude-opus-5[1m])