Review SEQUENCE_RELAXATION_SECONDS (2h): one object's track can span multi-hour gaps on a single pixel of overlap

[Chouffe]

Context

Follow-up to #276. While investigating why /localize/57 (alert 55664) showed a smoke object and a false-positive object 119 minutes apart, the 2-hour attach window in the offline clustering replay turned out to be the mechanism worth reviewing on its own.

Audit run 2026-08-04 against the local stack (475 lanes).

Mechanism

object_clustering.cluster_objects replays pyro-api's association rule offline. A detection attaches to an existing open object when two conditions hold (object_clustering.py:168-183):

candidates = sorted(
    (obj for obj in open_objects
     if (det.recorded_at - obj.last_seen).total_seconds() <= relaxation_seconds),
    key=lambda obj: obj.last_seen, reverse=True)
for obj in candidates:
    if bboxes_overlap(obj.last_box, det.box):
        matched = obj
        break

with SEQUENCE_RELAXATION_SECONDS = 7200 (object_clustering.py:35) and bboxes_overlap requiring only any pixel overlap, no IoU threshold (object_clustering.py:84).

So: a box appearing up to two hours after an object was last seen joins that object's track, on the strength of a single pixel of overlap. There is no check that the intervening time is plausible for one physical object.

The module's own header flags this as a known simplification:

Offline note: all frames of one alert sequence already share a camera/pose, so the per-camera filtering collapses; the 2h relaxation window is effectively always satisfied, while the 5min/3-detection spawn rule still shapes how many distinct objects are carved out. The thresholds remain parameters so the behaviour can be tightened.

"Effectively always satisfied" is the crux: within one alert sequence, the temporal gate does no work at all, leaving a single pixel of overlap as the only thing joining two boxes into one object.

Evidence

Lanes whose own consecutive frames jump by more than a plausible cadence, out of 475:

max internal gap	lanes
> 10 min	88
> 30 min	34
> 60 min	12

Worked example: lane 51 (alert 55738)

has_smoke = true, processing_stage = SEQ_ANNOTATION_DONE — already classified, currently sitting in the localize queue. One object, one lane, one classification:

Segment	Frames	Gap to next
13:39:31 → 13:40:31	8	115 min
15:35:31 → 15:37:31	5	28 min
16:06:00 → 16:15:01	17	—

Three detection episodes spanning 2.5 hours, merged into a single tracked object because each new burst happened to overlap the previous burst's last box.

Ruled out: this is not a --frames-limit sampling artifact

Many affected lanes have exactly 30 frames (the --frames-limit default), so the obvious objection is that we sampled across a long window. We don't — sequence_fetching.py:198-206 truncates:

for detection in detections:
    if unique_cap is not None and len(unique_detections) >= unique_cap:
        break

It takes a contiguous run of images and stops. The gaps are present in the source data, not introduced by the cap.

What I can't determine

Whether these merges are wrong. Two readings, and the data doesn't separate them:

1. Legitimate. A wildfire plume can persist for hours while the detector loses and reacquires it. Re-attaching is then correct, and the gap reflects detector dropout.
2. Spurious. Two unrelated events in the same image region — a morning plume and an afternoon cloud bank — merged because a single pixel overlapped after a 2-hour silence.

Deciding needs eyes on the imagery for a few of the 12 lanes with >60 min gaps.

Why it matters either way

Even under the generous reading, a merged lane is one annotation unit covering multiple episodes:

• One classification for several events. Lane 51 gets a single smoke/FP/unsure answer covering 13:39, 15:35, and 16:06. If the plume was real at 13:39 and the 16:06 boxes are a cloud, there is no way to say so.
• One localize submit. The frame grid renders a timeline with two multi-hour holes, and the per-object "N of M frames" progress treats episodes hours apart as one object's span.
• It compounds One platform_alert_id can group temporally disjoint detection episodes (up to 119 min apart) #276. That issue is about lanes grouped into one alert; this is about frames grouped into one object. Same root cause shape, one level down.

Questions to settle

1. Should the offline replay keep parity with pyro-api's 7200s, or diverge? The module was written for parity, so tightening is a deliberate departure and should be recorded as one.
2. If we tighten, what value? SEQUENCE_MIN_INTERVAL_SECONDS = 300 (the spawn window) is the natural candidate — attaching within 5 minutes, spawning a new object beyond that. That would split lane 51 into three objects.
3. Should bboxes_overlap gain an IoU floor for long-gap attachments specifically? A pixel of overlap is weak evidence after two hours, and much stronger after 30 seconds — the required overlap could scale with the gap.
4. Is the upstream pyro-api sequence itself the thing to fix? Lane 51's three episodes arrived inside one alert-API sequence, so pyro-api's own open-sequence window produced the 2.5-hour alert before our split ever ran.

Reproduction

Lanes with large internal gaps:

WITH g AS (
  SELECT d.sequence_id, extract(epoch FROM (d.recorded_at
         - lag(d.recorded_at) OVER (PARTITION BY d.sequence_id ORDER BY d.recorded_at))) AS gap
  FROM detections d)
SELECT g.sequence_id, s.platform_alert_id, sa.has_smoke, sa.processing_stage,
       round(max(g.gap) / 60) AS max_internal_gap_min, count(*) AS frames
FROM g
JOIN sequences s ON s.id = g.sequence_id
JOIN sequences_annotations sa ON sa.sequence_id = s.id
GROUP BY 1, 2, 3, 4 HAVING max(g.gap) > 600
ORDER BY 5 DESC;

One lane's frame-by-frame gaps:

SELECT d.recorded_at::time AS t,
       round(extract(epoch FROM (d.recorded_at
             - lag(d.recorded_at) OVER (ORDER BY d.recorded_at)))) AS gap_s
FROM detections d WHERE d.sequence_id = 51 ORDER BY d.recorded_at;

Related

• One platform_alert_id can group temporally disjoint detection episodes (up to 119 min apart) #276 — one platform_alert_id grouping temporally disjoint episodes (same shape, alert level)
• Import: sibling boxes from others_bboxes — lane coverage audit, silent drops, and cross-alert lanes #266 — below-threshold sibling boxes dropped silently at import (the other end of the same thresholds)
• Object-split cross-dedup leaves orphan others_bboxes and duplicates sibling objects across overlapping alerts #262 — object-split cross-dedup