Skip to content

feat: exclude bad sensors from all analysis (#56) - #93

Open
matt-pharr wants to merge 1 commit into
developfrom
feature/exclude-bad-sensors
Open

feat: exclude bad sensors from all analysis (#56)#93
matt-pharr wants to merge 1 commit into
developfrom
feature/exclude-bad-sensors

Conversation

@matt-pharr

Copy link
Copy Markdown
Collaborator

Closes #56.

The problem

A misbehaving probe — dead channel, railed signal, wrong gain, disconnected
integrator — flowed straight into both analyses and silently skewed the
fits. There was no way to say "ignore this channel on this shot" short of
editing code, and a bad sensor is not a rare edge case for real DIII-D data.

What you get

Click a sensor in the Sensors view to drop it from every analysis. Click it
again (or use the banner) to restore it. Exclusions are saved per shot
that is how bad channels actually behave, bad on a shot — so they survive a
reload and are seen by anyone opening that shot.

Excluded sensors stay on the map, greyed and still clickable, so you can see
what you dropped rather than wondering where a probe went.

Where the seam sits

Below both analyses, not per view (the issue was explicit about this):

  • Rotating_array_channels() drops them, so every toroidal/poloidal
    node inherits it through _toroidal_arr / _pick_pair / _toroidal_grid.
    A future call site cannot forget to honor it.
  • Quasi-stationary — unioned into the existing fit_exclude, so the QS
    tab's own per-fit deselect and the shot-wide bad list both reach the fit
    instead of overriding one another.

The subtle half: cache correctness

Most analysis caches key on an explicit channel-name list, which already moves
when the channel set changes. But _spec_result is lru_cached on
(shot, stft params) while calling _pick_pair internally — so excluding a
probe could change the chosen pair and still serve the cached pre-exclusion
STFT. It now takes the exclusion tuple as a key argument; a test pins exactly
this.

Client-side, useNode folds a store revision counter into its fetch key, so one
toggle re-fits every open view without threading state through ~30 call
sites.

API

  • GET /api/exclusions/{shot}{shot, excluded: []}
  • PUT /api/exclusions/{shot} {excluded: [...]} (empty clears)

Stored in one small JSON file next to the shot data, written atomically
(temp + rename) so a crash can't truncate it. Gitignored — it's local operator
state, not repo content.

Verification

20 backend tests (store, API, both analysis seams, the cache-key trap) and
9 frontend tests (including that a failed save rolls back, so the map never
claims an exclusion the analyses aren't honoring).

Playwright, against the running app on real shot 184927 — because this is a
behavior change, per CLAUDE.md:

  • click a sensor → banner appears, server persists exactly 1 exclusion (MPID66M340)
  • geometry node reports it excluded and still returns it as a point (greyed, not hidden)
  • survives a full reload (persisted, not session state)
  • restore via the banner clears it server-side

All 8 checks passed. Gates: Python 504 tests / 79.68% coverage / ruff + ty; frontend
55 tests / eslint / tsc / build.

Decisions taken (the issue's open questions)

  • Per-shot, saved rather than session-only or device-file date ranges. The
    device-file option remains sensible later for a probe dead for a whole
    campaign; it is not in this PR.
  • Excluded sensors still render, greyed — you need to see what was dropped.
  • Auto-flagging (railed/flatlined heuristics) deliberately left as the
    follow-up the issue suggested.

🤖 Generated with Claude Code

A misbehaving probe — dead channel, railed signal, wrong gain — flowed
straight into both analyses and silently skewed the fits, with no way to
drop it short of editing code.

Click a sensor in the Sensors view to exclude it. Exclusions are saved per
shot (that is how bad channels behave: bad *on a shot*), so they survive a
reload and are shared by anyone opening that shot.

The seam sits below both analyses rather than per view:
  * rotating — _array_channels() drops them, so every toroidal/poloidal node
    inherits it via _toroidal_arr/_pick_pair/_toroidal_grid; a new call site
    cannot forget to honor it;
  * quasi-stationary — unioned into the existing fit_exclude, so the QS tab's
    own per-fit deselect and the shot-wide bad list both reach the fit.

Cache correctness is the subtle half. Most analysis caches key on an explicit
channel-name list, which already moves when the channel set changes — but _spec_result
keys on (shot, stft params) while calling _pick_pair internally, so excluding
a probe could change the pair and still serve the cached pre-exclusion STFT.
It now takes the exclusion tuple as a key argument (test pins this).
Client-side, useNode folds a store revision counter into its fetch key, so
one toggle re-fits every open view without threading it through ~30 calls.

Excluded sensors stay on the map, greyed and clickable, so you can see what
you dropped and restore it — plus a banner listing them with restore/clear.

Verified live with Playwright against real shot 184927: click → persisted →
honored by the geometry node → survives reload → restore clears it.

Closes #56

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Exclude specific sensors/channels from all analysis (bad-data drop)

1 participant