Pulse never prunes scheduler state for removed jobs, so a deleted failing job reds the menu bar forever

[xmasyx]

Summary

Pulse persists per-job scheduler state in state/state.json keyed by job name, but nothing ever removes entries for jobs that no longer exist in the config. If a job is deleted from PULSE.toml (or a user overlay TOML) while it is in a latched-failure state, its record survives forever — and the menu bar keeps reporting it as failing, permanently, with no way to clear it from the UI.

Root cause

LIFEOS/PULSE/pulse.ts reads state once at startup and never reconciles it against the loaded config:

const config = await loadPulseConfig()
let state = await readState(STATE_PATH)
state.startedAt = Date.now()

const enabledJobs = config.jobs.filter((j) => j.enabled)

Writes are keyed by name and additive only:

state.jobs[job.name] = { lastRun: Date.now(), lastResult: "error", consecutiveFailures: failures }

The menu bar then counts every record in the map, regardless of whether the job still exists:

// LIFEOS/PULSE/MenuBar/PulseMenuBar.swift:152
let failing = state.jobs.values.filter { $0.consecutiveFailures >= 3 }.count
if failing > 0 { return ("Failing — \(failing) job\(failing == 1 ? "" : "s")", .systemRed, state) }

Reproduction

1. Add a job that always fails (e.g. one whose command hangs past the runner timeout).
2. Let it reach consecutiveFailures >= 3. Menu bar goes red.
3. Delete that job block from the TOML and restart Pulse.
4. Menu bar is still red, forever. The job no longer exists anywhere in the config.

The only remedy today is manual surgery: stop the daemon, hand-edit state/state.json, restart. It has to be done with the daemon stopped, because state is held in memory and rewritten on every tick, so an edit against a running Pulse is silently overwritten.

This interacts badly with the existing retry-cooldown comment at pulse.ts:425, which explicitly frames "manual state surgery" as the thing that design was meant to avoid — the same escape hatch is still required here, and for orphans there is no cooldown that will ever clear it.

Proposed fix

Reconcile state against config at startup, right after both are loaded:

 const config = await loadPulseConfig()
 let state = await readState(STATE_PATH)
 state.startedAt = Date.now()
+
+// Drop scheduler state for jobs that no longer exist in the config. Without
+// this, a removed job keeps its latched failure record forever and the menu
+// bar counts it as failing with no way to clear it from the UI.
+const configured = new Set(config.jobs.map((j) => j.name))
+for (const name of Object.keys(state.jobs)) {
+  if (!configured.has(name)) {
+    log("info", "Pruning state for job no longer in config", { job: name })
+    delete state.jobs[name]
+  }
+}

Pruning against config.jobs rather than enabledJobs is deliberate: a job that is merely enabled = false should keep its history, since re-enabling it later should not look like a brand-new job. Only a job that has actually been removed from the config gets dropped.

[waveman2020-sudo]

This is Shiva, Eugene's AI Assistant, reporting on Eugene's behalf.

Confirmed the failure mode and applied your proposed diff essentially as
written — reconciling state.jobs against config.jobs at startup, pruning
against every configured job rather than just enabled ones so a merely
disabled job keeps its history. Verified with isolated tests: a removed job
gets pruned, a disabled-but-present job doesn't.