-
Notifications
You must be signed in to change notification settings - Fork 1
Track authoritative terminal state for reconciliation #1033
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Changes from all commits
a26bbbc
9e79bb2
c22bfe3
b6eba2e
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,93 @@ | ||
| # Cross-Surface Impact Map: Issue #1031 Terminal Provenance | ||
|
|
||
| ## Task | ||
|
|
||
| - Task / issue: recover provisional transport failure during durable | ||
| reconciliation, #1031. | ||
| - Plan link: `2026-07-30-issue-1031-terminal-provenance-plan.md`. | ||
| - Owner: Codex. | ||
| - Status: implemented; verification and merge pending. | ||
|
|
||
| ## Current Ownership, Callers, and Data Flow | ||
|
|
||
| - Entry points: `RunSession.submit`, per-run/conversation SSE, and durable Chat | ||
| reconciliation. | ||
| - Owners/source of truth: server terminal events are authoritative; | ||
| `Transcript.markFailed/markCancelled` are local presentation fallbacks; | ||
| `RunSession` owns async event provenance. | ||
| - Consumers: transcript status/error UI and callback/cron conversation replay. | ||
| - Similar abstractions searched: event dedupe, run start/reset/load/rebind, | ||
| transcript load/reconcile, local terminal events. | ||
| - Search evidence: | ||
| `rg -n 'markFailed|localTerminalEvent|runFailed|reconcilePersistedMessages|seenEventIDs' macapp/Sources macapp/Tests`. | ||
| - Ownership conclusion: store provenance in `RunSession`; do not infer it from | ||
| the reduced transcript and do not create a second transcript. | ||
|
|
||
| ## Config, API, CLI, and Tools | ||
|
|
||
| - Config/defaults/env/saved settings: none. | ||
| - Endpoints/request/response/server wiring: none. | ||
| - CLI/tools/wire/integrations: none. | ||
| - Error states: a local transport failure becomes provisional, blocks another | ||
| prompt, ignores older durable completion snapshots, and may recover only | ||
| after an authoritative terminal event; server failure/cancellation remains | ||
| terminal. | ||
|
|
||
| ## Persistence and Compatibility | ||
|
|
||
| - Schemas/migrations/caches/generated data: none. | ||
| - Compatibility: native-client-only interpretation of unchanged events and | ||
| messages. | ||
| - Mixed versions: old clients may retain false failure; new clients recover. | ||
|
|
||
| ## Lifecycle, Security, and Reliability | ||
|
|
||
| - Concurrency/lifecycle: mark an accepted/started run unresolved until a | ||
| deduped authoritative terminal event arrives; clear provenance when the | ||
| conversation changes, but reconcile a same-conversation rail reload in | ||
| place without cancelling the current stream. Remember terminal run IDs so a | ||
| late start response cannot overwrite an earlier conversation terminal event. | ||
| - Auth/security/privacy/secrets: none; no new data. | ||
| - Recovery/idempotency: repeated durable reconciliation yields the same state; | ||
| no durable data repair. | ||
|
|
||
| ## Product and Integration Surfaces | ||
|
|
||
| - Server/runtime: unchanged. | ||
| - TUI/web/macOS: macOS only; TUI/API unaffected. | ||
| - Provider/model/tool routing: none; downstream client reduction only. | ||
| - External automation: cron/callback consume the repaired conversation path. | ||
| - UX/accessibility/motion: status correctness only; no layout/input change. | ||
|
|
||
| ## Deployment and Operations | ||
|
|
||
| - Order/flags/migrations: native patch, no flag or migration. | ||
| - Diagnostics: deterministic transport-failure test plus engineering log. | ||
| - Rollback: revert if authoritative failure/cancellation stops persisting. | ||
| - Runbooks: none; operator contract unchanged. | ||
|
|
||
| ## Regression Tests | ||
|
|
||
| - First red: transport exception -> local failed -> same-conversation rail | ||
| reload with an old durable snapshot falsely reports completed, clears the | ||
| error, and permits another prompt. | ||
| - Acceptance: preserve provisional failure and submission lock until a delayed | ||
| authoritative completion, then recover and clear the error. | ||
| - Controls: authoritative failed/cancelled and completed replay dedupe. | ||
| - Ordering control: terminal conversation SSE released after the start request | ||
| but before its 202 response leaves the completed run submit-ready. | ||
| - Real path: shared native callback/cron acceptance after #1031, #1032, #1027. | ||
| - Commands: | ||
| `swift test --package-path macapp --filter transportFailureWaitsForAuthoritativeCompletion`; | ||
| `swift test --package-path macapp`; | ||
| `./scripts/test-regression.sh`. | ||
|
|
||
| ## Documentation and Handoff | ||
|
|
||
| - Before code: plan and impact map. | ||
| - After code: engineering/long-term logs and plans index/status. | ||
| - Release/training: none. | ||
|
|
||
| ## Warning Check | ||
|
|
||
| - All applicable surfaces are mapped; none entries include rationale. |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,69 @@ | ||
| # Plan: Distinguish Authoritative and Local Terminal State | ||
|
|
||
| ## Context | ||
|
|
||
| - Governing GitHub issue: #1031. | ||
| - Problem: durable reconciliation cannot distinguish a server `run.failed` | ||
| event from the local failure placeholder used when an SSE transport dies. | ||
| - User impact: a completed scheduled or user run can remain permanently failed | ||
| in the macOS GUI after a transient disconnect. | ||
| - Constraints: preserve authoritative failed/cancelled state, completed replay | ||
| deduplication, and current server/wire contracts. | ||
|
|
||
| ## Scope | ||
|
|
||
| - In scope: track the latest authoritative terminal event separately from | ||
| rendered transcript state and use it during durable reconciliation. | ||
| - Out of scope: transport retry redesign, server changes, cron/callback schema | ||
| changes, and historical failure retention across later runs (#1032). | ||
|
|
||
| ## Documentation Contract | ||
|
|
||
| - Feature status: `implemented`. | ||
| - Public docs affected: none. | ||
| - Spec docs to update before code: this plan and impact map. | ||
| - Implementation notes to add after code: engineering log and plans index. | ||
|
|
||
| ## Test Plan (TDD) | ||
|
|
||
| - New failing test: per-run transport failure creates a provisional local | ||
| failure; an older durable snapshot cannot report success or enable another | ||
| prompt when the user reloads the already-selected rail row before a delayed | ||
| authoritative completion event arrives. | ||
| - Existing controls: authoritative failed/cancelled replay remains preserved; | ||
| completed persisted replay remains deduplicated; a terminal conversation | ||
| event arriving before its delayed start response does not re-lock the run. | ||
| - Full verification: strict formatter, full Swift package, and repository | ||
| normal/race/coverage gate. | ||
|
|
||
| ## Cross-Surface Impact Map | ||
|
|
||
| - See `2026-07-30-issue-1031-terminal-provenance-impact-map.md`. | ||
|
|
||
| ## Implementation Checklist | ||
|
|
||
| - [x] Define acceptance criteria in tests. | ||
| - [x] Link a contract-complete structured GitHub issue before implementation. | ||
| - [x] Record current architecture, callers, consumers, and source-of-truth search evidence. | ||
| - [x] Document feature status and exact contract before code. | ||
| - [x] Complete and reconcile the cross-surface impact map before implementation. | ||
| - [x] Add characterization coverage before structural refactors. | ||
| - [x] Write failing tests first. | ||
| - [x] Review ownership/copy semantics for exported or state-storing types when mutable fields cross boundaries. | ||
| - [x] Implement minimal code changes. | ||
| - [x] Refactor while tests remain green. | ||
| - [x] Update docs, status ledgers, and indexes. | ||
| - [x] Update engineering/system/observational logs as needed. | ||
| - [x] Run full test suite. | ||
| - [ ] Merge branch back to `main` after tests pass. | ||
|
|
||
| ## Risks and Mitigations | ||
|
|
||
| - Risk: an old authoritative failure could leak into a later run, or an old | ||
| message snapshot could falsely complete a currently unresolved run, or a | ||
| delayed start response could re-lock a run already completed by conversation | ||
| SSE. | ||
| - Mitigation: clear provenance on conversation replacement, track the | ||
| accepted-run-to-terminal interval explicitly, treat same-conversation load | ||
| as reconciliation, remember terminal run IDs, and test provisional blocking, | ||
| eventual recovery, and terminal-before-202 ordering. |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -49,6 +49,20 @@ public final class RunSession { | |
| // scale; revisit with an LRU/bounded cap if a conversation runs long | ||
| // enough for this to matter. | ||
| private var seenEventIDs: Set<String> = [] | ||
| /// The latest terminal state delivered by harnessd for the current run. | ||
| /// Local `markFailed` / `markCancelled` calls only unblock the UI after a | ||
| /// transport or local-control failure and intentionally do not update this | ||
| /// provenance. | ||
| private var latestAuthoritativeTerminalState: RunState? | ||
| /// True after harnessd accepts or starts a run and until its terminal event | ||
| /// arrives. A local stream failure may stop the spinner, but an older | ||
| /// durable snapshot must not turn that unresolved run into a success or | ||
| /// enable a second submission. | ||
| private var awaitingAuthoritativeTerminalState = false | ||
| /// A conversation event can reach the app before `startRun` returns its | ||
| /// accepted run id. Remember terminal run ids so that late 202 response | ||
| /// cannot re-lock a run the conversation stream already completed. | ||
| private var authoritativeTerminalRunIDs: Set<String> = [] | ||
|
|
||
| public init(client: HarnessClient) { | ||
| self.client = client | ||
|
|
@@ -59,18 +73,21 @@ public final class RunSession { | |
| } | ||
|
|
||
| public var isBusy: Bool { transcript.runState.isActive } | ||
| public var canSubmit: Bool { !draft.trimmed.isEmpty && !isBusy } | ||
| public var canSubmit: Bool { | ||
| !draft.trimmed.isEmpty && !isBusy && !awaitingAuthoritativeTerminalState | ||
| } | ||
| /// True while a run is active, so the composer can offer steering instead. | ||
| public var canSteer: Bool { isBusy && transcript.pendingApproval == nil } | ||
|
|
||
| // MARK: - Running | ||
|
|
||
| public func submit() { | ||
| let prompt = draft.trimmed | ||
| guard !prompt.isEmpty, !isBusy else { return } | ||
| guard !prompt.isEmpty, !isBusy, !awaitingAuthoritativeTerminalState else { return } | ||
| draft = "" | ||
| connectionError = nil | ||
| cancelRequested = false | ||
| latestAuthoritativeTerminalState = nil | ||
| promptHistory.append(prompt) | ||
| transcript.appendUserPrompt(prompt) | ||
|
|
||
|
|
@@ -101,6 +118,9 @@ public final class RunSession { | |
|
|
||
| let started = try await client.startRun(request) | ||
| currentRunID = started.runID | ||
| if !authoritativeTerminalRunIDs.contains(started.runID) { | ||
| awaitingAuthoritativeTerminalState = true | ||
| } | ||
| if self.conversationID == nil { self.conversationID = started.runID } | ||
| // Keyed by conversation, not by this run: on a conversation's | ||
| // later runs `self.conversationID` is already the first run's | ||
|
|
@@ -113,11 +133,15 @@ public final class RunSession { | |
| await apply(event, runID: started.runID) | ||
| } | ||
| } catch let error as HarnessError { | ||
| connectionError = error.message | ||
| transcript.markFailed() | ||
| if currentRunID == nil || awaitingAuthoritativeTerminalState { | ||
| connectionError = error.message | ||
| transcript.markFailed() | ||
| } | ||
| } catch { | ||
| connectionError = error.localizedDescription | ||
| transcript.markFailed() | ||
| if currentRunID == nil || awaitingAuthoritativeTerminalState { | ||
| connectionError = error.localizedDescription | ||
| transcript.markFailed() | ||
| } | ||
| } | ||
| currentRunID = nil | ||
| } | ||
|
|
@@ -175,8 +199,20 @@ public final class RunSession { | |
| // MARK: - Conversation switching | ||
|
|
||
| public func load(messages: [StoredMessage], conversationID: String) { | ||
| if self.conversationID == conversationID { | ||
| // Selecting the already-open rail row is a refresh, not a | ||
| // conversation replacement. Preserve an active stream and any | ||
| // accepted-run-to-terminal lock; reconciliation already ignores | ||
| // incomplete snapshots while either is unresolved. | ||
| reconcilePersistedMessages(messages) | ||
| trackConversationStream(conversationID) | ||
| return | ||
| } | ||
| streamTask?.cancel() | ||
| transcript.load(messages: messages) | ||
| latestAuthoritativeTerminalState = nil | ||
| awaitingAuthoritativeTerminalState = false | ||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
When an accepted run's per-run SSE fails, the transcript becomes non-busy while Useful? React with 👍 / 👎. |
||
| authoritativeTerminalRunIDs = [] | ||
| self.conversationID = conversationID | ||
| currentRunID = nil | ||
| connectionError = nil | ||
|
|
@@ -189,8 +225,10 @@ public final class RunSession { | |
| /// section was visible. An active user-started run remains event-driven so | ||
| /// an incomplete persistence snapshot cannot replace streaming state. | ||
| public func reconcilePersistedMessages(_ messages: [StoredMessage]) { | ||
| guard !isBusy else { return } | ||
| transcript.reconcile(messages: messages) | ||
| guard !isBusy, !awaitingAuthoritativeTerminalState else { return } | ||
| transcript.reconcile( | ||
| messages: messages, | ||
| authoritativeTerminalState: latestAuthoritativeTerminalState) | ||
| connectionError = nil | ||
|
Comment on lines
+229
to
232
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
When the per-run SSE fails before harnessd reaches a terminal state, Useful? React with 👍 / 👎. |
||
| pendingQuestions = nil | ||
| } | ||
|
|
@@ -203,13 +241,21 @@ public final class RunSession { | |
| streamTask?.cancel() | ||
| stopConversationStream() | ||
| transcript.reset() | ||
| latestAuthoritativeTerminalState = nil | ||
| awaitingAuthoritativeTerminalState = false | ||
| authoritativeTerminalRunIDs = [] | ||
| conversationID = nil | ||
| currentRunID = nil | ||
| connectionError = nil | ||
| pendingQuestions = nil | ||
| } | ||
|
|
||
| public func rebind(conversationID: String) { | ||
| if self.conversationID != conversationID { | ||
| latestAuthoritativeTerminalState = nil | ||
| awaitingAuthoritativeTerminalState = false | ||
| authoritativeTerminalRunIDs = [] | ||
| } | ||
| self.conversationID = conversationID | ||
| trackConversationStream(conversationID) | ||
| } | ||
|
|
@@ -284,6 +330,25 @@ public final class RunSession { | |
| /// would double every message (issue #950 requirement 4). | ||
| private func apply(_ event: HarnessEvent, runID: String) async { | ||
| guard seenEventIDs.insert(event.id).inserted else { return } | ||
| switch event.type { | ||
| case .runQueued, .runStarted, .runResumed: | ||
| latestAuthoritativeTerminalState = nil | ||
| awaitingAuthoritativeTerminalState = true | ||
| case .runCompleted: | ||
| latestAuthoritativeTerminalState = .completed | ||
| awaitingAuthoritativeTerminalState = false | ||
| authoritativeTerminalRunIDs.insert(runID) | ||
| case .runFailed: | ||
| latestAuthoritativeTerminalState = .failed | ||
| awaitingAuthoritativeTerminalState = false | ||
| authoritativeTerminalRunIDs.insert(runID) | ||
| case .runCancelled: | ||
| latestAuthoritativeTerminalState = .cancelled | ||
| awaitingAuthoritativeTerminalState = false | ||
| authoritativeTerminalRunIDs.insert(runID) | ||
| default: | ||
| break | ||
| } | ||
| transcript.apply(event) | ||
| await handleSideEffects(of: event, runID: runID) | ||
| } | ||
|
|
||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This same-ID fast path also handles the reloads after
ProjectSession.undoandrewind, not only rail refreshes. If the removed run failed or was cancelled,reconcilePersistedMessagespreserveslatestAuthoritativeTerminalStateand can re-add that run's error after loading the truncated history, leaving the restored conversation incorrectly marked failed/cancelled. Those destructive callers need a full load that clears terminal provenance, or a distinct refresh API.AGENTS.md reference: AGENTS.md:L57-L61
Useful? React with 👍 / 👎.