Skip to content

feat(voice): integrate private work checkpoints, host approval, and cancellation - #1007

Draft
silverstein wants to merge 25 commits into
feat/voice-livefrom
feat/voice-continuity-integration-20260916
Draft

silverstein wants to merge 25 commits into
feat/voice-livefrom
feat/voice-continuity-integration-20260916

Conversation

@silverstein

Copy link
Copy Markdown
Owner

Product direction

Mac-first discovery, portable ownership of intent and memory. The first integrated workflow is explicit context sharing → discussion → reviewed checkpoint → local park/resume. This is a working CLI/Voice Live integration, not a claim that the native Tauri experience is finished.

Built on feat/voice-live at 5a9d59fc, importing the portable contracts from #1004 (7df6a400) without modifying either active branch. No main merge, release, installed-app replacement, automatic cloud opt-in, or real outward action.

Implemented runtime behavior

  • minutes talk --local-work: offline checkpoint creation, debrief, private save/list/resume over local commands. No microphone, provider, API key, MCP launch or agent process.
  • The same /work new/show/park/list/resume/debrief/forget commands inside live talk. Resumption remains local until explicit /work share. Model checkpoint proposals remain suggestions; a debrief does not rewrite the original conversation.
  • Immutable content-addressed checkpoint storage through existing capability-bound policy_fs, bounded IO, owner-private permissions, symlink/identity checks, hashes and atomic no-replace writes.
  • Exact-payload local review for enabled model-requested writes, desktop mutations, MCP calls, agent delegation, screen capture and music. The model cannot redeem the old spoken token. /approve ID is processed only through the host's typed-control path; permissions are rechecked at execution. Private capsule content is shown to the local reviewer, not leaked in the model's proposal response.
  • /share-selection [bundle-id]: explicit, read-only macOS accessibility selection capture, executed off the audio loop with target/element/selection consistency checks. No clipboard, whole-screen or broad text fallback. Other platforms report unsupported. This is execution-time selection, not a new global hotkey/overlay.
  • Queued-call cancellation and duplicate-call suppression connected to the real runner. Already-running actions are explicitly not claimed to be undone; their late results are withheld after cancellation.
  • Actual Gemini transport and runner use standard/Extended Thinking capability profiles, non-blocking declarations, model-appropriate response scheduling, decoded interaction status, and distinct playback/provider/pending-call state. Outbound messages are bounded.
  • Normal-only meeting reads for live tools; deny the derived-insights path until it has live source provenance.
  • Full runtime regression tests, 27 portable contract tests, and an offline actual-CLI smoke script. CI runs the integrated Voice Live feature on Linux/macOS/Windows.

Validation and remaining release gates

Source patch application and portable tests succeeded in isolated CI. Full integrated builds/tests/lint are being exercised by the attached workflow; this paragraph does not claim they already passed.

No real microphone, accessibility selection, provider session, AppleScript/message send or delegated process has been exercised here. The Tauri hotkey/overlay and native click test in ~/Applications/Minutes Dev.app are not implemented/verified in this PR. It intentionally remains draft for native and live-provider verification.

This does not add an OS sandbox for external agents, hard cancellation of already-running external processes, a Herdr task service, general UI clicking, or full information-flow attestation of every existing brain/prep/calendar path. Approval is not a claim of sandboxing; read-only prompts are not a security boundary. Explicit /work share intentionally discloses the selected checkpoint, including its private interpretations.

Try without a provider

cargo build -p minutes-cli --no-default-features --features voice-live --bin minutes
./target/debug/minutes talk --local-work

Then /work new Simplify the offer, /work debrief Interested does not mean agreed, /work park, /work list, and /work resume <checkpoint-id>.

Architecture and actual commands: docs/architecture/voice-continuity-integration.md.

The local-only beads store was inaccessible, so no claims about shared issue state were made. Temporary source-transfer workflow and payload files have been removed from the resulting tree; the remaining CI workflow is read-only.

silverstein and others added 25 commits September 15, 2026 22:13
…erprint

The complete CLI manifest differs from the prior golden only by two comments and the opt-in voice-live feature. Worker binary entries, default features, dependencies, XPC authority and signing remain unchanged. All packaging and mutation self-tests remain enabled and pass.

Copy link
Copy Markdown
Owner Author

Integration and release plan

This branch implements the first CLI/runtime loop rather than expanding the desktop verb catalog: explicit selected context → conversation → locally reviewed checkpoint → private park/resume → deliberate sharing. The portable state remains useful without a live model or Mac adapter.

Acceptance boundary

  1. Portable/local loop: actual minutes talk --local-work creates, debriefs, parks, lists and resumes across separate processes without a provider credential or microphone. The same persistence implementation is used by Live.
  2. Live safety integration: model tool calls cannot authorize themselves; the typed host channel releases exact stored payloads. Queued cancellation prevents dispatch; already-running work is not described as undone. Restricted meeting exact reads fail closed. A private resumed capsule is not echoed to the model unless explicitly shared.
  3. Native usability gate (not completed here): wire the future Tauri hotkey/overlay into these existing host operations, not a parallel approval system. Capture the target before overlay focus, show the actual shared source and proposal, and verify Accessibility denial, stale selection, rejection, interruption and reconnect in Minutes Dev.
  4. Provider gate (not completed here): exercise both supported Live profiles with a real account. Compare wrong references, repeated context, sticky corrections, time to useful progress and verified action results—not first-audio filler latency.
  5. Delegation gate (not completed here): before advertising interruptible agent supervision, use a real executor with enforced permissions, durable task identity, stop acknowledgement and execution receipts. The present queue cancellation and human review are not a process sandbox or a Herdr task service.

The next product investment is the single native point-and-talk → park/resume experience. Continuous recording, autonomous clicking, arbitrary Shortcuts and multi-machine fleet management remain outside this experiment. Existing capture/CLI/MCP behavior stays a separate regression obligation; native feature parity is not required to test the new Mac interaction.

Packaging guard review

The inherited graph-packaging fingerprint failure was reviewed against the complete CLI manifest. Its only difference from the previously approved bytes is two comments plus voice-live = ["minutes-core/voice-live"]. Worker binary entries, default features and dependencies are unchanged. The fingerprint was updated only after verifying that removing those exact added bytes reproduces the old SHA-256; all graph/Apple Speech packaging and mutation self-tests pass locally and in the isolated review job. The guard itself was not removed or weakened.

Final CI results will be recorded separately against the final head. This remains a draft; no main merge, release, installed-app change or real outward action has occurred.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant