feat: make a Provider outage visible and recoverable

[tatelilith]

Summary

A deployment with a single localSession Provider ran for hours with a green
「已登录」chip while every Run failed on your access token could not be refreshed because your refresh token was revoked — 36 identical failures in 24
hours. Nothing on the page contradicted the credential, and recovering meant
opening each failed Run's drawer one at a time.

This PR makes that class of outage visible, honest, and recoverable.

1. The Provider session is proven, not assumed.
codex login status answers from ~/.codex/auth.json alone, so a revoked token
still reads as a session. codex checkLoginStatus now follows a positive local
verdict with codex doctor and takes its verdict from the row that actually
reaches the vendor — the WebSocket handshake (connected (HTTP 101 …)), not the
auth row, which only reports that a credential file is configured. A refusal
(401 / 403 on the handshake) becomes a distinct CREDENTIALS_REJECTED state
(「登录态已失效」): nothing to install, nothing missing — log in again. Any other
transport failure (proxy, DNS, disabled WebSocket policy) is reported as
unverified rather than blamed on the token, and a verifier that cannot answer at
all (older CLI without the subcommand, timeout) leaves the previous behaviour
untouched — which is why codex still needs no minVersion floor.

LoginStatus.verified is surfaced, so a session is labelled 「已验证」or
「仅本地凭证」. Every other Provider CLI answers from disk, and none of them is
allowed to look like a confirmed session.

2. The evidence a probe cannot contradict. The Provider section now shows
the Agent's most recent failed Run and its error, calling out auth-shaped errors
outright. One real 401 beats any local check.

3. Failures are retried in place. POST /runs/:id/rerun filed a new row for
every replay, so the failed row stayed failed forever and one outage turned the
Runs list into pairs of identical intents with nothing marking which failures
were already handled. A failed run is now re-executed on its own row (status
CAS failed → pending, normal admission; the attempt lands as another
run_steps row — executeChatRun already numbers steps by MAX(order)+1 for
multi-turn chat, so it needed no change). The failed attempt is not erased: its
step, output and logs stay. completed / cancelled keep the new-row rerun,
and the automatic job-retry chain still files new rows — it runs unattended, and
erasing the failure it is recovering from leaves nothing to diagnose.

4. Bulk recovery. A 「全部重试」button next to the Runs refresh retries every
failed Run on the current page, oldest first, behind a confirm, disabled for
viewers (a rerun is a write on the Agent).

5. A how-to where the credential is configured. An info icon on
「使用服务器登录态」explains how to establish the session on the server —
including handing those steps to an Agent that holds a shell there.

Non-obvious failure modes this had to handle

• A retry must not become a resume. executionMetadata is rebuilt from an
  allowlist. A surviving liveChatId makes resolveQueuedChatId read the retry
  as a resume: the Agent is sent a continuation prompt instead of the intent and
  no chat message is recorded. The allowlist also drops whatever transient field
  is added next, which a denylist would carry forward. oauthResetSession /
  nativeChatResetSession are kept — being consume-once, still finding them
  means the failed attempt never spent them.
• gitTriggerOrigin.queued is forced false, or the staleness probe cancels
  a retry a human just asked for, turning a failure record into a cancellation.
• Restart recovery must tell a retry from an unconsumed event. A retried
  native-Feishu run keeps the original event's triggerSessionId while that
  event was consumed by the first attempt, so both the running and queued
  branches would fail it awaiting a replay that never comes. retryAttempt
  marks the row (awaitsFeishuEventReplay()), and requeueForResume restores
  its reply context on the same rule.
• A retry must not overlap the previous attempt's cleanup. failed is
  written before the execution lease is released, and the durable SCM release
  the lease triggers is fire-and-forget with retries. Re-admitting the same id
  inside that window let the dying owner's cleanup tear down the retry's
  bindings and release its SCM lease. The registry now tracks a run as settling
  until that release lands; the retry refuses with 409 until then.
• Admission can throw, not only return queue_full (worktree teardown, SCM
  arbitration). Either way the row is restored to failed with its original
  error instead of being stranded in pending with the failure erased.
• The bulk-retry memory expires. It exists only to bridge the click and the
  refetch; now that a retry reuses the row, the same id can fail again, so the
  memory is keyed to the list snapshot the batch was judged from.

Type of change

• Bug fix (non-breaking change that fixes an issue)
• New feature (non-breaking change that adds functionality)
• Breaking change (fix or feature that changes existing behavior)
• Documentation only
• Refactor / chore

Behaviour change worth calling out even though it is not breaking: a retried run
keeps its original createdAt, so a recovered failure no longer counts as a
failure in byStatus, and its retry does not appear as "today's activity".

Testing checklist

• pnpm lint passes (0 errors in the changed areas; repo-wide warning count
  unchanged from main)
• pnpm typecheck passes
• pnpm test passes — API 7168, web 1098, shared 271. 31 new cases:
  buildRetryMetadata (12), the rerun route's in-place branch (8), restart
  recovery and requeue context (5), the codex verification verdict (6), plus
  the web probe/notice/bulk-retry/drawer suites
• E2E updated/run: bash scripts/e2e/restart-recovery.sh — 4/4 scenarios
• User-facing docs, i18n copy (zh.json and en.json), and the in-app
  manual updated (06-providers, 15-runs, zh + en), plus
  docs/agent/core-concepts-notes.md and the run.retry audit copy

Cross-cutting change matrix

• Create / import / bootstrap paths checked
• PATCH / enable-disable / cancellation paths checked — the retry CAS makes
  a double click, a bulk retry and a concurrent cancel resolve to one claim
• DELETE / cleanup / audit paths checked — run.retry is its own audit
  action, written only after admission succeeds
• Startup recovery and upgrade compatibility checked — retryAttempt is
  absent on every existing row, which reads exactly as today's behaviour
• Git and P4 behavior checked where applicable
• SQLite and PostgreSQL behavior checked where applicable — the claim counts
  .returning() rows rather than a driver row count, which is the one form
  that means the same on both
• Named volume, default bind, explicit bind, and macOS bind behavior checked
  where applicable
• Failure recovery and operator remediation are documented

AI assistance disclosure

• This change was substantially AI-generated. If checked, I confirm I
  understand the code and have tested it myself (see CONTRIBUTING.md).

Additional notes

Reviewed with codex exec in two rounds; ten findings were confirmed against
the code and fixed here (the false-green auth row, the execution-lease and
SCM-release overlap, the Feishu recovery gaps, the dropped session-reset flags,
the throwing-admission path, and the bulk-retry memory). The last two commits
are those fixes.

Follow-ups deliberately not in this PR:

• An E2E scenario for a restart landing while an in-place retry sits queued.
  The predicate is pinned by unit tests; the end-to-end script is not written.
• Artifacts are keyed by run id, so a second attempt writing a same-named file
  overwrites the first attempt's copy while its artifacts row remains. This
  pre-dates the PR (multi-turn chat has the same shape) and is untouched here.
• verified is only ever true for codex today. The contract exists for any CLI
  that grows a real validation command.