Mark G5 (widget frontend e2e) closed

docs/Planning/Feature Gaps.md

@@ -58,9 +58,13 @@ Mature knowledge platforms sync per-connector permissions and filters retrieval
 Mature knowledge platforms have a `regression/` layer + nightly LLM-provider-chat. We're adding LLM-judge evals — formalize it.
 - **TDD**: grow `rust/evals` into the regression layer — a fixed scenario set with rubric thresholds (grounding, **anti-hallucination/honest-don't-know**, tool-use appropriateness, multi-turn reasoning), plus a **retrieval-quality** eval (seed a corpus, assert recall@k / MRR on labeled queries — deterministic, no LLM). Add a `nightly` CI job that runs the judged evals across models. Track score history to catch regressions.
 
-### G5. Frontend e2e (Playwright) for the chat widget
-Mature platforms ship extensive web + Playwright suites. Our new `SmooAI/chat-widget` has none yet.
+### G5. Frontend e2e (Playwright) for the chat widget — ✅ running in CI
+Mature platforms ship extensive web + Playwright suites.
 - **TDD**: a Playwright spec first — load the widget against a locally-booted `smooth-operator-server`, send a message, assert streamed assistant tokens render + a grounded answer appears. Wire it into the widget repo's CI.
+- ✅ **Done — and the interesting part was that the specs already existed but had never run.** `SmooAI/chat-widget` had 6 spec files; its `ci.yml` ran typecheck/unit/build and **never invoked `test:e2e`**, so the suite was standing coverage that had never executed. Run for the first time, **2 of 8 were red**: `repro-stream-mock.spec.ts` mocked `WebSocket` but not `fetch`, so on mount the widget POSTed the **real production** `/internal/resume-by-fingerprint`, got a 403, and the console error tripped its own page-error assertion — a "mock" spec that depended on prod being reachable. Fixed with a `page.route` stub.
+- **Streaming had no real guard.** The existing spec asserted only the *final* assistant text, which a widget that ignores every `stream_token` frame and paints the `eventual_response` blob also satisfies — confirmed by deleting streaming from the mock and watching it stay green. `e2e/streaming.spec.ts` closes it: the mock **withholds `eventual_response`** and all assertions sample inside that window, so only stream tokens can have produced on-screen text; it asserts incremental, monotonic growth to the full reply.
+- **CI split so a missing secret cannot look like a pass**: `ci.yml` → `E2E (credential-free)` runs the hermetic specs on every PR (9 tests, ~6s); `e2e-live.yml` (nightly) builds a lean in-memory `smooth-operator-server` (`--no-default-features`) with `SMOOTH_AGENT_SEED_KB=1` and asserts the grounded **17-day return window** answer, **failing loudly** when `SMOOAI_GATEWAY_KEY` is absent rather than skipping green. `SMOOTH_AGENT_SERVER_BIN` replaced a hardcoded developer-local `shared-target` path that had never existed on a runner. (SmooAI/chat-widget#46)
+- **Lesson worth generalising**: a suite that has never run is not coverage. `grep -c` on a spec file counts *attempts*, not passes — the only proof is a CI job whose green depends on those assertions executing.
 
 ### G6. Deployment-integration tests in CI — ✅ shipped
 - ✅ **Done.** `.github/workflows/pr-kind-deploy-smoke.yml` runs the planned `kind` job on every PR: `helm install` into an ephemeral cluster, then the protocol smoke against the live pod. It is a required-looking check on current PRs (observed green on #526, 2026-08-22).