Skip to content

docs: nightly production health-check 2026-06-16 (0 fixes, prod healthy) #305

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
njrini99-code wants to merge 1 commit into from
Closed

docs: nightly production health-check 2026-06-16 (0 fixes, prod healthy) #305

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
103 changes: 103 additions & 0 deletions docs/daily-briefs/2026-06-16.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,103 @@
# Daily Review Brief — 2026-06-16

**Author:** nightly health-check (autonomous) · **Repo:** `njrini99-code/helmv3` · **Branch:** `claude/awesome-babbage-r0nfjc`
**Sources reviewed:** Vercel runtime logs + deployments (`helmv3`, `prj_qPgC4eErTUsaSmv40EiQMNuTpuEV`, production target `dpl_CWeGKBJ97QwkJnzh9i4oTFddMJZx` on `main`)
**Sources unavailable:** Supabase MCP (`mcp.supabase.com` not in this routine's network allowlist → no `admin_events`, `error_logs`, or `pg_stat_statements` query this run). Sentry MCP requires per-session OAuth and was not authenticated.

---

## TL;DR

- **Production is healthy.** Main is on `b310847…` (READY; deployed 2026-06-15 ~22:27 UTC). No 5xx, no runtime fatals, no failed cron in the last 24h.
- **Only 4 error-level runtime log lines** in the full 24-hour window — all from a single user at **14:02:19 UTC** with a stale Supabase refresh-token cookie. The auth path handles it correctly (server pages call `getGolfSessionProfile()` → returns null on `userError` → `redirect('/golf/login')`); the user got a 307 to login as designed.
- **One cron warning** (`/api/cron/coachhelm-roster-sweep`, 03:45 UTC, status 200, message starts `[insights.triggerPlayerInsi…`): a single player's insight trigger rejected, the sweep continued. The cron explicitly logs this as `warning`, not `error`, via `logServerError(…, 'warning')`. Single occurrence — not a recurring pattern.
- **Three FAILED preview deployments in the window** — all on PR #304 (`feat/coachhelm-stats-roundup`), all the same build error: parallel-route conflict between `src/app/golf/(auth)/demo/page.tsx` and `src/app/golf/demo/route.ts`. **Not a `main` problem** — those paths do not co-exist on `main` (`(auth)/demo/page.tsx` doesn't exist here; only `demo/route.ts`). The conflict is introduced by a merge inside PR #304 and must be resolved on that PR's branch — out of scope for this nightly run.
- **Net actionable issues for this PR: 0.** Per the routine's `<2 actionable issues` clause, this PR ships the brief alone so the signal still reaches the routine UI.

---

## 1. 24h totals

| Metric | Count |
|---|---|
| Total error-level runtime logs (production) | **4** |
| Distinct error patterns | **1** |
| Distinct users affected (estimated, single timestamp/route cluster) | **1** |
| Warning-level runtime logs | **1** |
| 5xx responses | **0** |
| Failed production deployments | **0** |
| Failed preview deployments | **3** (all PR #304, same root cause) |
| Cron failures | **0** (`event-reminders`, `coachhelm-roster-sweep`, `coachhelm-calibration`, `coachhelm-insight-lifecycle`, `v3/standing-refresh`, `v3/goal-suggestions-evaluate`, `v3/goal-suggestions-write` all returned 200) |

---

## 2. Top patterns + diagnosis

### Pattern 1 · `Error [AuthApiError]: Invalid Refresh Token` — **4 occurrences, 1 user, NOT a bug**

| Time (UTC) | Method | Path | Status |
|---|---|---|---|
| 14:02:19 | POST | `/golf/dashboard/rounds/new` | 307 (redirect to `/golf/login`) |
| 14:02:19 | HEAD | `/api/health` | 200 |
| 14:02:19 | HEAD | `/api/health` | 200 |
| 14:02:19 | HEAD | `/api/health` | 200 |

**Root cause:** Single browser tab arriving with a rotated/expired Supabase refresh-token cookie. `supabase.auth.getUser()` returns `{ user: null, error: AuthApiError }`; the call sites do the right thing (page redirects to login, health endpoint returns 200 with `database: ok`).

**Why it appears as `error` in Vercel runtime logs:** `@supabase/auth-js` itself logs the failed `/token?grant_type=refresh_token` round-trip to `console.error` before returning the error in the `.error` field. Our `middleware.ts` catches and downgrades to `console.warn`, but the supabase-js library's own log line still surfaces as `error` in Vercel's stream. Sentry **already** filters this — `src/instrumentation.ts` lists `AuthApiError`, `Invalid Refresh Token: Refresh Token Not Found`, and `Refresh Token Not Found` in `sharedIgnoreErrors` — so no alert fired.

**Diagnosis:** expected behavior on rotated refresh tokens. Not actionable as a bug. (A pure-cosmetic noise-reduction option is documented in §5.)

### Pattern 2 · `[insights.triggerPlayerInsights…]` warning from roster-sweep cron — **1 occurrence**

| Time (UTC) | Path | Status | Level |
|---|---|---|---|
| 03:45:34 | `GET /api/cron/coachhelm-roster-sweep` | 200 | warning |

**Source:** `src/app/api/cron/coachhelm-roster-sweep/route.ts:100-108`. The sweep logs `warning` (not `error`) when an individual player's `triggerPlayerInsightsAfterRound` rejects, then continues with the next player. The cron itself returned 200 and the route reports `failed`/`analyzed`/`skipped` counts in its JSON response.

**Diagnosis:** designed degradation, single player, single occurrence. Per the routine's skip-criteria ("Pattern-miner threshold warnings — those are tunable info logs, not bugs" and "Single-occurrence transients … unless they correlate to a recurring pattern"), explicitly not actionable in this run. If the same player ID recurs in tomorrow's sweep, escalate.

---

## 3. Performance — slow queries

**Not available this run.** Supabase MCP (`mcp.supabase.com`) is not in this routine's network allowlist (`Host not in allowlist`), so `pg_stat_statements` couldn't be queried. To restore visibility, the user can add `mcp.supabase.com` to the routine environment's egress allowlist and re-authenticate the Supabase MCP. Vercel/runtime side shows no slow-function symptoms (no `FUNCTION_INVOCATION_TIMEOUT`, no 5xx).

---

## 4. Deployment health

### Production (target=production, branch=main)

- **Current:** `dpl_CWeGKBJ97QwkJnzh9i4oTFddMJZx` · commit `b310847…` (`feat(ios): App Store submission polish for v1.7 (build 7) (#298)`) · state `READY` · created 2026-06-15 ~22:27 UTC. Serving fine.
- **No failed production deployments in the 24h window.**

### Preview failures (3, all same PR, all same root cause — NOT production)

| Deployment | Commit | Branch | First failing line |
|---|---|---|---|
| `dpl_EmBN5hyccNFTnryeQj1gs7CUxtWL` | `82ac0d1` | `feat/coachhelm-stats-roundup` (PR #304) | `Failed to compile.` → `src/app/golf/(auth)/demo/page.tsx — You cannot have two parallel pages that resolve to the same path. Please check /golf/(auth)/demo/page and /golf/demo/route.` |
| `dpl_HgjJdbgZZ6d4fU8fDPQ4FFGsnxuL` | `431afc7` | `feat/coachhelm-stats-roundup` | same `parallel pages` error |
| `dpl_8tD32JFyQFuWcXtAqsNsRvfLrbYo` | `1b4deb6` | `feat/coachhelm-stats-roundup` | same `parallel pages` error |

**Triage:** `src/app/golf/(auth)/demo/page.tsx` does **not** exist on `main`; only `src/app/golf/demo/route.ts` does. The conflicting `page.tsx` was introduced by a merge inside PR #304 and the conflict must be resolved on the PR #304 branch (delete the duplicate `page.tsx`, or move the existing `route.ts`). **Not actionable from `main`/this nightly PR** — fixing it from a separate branch wouldn't unblock PR #304's preview.

---

## 5. Things deliberately not fixed (with reasoning)

1. **Stale-refresh-token noise from `@supabase/auth-js` console.error.** A cosmetic mitigation exists — `src/proxy.ts` already has the explicit `if (/refresh token/i.test(message)) console.warn(…)` handling that the live `/middleware.ts` lacks, *and* `src/proxy.ts` is **dead code** (no importer found via grep across `src/**`; the real Next.js middleware entry is `/middleware.ts`). Porting that branch into `middleware.ts` wouldn't actually silence the lib-side log either (it fires inside `getUser()` before our `try/catch` sees it), so the change would be cleanup without effect. Per the harness guidance "a bug fix doesn't need surrounding cleanup," skipped. If we want to genuinely cut the noise, the right fix is either:
- patch `console.error` in `src/instrumentation.ts` to swallow `Invalid Refresh Token` messages at the Node runtime level, or
- exclude `/api/health` and `/api/cron/*` from the middleware matcher so the unauthenticated request never refreshes a token in the first place.

Both are real changes deserving their own PR + review, not a piggyback on a nightly run.
2. **PR #304 build failure** — fixing it requires committing to PR #304's branch (`feat/coachhelm-stats-roundup`), not to `main`. Out of nightly scope.
3. **`golf_team_members` membership rows / player-id null check etc.** Database-side health was not inspectable this run (see §3); deferring to tomorrow's run after MCP allowlist is restored.

---

## 6. Routine limitations encountered (please address for next run)

- **Add `mcp.supabase.com` to the environment's network egress allowlist** so the Supabase MCP can authenticate and the nightly run can query `admin_events`, `error_logs`, and `pg_stat_statements` directly (the brief promises Step 1 evidence from those tables and got none this run).
- **Sentry MCP** is connected but needs interactive OAuth on first use — not viable for an unattended routine. If Sentry visibility is wanted in nightlies, swap to a service-token-based integration or pre-auth the OAuth flow.
Loading