Make the walking skeleton fast by injecting its clocks (CL-7250)

[TheGreatAxios]

Summary

The walking-skeleton e2e suite (40 tests, 314s in CI) is about to become CI's critical path once #CL-7226 lands (typecheck drops to ~2min). The cost is concentrated in tests that sleep, not tests that do real work — the top five are 45% of the total time.

The instinct to delete the slow tests is wrong: they're the highest-value coverage in the repo (scheduling, onboarding, webhook delivery). This PR speeds them up without touching a single assertion.

Key finding (from CL-7250's own investigation comment): Bun 1.3.14's fake timers (jest.useFakeTimers/advanceTimersByTime) work, but only in-process — the e2e harness spawns the hub as a separate child process (Bun.spawn), so fake timers in the test process never reach it. That splits the fix in two:

1. Scheduler decision logic was already in-process. apps/hub/test/routine-scheduler.test.ts already drove tickRoutineScheduler directly against an in-memory store, covering claim-and-fire, backoff, retry-after-backoff, and dead-letter-at-max — in milliseconds. No gap there; confirmed, not newly added.

2. The real e2e cost was the hub's hardcoded 30s setInterval (apps/hub/src/routine-scheduler.ts's POLL_INTERVAL_MS), which spawned-hub tests had to wait out for real. Made it injectable via ROUTINE_SCHEDULER_POLL_INTERVAL_MS (parsed at config.ts's one arktype env boundary; production default unchanged at 30s), and proved both the default and the override entirely in-process with Bun's fake timers — no live hub needed to verify this wiring. The e2e harness now defaults every spawned test hub to a 300ms interval.

3. Tightened every fixed retry-sleep loop (sidecar-not-ready 502 retries, mailbox/deployment-appears polls) from 500-1000ms to 200ms across the slow suites, so a loop returns as soon as its condition holds instead of averaging most of a full interval per retry.

No test was deleted, skipped, or had an assertion removed or weakened. Same HTTP responses, same database state — observed sooner.

Commits

1. Make the routine scheduler's poll interval injectable — production change + 2 new fake-timer tests proving default/override cadence in-process
2. e2e harness: default spawned test hubs to a fast scheduler poll interval — harness wiring
3. e2e: tighten fixed retry-sleep intervals in the slow walking-skeleton tests — no production change

Timings

This machine runs several concurrent agent workloads, so load average swings 4x-8x between runs — noted per measurement. CI's numbers will differ, but the relative improvement should hold since it comes from waiting less wall-clock time for the same real work, not from a faster machine.

Controlled comparison (same machine, same 10-second window, load ~4-10 for both runs — the only fully apples-to-apples pair I could get given the noise):

Test	Before (real 30s cadence)	After (300ms cadence)
routine-trigger-input.test.ts	38.0s	15.6s

Individual runs (load noted, not directly comparable to each other or to the CI-reported baseline due to machine noise):

Test	CI baseline (ticket)	This machine, after
routine stored trigger input reaches the run	44.6s	15.6s (load ~4)
local-rip: onboard → connect	30.4s	18.2s (load ~4.5)
greeting delivery: agent speaks first	27.2s	15.4s (load ~4.5)
routine repeat fires twice	21.3s	13.5s (load ~4)

Full suite (40 tests): 280s on this machine at load ~5, vs. the ticket's reported 314s CI baseline. A reliable before/after full-suite comparison on this box wasn't possible — a matched-load control run of the unmodified suite came back at 185s, faster than my "after" run at lower load, purely from other processes' noise swamping the signal at that granularity. The single-test controlled comparison above is the trustworthy number; CI will give the real full-suite before/after.

What still needs a live hub

routine-trigger-input.test.ts's three fires (run-now, scheduled, webhook) and smoke-webhook.test.ts's delivery still spawn a real hub+sidecar+Postgres, because they prove the actual mail-delivery wiring (routine-launcher.ts's sendFoldedMailWithRetry, webhook trigger HMAC verification, insights trace population) — none of that has an in-memory-store equivalent, nor should it; it's exactly the coverage the walking skeleton exists for.

Nothing recommended for deletion

Every test kept its assertions. No coverage gap was found or created.

Unmet outcomes from the ticket

• "Per-test hub and schema setup is shared within a file where the tests do not require isolation" — not attempted in this PR; each e2e file still boots its own hub+sidecar+schema. Left for a follow-up, since it's a larger structural change than the clock-injection work here and each file's current isolation is deliberate (fresh tenant/routine state per scenario).
• The exact "well under two minutes" every-PR-tier target is not yet reached/verified in CI by this PR alone — the full suite still runs as one tier. Splitting into fast/slow CI tiers (this ticket's other checkbox) is not done here; this PR is the clock-injection foundation that split would build on.

Interchange defects found

None.