Skip to content

ci: workarounds for scale to zero - #238

Open
AlexAxthelm wants to merge 7 commits into
mainfrom
ci/cooldown
Open

ci: workarounds for scale to zero#238
AlexAxthelm wants to merge 7 commits into
mainfrom
ci/cooldown

Conversation

@AlexAxthelm

@AlexAxthelm AlexAxthelm commented Aug 17, 2026

Copy link
Copy Markdown
Collaborator

Alex's TL;DR here, Claude's fuller summary below the fold:

Last pinged around 1510 Belgrade time. Update this description as needed.

Responding to changes in #234, where we set more environments to scale to zero (fall asleep) this PR is a collection of workarounds to mitigate the impact of that in practice:

  • (up for discussion after we try it out): Set timeout before falling asleep to 15 min (900s) up from 5 min (300s)
  • Add "waking up" message on env banner (this only affects the same environments as get env banner; prod is always-on). See screenshot below
  • ping the API as part of the initial page load (prior to login) so server can be spinning up while user is (maybe) running through login
  • add utility to frontend to help it understand which erros should be retried.
CleanShot 2026-08-20 at 15 01 00@2x

Why

#234 narrows the always-on policy so that only pushes to production keep a warm replica. Everything else — main, demo lanes, PR previews — now scales to zero and pays a cold start on the first request of a session. This PR makes that acceptable: slightly less frequent, and no longer indistinguishable from the app being broken.

Widening the cooldown

Cooldown is how long a Container App stays up after its last request. Azure's default is 300s, so a five-minute pause costs a cold start. This raises it to 900s for apps that actually scale to zero, reasserted after every deploy.

Two things a reviewer should know about it:

There is no CLI flag. az containerapp update has no --cooldown-period and no generic --set, only --yaml. The step therefore does the same show → jq → update --yaml round-trip already used for ETL volume mounts, and runs after that step so it re-reads its result. The jq transform was dry-run against the live main-api spec: cooldownPeriod 300 → 900, systemData stripped (ARM rejects it on write), image digest and the rest of the template intact.

Absorbing the cold start in the browser

Retry transient failures instead of surfacing them. The global retry: 1 retried once after ~1s — still inside an ACA activation — then showed the user an error. Now a predicate retries only 5xx, 408, 429 and the TypeError a dropped fetch rejects with, up to 3 times.

Wake the API during bootstrap. A throwaway GET fired from bootstrap() and not awaited. It runs before Auth0 mounts, so the container starts while the user is still being redirected through login. Targets /health/details rather than /health because it resolves the database engine, so it opens a connection too — the two costs of a cold start, both paid during dead time. Neither endpoint requires auth.

Say so when it is still slow. After 2s of a request in flight, the environment banner shows "Server is waking up — this can take a few seconds" as a role="status" live region. Nothing to fix in that moment; it just needs saying, so the wait reads as starting up rather than nothing happening.

Details that look like omissions but are deliberate

Each is commented at the call site and pinned by a test, so they are not tidied away later.

Choice Reason
The cooldown step is gated on min-replicas == '' An app that never scales to zero has no use for a cooldown, and skipping it saves a round-trip per deploy
Prewarm has no abort timer Azure holds the request open while it activates a replica — that wait is the thing doing the work
Prewarm sends no headers and no auth Keeps it CORS-simple, so it reaches the server without a preflight even when the deployment's single allowed origin does not match the browser's. The response is discarded, so being blocked from reading it costs nothing — which matters during the stitch-dev.rmi.org cutover
Prewarm never rejects A failed prewarm has cost the user nothing; the real request behind it carries its own retry policy
The warming hook keys on a boolean, not the fetch count Otherwise a second query starting restarts the 2s clock on the request that was already waiting
Mutations keep retry: 0 They are not idempotent; replaying a write after a timeout is worse than the error

Things to look at

A 4xx is no longer retried. retry: 1 retried every failure once, including 404 and 401. Re-sending an unchanged bad request only delays an error that was never going to change — but this is a change beyond "retry more", so it is worth a reviewer's eye.

getResources and getResourceFilterOptions now attach error.status. Every other throw site already did; these two carried the status only in the message string, so a status-based retry predicate could not see them. They back the resources list and its filter options — the first requests after sign-in, and so exactly the ones that meet a cold container. The retry fix does not work without this.

The cooldown step costs CI time. A show plus an update per scale-to-zero app per deploy — four apps on a PR lane, so perhaps 30–60s. Cheap, but not free, and easy to forget once buried in a reusable workflow.

Existing apps stay at 300s until their next deploy.

Not in this PR

  • Prewarming entity-linkage and stitch-llm. Built, then reverted — better approaches are being explored, and the API is the one every session needs.
  • An HTTP startup probe on /health. No probes are configured on any container app, so ACA falls back to TCP-open, and uvicorn can accept connections before the FastAPI lifespan finishes. That is a plausible source of the 503s the retry policy is defending against. It needs the same --yaml round-trip, so it is CI work and belongs in its own PR.
  • Tuning retryDelay. Left at the TanStack default (1s/2s/4s). It is the knob to turn if the retries prove too eager against a real cold start, but picking a curve now would be guessing.

Notes for review

Stacked on #234 — base is ci/hostnames, so that needs to merge first.

AI was used to write the frontend changes, the CI step, the tests and this description. The measurements are from az monitor metrics list against main-api, not estimates. The new banner tests were mutation-checked: forcing the delay never to elapse kills exactly the two tests asserting the notice appears and clears, and leaves the other six green.

@github-actions

Copy link
Copy Markdown

CD summary 6510986

Database (1)
db_name postgres_host postgres_port postgres_db
pr_0238 stitch-dev.postgres.database.azure.com 5432 pr_0238
Jobs (1)
job image postgres_db
db-migrations ghcr.io/rmi/stitch-api:pr-0238@sha256:652799578d8e819f128c2dc72fc4bf5e1414ad1f77340296893a0490ae86b7e6 pr_0238
Images (4)
build_time commit_time git_sha image image_digest
2026-08-17T16:46:57Z 2026-08-17T16:46:36Z c20e793 ghcr.io/rmi/stitch-api:pr-0238 ghcr.io/rmi/stitch-api:pr-0238@sha256:652799578d8e819f128c2dc72fc4bf5e1414ad1f77340296893a0490ae86b7e6
2026-08-17T16:46:57Z 2026-08-17T16:46:36Z c20e793 ghcr.io/rmi/stitch-entity-linkage:pr-0238 ghcr.io/rmi/stitch-entity-linkage:pr-0238@sha256:e2263ae5a66f6b319526391f83f71fe4d7a904a35be4a0c450eff43414a9d3df
2026-08-17T16:46:55Z 2026-08-17T16:46:36Z c20e793 ghcr.io/rmi/stitch-seed:pr-0238 ghcr.io/rmi/stitch-seed:pr-0238@sha256:0c78a5d335d1a44b30f471def0f0b2193f6bebda7e4e6880b14f980328d0d6e9
2026-08-17T16:46:56Z 2026-08-17T16:46:36Z c20e793 ghcr.io/rmi/stitch-stitch-llm:pr-0238 ghcr.io/rmi/stitch-stitch-llm:pr-0238@sha256:7ae51cc8127f51511af7438996c089b4fe84676a45407c3a0c996be1c6073f6a

@github-actions

Copy link
Copy Markdown

CD summary 9556f57

Frontend: https://witty-mushroom-017a3dc1e-238.westus2.1.azurestaticapps.net

Deployments (4)
service url fqdn
api open pr-0238-api.purplegrass-c07d0a94.westus2.azurecontainerapps.io
entity-linkage open pr-0238-el.purplegrass-c07d0a94.westus2.azurecontainerapps.io
frontend https://witty-mushroom-017a3dc1e-238.westus2.1.azurestaticapps.net
stitch-llm open pr-0238-llm.purplegrass-c07d0a94.westus2.azurecontainerapps.io
Database (1)
db_name postgres_host postgres_port postgres_db
pr_0238 stitch-dev.postgres.database.azure.com 5432 pr_0238
Jobs (1)
job image postgres_db
db-migrations ghcr.io/rmi/stitch-api:pr-0238@sha256:8064a9e16c4a562914933878eaff4a38ffbad5b07f79128f14d32500e22ac884 pr_0238
Images (4)
build_time commit_time git_sha image image_digest
2026-08-17T16:49:00Z 2026-08-17T16:48:25Z ad14cdb ghcr.io/rmi/stitch-api:pr-0238 ghcr.io/rmi/stitch-api:pr-0238@sha256:8064a9e16c4a562914933878eaff4a38ffbad5b07f79128f14d32500e22ac884
2026-08-17T16:49:03Z 2026-08-17T16:48:25Z ad14cdb ghcr.io/rmi/stitch-entity-linkage:pr-0238 ghcr.io/rmi/stitch-entity-linkage:pr-0238@sha256:b0531bf3b6d702b20d1f646367cf38bfaa41afc550933353c530f83812d99bd8
2026-08-17T16:49:02Z 2026-08-17T16:48:25Z ad14cdb ghcr.io/rmi/stitch-seed:pr-0238 ghcr.io/rmi/stitch-seed:pr-0238@sha256:c15480b89ce79515ccf22e2907d01e02e046eab03e24c5ac7c6ca46841683f0a
2026-08-17T16:48:57Z 2026-08-17T16:48:25Z ad14cdb ghcr.io/rmi/stitch-stitch-llm:pr-0238 ghcr.io/rmi/stitch-stitch-llm:pr-0238@sha256:546cc49b3c9d23db85854dd7b2408f7215ca63065b0553e3fb146b16b4ebdfcc

@github-actions

Copy link
Copy Markdown

CD summary f6daee1

Frontend: https://witty-mushroom-017a3dc1e-238.westus2.1.azurestaticapps.net

Deployments (4)
service url fqdn
api open pr-0238-api.purplegrass-c07d0a94.westus2.azurecontainerapps.io
entity-linkage open pr-0238-el.purplegrass-c07d0a94.westus2.azurecontainerapps.io
frontend https://witty-mushroom-017a3dc1e-238.westus2.1.azurestaticapps.net
stitch-llm open pr-0238-llm.purplegrass-c07d0a94.westus2.azurecontainerapps.io
Database (1)
db_name postgres_host postgres_port postgres_db
pr_0238 stitch-dev.postgres.database.azure.com 5432 pr_0238
Jobs (1)
job image postgres_db
db-migrations ghcr.io/rmi/stitch-api:pr-0238@sha256:340f53c0a867908e79a15173f8b6c7b2d680aac6ade02530db89228ad5ebacb5 pr_0238
Images (4)
build_time commit_time git_sha image image_digest
2026-08-18T16:51:04Z 2026-08-18T16:50:47Z 6de44b9 ghcr.io/rmi/stitch-api:pr-0238 ghcr.io/rmi/stitch-api:pr-0238@sha256:340f53c0a867908e79a15173f8b6c7b2d680aac6ade02530db89228ad5ebacb5
2026-08-18T16:51:00Z 2026-08-18T16:50:47Z 6de44b9 ghcr.io/rmi/stitch-entity-linkage:pr-0238 ghcr.io/rmi/stitch-entity-linkage:pr-0238@sha256:aac380f27bb220fd9d93c5b6441469e57652099daf0a4dbef5aead440b58f044
2026-08-18T16:51:00Z 2026-08-18T16:50:47Z 6de44b9 ghcr.io/rmi/stitch-seed:pr-0238 ghcr.io/rmi/stitch-seed:pr-0238@sha256:d4a285f9915b8c460b40ed2e3791fa90fbf49f7d96d47fe47188d621391836a4
2026-08-18T16:51:02Z 2026-08-18T16:50:47Z 6de44b9 ghcr.io/rmi/stitch-stitch-llm:pr-0238 ghcr.io/rmi/stitch-stitch-llm:pr-0238@sha256:0d7f3f47f1741eda61c31333aa593df92c01ba258cc9649aadac688bf92db5a0

@github-actions

Copy link
Copy Markdown

CD summary de1a696

Frontend: https://witty-mushroom-017a3dc1e-238.westus2.1.azurestaticapps.net

Deployments (4)
service url fqdn
api open pr-0238-api.purplegrass-c07d0a94.westus2.azurecontainerapps.io
entity-linkage open pr-0238-el.purplegrass-c07d0a94.westus2.azurecontainerapps.io
frontend https://witty-mushroom-017a3dc1e-238.westus2.1.azurestaticapps.net
stitch-llm open pr-0238-llm.purplegrass-c07d0a94.westus2.azurecontainerapps.io
Database (1)
db_name postgres_host postgres_port postgres_db
pr_0238 stitch-dev.postgres.database.azure.com 5432 pr_0238
Jobs (1)
job image postgres_db
db-migrations ghcr.io/rmi/stitch-api:pr-0238@sha256:a9d1c1e9f236d77e9a19c88a473f2aab32854b72f58cef89fe5c1e9132acb8b6 pr_0238
Images (4)
build_time commit_time git_sha image image_digest
2026-08-20T11:31:26Z 2026-08-20T11:30:56Z 32a5be5 ghcr.io/rmi/stitch-api:pr-0238 ghcr.io/rmi/stitch-api:pr-0238@sha256:a9d1c1e9f236d77e9a19c88a473f2aab32854b72f58cef89fe5c1e9132acb8b6
2026-08-20T11:31:13Z 2026-08-20T11:30:56Z 32a5be5 ghcr.io/rmi/stitch-entity-linkage:pr-0238 ghcr.io/rmi/stitch-entity-linkage:pr-0238@sha256:52d782b88bff07e7153db9dffe59b2330c5181d90f0b1454cfc552fdc540a85c
2026-08-20T11:31:13Z 2026-08-20T11:30:56Z 32a5be5 ghcr.io/rmi/stitch-seed:pr-0238 ghcr.io/rmi/stitch-seed:pr-0238@sha256:8caeeb742504b5736bc0266c549fbc2d0f1af6412128f6f9cae56571b144b0cb
2026-08-20T11:31:11Z 2026-08-20T11:30:56Z 32a5be5 ghcr.io/rmi/stitch-stitch-llm:pr-0238 ghcr.io/rmi/stitch-stitch-llm:pr-0238@sha256:2057afd2acc3130f5a813ed4d43cb40396cd19ef7268a3e6397015b24c825fe5

@github-actions

Copy link
Copy Markdown

CD summary de1a696

Frontend: https://witty-mushroom-017a3dc1e-238.westus2.1.azurestaticapps.net

Deployments (4)
service url fqdn
api open pr-0238-api.purplegrass-c07d0a94.westus2.azurecontainerapps.io
entity-linkage open pr-0238-el.purplegrass-c07d0a94.westus2.azurecontainerapps.io
frontend https://witty-mushroom-017a3dc1e-238.westus2.1.azurestaticapps.net
stitch-llm open pr-0238-llm.purplegrass-c07d0a94.westus2.azurecontainerapps.io
Database (1)
db_name postgres_host postgres_port postgres_db
pr_0238 stitch-dev.postgres.database.azure.com 5432 pr_0238
Jobs (2)
job image postgres_db api_url auth_mode
db-migrations ghcr.io/rmi/stitch-api:pr-0238@sha256:d616dfd54d329f9d3bbc40dfdee7830ca910244b1e1481a221cdaa246a01fb0d pr_0238
seed ghcr.io/rmi/stitch-seed:pr-0238@sha256:144be6d630fdee428763a90bb45978b617fe8642d766bcdc4e7832daf0e8103c https://pr-0238-api.purplegrass-c07d0a94.westus2.azurecontainerapps.io/api/v1 stitch-client-bearer-token
Images (4)
build_time commit_time git_sha image image_digest
2026-08-20T13:16:05Z 2026-08-20T13:15:36Z 32a5be5 ghcr.io/rmi/stitch-api:pr-0238 ghcr.io/rmi/stitch-api:pr-0238@sha256:d616dfd54d329f9d3bbc40dfdee7830ca910244b1e1481a221cdaa246a01fb0d
2026-08-20T13:15:51Z 2026-08-20T13:15:36Z 32a5be5 ghcr.io/rmi/stitch-entity-linkage:pr-0238 ghcr.io/rmi/stitch-entity-linkage:pr-0238@sha256:31efc37cc73e8a182bdb23a4630bc9e6ce8d39715739b7c115f81311a33391ae
2026-08-20T13:15:53Z 2026-08-20T13:15:36Z 32a5be5 ghcr.io/rmi/stitch-seed:pr-0238 ghcr.io/rmi/stitch-seed:pr-0238@sha256:144be6d630fdee428763a90bb45978b617fe8642d766bcdc4e7832daf0e8103c
2026-08-20T13:15:49Z 2026-08-20T13:15:36Z 32a5be5 ghcr.io/rmi/stitch-stitch-llm:pr-0238 ghcr.io/rmi/stitch-stitch-llm:pr-0238@sha256:a3a82adf319f5c8838b02896d2c4eda05975b5eac17b36582dbe0beae5983ab5

@AlexAxthelm
AlexAxthelm marked this pull request as ready for review August 20, 2026 13:21
Copilot AI lite review requested due to automatic review settings August 20, 2026 13:21
@AlexAxthelm
AlexAxthelm requested review from john and mbarlow12 August 20, 2026 13:22

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR mitigates the user-facing impact of Azure Container Apps scaling to zero (cold starts) by (1) widening the cooldown period in CI/CD and (2) making the frontend proactively “wake” the API and retry only genuinely transient failures, while also surfacing an explicit “server is waking up” message during stalled requests.

Changes:

  • CI: add a post-deploy step to set cooldownPeriod to 900s for scale-to-zero Container Apps via show → jq → update --yaml.
  • Frontend: prewarm the API during bootstrap and introduce a TanStack Query retry predicate that retries only transient failures (5xx/408/429 + failed fetch).
  • UI/tests: show an accessible “Server is waking up” notice after a short delay while requests are pending, with accompanying tests.

Reviewed changes

Copilot reviewed 11 out of 11 changed files in this pull request and generated 3 comments.

Show a summary per file
File Description
.github/workflows/deploy-container.yml Adds a deploy step to widen ACA scale-to-zero cooldown to reduce cold starts within a session.
deployments/CI_DEPLOYMENTS.md Documents the widened cooldown and the frontend mitigations; adds a metrics command to measure impact.
deployments/stitch-frontend/src/AppProviders.jsx Switches TanStack Query retry from a fixed count to a predicate for transient-only retries.
deployments/stitch-frontend/src/main.jsx Fires a non-blocking API prewarm call during bootstrap so warm-up overlaps with Auth0 redirect time.
deployments/stitch-frontend/src/queries/prewarm.js Implements a best-effort prewarm request to /health/details that should never block bootstrap.
deployments/stitch-frontend/src/queries/prewarm.test.js Adds unit tests for prewarm behavior (URL, CORS-simplicity, non-blocking semantics).
deployments/stitch-frontend/src/queries/retry.js Adds isRetriableError and shouldRetryQuery utilities to classify retryable failures.
deployments/stitch-frontend/src/queries/retry.test.js Adds unit tests for retry classification and retry cap behavior.
deployments/stitch-frontend/src/queries/api.js Ensures thrown HTTP errors include error.status so retry logic can inspect status codes.
deployments/stitch-frontend/src/components/EnvironmentBanner.jsx Adds a delayed “waking up” live region message when queries appear stalled.
deployments/stitch-frontend/src/components/EnvironmentBanner.test.jsx Adds tests for the “waking up” notice timing and clearing behavior.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment on lines +30 to +49
export function prewarmApi(config) {
const baseUrl = config?.apiBaseUrl;

if (!baseUrl) {
return Promise.resolve();
}

// Deliberately not aborted on a timer: Azure holds the request open while it
// activates a replica, and that wait is the thing doing the work.
//
// Deliberately unauthenticated and header-free, which keeps this a
// CORS-simple request. It reaches the server without a preflight even when
// the deployment's single allowed origin does not match the browser's -- and
// since the response is discarded, being blocked from reading it costs
// nothing. That matters while a custom domain is being cut over.
return fetch(healthUrl(baseUrl), { cache: "no-store" }).then(
() => undefined,
() => undefined,
);
}
Comment on lines +1 to +11
import { describe, it, expect, vi, beforeEach } from "vitest";
import { prewarmApi } from "./prewarm";

describe("prewarmApi", () => {
let fetchMock;

beforeEach(() => {
fetchMock = vi.fn().mockResolvedValue({ ok: true });
vi.stubGlobal("fetch", fetchMock);
});

Comment thread deployments/CI_DEPLOYMENTS.md Outdated
lane is in real daily use:

```bash
az monitor metrics list --resource "$(az containerapp show -n main-api -g STITCH-DEV-RG --query id -o tsv)" --metric Replicas --interval PT1H --start-time "$(date -u -v-13d +%Y-%m-%dT%H:%M:%SZ)" --end-time "$(date -u +%Y-%m-%dT%H:%M:%SZ)" --aggregation Maximum -o table
* Guard `prewarmApi` against a non-string base URL. Unreachable via
  `loadConfig`, which already rejects one, but the function documents that
  it never rejects and `main.jsx` relies on that -- a synchronous throw
  would escape `bootstrap()` and swap the app for the config-error screen.
* Unstub `fetch` in `afterEach`, matching the other suites that stub
  globals. `vitest.config.js` sets no `unstubGlobals`, and the shared
  `restoreAllMocks` does not undo `stubGlobal`.
* Replace the BSD-only `date -v-13d` in the metrics snippet with
  `az monitor metrics list --offset 13d`, which needs no date arithmetic
  and works on GNU too. Phrase the sample as a ratio rather than an exact
  hour count, which drifts as the window moves.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@github-actions

Copy link
Copy Markdown

CD summary 49effe8

Frontend: https://witty-mushroom-017a3dc1e-238.westus2.1.azurestaticapps.net

Deployments (4)
service url fqdn
api open pr-0238-api.purplegrass-c07d0a94.westus2.azurecontainerapps.io
entity-linkage open pr-0238-el.purplegrass-c07d0a94.westus2.azurecontainerapps.io
frontend https://witty-mushroom-017a3dc1e-238.westus2.1.azurestaticapps.net
stitch-llm open pr-0238-llm.purplegrass-c07d0a94.westus2.azurecontainerapps.io
Database (1)
db_name postgres_host postgres_port postgres_db
pr_0238 stitch-dev.postgres.database.azure.com 5432 pr_0238
Jobs (1)
job image postgres_db
db-migrations ghcr.io/rmi/stitch-api:pr-0238@sha256:2fef95c5931453385318a21fef949768b8c3482a293edf860f798ab64acee528 pr_0238
Images (4)
build_time commit_time git_sha image image_digest
2026-08-20T17:15:41Z 2026-08-20T17:15:22Z 77bfdae ghcr.io/rmi/stitch-api:pr-0238 ghcr.io/rmi/stitch-api:pr-0238@sha256:2fef95c5931453385318a21fef949768b8c3482a293edf860f798ab64acee528
2026-08-20T17:15:40Z 2026-08-20T17:15:22Z 77bfdae ghcr.io/rmi/stitch-entity-linkage:pr-0238 ghcr.io/rmi/stitch-entity-linkage:pr-0238@sha256:a614c1d6491d8693105e4f6853140840520fbd67675adc3920dd8f148536105c
2026-08-20T17:15:44Z 2026-08-20T17:15:22Z 77bfdae ghcr.io/rmi/stitch-seed:pr-0238 ghcr.io/rmi/stitch-seed:pr-0238@sha256:2dc781685f1ea429c23d5af4e04fe551804ab0be8a4a6382b864b80261404adc
2026-08-20T17:15:40Z 2026-08-20T17:15:22Z 77bfdae ghcr.io/rmi/stitch-stitch-llm:pr-0238 ghcr.io/rmi/stitch-stitch-llm:pr-0238@sha256:1e77b6b24e14440ac48dcb2513ab08bd75797ee28a0d3bdda80802c066754b7e

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants