ci: workarounds for scale to zero - #238
Conversation
CD summary
|
| db_name | postgres_host | postgres_port | postgres_db |
|---|---|---|---|
| pr_0238 | stitch-dev.postgres.database.azure.com |
5432 |
pr_0238 |
Jobs (1)
| job | image | postgres_db |
|---|---|---|
| db-migrations | ghcr.io/rmi/stitch-api:pr-0238@sha256:652799578d8e819f128c2dc72fc4bf5e1414ad1f77340296893a0490ae86b7e6 |
pr_0238 |
Images (4)
| build_time | commit_time | git_sha | image | image_digest |
|---|---|---|---|---|
| 2026-08-17T16:46:57Z | 2026-08-17T16:46:36Z | c20e793 | ghcr.io/rmi/stitch-api:pr-0238 |
ghcr.io/rmi/stitch-api:pr-0238@sha256:652799578d8e819f128c2dc72fc4bf5e1414ad1f77340296893a0490ae86b7e6 |
| 2026-08-17T16:46:57Z | 2026-08-17T16:46:36Z | c20e793 | ghcr.io/rmi/stitch-entity-linkage:pr-0238 |
ghcr.io/rmi/stitch-entity-linkage:pr-0238@sha256:e2263ae5a66f6b319526391f83f71fe4d7a904a35be4a0c450eff43414a9d3df |
| 2026-08-17T16:46:55Z | 2026-08-17T16:46:36Z | c20e793 | ghcr.io/rmi/stitch-seed:pr-0238 |
ghcr.io/rmi/stitch-seed:pr-0238@sha256:0c78a5d335d1a44b30f471def0f0b2193f6bebda7e4e6880b14f980328d0d6e9 |
| 2026-08-17T16:46:56Z | 2026-08-17T16:46:36Z | c20e793 | ghcr.io/rmi/stitch-stitch-llm:pr-0238 |
ghcr.io/rmi/stitch-stitch-llm:pr-0238@sha256:7ae51cc8127f51511af7438996c089b4fe84676a45407c3a0c996be1c6073f6a |
CD summary
|
| service | url | fqdn |
|---|---|---|
| api | open | pr-0238-api.purplegrass-c07d0a94.westus2.azurecontainerapps.io |
| entity-linkage | open | pr-0238-el.purplegrass-c07d0a94.westus2.azurecontainerapps.io |
| frontend | https://witty-mushroom-017a3dc1e-238.westus2.1.azurestaticapps.net | |
| stitch-llm | open | pr-0238-llm.purplegrass-c07d0a94.westus2.azurecontainerapps.io |
Database (1)
| db_name | postgres_host | postgres_port | postgres_db |
|---|---|---|---|
| pr_0238 | stitch-dev.postgres.database.azure.com |
5432 |
pr_0238 |
Jobs (1)
| job | image | postgres_db |
|---|---|---|
| db-migrations | ghcr.io/rmi/stitch-api:pr-0238@sha256:8064a9e16c4a562914933878eaff4a38ffbad5b07f79128f14d32500e22ac884 |
pr_0238 |
Images (4)
| build_time | commit_time | git_sha | image | image_digest |
|---|---|---|---|---|
| 2026-08-17T16:49:00Z | 2026-08-17T16:48:25Z | ad14cdb | ghcr.io/rmi/stitch-api:pr-0238 |
ghcr.io/rmi/stitch-api:pr-0238@sha256:8064a9e16c4a562914933878eaff4a38ffbad5b07f79128f14d32500e22ac884 |
| 2026-08-17T16:49:03Z | 2026-08-17T16:48:25Z | ad14cdb | ghcr.io/rmi/stitch-entity-linkage:pr-0238 |
ghcr.io/rmi/stitch-entity-linkage:pr-0238@sha256:b0531bf3b6d702b20d1f646367cf38bfaa41afc550933353c530f83812d99bd8 |
| 2026-08-17T16:49:02Z | 2026-08-17T16:48:25Z | ad14cdb | ghcr.io/rmi/stitch-seed:pr-0238 |
ghcr.io/rmi/stitch-seed:pr-0238@sha256:c15480b89ce79515ccf22e2907d01e02e046eab03e24c5ac7c6ca46841683f0a |
| 2026-08-17T16:48:57Z | 2026-08-17T16:48:25Z | ad14cdb | ghcr.io/rmi/stitch-stitch-llm:pr-0238 |
ghcr.io/rmi/stitch-stitch-llm:pr-0238@sha256:546cc49b3c9d23db85854dd7b2408f7215ca63065b0553e3fb146b16b4ebdfcc |
CD summary
|
| service | url | fqdn |
|---|---|---|
| api | open | pr-0238-api.purplegrass-c07d0a94.westus2.azurecontainerapps.io |
| entity-linkage | open | pr-0238-el.purplegrass-c07d0a94.westus2.azurecontainerapps.io |
| frontend | https://witty-mushroom-017a3dc1e-238.westus2.1.azurestaticapps.net | |
| stitch-llm | open | pr-0238-llm.purplegrass-c07d0a94.westus2.azurecontainerapps.io |
Database (1)
| db_name | postgres_host | postgres_port | postgres_db |
|---|---|---|---|
| pr_0238 | stitch-dev.postgres.database.azure.com |
5432 |
pr_0238 |
Jobs (1)
| job | image | postgres_db |
|---|---|---|
| db-migrations | ghcr.io/rmi/stitch-api:pr-0238@sha256:340f53c0a867908e79a15173f8b6c7b2d680aac6ade02530db89228ad5ebacb5 |
pr_0238 |
Images (4)
| build_time | commit_time | git_sha | image | image_digest |
|---|---|---|---|---|
| 2026-08-18T16:51:04Z | 2026-08-18T16:50:47Z | 6de44b9 | ghcr.io/rmi/stitch-api:pr-0238 |
ghcr.io/rmi/stitch-api:pr-0238@sha256:340f53c0a867908e79a15173f8b6c7b2d680aac6ade02530db89228ad5ebacb5 |
| 2026-08-18T16:51:00Z | 2026-08-18T16:50:47Z | 6de44b9 | ghcr.io/rmi/stitch-entity-linkage:pr-0238 |
ghcr.io/rmi/stitch-entity-linkage:pr-0238@sha256:aac380f27bb220fd9d93c5b6441469e57652099daf0a4dbef5aead440b58f044 |
| 2026-08-18T16:51:00Z | 2026-08-18T16:50:47Z | 6de44b9 | ghcr.io/rmi/stitch-seed:pr-0238 |
ghcr.io/rmi/stitch-seed:pr-0238@sha256:d4a285f9915b8c460b40ed2e3791fa90fbf49f7d96d47fe47188d621391836a4 |
| 2026-08-18T16:51:02Z | 2026-08-18T16:50:47Z | 6de44b9 | ghcr.io/rmi/stitch-stitch-llm:pr-0238 |
ghcr.io/rmi/stitch-stitch-llm:pr-0238@sha256:0d7f3f47f1741eda61c31333aa593df92c01ba258cc9649aadac688bf92db5a0 |
CD summary
|
| service | url | fqdn |
|---|---|---|
| api | open | pr-0238-api.purplegrass-c07d0a94.westus2.azurecontainerapps.io |
| entity-linkage | open | pr-0238-el.purplegrass-c07d0a94.westus2.azurecontainerapps.io |
| frontend | https://witty-mushroom-017a3dc1e-238.westus2.1.azurestaticapps.net | |
| stitch-llm | open | pr-0238-llm.purplegrass-c07d0a94.westus2.azurecontainerapps.io |
Database (1)
| db_name | postgres_host | postgres_port | postgres_db |
|---|---|---|---|
| pr_0238 | stitch-dev.postgres.database.azure.com |
5432 |
pr_0238 |
Jobs (1)
| job | image | postgres_db |
|---|---|---|
| db-migrations | ghcr.io/rmi/stitch-api:pr-0238@sha256:a9d1c1e9f236d77e9a19c88a473f2aab32854b72f58cef89fe5c1e9132acb8b6 |
pr_0238 |
Images (4)
| build_time | commit_time | git_sha | image | image_digest |
|---|---|---|---|---|
| 2026-08-20T11:31:26Z | 2026-08-20T11:30:56Z | 32a5be5 | ghcr.io/rmi/stitch-api:pr-0238 |
ghcr.io/rmi/stitch-api:pr-0238@sha256:a9d1c1e9f236d77e9a19c88a473f2aab32854b72f58cef89fe5c1e9132acb8b6 |
| 2026-08-20T11:31:13Z | 2026-08-20T11:30:56Z | 32a5be5 | ghcr.io/rmi/stitch-entity-linkage:pr-0238 |
ghcr.io/rmi/stitch-entity-linkage:pr-0238@sha256:52d782b88bff07e7153db9dffe59b2330c5181d90f0b1454cfc552fdc540a85c |
| 2026-08-20T11:31:13Z | 2026-08-20T11:30:56Z | 32a5be5 | ghcr.io/rmi/stitch-seed:pr-0238 |
ghcr.io/rmi/stitch-seed:pr-0238@sha256:8caeeb742504b5736bc0266c549fbc2d0f1af6412128f6f9cae56571b144b0cb |
| 2026-08-20T11:31:11Z | 2026-08-20T11:30:56Z | 32a5be5 | ghcr.io/rmi/stitch-stitch-llm:pr-0238 |
ghcr.io/rmi/stitch-stitch-llm:pr-0238@sha256:2057afd2acc3130f5a813ed4d43cb40396cd19ef7268a3e6397015b24c825fe5 |
CD summary
|
| service | url | fqdn |
|---|---|---|
| api | open | pr-0238-api.purplegrass-c07d0a94.westus2.azurecontainerapps.io |
| entity-linkage | open | pr-0238-el.purplegrass-c07d0a94.westus2.azurecontainerapps.io |
| frontend | https://witty-mushroom-017a3dc1e-238.westus2.1.azurestaticapps.net | |
| stitch-llm | open | pr-0238-llm.purplegrass-c07d0a94.westus2.azurecontainerapps.io |
Database (1)
| db_name | postgres_host | postgres_port | postgres_db |
|---|---|---|---|
| pr_0238 | stitch-dev.postgres.database.azure.com |
5432 |
pr_0238 |
Jobs (2)
| job | image | postgres_db | api_url | auth_mode |
|---|---|---|---|---|
| db-migrations | ghcr.io/rmi/stitch-api:pr-0238@sha256:d616dfd54d329f9d3bbc40dfdee7830ca910244b1e1481a221cdaa246a01fb0d |
pr_0238 |
||
| seed | ghcr.io/rmi/stitch-seed:pr-0238@sha256:144be6d630fdee428763a90bb45978b617fe8642d766bcdc4e7832daf0e8103c |
https://pr-0238-api.purplegrass-c07d0a94.westus2.azurecontainerapps.io/api/v1 |
stitch-client-bearer-token |
Images (4)
| build_time | commit_time | git_sha | image | image_digest |
|---|---|---|---|---|
| 2026-08-20T13:16:05Z | 2026-08-20T13:15:36Z | 32a5be5 | ghcr.io/rmi/stitch-api:pr-0238 |
ghcr.io/rmi/stitch-api:pr-0238@sha256:d616dfd54d329f9d3bbc40dfdee7830ca910244b1e1481a221cdaa246a01fb0d |
| 2026-08-20T13:15:51Z | 2026-08-20T13:15:36Z | 32a5be5 | ghcr.io/rmi/stitch-entity-linkage:pr-0238 |
ghcr.io/rmi/stitch-entity-linkage:pr-0238@sha256:31efc37cc73e8a182bdb23a4630bc9e6ce8d39715739b7c115f81311a33391ae |
| 2026-08-20T13:15:53Z | 2026-08-20T13:15:36Z | 32a5be5 | ghcr.io/rmi/stitch-seed:pr-0238 |
ghcr.io/rmi/stitch-seed:pr-0238@sha256:144be6d630fdee428763a90bb45978b617fe8642d766bcdc4e7832daf0e8103c |
| 2026-08-20T13:15:49Z | 2026-08-20T13:15:36Z | 32a5be5 | ghcr.io/rmi/stitch-stitch-llm:pr-0238 |
ghcr.io/rmi/stitch-stitch-llm:pr-0238@sha256:a3a82adf319f5c8838b02896d2c4eda05975b5eac17b36582dbe0beae5983ab5 |
There was a problem hiding this comment.
Pull request overview
This PR mitigates the user-facing impact of Azure Container Apps scaling to zero (cold starts) by (1) widening the cooldown period in CI/CD and (2) making the frontend proactively “wake” the API and retry only genuinely transient failures, while also surfacing an explicit “server is waking up” message during stalled requests.
Changes:
- CI: add a post-deploy step to set
cooldownPeriodto 900s for scale-to-zero Container Apps viashow → jq → update --yaml. - Frontend: prewarm the API during bootstrap and introduce a TanStack Query retry predicate that retries only transient failures (5xx/408/429 + failed fetch).
- UI/tests: show an accessible “Server is waking up” notice after a short delay while requests are pending, with accompanying tests.
Reviewed changes
Copilot reviewed 11 out of 11 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
.github/workflows/deploy-container.yml |
Adds a deploy step to widen ACA scale-to-zero cooldown to reduce cold starts within a session. |
deployments/CI_DEPLOYMENTS.md |
Documents the widened cooldown and the frontend mitigations; adds a metrics command to measure impact. |
deployments/stitch-frontend/src/AppProviders.jsx |
Switches TanStack Query retry from a fixed count to a predicate for transient-only retries. |
deployments/stitch-frontend/src/main.jsx |
Fires a non-blocking API prewarm call during bootstrap so warm-up overlaps with Auth0 redirect time. |
deployments/stitch-frontend/src/queries/prewarm.js |
Implements a best-effort prewarm request to /health/details that should never block bootstrap. |
deployments/stitch-frontend/src/queries/prewarm.test.js |
Adds unit tests for prewarm behavior (URL, CORS-simplicity, non-blocking semantics). |
deployments/stitch-frontend/src/queries/retry.js |
Adds isRetriableError and shouldRetryQuery utilities to classify retryable failures. |
deployments/stitch-frontend/src/queries/retry.test.js |
Adds unit tests for retry classification and retry cap behavior. |
deployments/stitch-frontend/src/queries/api.js |
Ensures thrown HTTP errors include error.status so retry logic can inspect status codes. |
deployments/stitch-frontend/src/components/EnvironmentBanner.jsx |
Adds a delayed “waking up” live region message when queries appear stalled. |
deployments/stitch-frontend/src/components/EnvironmentBanner.test.jsx |
Adds tests for the “waking up” notice timing and clearing behavior. |
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
| export function prewarmApi(config) { | ||
| const baseUrl = config?.apiBaseUrl; | ||
|
|
||
| if (!baseUrl) { | ||
| return Promise.resolve(); | ||
| } | ||
|
|
||
| // Deliberately not aborted on a timer: Azure holds the request open while it | ||
| // activates a replica, and that wait is the thing doing the work. | ||
| // | ||
| // Deliberately unauthenticated and header-free, which keeps this a | ||
| // CORS-simple request. It reaches the server without a preflight even when | ||
| // the deployment's single allowed origin does not match the browser's -- and | ||
| // since the response is discarded, being blocked from reading it costs | ||
| // nothing. That matters while a custom domain is being cut over. | ||
| return fetch(healthUrl(baseUrl), { cache: "no-store" }).then( | ||
| () => undefined, | ||
| () => undefined, | ||
| ); | ||
| } |
| import { describe, it, expect, vi, beforeEach } from "vitest"; | ||
| import { prewarmApi } from "./prewarm"; | ||
|
|
||
| describe("prewarmApi", () => { | ||
| let fetchMock; | ||
|
|
||
| beforeEach(() => { | ||
| fetchMock = vi.fn().mockResolvedValue({ ok: true }); | ||
| vi.stubGlobal("fetch", fetchMock); | ||
| }); | ||
|
|
| lane is in real daily use: | ||
|
|
||
| ```bash | ||
| az monitor metrics list --resource "$(az containerapp show -n main-api -g STITCH-DEV-RG --query id -o tsv)" --metric Replicas --interval PT1H --start-time "$(date -u -v-13d +%Y-%m-%dT%H:%M:%SZ)" --end-time "$(date -u +%Y-%m-%dT%H:%M:%SZ)" --aggregation Maximum -o table |
* Guard `prewarmApi` against a non-string base URL. Unreachable via `loadConfig`, which already rejects one, but the function documents that it never rejects and `main.jsx` relies on that -- a synchronous throw would escape `bootstrap()` and swap the app for the config-error screen. * Unstub `fetch` in `afterEach`, matching the other suites that stub globals. `vitest.config.js` sets no `unstubGlobals`, and the shared `restoreAllMocks` does not undo `stubGlobal`. * Replace the BSD-only `date -v-13d` in the metrics snippet with `az monitor metrics list --offset 13d`, which needs no date arithmetic and works on GNU too. Phrase the sample as a ratio rather than an exact hour count, which drifts as the window moves. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
CD summary
|
| service | url | fqdn |
|---|---|---|
| api | open | pr-0238-api.purplegrass-c07d0a94.westus2.azurecontainerapps.io |
| entity-linkage | open | pr-0238-el.purplegrass-c07d0a94.westus2.azurecontainerapps.io |
| frontend | https://witty-mushroom-017a3dc1e-238.westus2.1.azurestaticapps.net | |
| stitch-llm | open | pr-0238-llm.purplegrass-c07d0a94.westus2.azurecontainerapps.io |
Database (1)
| db_name | postgres_host | postgres_port | postgres_db |
|---|---|---|---|
| pr_0238 | stitch-dev.postgres.database.azure.com |
5432 |
pr_0238 |
Jobs (1)
| job | image | postgres_db |
|---|---|---|
| db-migrations | ghcr.io/rmi/stitch-api:pr-0238@sha256:2fef95c5931453385318a21fef949768b8c3482a293edf860f798ab64acee528 |
pr_0238 |
Images (4)
| build_time | commit_time | git_sha | image | image_digest |
|---|---|---|---|---|
| 2026-08-20T17:15:41Z | 2026-08-20T17:15:22Z | 77bfdae | ghcr.io/rmi/stitch-api:pr-0238 |
ghcr.io/rmi/stitch-api:pr-0238@sha256:2fef95c5931453385318a21fef949768b8c3482a293edf860f798ab64acee528 |
| 2026-08-20T17:15:40Z | 2026-08-20T17:15:22Z | 77bfdae | ghcr.io/rmi/stitch-entity-linkage:pr-0238 |
ghcr.io/rmi/stitch-entity-linkage:pr-0238@sha256:a614c1d6491d8693105e4f6853140840520fbd67675adc3920dd8f148536105c |
| 2026-08-20T17:15:44Z | 2026-08-20T17:15:22Z | 77bfdae | ghcr.io/rmi/stitch-seed:pr-0238 |
ghcr.io/rmi/stitch-seed:pr-0238@sha256:2dc781685f1ea429c23d5af4e04fe551804ab0be8a4a6382b864b80261404adc |
| 2026-08-20T17:15:40Z | 2026-08-20T17:15:22Z | 77bfdae | ghcr.io/rmi/stitch-stitch-llm:pr-0238 |
ghcr.io/rmi/stitch-stitch-llm:pr-0238@sha256:1e77b6b24e14440ac48dcb2513ab08bd75797ee28a0d3bdda80802c066754b7e |
Alex's TL;DR here, Claude's fuller summary below the fold:
Last pinged around 1510 Belgrade time. Update this description as needed.
Responding to changes in #234, where we set more environments to scale to zero (fall asleep) this PR is a collection of workarounds to mitigate the impact of that in practice:
Why
#234 narrows the always-on policy so that only pushes to
productionkeep a warm replica. Everything else —main, demo lanes, PR previews — now scales to zero and pays a cold start on the first request of a session. This PR makes that acceptable: slightly less frequent, and no longer indistinguishable from the app being broken.Widening the cooldown
Cooldown is how long a Container App stays up after its last request. Azure's default is 300s, so a five-minute pause costs a cold start. This raises it to 900s for apps that actually scale to zero, reasserted after every deploy.
Two things a reviewer should know about it:
There is no CLI flag.
az containerapp updatehas no--cooldown-periodand no generic--set, only--yaml. The step therefore does the sameshow → jq → update --yamlround-trip already used for ETL volume mounts, and runs after that step so it re-reads its result. The jq transform was dry-run against the livemain-apispec:cooldownPeriod300 → 900,systemDatastripped (ARM rejects it on write), image digest and the rest of the template intact.Absorbing the cold start in the browser
Retry transient failures instead of surfacing them. The global
retry: 1retried once after ~1s — still inside an ACA activation — then showed the user an error. Now a predicate retries only 5xx, 408, 429 and theTypeErrora droppedfetchrejects with, up to 3 times.Wake the API during bootstrap. A throwaway GET fired from
bootstrap()and not awaited. It runs before Auth0 mounts, so the container starts while the user is still being redirected through login. Targets/health/detailsrather than/healthbecause it resolves the database engine, so it opens a connection too — the two costs of a cold start, both paid during dead time. Neither endpoint requires auth.Say so when it is still slow. After 2s of a request in flight, the environment banner shows "Server is waking up — this can take a few seconds" as a
role="status"live region. Nothing to fix in that moment; it just needs saying, so the wait reads as starting up rather than nothing happening.Details that look like omissions but are deliberate
Each is commented at the call site and pinned by a test, so they are not tidied away later.
min-replicas == ''stitch-dev.rmi.orgcutoverretry: 0Things to look at
A 4xx is no longer retried.
retry: 1retried every failure once, including 404 and 401. Re-sending an unchanged bad request only delays an error that was never going to change — but this is a change beyond "retry more", so it is worth a reviewer's eye.getResourcesandgetResourceFilterOptionsnow attacherror.status. Every other throw site already did; these two carried the status only in the message string, so a status-based retry predicate could not see them. They back the resources list and its filter options — the first requests after sign-in, and so exactly the ones that meet a cold container. The retry fix does not work without this.The cooldown step costs CI time. A
showplus anupdateper scale-to-zero app per deploy — four apps on a PR lane, so perhaps 30–60s. Cheap, but not free, and easy to forget once buried in a reusable workflow.Existing apps stay at 300s until their next deploy.
Not in this PR
/health. No probes are configured on any container app, so ACA falls back to TCP-open, and uvicorn can accept connections before the FastAPI lifespan finishes. That is a plausible source of the 503s the retry policy is defending against. It needs the same--yamlround-trip, so it is CI work and belongs in its own PR.retryDelay. Left at the TanStack default (1s/2s/4s). It is the knob to turn if the retries prove too eager against a real cold start, but picking a curve now would be guessing.Notes for review
Stacked on #234 — base is
ci/hostnames, so that needs to merge first.AI was used to write the frontend changes, the CI step, the tests and this description. The measurements are from
az monitor metrics listagainstmain-api, not estimates. The new banner tests were mutation-checked: forcing the delay never to elapse kills exactly the two tests asserting the notice appears and clears, and leaves the other six green.