Skip to content

Load less JavaScript, cache shared reads, and bound server memory - #100

Merged
poitee merged 23 commits into
mainfrom
perf/print-partner-optimizations
Sep 28, 2026
Merged

poitee merged 23 commits into
mainfrom
perf/print-partner-optimizations

Conversation

@poitee

@poitee poitee commented Sep 28, 2026 •

Copy link
Copy Markdown
Owner

Summary

Performance and scaling work, stacked on #99. Pages load less JavaScript, repeat fewer requests, and the server holds less memory under uploads. It also removes S3 support that nothing used and sets v3.3.0 as the oldest version you can upgrade from. npm run quality passes.

What changes for users

JavaScript a page downloads before it renders:

Page Before After
Checkoff 884 KB (246 KB gzipped) 258 KB (84 KB gzipped)
Production 842 KB (230 KB gzipped) 200 KB (65 KB gzipped)
Plan 679 KB (183 KB gzipped) 143 KB (48 KB gzipped)
Printers 179 KB (59 KB gzipped) 86 KB (32 KB gzipped)
  • three.js loads when a thumbnail must be drawn or a 3D view opens; jszip loads when a zip or spreadsheet is read. scripts/route-bundles.mjs prints this table from the Vite manifest and fails if three.js reaches any page's initial load. npm run quality runs it after the build.
  • Printers, integrations, filaments, spools, slicer options and the date format now read through TanStack Query, so pages share one cache. Each printer on Settings costs 2 requests instead of 3, and reloading the Build list sends one /plans request instead of two. src/queries/readRequests.test.tsx counts them.
  • Printer and Checkoff events refresh only the affected Build. Hosts that stop answering are polled at 10, 20, 40, then every 60 seconds, and return to the normal cadence on the first answer.

Server

  • JSON bodies default to 1 MiB. Plan save, drafts, progress import, plates, production setup, kit and repo manifests, and import rules allow 8 MiB. Assistant actions allow 96 MiB, MCP inline files 64 MiB, thumbnails 1 MiB.
  • Source uploads stream to disk with the byte budget enforced while streaming, and nested 3MF packages are read in chunks. A 96 MiB upload peaks at about 40 MiB of memory instead of about 200 MiB (src/upload-memory.test.ts).
  • S3 storage is removed. S3_BUCKET made /health report artifactStore: "s3" while every blob stayed on local disk. @aws-sdk/client-s3 is gone, along with the RustFS service in docker-compose.saas.yml.

Upgrade floor (breaking for very old installs)

A database below schema 31 (v3.3.0, the only tagged release) is refused before any backup or DDL, with instructions to install v3.3.0, start it once, then upgrade. This removed about 3,200 lines of pre-31 migrations and repairs. A v3.3.0 database upgraded by the old and the new code produces identical SQLite sqlite_master and rows, and identical Postgres catalogs and rows (checked in PGlite). CHANGELOG and OPERATIONS describe the step-through upgrade.

Verification

lint, knip, typecheck, npm test (contracts 101, domain 141, web 1320, server 2131), workflow smoke, build, route bundle gate, runtime and browser tests all pass.

Not included

Moving the contracts to zod/mini is on perf/zod-mini and held back. It would save about 14 KB gzipped per visit, but zod/mini overflows the stack when an object field collects more than about 120k issues, turning some malformed requests from 400 into 500, and it parses large Plan saves about twice as slowly on the server because it skips classic's compiled object path. The overflow comes from issues.push(...) spreads in zod's shared core, still present in zod 4.6.5, and classic hits it too for nested arrays, records of arrays and tuples. Revisit once zod fixes that upstream.

🤖 Generated with Claude Code


Note

High Risk
Breaking upgrade floor and removed pre-31 migration code can brick old databases; upload limit changes may reject previously accepted large MCP/JSON payloads.

Overview
This release removes unused S3 blob storage end to end: @aws-sdk/client-s3, StoragePort wiring in self-host and SaaS adapters, S3_* config, RustFS from docker-compose.saas.yml, and related deploy/docs. Artifacts stay on local disk under the data directory.

Upgrade policy is stricter. Startup now refuses SQLite or Postgres below schema 31 (v3.3.0) via upgrade-guard before migrations run, and drops thousands of lines of incremental migration/repair paths (compatibility dirty repair, legacy print-plan removal, v19–v31 cutover modules). Operators must run v3.3.0 once, then this release. CHANGELOG and OPERATIONS document the step-through path.

Request and upload limits are retuned in upload-limits.ts and docs: default JSON 1 MiB, bulk Plan/manifest routes 8 MiB, assistant/MCP bodies 96 MiB, MCP inline files 64 MiB (down from 256 MiB for propose_import_source_files). Source uploads stream to disk; MCP imports stage to temp files; 3MF parsing uses chunked reads (byte-chunks.ts). SECURITY and API docs describe what still buffers in memory vs streams.

Tests and Postgres DDL are aligned with the slimmer migration story (fewer legacy upgrade scenarios, consolidated plan_revision_inputs shape in fresh migrations).

Reviewed by Cursor Bugbot for commit cc907fc. Bugbot is set up for automated code reviews on this repo. Configure here.

poitee and others added 23 commits September 27, 2026 16:25
Checkoff, Production, and Plan imported three.js statically through the
part thumbnail generator and the plate 3D preview, and jszip through the
sliced-file and xlsx parsers, so every visit to those pages downloaded
both. Thumbnail generation, the plate preview, and both zip readers now
load on demand; the plate preview shows its loading box meanwhile.

JavaScript each route loads before rendering (uncompressed, gzip):
  Checkoff    +884 KB / 246 KB  ->  +258 KB / 84 KB
  Production  +842 KB / 230 KB  ->  +200 KB / 65 KB
  Plan        +679 KB / 183 KB  ->  +143 KB / 48 KB
  Printers    +179 KB /  59 KB  ->   +86 KB / 32 KB

scripts/route-bundles.mjs prints that table from the Vite manifest and
fails when three.js reaches any route's static import closure; npm run
quality runs it after the build.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…anifest routes to 8 MiB

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ion bodies at 96 MiB

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The web renderer produces 256 by 256 PNGs, well under 1 MiB, so the route
no longer buffers up to 64 MiB per request.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…them in memory

Source ZIP and file uploads now stream into a staging directory beside the
Source directories while the byte budget is enforced, then move into place.
ZIP extraction reads the archive from disk in chunks. MCP file imports use
the same staging path.

A 96 MiB upload-zip request peaked at about 200 MiB of heap plus external
memory before and about 35 MiB after.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…l document

Source uploads no longer load each nested 3MF package whole. The reader
streams the package from disk, refuses an oversized model document from its
declared size, and stops once the model document is complete.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… copying it

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The file picker saves one rule per selected file and the route has no
rule-count cap, so a large Source could exceed the new 1 MiB default.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Printers, Settings, Checkoff, and Production screens each fetched the
printer fleet and host integrations on their own. They now read shared
printers and integrations queries, and fleet writes in Settings update or
invalidate those caches.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The Settings printer card fetched plans on its own and the unmatched print
card kept a fallback fetch that no screen used. The printer card now reads
the profiles query, and the unmatched print card takes plans from its
parent.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The role color picker, part color dialog, Settings printer card, and
Review sheet each fetched role filaments, the filament catalog, or
Spoolman spools on their own. They now read shared queries. Custom
filament and Spoolman setting writes invalidate the catalog, and a Plan
rebuild invalidates that Build's role filaments.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Each printer's slicer profile section fetched the same option lists, so
Settings made one request per printer. The sections now share one query.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
All Production fetched printer checkoff links and unmatched prints by hand.
It now reads them through queries, and the Printers page shares the same
checkoff links cache.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… events

Printer reconcile results, claimed unmatched prints, and Plan workspace
refreshes reloaded the summary of every Build. The events already carry
the Build id, so All Production and the Plan workspace now refetch that
one Build's summary. Unmatched-file and dismiss events refresh farm state
without touching Build summaries, and one reconcile notifies each
affected Build once.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Status and reconcile polls ran at the configured cadence even when a
host kept failing. Each consecutive offline answer now doubles that
host's poll interval up to five minutes, and the first answer from the
host restores the configured cadence. Healthy hosts keep their cadence,
and a host never starts a poll while its previous one is running.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
reloadProfiles invalidated the profiles query, which refetched it, and
then refetched it again. It now refetches once.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A five-minute cap left a printer that was switched back on showing
offline for up to five minutes. At the default five-second cadence an
unreachable host is now probed at 10, 20, 40, then every 60 seconds.
The backoff tests run at that default cadence.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The S3 and local storage ports were constructed but never read, so every
blob went to local disk while /health reported artifact_store "s3". Remove
the ports, S3 config, the @aws-sdk/client-s3 dependency, the RustFS
development service, and the S3 deployment documentation.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…eir migrations

Print Partner v3.3.0 ships schema 31, so it becomes the oldest release this
one upgrades in place. SQLite and Postgres startup now refuse an older stored
schema before touching the database and tell the operator to step through
v3.3.0 first. The version-gated repairs below 31 (accepted revision and
required-unit backfills, the v26 compatibility cutover, legacy print Plan
removal, the filament and plate-edit repairs, the v17 Plan-input rebuild) and
their tests are gone. Fresh installs build the same schema as before.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@coderabbitai

coderabbitai Bot commented Sep 28, 2026 •

Copy link
Copy Markdown

Review in Change Stack →

Navigate logical layers of code changes, visualize relationships, and explore their blast radius.

Note

Currently processing new changes in this PR. This may take a few minutes, please wait...

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Team

Run ID: 9a77f9c6-9edf-4031-8034-ca15350801c2

📥 Commits

Reviewing files that changed from the base of the PR and between 3d16e62 and cc907fc.

⛔ Files ignored due to path filters (1)
  • web/package-lock.json is excluded by !**/package-lock.json
📒 Files selected for processing (116)
  • CHANGELOG.md
  • OPERATIONS.md
  • README.md
  • SECURITY.md
  • THIRD_PARTY_NOTICES.md
  • docker-compose.hosted.yml
  • docker-compose.saas.yml
  • docs/API.md
  • docs/ARCHITECTURE.md
  • docs/README.md
  • docs/audits/2026-09-04-codebase-rehabilitation.md
  • hosted-planning.env.example
  • scripts/docker-startup.test.mjs
  • web/DEPLOY.md
  • web/README.md
  • web/apps/server/package.json
  • web/apps/server/src/accepted-plate-cutover-inventory.test.ts
  • web/apps/server/src/adapters/saas/index.ts
  • web/apps/server/src/adapters/saas/storage-s3.ts
  • web/apps/server/src/adapters/self-host/index.ts
  • web/apps/server/src/assistant/tools.test.ts
  • web/apps/server/src/assistant/tools.ts
  • web/apps/server/src/config.ts
  • web/apps/server/src/data/legal/THIRD_PARTY_NOTICES.md
  • web/apps/server/src/db/accepted-filament-assignment-schema.ts
  • web/apps/server/src/db/accepted-plan-revisions.test.ts
  • web/apps/server/src/db/accepted-plan-revisions.ts
  • web/apps/server/src/db/accepted-plate-edit-schema.ts
  • web/apps/server/src/db/client-postgres.ts
  • web/apps/server/src/db/client.ts
  • web/apps/server/src/db/compatibility-dirty-repair.ts
  • web/apps/server/src/db/legacy-print-plan-removal.ts
  • web/apps/server/src/db/migrations-pg.ts
  • web/apps/server/src/db/migrations-sqlite.ts
  • web/apps/server/src/db/plan-drafts.test.ts
  • web/apps/server/src/db/required-units.test.ts
  • web/apps/server/src/db/required-units.ts
  • web/apps/server/src/db/schema-v9.test.ts
  • web/apps/server/src/db/upgrade-guard.test.ts
  • web/apps/server/src/db/upgrade-guard.ts
  • web/apps/server/src/json-body-limits.test.ts
  • web/apps/server/src/lib/byte-chunks.ts
  • web/apps/server/src/lib/deployment-capability.test.ts
  • web/apps/server/src/lib/deployment-capability.ts
  • web/apps/server/src/phase5.test.ts
  • web/apps/server/src/ports/index.ts
  • web/apps/server/src/routes/accepted-part-media-routes.test.ts
  • web/apps/server/src/routes/accepted-plates.ts
  • web/apps/server/src/routes/health.ts
  • web/apps/server/src/routes/plan-drafts.ts
  • web/apps/server/src/routes/plans.ts
  • web/apps/server/src/routes/printer-checkoff.ts
  • web/apps/server/src/routes/production-setup.ts
  • web/apps/server/src/routes/repo-manifest.ts
  • web/apps/server/src/routes/sources.ts
  • web/apps/server/src/services/archive-import.test.ts
  • web/apps/server/src/services/archive-import.ts
  • web/apps/server/src/services/legacy-source-manifest.ts
  • web/apps/server/src/services/plan-publication.ts
  • web/apps/server/src/services/three-mf-import.test.ts
  • web/apps/server/src/services/three-mf-import.ts
  • web/apps/server/src/services/upload-limits.ts
  • web/apps/server/src/upload-boundaries.test.ts
  • web/apps/server/src/upload-memory.test.ts
  • web/apps/web/package.json
  • web/apps/web/scripts/route-bundles.mjs
  • web/apps/web/src/App.site-map.test.tsx
  • web/apps/web/src/api/endpoints/slicers.ts
  • web/apps/web/src/components/RoleFilamentPicker.tsx
  • web/apps/web/src/components/checkoff/PastPrintIntakePanel.test.tsx
  • web/apps/web/src/components/checkoff/PastPrintIntakePanel.tsx
  • web/apps/web/src/components/checkoff/PrinterLiveStrip.test.tsx
  • web/apps/web/src/components/checkoff/PrinterLiveStrip.tsx
  • web/apps/web/src/components/checkoff/UnattributedPrintCard.test.tsx
  • web/apps/web/src/components/checkoff/UnattributedPrintCard.tsx
  • web/apps/web/src/components/export/PrinterSendPanel.tsx
  • web/apps/web/src/components/export/ProductionRulesPanel.test.tsx
  • web/apps/web/src/components/export/ProductionRulesPanel.tsx
  • web/apps/web/src/components/export/accepted-plates/AcceptedPlateBed.test.tsx
  • web/apps/web/src/components/export/accepted-plates/AcceptedPlateBed.tsx
  • web/apps/web/src/components/export/accepted-plates/AcceptedPlateGallery.test.tsx
  • web/apps/web/src/components/export/useProductionSendFleet.ts
  • web/apps/web/src/components/parts/PartThumb.tsx
  • web/apps/web/src/components/review/PartColorDialog.tsx
  • web/apps/web/src/components/review/ReviewPartsSheet.accessibility.test.tsx
  • web/apps/web/src/components/review/ReviewPartsSheet.tsx
  • web/apps/web/src/components/settings/IntegrationsSettingsCard.tsx
  • web/apps/web/src/components/settings/PrinterProfileAssignmentSection.tsx
  • web/apps/web/src/components/settings/PrintersSettingsCard.tsx
  • web/apps/web/src/components/share/ReferenceShareImport.test.tsx
  • web/apps/web/src/components/share/ReferenceShareImport.tsx
  • web/apps/web/src/context/DateFormatContext.tsx
  • web/apps/web/src/context/PlanWorkspaceContext.test.tsx
  • web/apps/web/src/context/PlanWorkspaceContext.tsx
  • web/apps/web/src/context/ProfileContext.reload.test.tsx
  • web/apps/web/src/context/ProfileContext.tsx
  • web/apps/web/src/hooks/useSpoolmanEnabled.ts
  • web/apps/web/src/lib/parseSlicedObjects.ts
  • web/apps/web/src/lib/partsManifestXlsx.ts
  • web/apps/web/src/pages/BoardPage.test.tsx
  • web/apps/web/src/pages/CheckoffPage.tsx
  • web/apps/web/src/pages/GlobalProductionPage.test.tsx
  • web/apps/web/src/pages/GlobalProductionPage.tsx
  • web/apps/web/src/pages/PrintersPage.tsx
  • web/apps/web/src/pages/SettingsPage.tsx
  • web/apps/web/src/queries/dateFormat.ts
  • web/apps/web/src/queries/filaments.ts
  • web/apps/web/src/queries/keys.ts
  • web/apps/web/src/queries/printerCheckoff.ts
  • web/apps/web/src/queries/printerFleet.ts
  • web/apps/web/src/queries/printerStatuses.integration.test.tsx
  • web/apps/web/src/queries/printerStatuses.ts
  • web/apps/web/src/queries/readRequests.test.tsx
  • web/apps/web/src/queries/slicerProfileOptions.ts
  • web/package.json
  • web/scripts/server-runtime.test.mjs
 __________________________________________________________
< Sometimes, I pretend to be a compiler to feel important. >
 ----------------------------------------------------------
  \
   \   (\__/)
       (•ㅅ•)
       /   づ
✨ Finishing Touches
📝 Generate docstrings
  • Commit to this branch
  • Create a new PR
🧪 Generate unit tests (beta)
  • Commit to this branch
  • Create a new PR

Comment @coderabbitai help to get the list of available commands.

@cursor

cursor Bot commented Sep 28, 2026

Copy link
Copy Markdown
Contributor

Bugbot couldn't run - usage limit reached

Bugbot is counted against Cursor usage for this user or team, and this run hit a usage or spend limit.

A user or team admin can review and increase usage limits in the Cursor dashboard.

(requestId: serverGenReqId_249b912c-34cd-4146-84fb-3fca81321abd)

@poitee

poitee commented Sep 28, 2026

Copy link
Copy Markdown
Owner Author

Independent pre-merge verification: PASS+NOTES for cc907fcd3570b2510c08bf46d78b7294824de78c against 5f637a9f7fa8eea8db33db1a0d34c5ebeed66781.

An independent Codex agent that did not author this PR inspected shared reads, invalidation, polling, dynamic imports, disk-backed uploads, archive boundaries and upgrade guards. All 78 fresh focused tests passed. A real 96 MiB ZIP upload succeeded with 50.1 MiB peak heap-plus-external growth. The existing built bundle passed its import-closure check with no static Three.js route imports. No concrete merge blocker was found.

Limits: tests used integrated stack head 94e42d00; reviewed implementations and focused tests match this PR's head. Exact-head CI and the integrated 3,720-test quality pass provide separate supporting evidence. The schema-31 upgrade floor and removal of the unused S3 port are intentional changes. Cursor cloud verification was unavailable; this is a Codex review, not human approval. No production deployment or physical printer action was tested.

@poitee
poitee changed the base branch from cleanup/audit-debloat to main September 28, 2026 04:28
@poitee
poitee merged commit 59fec59 into main Sep 28, 2026
5 of 6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant