Dispatch Lambda processing by instrument_type instead of ID allowlists

[wasimxyz]

Summary

Lambda file processing no longer keys off hardcoded instrument IDs. The handler now resolves an instrument's instrument_type through the API and looks up a processor in a single registry, so onboarding an instrument that reuses an existing vendor format needs no Lambda code change, no Instrument enum entry, and no new S3 trigger.

Three allowlists collapse into one registry plus one mirrored TypeScript constant:

• lambda/src/data_hub_lambda/processors.py: maps instrument_type to a process_file function and a filename gate.
• web/lib/instruments/processable-types.ts: replaces processable-ids.ts and gates the reprocess UI and API on type.
• infra/template.yaml: nine per-instrument LambdaConfiguration filter blocks collapse to one catch-all ObjectCreated:* notification.

Why

Adding an instrument previously meant editing the shared Instrument enum, INSTRUMENT_ID_TO_NAME_MAP, an elif chain in handler.py, PROCESSABLE_INSTRUMENT_IDS, and a prefix/suffix filter pair in the SAM template. Every one of those was keyed on a specific instrument ID, so a second SpectraMax or a second gel doc required touching all five. Instrument type already exists as a Postgres enum on the instruments table and is the property that actually determines which parser applies.

Behavior changes worth reviewing

• Filename gating moved from S3 into the handler. The catch-all notification means the Lambda now sees every ObjectCreated event on the raw bucket. A union gate over all processor suffixes runs before any API call, so non-matching files return without network I/O.
• Reprocess skips filename gates. Function URL invocations bypass both the union gate and the per-type gate. A user clicking Reprocess has stated intent, and gating there would strand the file in processing after the web app already transitioned it. If the Function URL path still cannot dispatch (unmapped type / instrument 404), the handler PATCHes the file to failed so it is not stranded.
• The Lambda now calls GET /instruments/:id per event (cached ~60s on a warm container). One short in-invocation retry (0.5s) covers blips; exhausted transient errors re-raise so S3 async retries can take over. 404 skips (and fails the file on reprocess); 401/403 log a scope hint and re-raise.
• New fplc instrument type added to instrumentTypeEnum (migration 0034). Seeded / live jolene-fplc deliberately stays generic: its PDFs have not been confirmed against the ÄKTA parser, and mistyping it would feed non-ÄKTA files into that processor.
• instrument_type added to the run list API response so canReprocessRun can gate on type client-side.

Test plan

• lambda/tests/test_processors.py: registry lookup, gates, union gate, web/Python allowlist sync, instrument cache, Function URL fail-to-failed, single transient retry.
• lambda/tests/integration/test_lambda_api.py: updated for type dispatch, 404/401/403 paths, and gate-skipping on reprocess.
• web/tests/integration/files.test.ts: reprocess eligibility on type, including 409 for unprocessable types.
• make check-all passes.
• Staging smoke tests from the operator plan above.
• Production smoke tests from the operator plan above.

Docs

developer-docs/ updated across lambda.md, conventions.md, ci-and-deployment.md, first-time-deployment.md, and shared-library.md to describe type dispatch and drop the enum-registration steps.

Made with Cursor

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
data-hub	Ready	Preview, Comment	Jul 27, 2026 11:18pm