Improve study startup assignment performance

[JackWilb]

Summary

This PR improves study-startup performance and sequence-assignment correctness.

It removes participant-history-dependent collection scans from normal startup, eliminates redundant startup reads and large object downloads, and assigns stable sequence and participant indexes using provider-specific concurrency controls. Once the allocator is initialized, the amount of assignment work performed for a new or returning participant is bounded and does not grow with the number of prior participants.

Closes #1176.

Performance changes

• Replace up to three complete sequence-assignment collection scans during participant startup with participant-specific reads, bounded queries, aggregate counts for legacy records, and allocator state.
• Persist zero-based sequenceIndex and creationIndex values on new assignments so returning participants can use a point read.
• Reuse the sequence array and modes document already fetched during Shell initialization.
• Use one participant-specific assignment lookup for completion status.
• Skip downloading an unchanged stored config when its active hash already matches.
• Narrow Supabase startup verification from selecting every study row to selecting only the connection row.

The important bound is on client-visible database work: after allocator initialization, normal assignment startup uses a fixed number of point reads, bounded queries, and writes rather than downloading work proportional to participant history. This does not impose a literal wall-clock timeout—authentication, provider latency, retries, and network throughput can still vary—and Firestore transactions may produce more raw HTTP protocol entries even while eliminating redundant logical reads and unbounded document downloads.

Measured effect

For CALVI with approximately 33 assignments, cache-disabled Firefox HARs showed:

• dev downloaded the approximately 3.35 MB sequence array twice per load.
• This PR downloads it once, reducing response bytes from approximately 9.69 MB to 6.41 MB, or about 34%.
• The sequence object was served with Cache-Control: private, max-age=0, so the second dev download was not expected to become a browser-cache hit when cache was enabled.
• Fast-network first-render timing was effectively unchanged at this study size: median approximately 4.03 seconds on dev versus 4.17 seconds on the PR. At roughly 33 assignments, the removed collection scans were inexpensive and the bounded Firebase transaction replaced much of the saved elapsed time.

The immediate benefit is therefore lower transferred data and less duplicate parsing, especially on cold or mobile connections. The database benefit grows with participant history because normal startup no longer downloads the complete assignment collection.

Config-change startup still regenerates and uploads the expanded sequence array; that work is intentionally deferred to the follow-up issues listed below.

Correctness changes

• Firebase commits the participant assignment, allocator update, and rejected-slot claim in one Firestore transaction. The transaction re-reads the participant, allocator, and candidate rejected assignment so contention causes a retry rather than a duplicate allocation.
• Local storage serializes allocation within the application.
• Supabase uses a versioned compare-and-swap allocator, which prevents duplicate counter reservations under the tested concurrent-start scenario.
• Rejected-slot reuse remains FIFO by assignment timestamp and supports legacy assignments without allocator metadata and chained replacement rejection.
• Existing studies require no assignment migration; allocator state is initialized lazily using provider-side counts.

Automated tests start 12 participants concurrently for Firebase, Supabase, and local storage and verify unique sequence and creation indexes. Provider tests also cover fresh startup, initialized allocators, legacy returning assignments, legacy rejected-slot reuse, oldest rejected-slot selection, chained replacement rejection, and participant-specific completion lookup.

Correctness caveats

• The Firebase concurrency test uses the repository's serialized in-memory Firestore transaction mock. Rejected-slot behavior is covered functionally, but there is not a dedicated Firebase emulator test in which two simultaneous participants compete for the same rejected slot. The production transaction is designed to handle that race by re-reading and atomically claiming the candidate before commit.
• Supabase allocation is not atomic end to end: rejected-slot claim, allocator reservation, and participant-assignment upsert are separate database operations. The counter reservation is race-safe, but a failure between those operations can leave partial state. Make Supabase sequence assignment allocation atomic #1368 tracks moving the complete operation into an atomic, idempotent database transaction/RPC.

Firebase deployment requirement

This PR introduces two Firestore compound queries that were not required by the previous full-scan implementation. Firebase's existing automatic single-field indexes, and any unrelated composite indexes already in a project, do not replace them.

Each Firebase project/database must create these collection-scoped composite indexes once:

1. Legacy sequence-position lookup
   • rejected ascending
   • timestamp ascending
2. Deterministic rejected-slot lookup
   • rejected ascending
   • claimed ascending
   • timestamp ascending

The first index is used only when deriving the position of a legacy assignment without stored allocator metadata. The second query runs during fresh Firebase participant allocation even when no rejected assignment is found, so a fresh Firebase project must create that index before its first participant starts.

These indexes are required once per Firebase project/database, not once per study. They apply to the shared sequenceAssignment collection scope and serve dynamically named prod-{studyId} and dev-{studyId} study collections. If an index already exists in a test project, it may have been created while testing this PR and remains available regardless of which application branch is deployed.

When Firestore reports a missing index, the application shows the exact Firebase-generated creation link, explains that setup is a one-time action for the study owner, and provides open/copy controls.

Follow-up work

• Move sequence-array publication off participant startup's critical path #1319 moves sequence-array publication off participant startup's critical path.
• Render the study interface while participant startup resolves #1320 renders the study interface behind a blocking modal while participant startup resolves.
• Replace expanded sequence arrays with a compact deterministic representation #1321 replaces expanded sequence arrays with a compact deterministic representation.
• Reduce bounded sequence-assignment allocator round trips #1322 reduces remaining bounded allocator round trips.
• Profile and optimize application bundle and authentication startup #1323 profiles and optimizes bundle and authentication startup.
• Make Supabase sequence assignment allocation atomic #1368 makes Supabase sequence allocation atomic end to end.

Validation

• yarn unittest --run — 1,980 passed, 1 skipped
• yarn typecheck
• yarn lint
• yarn build
• Focused provider regression suite — 168 passed
• Local NASA-TLX fresh-load smoke test — 672, 645, 642, 628, 641 ms; median 642 ms
• Firebase fresh and returning-participant HAR captures
• Firebase CALVI before/after HAR captures with browser cache disabled
• Live Firebase legacy allocator/index query smoke test; the attempted returning-participant test bypassed allocation and therefore did not exercise this query

[github-actions]

A preview of 8c15319 is uploaded and can be seen here:

✨ https://revisit.dev/study/PR1311 ✨

Changes may take a few minutes to propagate.

[Copilot]

Pull request overview

This PR optimizes normal participant startup by replacing repeated sequence-assignment collection scans with bounded, race-safe allocation using stable sequenceIndex and creationIndex metadata across Firebase, Supabase, and LocalStorage. It also reduces redundant reads by reusing prefetched sequence/modes data, switching completion checks to a single assignment lookup, and skipping stored-config downloads when the active config hash is unchanged.

Changes:

• Add atomic/bounded sequence-assignment allocation (sequenceIndex, creationIndex) with legacy compatibility and rejected-slot reuse across providers.
• Reduce startup I/O by reusing prefetched modes + sequence array in Shell, switching completion to a single assignment lookup, and narrowing Supabase verification reads.
• Add/extend contract and concurrency tests to lock in “no scans” behavior and uniqueness under concurrent starts.

Reviewed changes

Copilot reviewed 12 out of 12 changed files in this pull request and generated 4 comments.

Show a summary per file

File	Description
src/tests/utils.ts	Extends the test storage engine with the new allocation primitive.
src/storage/tests/primitivesSupabase.spec.ts	Enhances Supabase mock query builder to support new bounded operations used by allocator logic.
src/storage/tests/primitivesFirebase.spec.ts	Extends Firestore mock surface (query/count/transaction) needed by new allocator paths.
src/storage/tests/highLevelSupabase.spec.ts	Adds high-level Supabase tests for “no scans”, legacy compatibility, and concurrency uniqueness.
src/storage/tests/highLevelFirebase.spec.ts	Adds high-level Firebase tests for “no scans”, legacy compatibility, and concurrency uniqueness.
src/storage/tests/highLevel.spec.ts	Adds provider-agnostic tests for no-scan startup, bootstrap reuse, and concurrency behavior.
src/storage/engines/types.ts	Introduces allocator metadata/types and bootstrapping data plumbing; updates startup and completion logic.
src/storage/engines/SupabaseStorageEngine.ts	Implements Supabase allocator (CAS/version counter), legacy index derivation, and bounded verification/lookup behavior.
src/storage/engines/LocalStorageEngine.ts	Implements serialized local allocation and updates rejection handling for reusable slots.
src/storage/engines/FirebaseStorageEngine.ts	Implements Firestore transaction-based allocator, legacy index derivation via count aggregates, and bounded completion lookup.
src/components/tests/Shell.spec.tsx	Asserts Shell passes prefetched modes + sequence array into participant initialization.
src/components/Shell.tsx	Reuses the prefetched sequence array and passes bootstrap data into participant initialization.

Comments suppressed due to low confidence (1)

src/storage/engines/FirebaseStorageEngine.ts:322

• The reusable rejected assignment lookup uses limit(1) without an explicit sort. Firestore does not guarantee any stable order here, so this can change which rejected slot is reused and break the existing "oldest rejected first" behavior.

      getDocs(query(
        sequenceAssignmentCollection,
        where('rejected', '==', true),
        where('claimed', '==', false),
        limit(1),

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[jaykim1213]

The PR preview currently fails during participant startup because the required sequenceAssignment composite index has not been created in Firebase