Skip to content

fix(persistence,rest): _lastUpdated on Postgres; server-populated meta in response bodies - #876

Merged
smunini merged 6 commits into
mainfrom
fix/871-873-read-correctness
Sep 3, 2026
Merged

fix(persistence,rest): _lastUpdated on Postgres; server-populated meta in response bodies#876
smunini merged 6 commits into
mainfrom
fix/871-873-read-correctness

Conversation

@angela-helios

Copy link
Copy Markdown
Contributor

Closes #871
Closes #873

Both found by the #448 Postgres sanity leg; base is main (independent of the bulk-progress stack).

#871_lastUpdated 500s on Postgres

build_last_updated_condition bound the raw query string as a text parameter against the TIMESTAMPTZ last_updated column — tokio-postgres refuses to serialize it ("error serializing parameter 2"), so every precision and prefix returned an opaque 500. The path also predated the precision-range semantics dates got in #463.

Now it binds real timestamps and mirrors build_date_condition exactly: eq at day precision means [day, day+1), ne its complement, gt/sa exclude the named period, lt/le/ge use the correct boundary, and a full-precision instant falls back to scalar comparison. Unit tests pin the SQL shape, the bound param types, and the period-end arithmetic.

#873 — response bodies carried no server-populated meta

versionId/lastUpdated lived only in the ETag/Last-Modified headers: search entries, reads, vreads, history bundles, and batch/transaction responses all returned resources with no server metadata — R4 requires it at least on vread, Bundles have no per-resource headers, and the Resources workspace's UPDATED column rendered blank on every backend.

Fix is read-side and central: StoredResource::content_with_meta() merges the row's version and timestamp into the returned body — client-supplied meta members (profile, tag, security) survive, versionId/lastUpdated are overwritten because the row is authoritative — and every echo point uses it: read, vread, history entries, search to_bundle (match + include), create/update/patch responses, batch/transaction BundleEntryResult, and subscription event payloads. Read-side means pre-existing rows are fixed immediately, no migration.

An integration test walks the whole lifecycle: create (v1 meta in the response, profile preserved) → update → read (v2) → vread (v1 keeps its version) → history (per-version meta) → search (entries carry meta).

Verified

…nse bodies

Two findings from the #448 Postgres leg.

_lastUpdated searches returned an opaque 500 on the Postgres backend at
every precision and prefix: build_last_updated_condition bound the raw
query string as a text parameter against the TIMESTAMPTZ last_updated
column, which tokio-postgres refuses to serialize, and the path had
none of the precision-range semantics the date parameters got in #463.
It now binds real timestamps and mirrors build_date_condition: eq at
day precision means [day, day+1), gt excludes the named period, and a
full-precision instant falls back to scalar comparison.

Response bodies carried no server-populated meta on any read path —
versionId and lastUpdated lived only in the ETag and Last-Modified
headers, so search entries, reads, vreads, history bundles, and
batch/transaction responses returned resources with no server
metadata, and the Resources workspace's UPDATED column rendered blank
on every backend. StoredResource::content_with_meta now merges the
row's version and timestamp into the returned body (client-supplied
meta members like profile survive), and every handler and bundle
builder that echoed stored content uses it.

Verified live against postgres:16 with the #448 dataset: the full
battery passes 22/22, and a Patient read returns
meta.profile + versionId + lastUpdated together.

Closes #871
Closes #873
Resolve conflict in core/search.rs: main split SearchResult::to_bundle into
a shared bundle_shell plus a copying to_bundle and a consuming into_bundle.
Apply the #873 server-populated meta merge to both paths, adding a
consuming StoredResource::into_content_with_meta so into_bundle keeps its
clone-free behaviour.

Claude-Session: https://claude.ai/code/session_01334x1QZSmUsznAtEVcnVy7
test_prefix_operators asserted the scalar operators the pre-#871 code
emitted — the exact semantics the fix replaces. Day-precision prefixes
now assert their [day, day+1) range shapes and bound counts, and a
full-precision instant pins the scalar fallback.
@angela-helios

Copy link
Copy Markdown
Contributor Author

CI fix pushed: query_builder_tests::test_prefix_operators asserted the scalar operators the pre-#871 code emitted for day-precision _lastUpdated — the exact broken semantics this PR replaces (my local run had filtered on postgres_ and missed that module). The test now pins the range shapes (eq[day, day+1), ne → its complement, gt → period end, etc.), the bound counts, and the scalar fallback for full-precision instants. Full postgres_tests query-builder module: 26/26 locally.

angela-helios and others added 3 commits September 2, 2026 09:09
… manifest

`postgres_bulk_submit_import_directives_round_trip` called
`claim_next_manifest` and asserted the claimed view carried the directives
it had just set. The claim queue is cross-tenant and ordered by `added_at`,
the test binary shares one container database, and the batch test that
#880 added leaves its manifest as `processing` with no lease — which the
claim query treats as an orphan to reclaim. Whenever that manifest was
added first, the directives test claimed it instead of its own and failed
with `left: []`; main's coverage job has been red on most runs since.

The test now claims through `claim_specific_manifest`, the submit-side twin
of the export tests' `claim_specific`: loop until the target manifest comes
back, hold any foreign lease picked up along the way so it cannot be
re-claimed, then release those back to the queue. The batch test's comment
no longer presents its `process_entries` call as a defence against
concurrent claims. No product code changes.

Tests: the two submit tests pass five consecutive runs together; the full
postgres_tests binary passes (156).
@angela-helios

Copy link
Copy Markdown
Contributor Author

CI here was failing on postgres_bulk_submit_import_directives_round_trip — the known flake #911 fixes (another test's worker claims the manifest before this test reads it back; it has been failing main too since #880). Nothing in this branch touches that path, and a rerun failed identically, so I merged main and cherry-picked #911's commit (d4a376d) onto this branch to unblock it. Both bulk-submit tests pass locally against a fresh testcontainer. When #911 merges, the identical change resolves cleanly.

@codecov

codecov Bot commented Sep 3, 2026

Copy link
Copy Markdown

@smunini
smunini merged commit 04c9c83 into main Sep 3, 2026
19 checks passed
@smunini
smunini deleted the fix/871-873-read-correctness branch September 3, 2026 02:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

3 participants