Skip to content

fix(persistence): ES _lastUpdated honours ne/sa/eb/ap and OR-lists - #893

Merged
smunini merged 1 commit into
mainfrom
fix/892-es-last-updated
Sep 2, 2026
Merged

fix(persistence): ES _lastUpdated honours ne/sa/eb/ap and OR-lists#893
smunini merged 1 commit into
mainfrom
fix/892-es-last-updated

Conversation

@smunini

@smunini smunini commented Sep 2, 2026

Copy link
Copy Markdown
Contributor

Closes #892

Follow-up from #876. Elasticsearch was the one backend whose _lastUpdated clause did not go through a precision-aware date builder.

What was wrong

build_last_updated_clause wrote every SearchValue into a single range map keyed only by gte/gt/lt/lte:

  • ne, sa, eb, ap fell into the default arm and behaved as eq. _lastUpdated=ne2026-09-01 returned exactly the resources updated that day.
  • Comma-separated OR values overwrote each other, so _lastUpdated=2026-09-01,2026-09-03 silently became a single-value match.

Fix

The indexed date handler's precision logic is now exposed as date::field_range(field, value, prefix) -> DateRange, returning either a plain range (Within) or one the caller must negate (Outside, for ne). Both call sites build on it:

  • Indexed date params wrap it in the existing nested query; ne becomes must_not inside the nested bool, as before.
  • _lastUpdated applies it to the top-level last_updated field, one clause per value, OR-combined with bool.should / minimum_should_match: 1.

Semantics now match SQLite, MongoDB, and Postgres (#876): eq at day precision is [day, day+1), ne its complement, gt/sa start after the whole period, lt/eb end before it, le reaches the end of the period.

One incidental fix: a full-precision instant (2024-01-15T10:00:00Z) used to produce gte X, lt X, an empty range, for eq on indexed date parameters as well. It now compares as a scalar (gte X, lte X), mirroring the degenerate-range fallback in the Postgres builder.

Tests

  • date.rs units: ne is the negated precision range; sa/eb mirror gt/lt on the whole period; full-precision instants are scalar, not empty.
  • query_builder.rs units pin the emitted JSON for _lastUpdated eq, ne, sa/eb, and a two-value OR list.
  • ES integration: new es_integration_search_last_updated_ne_excludes_today creates a resource, asserts eq on today matches it and ne on today excludes it.

Verified

  • cargo test -p helios-persistence --lib --features elasticsearch: 888 passed.
  • ES query-builder / parameter-handler tests in elasticsearch_tests.rs: 32 passed.
  • Clippy clean on the touched files.
  • The ES integration tests could not run locally: the ES 7.16 image fails to boot on this arm64 Docker host with a JDK cgroup NPE before any test code runs, and the pre-existing es_integration_search_last_updated fails identically. CI's runner covers that leg.

https://claude.ai/code/session_01334x1QZSmUsznAtEVcnVy7

The Elasticsearch _lastUpdated clause folded every value into one range
map keyed only by gte/gt/lt/lte: ne, sa, eb and ap all degraded to eq,
and a second comma-separated value overwrote the first.

Expose the indexed date handler's precision logic as date::field_range
and build _lastUpdated on it against the top-level last_updated field:
eq at day precision is [day, day+1), ne its complement via must_not,
sa/eb mirror gt/lt on the whole period, and OR lists become
bool.should with minimum_should_match 1. A full-precision instant now
compares as a scalar instead of an empty half-open range, which also
fixes eq on indexed date parameters at full precision.

Closes #892

Claude-Session: https://claude.ai/code/session_01334x1QZSmUsznAtEVcnVy7
@claude

claude Bot commented Sep 2, 2026

Copy link
Copy Markdown

Code review

No issues found. Checked for bugs and CLAUDE.md compliance.

@codecov

codecov Bot commented Sep 2, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

@smunini
smunini merged commit d65d7e1 into main Sep 2, 2026
19 checks passed
@smunini
smunini deleted the fix/892-es-last-updated branch September 2, 2026 17:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

fix(persistence): ES _lastUpdated ignores ne/sa/eb/ap prefixes and OR-lists

1 participant