Skip to content

Resolve physicalName in servicelevel freshness and retention checks - #1489

Open
erikgrip2 wants to merge 1 commit into
datacontract:mainfrom
erikgrip2:fix/servicelevel-physical-name
Open

Resolve physicalName in servicelevel freshness and retention checks#1489
erikgrip2 wants to merge 1 commit into
datacontract:mainfrom
erikgrip2:fix/servicelevel-physical-name

Conversation

@erikgrip2

Copy link
Copy Markdown

Fixes #1488.

datacontract test's freshness and retention checks read the relation by the schema object's logical name; every other check resolves physicalName via to_schema_name(). A contract whose object name differs from the relation name — e.g. a versioned dbt model, aliased to <model>_v<n> — failed while all schema checks in the same run passed against the correct relation:

│ failed │ Freshness of events.ts < 1h │ ts │ Could not read model 'events': events │

datacontract export sodacl emitted its servicelevel blocks under the logical name the same way.

Changes

  • Engine: a _resolve_sla_element helper folds the element split, the model existence gate, and name resolution — the object via to_schema_name() (keeping the kafka carve-out, which reads the Spark SQL view by logical name), the property via its physicalName. Both _freshness_check and _retention_check use it.
  • Sodacl export: a _resolve_physical_names helper of the same shape, used by both servicelevel builders.
  • The postgres servicelevels fixture now has an object name that differs from physicalName, so the e2e can distinguish "measured and stale" from "could not read the table"; it now also asserts the freshness diagnostics (age_seconds).
  • New unit tests for both layers: resolution, fallback without physicalName, kafka.

Behavior notes

  1. Reported names become physical for servicelevel checks (Check.model and the check display name) — consistent with what schema checks already report, but different from the sla.element spelling.
  2. Row filters (--filter/--filters) now apply to servicelevel checks when physicalName is set. Filters are normalized into a mapping keyed by physical name, and the servicelevel specs' logical-name key never matched, so they silently ran unfiltered.
  3. If the logical name happened to also name a readable relation, freshness previously measured that (wrong) table silently; it now measures the right one.
  4. File servers (local/s3/gcs/azure): the duckdb views are registered under the logical name, so a contract with a differing physicalName already fails every schema check there; servicelevel checks now fail consistently with them instead of accidentally measuring the logical-name view. (Whether those views should be created via to_schema_name() instead may deserve a separate look.)

No change for contracts where physicalName is unset or equals the name.

Testing

  • New unit tests fail before the fix and pass after.
  • The postgres e2e fails pre-fix with Could not read model '<logical name>' and passes post-fix with freshness measurement diagnostics.

  • Tests pass (uv run pytest)
  • Code formatted (uv run ruff check --fix && uv run ruff format)
  • Docs updated (if relevant)
  • CHANGELOG.md entry added

🤖 Generated with Claude Code

The engine's freshness/retention checks read the relation by the schema
object's logical name; the sodacl exporter emitted its servicelevel
blocks the same way. Both now resolve the object's and property's
physicalName like the schema checks do.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

datacontract test freshness/retention checks ignore physicalName and read the logical schema object name

1 participant