Spike #1759 - How to organize the Seal of Reliability endpoint(s)

[davidgamez]

1. Summary of the proposal

Decision	Recommendation
How many endpoints?	One dedicated read endpoint, not per-criterion endpoints.
Endpoint	GET /v1/gtfs_feeds/{id}/reliability → full breakdown (summary + 6 criteria).
Badge support	Embed a small reliability_seal summary object in the existing feed-detail (GtfsFeed) and search (SearchFeedItemResult) responses, so the "verified feed" badge needs no extra call.
Feed types	GTFS only.
Computation	None at request time - the API reads the stored nightly-computed values as-is.
Gating	Viewing a feed's seal (embedded summary + /reliability detail) is public. Filtering/searching feeds by seal status is gated (requires special permissions).

The seal data model (#1760) already dictates a single-aggregate shape: one feed_reliability_seal row + six seal_criterion rows per feed. The endpoint simply surfaces those two tables. Splitting into multiple endpoints would only add round-trips for a UI that renders all six criteria together.

---

2. Findings

2.1 This spike is the API layer of an already-designed epic

The backend data model and evaluation logic are defined in sibling tickets, and #1761 is explicit that the API is read-only and stored-as-is ("has_seal and grace_failing are stored directly and read as-is - no on-the-fly recomputation needed").

Tables (#1760):

• feed_reliability_seal - one row per feed: has_seal, seal_earned_at, seal_lost_at.
• seal_criterion - six rows per feed: raw_failing, grace_failing, evaluated_at, first_raw_failure_at, last_raw_failure_at, last_grace_failure_at.

Criteria (#1783 / #1784 / #1782): official, stable, available, compliant, fresh_coverage, fresh_continuous.

has_seal is true iff all six criteria have grace_failing = false. The grace period is absorbed inside the nightly job: a criterion failing but still within its grace window keeps grace_failing = false, so it still counts toward the seal.

2.2 Backing data already exists for every criterion

Criterion	DB source	Grace	Notes
official	feed.official (+ officialstatushistory audit)	none	point-in-time, no 6-month window
stable	gtfsdataset.created_at (latest) + feed.is_producer_url_unstable	none	see schema-drift risk in §5
available	gtfs_feed_availability_check.success	14 d	GTFS only
compliant	validationreport.total_error on latest dataset	30 d	total_error == 0
fresh_coverage	gtfsdataset.service_date_range_end (latest)	7 d	skipped for seasonal feeds
fresh_continuous	gtfsdataset.service_date_range_* + new feed_info_date_range_* (#1775)	7 d	algorithm in #1782

No existing endpoint or schema aggregates these into a verdict - that is the net-new surface.

2.3 Existing public API surface (docs/DatabaseCatalogAPI.yaml)

• GET /v1/gtfs_feeds/{id} (GtfsFeed) already exposes official, official_updated_at, status, and latest_dataset (validation counts + service dates). Nothing aggregates a seal.
• Sub-resource precedent already exists: GET /v1/gtfs_feeds/{id}/availability and GET /v1/gtfs_feeds/{id}/datasets. A /reliability sub-resource fits this pattern exactly.
• Auth: a single global bearer-JWT scheme (Authentication). The spec has no tiered/scoped/gated concept. "Restricted" access is enforced in the impl layer via is_user_email_restricted() (api/src/middleware/request_context.py), not in the spec. Privileged/admin operations live in a separate docs/OperationsAPI.yaml (ApiKeyAuth).

---

3. Proposed endpoint & schema

3.1 New endpoint

GET /v1/gtfs_feeds/{id}/reliability   ->   FeedReliabilityReport

Response codes: 200 (evaluated, or not-yet-evaluated with empty / not_evaluated criteria), 404 unknown feed. Public - no special permission required.

3.2 Embedded summary (drives the public badge)

Added to GtfsFeed and SearchFeedItemResult:

FeedReliabilitySummary:
  type: object
  nullable: true            # null when the feed has never been evaluated
  properties:
    has_seal:     { type: boolean }
    earned_at:    { type: string, format: date-time, nullable: true }
    lost_at:      { type: string, format: date-time, nullable: true }
    evaluated_at: { type: string, format: date-time, nullable: true }

3.3 Detail schema

FeedReliabilityReport:
  type: object
  properties:
    feed_id:      { type: string }
    has_seal:     { type: boolean }
    earned_at:    { type: string, format: date-time, nullable: true }
    lost_at:      { type: string, format: date-time, nullable: true }
    evaluated_at: { type: string, format: date-time, nullable: true }
    criteria:
      type: array
      items: { $ref: "#/components/schemas/ReliabilityCriterion" }

ReliabilityCriterion:
  type: object
  properties:
    criterion:
      type: string
      enum: [official, stable, available, compliant, fresh_coverage, fresh_continuous]
    status:
      type: string
      enum: [pass, fail, not_applicable, not_evaluated]
      # Seal-contributing verdict only (mirrors NOT grace_failing).
      # A feed inside its grace period is STILL passing, so grace is not a status value.
      #   pass           = grace_failing false (includes "failing raw but still inside grace window")
      #   fail           = grace_failing true (contributes to seal loss)
      #   not_applicable = criterion not run for this feed (e.g. seasonal feed skips fresh_coverage)
      #   not_evaluated  = raw_failing IS NULL (nightly job has not evaluated it yet)
    in_grace_period:
      type: boolean
      # raw_failing true but grace_failing false -> passing-but-at-risk; drives the warning banner.
      # Can only be true when status = pass.
    grace_period_ends_at:
      type: string
      format: date-time
      nullable: true        # first_raw_failure_at + grace window; only meaningful while in_grace_period = true
    evaluated_at:  { type: string, format: date-time, nullable: true }
    first_failure_at: { type: string, format: date-time, nullable: true }  # <- first_raw_failure_at
    last_failure_at:  { type: string, format: date-time, nullable: true }  # <- last_raw_failure_at

Why status and in_grace_period are orthogonal: grace is not a failing state. While a feed is within its grace window grace_failing = false, so the criterion still passes and the feed still holds the seal. The web renders three states from these two fields: healthy (pass, not in grace), at-risk (pass + in_grace_period → warning), and lost (fail); plus not_applicable / not_evaluated.

in_grace_period and grace_period_ends_at are derived server-side from first_raw_failure_at + the per-criterion grace window (14/30/7 days), so the web gets #141's grace-period warning without knowing the rules.

3.4 Example payloads

Feed-detail addition:

"reliability_seal": {
  "has_seal": true,
  "earned_at": "2026-01-15T00:00:00Z",
  "lost_at": null,
  "evaluated_at": "2026-07-30T04:00:00Z"
}

GET /v1/gtfs_feeds/mdb-1210/reliability:

{
  "feed_id": "mdb-1210",
  "has_seal": false,
  "earned_at": "2026-01-15T00:00:00Z",
  "lost_at": "2026-07-20T04:00:00Z",
  "evaluated_at": "2026-07-30T04:00:00Z",
  "criteria": [
    {"criterion": "official",         "status": "pass",           "in_grace_period": false, "grace_period_ends_at": null,                   "first_failure_at": null,                  "last_failure_at": null},
    {"criterion": "stable",           "status": "pass",           "in_grace_period": false, "grace_period_ends_at": null,                   "first_failure_at": null,                  "last_failure_at": null},
    {"criterion": "available",        "status": "fail",           "in_grace_period": false, "grace_period_ends_at": null,                   "first_failure_at": "2026-06-01T04:00:00Z", "last_failure_at": "2026-07-30T04:00:00Z"},
    {"criterion": "compliant",        "status": "pass",           "in_grace_period": true,  "grace_period_ends_at": "2026-08-24T04:00:00Z", "first_failure_at": "2026-07-25T04:00:00Z", "last_failure_at": "2026-07-30T04:00:00Z"},
    {"criterion": "fresh_coverage",   "status": "not_applicable", "in_grace_period": false, "grace_period_ends_at": null,                   "first_failure_at": null,                  "last_failure_at": null},
    {"criterion": "fresh_continuous", "status": "pass",           "in_grace_period": false, "grace_period_ends_at": null,                   "first_failure_at": null,                  "last_failure_at": null}
  ]
}

Here has_seal is false only because available = fail; compliant is inside its 30-day grace window so it still passes.

3.5 Gated search filter

This is where #1759's "requires special permissions" applies. A new query parameter lets a caller filter/search the catalog by seal status:

has_seal_query_param:
  name: has_seal
  in: query
  required: false
  schema: { type: boolean, default: null }
  description: >
    If true, only return feeds that currently hold the Seal of Reliability;
    if false, only feeds without it. This feature is available to selected users. To request access or learn more, contact us at <<email>>.

---

4. UI mockup → API data-point mapping

Feed-detail "Data Quality" tab (MobilityData/mobilitydatabase-web#141) mapped to the API. "new" = the proposed GET /v1/gtfs_feeds/{id}/reliability endpoint / embedded reliability_seal object (spike #1759); everything else already exists.

Mockup region	Data	Endpoint to use
Header / identity	type, id, provider, country, timezone, error/warning/info chips, Download + Full-Report links	GET /v1/gtfs_feeds/{id} (existing); Full-Report URL via GET /v1/datasets/gtfs/{id}
Seal banner	overall state, "N of 5 pass", grace countdown, criterion chips	GET /v1/gtfs_feeds/{id}/reliability (new)
Official / Fresh cards	pass verdicts + grace	/reliability (new); dates ("extends to Jul 28 2027") from GET /v1/gtfs_feeds/{id} (existing)
Available card	pass verdict	/reliability (new)
→ 90-day uptime chart	daily success/fail bars	GET /v1/gtfs_feeds/{id}/availability (existing)
Continuous coverage card	verdict + "max gap"	/reliability (new)
→ dataset timeline Gantt	per-dataset service ranges	GET /v1/gtfs_feeds/{id}/datasets (existing)
Compliant card	verdict + "N days left"	/reliability (new)
→ notice list (code/file/severity/count)	per-notice detail	validator url_json (existing, public)
Badge on Overview tab	has_seal	reliability_seal embedded in GET /v1/gtfs_feeds/{id} (new field)

Summary - the whole tab needs 3 primary calls

1. GET /v1/gtfs_feeds/{id} - identity, Fresh dates, validation counts, and the embedded reliability_seal badge.
2. GET /v1/gtfs_feeds/{id}/reliability (new) - every pass/fail verdict and grace-period countdown.
3. The two existing sub-resources for the two charts: /availability (uptime bars) and /datasets (coverage Gantt); notice list read straight from the validator url_json.