Skip to content

Decision: how do translations reach published entries, given immutability? #102

Description

@stiebitzhofer

Raised by #76 (24 published entries carry no German definition) but the question outlives that issue: the dictionary currently has no mechanism by which an existing entry can gain a language. Every option available today either mints identifiers for a change that carries no new meaning, or breaks a guarantee we sell. This issue lays out the requirements, what the implementation actually does, the options, and a recommendation.

1. Why translations matter here

ESPR Art. 7(8): "The information to be supplied pursuant to information requirements shall be provided in a language which can be easily understood by customers, as determined by the Member State on whose market the product is to be made available or in which it is to be put into service."

The dictionary is not itself the passport, so that obligation does not bind us directly. But a passport renders its element labels and definitions from the dictionary — the consuming repo resolves them exclusively from the vendored entries and never authors its own. So a German passport whose element definitions are English is a rendering our data caused. That is exactly what #76 reports.

The envelope was designed for this: preferredName, definition and symbol are all langMaps. All 24 entries already carry a German preferredName. Only definition is missing — the mechanism gap, not the data model, is what blocks us.

2. What our own requirements say

Statement Bearing on this question
R4 resolves for decades a reference minted today must still resolve in 20 years — argues for stability of identifiers
R5 reference = one version, one file per version a pinned reference must keep meaning exactly one thing
R6 published never changes; published/ append-only, enforced by CI + branch protection; served Cache-Control: public, max-age=31536000, immutable the direct obstacle
Definition of done "no process exists by which the JSON response can ever change" the promise in the README and plan §8
Versioning test a new identifier is issued only on material semantic change — definition, data type, unit a translation is not one

The pincer is the whole problem: the versioning test says do not mint; R6 says do not edit. Both are right, and together they say the translation cannot live where it currently would have to live.

3. What the implementation actually does today

  • Check 1 fails any M, D or T under published/. An in-place edit cannot merge without disabling the check that carries R6.
  • The Worker serves /def/<uuid> with max-age=31536000, immutable (verified live). A consumer that has fetched an entry keeps those bytes for a year and will not revalidate.
  • emit.ts guarantees byte-stable canonical JSON; future hash manifests (trust-ladder step 2) are specified to depend on it.
  • The consuming repo vendors the closure byte-exact and digests it. verifyDppDictionary treats an unreachable dictionary as a warning and a changed entry as a failure — deliberately, because conflating the two "would train everyone to ignore the one alarm that means something".
  • REVIEW.md §2 already requires German to be technically correct where present, and the request form takes definition (de) — so new entries are expected to be born bilingual. This issue is only about the backlog and about later languages.
  • Renderers (/def/<uuid> HTML, /tree, /dictionary.ttl) already emit every language a langMap contains; nothing downstream needs teaching.

4. Options

A — Supersede: a new entry with replaces per translated entry

What #76 asked for. Mechanically supported today; no code changes.

  • Cascade: superseding a member while its parent still points at the old id leaves the current collection referencing superseded children. So it is not 24 entries but the transitive closure, minted children-first: substancesOfConcern → substanceEntry → {substanceName, casNumber, concentrationRange, location}, mechanicalProperties → {tensileStrength, yieldStrength, elongationAtFracture}, steelmakingRoute → {BF-BOF, EAF, OHF}.
  • Consumer: re-pins the whole closure in its next content-spec version.
  • Precedent: every further language re-mints the closure again. At 24 EU languages that is arithmetic nobody wants, and each round leaves the previous generation permanently resolvable — the dictionary grows by a factor for zero new meaning.
  • Contradicts the versioning test we apply everywhere else, including in the argument that access categories and value restrictions must not force new versions.

B — A derived translations surface (recommended)

translations/<uuid>.yaml tracked in the repo; the build merges it into every rendered surface and emits one derived /translations.json.

  • Identifiers minted: 0. Cost per additional language: 0 identifiers.
  • /def/<uuid>.json stays byte-identical forever; R6, the cache header, the digests and the consumer's tamper alarm are all untouched.
  • Same pattern the project already uses for everything derived and time-varying: the index, /feed.xml, /superseded.json, /tree, /dictionary.ttl.
  • Cost: the canonical JSON is no longer the whole truth about an entry — a consumer wanting German fetches a second, short-cache file. That weakens "one immutable GET gives you everything", which is a real selling point. The consuming repo vendors one more file (and it is mutable, so its digest discipline needs a different rule for it).
  • Open sub-question: one aggregate /translations.json, or /def/<uuid>.translations.json per entry? The aggregate is one fetch for a whole closure; the per-entry form matches how everything else is addressed.

C — Edit the published file in place

The intuitive reading of "a translation is a non-material change". Rejected, and worth recording why so it is not re-proposed:

  1. It does not reach production. max-age=31536000, immutable means existing consumers keep the old bytes for up to a year and never revalidate. The edit would be real in git and invisible in the wild, with no way to tell who sees which.
  2. It fires the consumer's tamper alarm. A changed entry is a hard failure in verifyDppDictionary, and an added translation is byte-indistinguishable from a compromise.
  3. CI refuses it (check 1).
  4. It contradicts the definition of done verbatim.

D — Content negotiation: merge translations at the Worker on Accept-Language

Rejected. /def/<uuid> would return different bytes per request, requiring Vary: Accept-Language, destroying byte-stability and making immutable caching a lie in a second way. It also puts entry knowledge into the Worker, which plan §2.2 forbids — the mapping must stay reproducible on any reverse proxy.

E — Redefine R6 as additive immutability

The principled version of C: the rule becomes "no fact about an entry ever changes; language keys may be added", enforced by a check that the diff is purely additive inside langMaps — which is mechanically checkable, so this is not hand-waving.

Honest case in favour: an added translation is monotonic. A consumer with stale bytes is not wrong, only less complete, so the staleness in (C1) arguably harms nothing.

What it would cost:

  • The cache policy would have to change — immutable for a year cannot coexist with "bytes may gain content". Weakening it costs the strongest property we advertise, and R4's "resolves for decades" is what that property serves.
  • Byte-exact vendoring and digests downstream would need to become semantic comparison; the consumer's deliberately dumb alarm would have to get clever, which is precisely what they argued against.
  • Future hash manifests would need to hash semantic content rather than bytes.
  • "Immutable" becomes conditional and needs a paragraph of explanation everywhere it is claimed. A guarantee that needs a footnote is a weaker guarantee.

5. Recommendation

Option B. It is the only one that satisfies both halves of the pincer instead of choosing a side: the versioning test says do not mint, R6 says do not edit — therefore the translation does not belong in the immutable representation at all. It costs zero identifiers now and zero per language later, and it reuses a pattern already load-bearing five times over.

Explicitly accepted trade: the canonical JSON stops being the complete picture of an entry, and consumers who want non-English text make a second request. If that trade is judged wrong — if "one GET, everything, forever" is worth more than identifier stability — then E, not A, is the honest alternative, and it should be taken deliberately with its cache and consumer consequences priced in. A is the option that looks cheapest today and is the most expensive per language.

Acceptance

  • Decision recorded here as an outcome comment
  • If B: translations/ source layout and the aggregate-vs-per-entry question settled; build merges into HTML, /dictionary.ttl and the derived JSON; REVIEW.md gains a line on reviewing translations; CLAUDE.md records that a translation never mints an identifier
  • If E: R6 restated, check 1 extended to allow additive-only langMap diffs, cache policy revised, consumer notified before anything ships
  • Either way: the 24 drafted German definitions get a native technical review (drafted, posted on German definitions missing on every entry in the steel DPP closure (schema-repo O10) #76)
  • The consuming repo is told which mechanism won, since it re-pins or re-vendors accordingly

Out-of-scope test (plan §7)

The decision itself changes nothing. Option B adds derived outputs and changes no identifier and no published byte. Option E would change published/ semantics and belongs to a separate, explicit amendment of R6 rather than to a translation ticket.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    documentationImprovements or additions to documentationsize:MMedium — about a daytype:featureNew capability

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions