Resolve Gregorian weekday/month names with CLDR inheritance - #582
Draft
nightsurgex2 wants to merge 1 commit into
Draft
Resolve Gregorian weekday/month names with CLDR inheritance#582nightsurgex2 wants to merge 1 commit into
nightsurgex2 wants to merge 1 commit into
Conversation
CLDR resolves locale data item by item: a locale's effective data is the union of its own data with each of its ancestors', the more specific locale winning per item (UTS Shopify#35 4.1). I18n's fallback backend returns the first non-nil node it finds in the chain instead, which is right for leaves and wrong for whole nodes. en-CA overrides exactly one abbreviated month name (Sept), so month_names(locale: "en-CA", width: :abbreviated) returned ["Sept"] rather than all twelve. 104 locales were affected across the two methods and three widths, including every en-001 descendant (en-GB, en-IN, en-AU, en-NZ, en-ZA, ...), ar-DZ/ar-MA/ar-TN, hi-Latn, sr-Latn-ME, sr-Latn-XK and se-FI. Worldwide::Cldr.resolved_hash merges the whole fallback chain and resolves CLDR <alias> nodes against the requested locale, matching what CLDR specifies. Gregorian now builds its names from that and raises MissingCalendarDataError if an entry is still missing, rather than letting the I18n exception handler degrade the key into a humanized copy of its last segment (a missing weekday lookup used to come back as the string "wide", and month_names then died on String#values). Regression coverage asserts complete data for all 805 known locales at all three widths; 103 of those cases fail without this change. Assisted-By: devx/2f3a0f30-70b8-4af7-ae28-67d5a94715cb
nightsurgex2
force-pushed
the
cldr-resolved-hash-inheritance
branch
from
August 26, 2026 02:48
62cdcb6 to
b9483c0
Compare
Author
|
Companion PR for the byte-identical copy in |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What are you trying to accomplish?
Fixes #581.
Worldwide::Calendar::Gregorian.month_namesand.weekday_namesreturn partial results for 104 locales, includingen-CA,en-GB,en-IN,en-AU,en-NZ,en-ZAand every otheren-001descendant:CLDR resolves locale data item by item: a locale's effective data is the union of its own data with each of its ancestors', the more specific locale winning per item (UTS #35 §4.1). The YAML in
data/cldr/locales/is stored as deltas to match —en-CA/calendars.ymlcontains onlymonths.stand_alone.abbreviated: {9: Sept}.I18n::Backend::Fallbacks#translatereturns the first non-nil node in the chain. Correct for leaves (...abbreviated.1still resolves to"Jan"fromen), wrong for a whole node: the lookup stops aten-CA's one-entry hash and the eleven inherited entries are never merged in.month_namesthen calls.valueson it, so the truncation is silent.Full affected-locale table is in the issue. 108 of 4830 method × width × locale combinations were wrong.
What approach did you choose and why?
Chosen: resolve structural keys with CLDR's inheritance rules at read time.
Worldwide::Cldr.resolved_hashwalks the fallback chain from the least specific ancestor to the most specific, deep-merging each locale's own contribution, and resolves CLDR<alias>nodes against the requested locale rather than the ancestor they were found in. That is what UTS #35 specifies, it is a handful of lines, and it leavest— and therefore every leaf lookup in the gem — untouched.Rejected alternatives:
rake/cldr/locale_generator.rb), writing fully-resolved calendar data into each locale file. Correct output, but it discards the delta structure the CLDR import produces and multipliesdata/cldr/locales/for 805 locales. It also has to be redone for every component that grows this problem, and diverges from whatcldr:data:importemits.I18n's first-match behaviour is the right one for leaves.stand_alone→formatinGregorian. This was my first read of the bug, and it is wrong: thestand_alone→formatalias already works (it lives inroot/calendars.ymlandI18nresolves symbol links againstfallback_original_locale). Hard-coding it would have masked the real problem while leavingse-FI,ar-MAand theen-001tree broken.weekday_names/month_namesnow also raiseMissingCalendarDataErrorwhen an entry is still missing after resolution. PreviouslyWorldwide::I18nExceptionHandler#degraded_translationturned an unresolved structural key into a humanized copy of its last segment, so a missing weekday table came back as the string"wide"andmonth_namesthen died onString#valuessome distance from the cause. Degrading a leaf into readable text is reasonable; degrading a hash-valued CLDR node into a plausible-looking string is not.What should reviewers focus on?
Worldwide::Cldr#unresolved_nodereads a single locale's own contribution withdefault: nil, fallback: false, resolve: false.default: nilis what keeps a miss returningnilinstead of going through the exception handler, andresolve: falseis what returns a CLDR alias as a rawSymbolso it can be re-resolved against the requested locale.seenset guards against two aliases pointing at each other; I did not find such a cycle in the current data, it is there so a future CLDR import cannot hang.resolved_hashis public because it is generally useful for structural CLDR keys, but nothing else in the gem needs it today —Cldr::DateFormatPatternonly reads leaves. Happy to make it private if you'd rather not add API surface.sun..sat,1..12) rather than the hash's insertion order.The impact of these changes
Bug fix, no API change for callers that were already getting complete data. Two behaviour changes worth calling out:
Worldwide::Calendar::Gregorian::MissingCalendarDataErrorinstead of returning a string or raisingNoMethodError. With this changerootalways supplies a complete set, so this should be unreachable with the shipped data.Testing
bundle exec rake test→ 2256 tests, 0 failures.bundle exec rubocop→ clean.New coverage:
test/worldwide/calendar/gregorian_test.rbasserts complete, non-empty weekday and month names for all 805 known locales at all three widths, plus explicit cases for the alias path (en,nb, which holds no calendar data of its own and inherits fromno) and the partial-override path (en-CA,en-GB,ar-MA,se-FI).test/worldwide/cldr_test.rbcoversresolved_hashdirectly, including that it returns{}for a missing key wheretreturns the degraded"wide".103 of the new per-locale assertions fail on
mainwithout thelib/change.Checklist