diff --git a/AGENTS.md b/AGENTS.md index 6aad23f6..3fdb6547 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -2,3 +2,4 @@ - When investigating a discrepant Microcosm calibration target, follow [the shared investigation workflow](docs/ai/workflows/investigate-microcosm-target.md) and its linked role-specific reviews. - Before starting the application for Cross-dataset work, configure exactly one DIR or URL artifact location for each country you will use, as described in [the Cross-dataset application configuration](docs/cross-dataset-api.md#configure-the-application). +- Adding a Microcosm country is one entry in `frontend/lib/microcosm/countries.ts`; pages are gated by the registration's capabilities and the release artifact's `country` block, never by country code (see [the spec-driven countries note](docs/spec-driven-countries.md)). diff --git a/CLAUDE.md b/CLAUDE.md index 74f83c4b..e8b7c29a 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -2,3 +2,4 @@ - When investigating a discrepant Microcosm calibration target, follow [the shared investigation workflow](docs/ai/workflows/investigate-microcosm-target.md) and its linked role-specific reviews. - Before starting the application for Cross-dataset work, configure exactly one DIR or URL artifact location for each country you will use, as described in [the Cross-dataset application configuration](docs/cross-dataset-api.md#configure-the-application). +- Adding a Microcosm country is one entry in `frontend/lib/microcosm/countries.ts`; pages are gated by the registration's capabilities and the release artifact's `country` block, never by country code (see [the spec-driven countries note](docs/spec-driven-countries.md)). diff --git a/docs/spec-driven-countries.md b/docs/spec-driven-countries.md index 5b5a4590..553603bc 100644 --- a/docs/spec-driven-countries.md +++ b/docs/spec-driven-countries.md @@ -20,10 +20,11 @@ from `cross_dataset.frontend_bundle.v1`. ## Code-owned metadata that remains -`COUNTRY_REPO` should be the only required registration point, but today adding a -country also touches closed country unions/parsers and selector lists, dataset and -national-geography labels, overview and target-browser copy, public/private link -behavior, and feature checks such as US-only staging and navigation. Publisher +`COUNTRY_REGISTRY` in `frontend/lib/microcosm/countries.ts` is the only required +registration point: country unions and parsers, selector lists, dataset and +national-geography labels, public/private link behavior, and page gating read +the registration or the artifact `country` block. Overview and target-browser +copy still sit in legacy per-country tables with a generic fallback. Publisher display names remain in a shared TypeScript map. Target decomposition still has a generic filter-pattern table plus legacy US name grammar (FIPS/state, filing status, return type, income band, and qualifying-child rules). @@ -36,8 +37,9 @@ should not grow another country branch. `frontend/lib/microcosm/third-country-conformance.test.ts` defines a synthetic fourth country, `zz`, with four targets, filter-coded facets, a release description, and Chronicle record IDs whose publisher prefix is unknown to the -label map. Registering its repository in `COUNTRY_REPO` must be the only country -specific code change. The same builders used by US/UK/BE must then produce: +label map. Registering its repository in `COUNTRY_REGISTRY` must be the only +country specific code change. The same builders used by US/UK/BE must then +produce: - the normal overview and targets response shapes and section order; - the artifact description in the existing provenance-note slot; @@ -59,6 +61,45 @@ be enabled without adding `zz` conditionals or tables. | Filter-pattern decomposition, region/sex/age value maps, and legacy US target-name parsing | A `dimensions` dictionary in `calibration_diagnostics` (label, semantic role, value labels, ordering) plus `targets[].dimensions` values. Geography dimensions also declare their level/id so no country geography fallback is needed. | | Source/variable guesses from flat target names | Structured `targets[].source` and `targets[].variable` identifiers, with the publisher still traceable to `metadata.chronicle_record_ids`. | +### Implemented: the `country` block + +`release_manifest.country` is read by `releaseCountry` in +`frontend/lib/microcosm/latest-artifact.ts` and served as `country` on the +overview summary and the target-diagnostics page (client type +`MicrocosmArtifactCountry`). Registration lives in +`frontend/lib/microcosm/countries.ts`; adding a country is one entry there. + +```json +{ + "country": { + "code": "be", + "label": "Belgium", + "geography_id": null, + "geography_label": "Belgium", + "repository_visibility": "private", + "capabilities": ["calibration", "targets", "compare", "cross_dataset"] + } +} +``` + +Merge rule: every field defaults to the registration; a well-typed string +field in the block overrides it (`label`, `geography_id`, `geography_label`, +and `repository_visibility` as `"public"` or `"private"`). `code`, when +present, must equal the selected country after lower-casing, otherwise the whole +block is ignored: the dashboard is selected by registry and an artifact cannot +re-route it. `capabilities` is filtered to the enumerated set and intersected +with the registration, so an artifact can narrow what a deployment serves but +never widen it. Unknown keys are ignored. The resolved `geography_label` is the +national geography for rows that carry none. + +Capabilities: `calibration`, `targets`, `compare`, `cross_dataset`, `staging`, +`model_coverage`, `pipeline`, `variables`, `external_checks`. Navigation, the +staging loaders and hooks, and the staging page gate on capability membership. + +A registration with `fixture: true` (the conformance country `zz`) is a valid +country for parsers and builders but is never listed in selectors or the +release-alert allowlist. + Schema readers must remain backward-compatible while published releases migrate. After migration, name/filter parsing is a legacy adapter selected by artifact schema version, never by country. diff --git a/frontend/app/api/hf-webhook/route.ts b/frontend/app/api/hf-webhook/route.ts index 75291d1a..a0f2443d 100644 --- a/frontend/app/api/hf-webhook/route.ts +++ b/frontend/app/api/hf-webhook/route.ts @@ -2,7 +2,8 @@ import { timingSafeEqual } from "node:crypto"; import { NextResponse } from "next/server"; -import type { MicrocosmCountry } from "@/lib/microcosm/latest-artifact"; +import { selectableCountries, type MicrocosmCountry } from "@/lib/microcosm/countries"; +import { microcosmRepo } from "@/lib/microcosm/latest-artifact"; import { postReleaseAlert } from "@/lib/slack"; export const runtime = "nodejs"; @@ -26,18 +27,19 @@ interface WebhookPayload { const TAG_PREFIX = "refs/tags/"; -// Only these repos may trigger a release alert. The webhook secret is shared -// across US and UK, so without an allowlist a valid caller could spoof an -// arbitrary repo name into either Slack channel. -const ALLOWED_REPOS: Record = { - // Deprecated upstream identifiers: Hugging Face webhook payloads still use - // the former Populace dataset repository names. - "policyengine/populace-us": "us", - "policyengine/populace-uk-private": "uk", -}; +// Only the registered country repositories may trigger a release alert. The +// webhook secret is shared across countries, so without an allowlist a valid +// caller could spoof an arbitrary repo name into any Slack channel. Fixture +// registrations are never allowlisted. Repositories are the env-resolved ones +// the dashboard actually reads (a POPULACE_*_HF_REPO override moves the +// allowlist with it); Hugging Face webhook payloads carry the repositories' +// former Populace names, as registered. +const ALLOWED_REPOS = new Map( + selectableCountries().map((country) => [microcosmRepo(country).toLowerCase(), country]), +); function countryForRepo(repoName: string): MicrocosmCountry | null { - return ALLOWED_REPOS[repoName.toLowerCase()] ?? null; + return ALLOWED_REPOS.get(repoName.toLowerCase()) ?? null; } // Constant-time secret check. HF sends the configured secret as the diff --git a/frontend/app/api/microcosm/staging/compare/route.ts b/frontend/app/api/microcosm/staging/compare/route.ts index f9fcc22f..df8ef2d9 100644 --- a/frontend/app/api/microcosm/staging/compare/route.ts +++ b/frontend/app/api/microcosm/staging/compare/route.ts @@ -1,5 +1,6 @@ import { NextResponse } from "next/server"; +import { hasCapability } from "@/lib/microcosm/countries"; import { loadPointerReleaseId, parseCountry, @@ -17,11 +18,11 @@ export async function GET(request: Request) { const country = parseCountry(url.searchParams.get("country")); const runId = url.searchParams.get("run")?.trim(); let release = url.searchParams.get("release")?.trim() || "latest"; - if (!runId && country === "us") { + if (!runId && hasCapability(country, "staging")) { return NextResponse.json({ detail: "Provide a staging run id via ?run=." }, { status: 400 }); } try { - if (release === "latest" && country === "us") { + if (release === "latest" && hasCapability(country, "staging")) { release = (await loadPointerReleaseId(300, country)).release_id; } return NextResponse.json( diff --git a/frontend/app/api/microcosm/staging/run/route.ts b/frontend/app/api/microcosm/staging/run/route.ts index a6cc4255..586e2827 100644 --- a/frontend/app/api/microcosm/staging/run/route.ts +++ b/frontend/app/api/microcosm/staging/run/route.ts @@ -1,5 +1,6 @@ import { NextResponse } from "next/server"; +import { hasCapability } from "@/lib/microcosm/countries"; import { parseCountry, scrub } from "@/lib/microcosm/latest-artifact"; import { loadStagingRun } from "@/lib/microcosm/staging-artifact"; @@ -12,7 +13,7 @@ export async function GET(request: Request) { const params = new URL(request.url).searchParams; const country = parseCountry(params.get("country")); const runId = params.get("id")?.trim(); - if (!runId && country === "us") { + if (!runId && hasCapability(country, "staging")) { return NextResponse.json({ detail: "Provide a staging run id via ?id=." }, { status: 400 }); } try { diff --git a/frontend/app/api/microcosm/staging/target-diagnostics/route.ts b/frontend/app/api/microcosm/staging/target-diagnostics/route.ts index 29ccb0af..8da6e8b2 100644 --- a/frontend/app/api/microcosm/staging/target-diagnostics/route.ts +++ b/frontend/app/api/microcosm/staging/target-diagnostics/route.ts @@ -1,5 +1,6 @@ import { NextResponse } from "next/server"; +import { hasCapability } from "@/lib/microcosm/countries"; import { parseCountry, scrub } from "@/lib/microcosm/latest-artifact"; import { loadStagingTargetDiagnostics } from "@/lib/microcosm/staging-artifact"; @@ -12,7 +13,7 @@ export async function GET(request: Request) { const params = new URL(request.url).searchParams; const country = parseCountry(params.get("country")); const runId = params.get("id")?.trim(); - if (!runId && country === "us") { + if (!runId && hasCapability(country, "staging")) { return NextResponse.json({ detail: "Provide a staging run id via ?id=." }, { status: 400 }); } try { diff --git a/frontend/components/layout/country-context.test.ts b/frontend/components/layout/country-context.test.ts index f8280ab5..10f378f4 100644 --- a/frontend/components/layout/country-context.test.ts +++ b/frontend/components/layout/country-context.test.ts @@ -1,6 +1,6 @@ import { expect, test } from "bun:test"; -import { countrySwitchUrl } from "./country-context"; +import { countrySwitchUrl, isCountry } from "./country-context"; test("country switching clears bundle-specific Cross-dataset state", () => { const switched = new URL( @@ -25,3 +25,14 @@ test("country switching preserves route state outside Cross-dataset", () => { expect(switched.pathname).toBe("/microcosm/targets"); expect(switched.searchParams.toString()).toBe("country=uk&variable=income_tax"); }); + +test("the client country parser accepts every registered country and nothing else", () => { + expect(isCountry("us")).toBe(true); + expect(isCountry("uk")).toBe(true); + expect(isCountry("be")).toBe(true); + expect(isCountry("zz")).toBe(true); + expect(isCountry("US")).toBe(false); + expect(isCountry("fr")).toBe(false); + expect(isCountry("")).toBe(false); + expect(isCountry(null)).toBe(false); +}); diff --git a/frontend/components/layout/country-context.tsx b/frontend/components/layout/country-context.tsx index 1ae58dd8..46f5f74c 100644 --- a/frontend/components/layout/country-context.tsx +++ b/frontend/components/layout/country-context.tsx @@ -8,13 +8,15 @@ import { type ReactNode, } from "react"; -export type Country = "us" | "uk" | "be"; +import { + DEFAULT_COUNTRY, + isCountry, + type MicrocosmCountry, +} from "@/lib/microcosm/countries"; -const COUNTRIES = new Set(["us", "uk", "be"]); +export type Country = MicrocosmCountry; -export function isCountry(value: string | null): value is Country { - return value != null && COUNTRIES.has(value as Country); -} +export { isCountry }; const STORAGE_KEY = "microcosm-country"; @@ -41,12 +43,12 @@ interface CountryContextValue { } const CountryContext = createContext({ - country: "us", + country: DEFAULT_COUNTRY, setCountry: () => {}, }); export function CountryProvider({ children }: { children: ReactNode }) { - const [country, setCountryState] = useState("us"); + const [country, setCountryState] = useState(DEFAULT_COUNTRY); useEffect(() => { const requested = new URLSearchParams(window.location.search).get("country"); diff --git a/frontend/components/layout/nav-items.test.ts b/frontend/components/layout/nav-items.test.ts index 35340fa5..fe001f4c 100644 --- a/frontend/components/layout/nav-items.test.ts +++ b/frontend/components/layout/nav-items.test.ts @@ -1,5 +1,7 @@ import { expect, test } from "bun:test"; +import { hasCapability, selectableCountries } from "@/lib/microcosm/countries"; + import { isActive, navGroupsForCountry, @@ -78,11 +80,28 @@ test("preserves the Cross-dataset navigation label and route", () => { expect(item).toEqual({ href: "/microcosm/datasets", label: "Cross-dataset", + capability: "cross_dataset", }); expect(isActive("/microcosm/datasets", item!)).toBe(true); }); -test("Belgium navigation keeps country-ready pages and hides US-only tools", () => { +test("every navigation item is gated by a capability, never by a country code", () => { + const items = NAV_GROUPS.flatMap((group) => group.items); + expect(items.every((item) => item.capability != null)).toBe(true); + expect(items.find((item) => item.href === "/microcosm/staging")?.capability).toBe("staging"); + expect(items.find((item) => item.href === "/microcosm/model-coverage")?.capability).toBe( + "model_coverage", + ); + expect(items.find((item) => item.href === "/microcosm/pipeline")?.capability).toBe("pipeline"); + expect(items.find((item) => item.href === "/microcosm/variables")?.capability).toBe( + "variables", + ); + expect(items.find((item) => item.label === "External checks")?.capability).toBe( + "external_checks", + ); +}); + +test("Belgium navigation keeps country-ready pages and hides pages it lacks capabilities for", () => { const items = navGroupsForCountry("be").flatMap((group) => group.items); expect(items.map((item) => item.href)).toEqual([ "/microcosm", @@ -90,7 +109,7 @@ test("Belgium navigation keeps country-ready pages and hides US-only tools", () "/microcosm/datasets", "/microcosm/compare", ]); - expect(items.every((item) => item.usOnly !== true)).toBe(true); + expect(items.every((item) => hasCapability("be", item.capability!))).toBe(true); expect(items.map((item) => navItemHref(item, "be"))).toEqual([ "/microcosm?country=be", "/microcosm/targets?country=be", @@ -99,8 +118,23 @@ test("Belgium navigation keeps country-ready pages and hides US-only tools", () ]); }); +test("US navigation lists every page", () => { + expect(navGroupsForCountry("us").flatMap((group) => group.items)).toEqual( + NAV_GROUPS.flatMap((group) => group.items), + ); +}); + +test("artifact-narrowed capabilities hide pages the release does not serve", () => { + const groups = navGroupsForCountry("us", ["calibration", "targets"]); + expect(groups.map((group) => group.label)).toEqual(["Dataset accuracy"]); + expect(groups[0].items.map((item) => item.href)).toEqual([ + "/microcosm", + "/microcosm/targets", + ]); +}); + test("shows Cross-dataset navigation for every selectable country", () => { - for (const country of ["us", "uk", "be"] as const) { + for (const country of selectableCountries()) { expect( navGroupsForCountry(country) .flatMap((group) => group.items) diff --git a/frontend/components/layout/nav-items.ts b/frontend/components/layout/nav-items.ts index 70655000..d3d48127 100644 --- a/frontend/components/layout/nav-items.ts +++ b/frontend/components/layout/nav-items.ts @@ -1,11 +1,17 @@ import type { Country } from "@/components/layout/country-context"; +import { + countryCapabilities, + type CountryCapability, +} from "@/lib/microcosm/countries"; -// usOnly pages run on US-specific data/runtimes (JCT scores, the PolicyEngine-US -// variable runtime) and aren't wired for UK yet. +// A page is shown when the country (or the release artifact, which can narrow +// a registration) serves its capability. Pages that run on country-specific +// data/runtimes (JCT scores, the PolicyEngine-US variable runtime, the staging +// repository) are granted only to the countries wired for them. export interface NavItem { href: string; label: string; - usOnly?: boolean; + capability?: CountryCapability; external?: boolean; // Extra path prefixes that keep this item highlighted (drill-down views). also?: string[]; @@ -17,17 +23,21 @@ export const NAV_GROUPS: { label: string; items: NavItem[] }[] = [ { label: "Dataset accuracy", items: [ - { href: "/microcosm", label: "Calibration fit" }, - { href: "/microcosm/targets", label: "Calibration targets" }, - { href: "/microcosm/model-coverage", label: "Validation reach", usOnly: true }, + { href: "/microcosm", label: "Calibration fit", capability: "calibration" }, + { href: "/microcosm/targets", label: "Calibration targets", capability: "targets" }, + { + href: "/microcosm/model-coverage", + label: "Validation reach", + capability: "model_coverage", + }, // External checks (reform scores vs JCT/fiscal notes/admin actuals) // moved to the PolicyEngine scorecard, which owns all external // comparisons; per-release history was ingested there (issue #15). - { href: "/microcosm/datasets", label: "Cross-dataset" }, + { href: "/microcosm/datasets", label: "Cross-dataset", capability: "cross_dataset" }, { href: "https://www.policyengine.org/scorecard", label: "External checks", - usOnly: true, + capability: "external_checks", external: true, }, ], @@ -35,23 +45,28 @@ export const NAV_GROUPS: { label: string; items: NavItem[] }[] = [ { label: "Releases", items: [ - { href: "/microcosm/compare", label: "Compare versions" }, - { href: "/microcosm/staging", label: "Staging candidates", usOnly: true }, + { href: "/microcosm/compare", label: "Compare versions", capability: "compare" }, + { href: "/microcosm/staging", label: "Staging candidates", capability: "staging" }, ], }, { label: "Reference", items: [ - { href: "/microcosm/pipeline", label: "Pipeline", usOnly: true }, - { href: "/microcosm/variables", label: "Variable lookup", usOnly: true }, + { href: "/microcosm/pipeline", label: "Pipeline", capability: "pipeline" }, + { href: "/microcosm/variables", label: "Variable lookup", capability: "variables" }, ], }, ]; -export function navGroupsForCountry(country: Country) { +export function navGroupsForCountry( + country: Country, + capabilities: readonly CountryCapability[] = countryCapabilities(country), +) { return NAV_GROUPS.map((group) => ({ ...group, - items: group.items.filter((item) => country === "us" || !item.usOnly), + items: group.items.filter( + (item) => item.capability == null || capabilities.includes(item.capability), + ), })).filter((group) => group.items.length > 0); } diff --git a/frontend/components/layout/nav-sidebar.tsx b/frontend/components/layout/nav-sidebar.tsx index cea0f372..be5963fd 100644 --- a/frontend/components/layout/nav-sidebar.tsx +++ b/frontend/components/layout/nav-sidebar.tsx @@ -3,26 +3,22 @@ import Link from "next/link"; import { usePathname } from "next/navigation"; -import { useCountry, type Country } from "@/components/layout/country-context"; +import { useCountry } from "@/components/layout/country-context"; import { isActive, navGroupsForCountry, navItemHref, navLinkAttributes, } from "@/components/layout/nav-items"; - -const DATASET: Record = { - // Deprecated upstream identifiers: Microcosm's HF repositories retain the - // former Populace slugs. - us: { label: "Microcosm US", repo: "policyengine/populace-us" }, - uk: { label: "Microcosm UK" }, - be: { label: "Microcosm Belgium" }, -}; +import { + countryRegistration, + selectableCountries, +} from "@/lib/microcosm/countries"; export function NavSidebar() { const pathname = usePathname(); const { country, setCountry } = useCountry(); - const dataset = DATASET[country]; + const dataset = countryRegistration(country); const groups = navGroupsForCountry(country); return (
@@ -37,7 +33,7 @@ export function NavSidebar() { aria-label="Country" className="mt-1.5 inline-flex rounded-lg bg-muted p-0.5" > - {(["us", "uk", "be"] as const).map((value) => { + {selectableCountries().map((value) => { const active = country === value; return (
diff --git a/frontend/components/microcosm/microcosm-overview-view.tsx b/frontend/components/microcosm/microcosm-overview-view.tsx index 762df186..3c11a149 100644 --- a/frontend/components/microcosm/microcosm-overview-view.tsx +++ b/frontend/components/microcosm/microcosm-overview-view.tsx @@ -9,7 +9,7 @@ import { } from "@/components/microcosm/calibration-explorer-map"; import { ArtifactDescriptionBanner } from "@/components/microcosm/artifact-description-banner"; import { WEIGHTED_TARGET_ERROR_HELP } from "@/components/microcosm/calibration-explorer-view"; -import { useCountry, type Country } from "@/components/layout/country-context"; +import { useCountry } from "@/components/layout/country-context"; import { EmptyState } from "@/components/shared/empty-state"; import { fmt, fmtCompact } from "@/components/shared/format"; import { HelpHint } from "@/components/shared/help-hint"; @@ -23,6 +23,7 @@ import { useMicrocosm, useMicrocosmReleases, } from "@/lib/api/hooks/use-microcosm"; +import type { MicrocosmCountry } from "@/lib/microcosm/countries"; import { microcosmPublicationUrl, microcosmSourceAttribution, @@ -40,9 +41,11 @@ function formatPublishedAt(value: string | null | undefined): string { }); } -const COUNTRY_OVERVIEW_COPY: Record< - Country, - { authorities: string; examples: string } +// Legacy per-country intro copy for releases published before a typed +// `release_manifest.presentation` block; countries without an entry get the +// generic sentences below. +const COUNTRY_OVERVIEW_COPY: Partial< + Record > = { us: { authorities: "the IRS, the Census Bureau, and CMS", @@ -120,7 +123,11 @@ export function MicrocosmOverviewView() { const normalizedLoss = isNormalizedLoss(lossKind); const diagnosticsStatus = cal.diagnostics_status ?? "ok"; const isNonDefault = cal.is_local_area === true || cal.is_default === false; - const sourceAttribution = microcosmSourceAttribution(country, data.source_repo); + const sourceAttribution = microcosmSourceAttribution( + country, + data.source_repo, + cal.country?.repository_visibility, + ); const publicationUrl = microcosmPublicationUrl(data.source_repo, data.release_id); const overviewCopy = COUNTRY_OVERVIEW_COPY[country]; @@ -133,12 +140,13 @@ export function MicrocosmOverviewView() { description={ <> Microcosm reweights survey microdata so it matches official statistics - from agencies like{" "} - {overviewCopy.authorities}. - Each tile in the Calibration fit explorer below is a category we calibrate to, - including{" "} - {overviewCopy.examples} - . Data is built live from{" "} + from{" "} + {overviewCopy + ? `agencies like ${overviewCopy.authorities}` + : "national statistical agencies and administrative sources"} + . Each tile in the Calibration fit explorer below is a category we + calibrate to{overviewCopy ? `, including ${overviewCopy.examples}` : ""}. + Data is built live from{" "} {sourceAttribution.href ? ( , string> = { - uk: "United Kingdom has no staging repository.", - be: "Belgium has no staging repository.", -}; - export function MicrocosmStagingView() { const { country } = useCountry(); - if (country !== "us") { + if (!hasCapability(country, "staging")) { return (
); } - return ; + return ; } -function MicrocosmUsStagingView() { +function MicrocosmStagingRunsView() { const { data: runsData, isLoading: runsLoading, error: runsError } = useMicrocosmStagingRuns(); const runs = runsData?.runs ?? []; const [selectedRun, setSelectedRun] = useState(""); diff --git a/frontend/components/microcosm/microcosm-targets-view.tsx b/frontend/components/microcosm/microcosm-targets-view.tsx index 7bc27408..e625b2f5 100644 --- a/frontend/components/microcosm/microcosm-targets-view.tsx +++ b/frontend/components/microcosm/microcosm-targets-view.tsx @@ -4,7 +4,7 @@ import { useEffect, useMemo, useState } from "react"; import { EmptyState } from "@/components/shared/empty-state"; import { fmt, fmtCompact, humanizeName, releaseLabel } from "@/components/shared/format"; -import { useCountry, type Country } from "@/components/layout/country-context"; +import { useCountry } from "@/components/layout/country-context"; import { ArtifactDescriptionBanner } from "@/components/microcosm/artifact-description-banner"; import { KpiCard } from "@/components/shared/kpi-card"; import { LoadingBlock } from "@/components/shared/LoadingBlock"; @@ -13,6 +13,7 @@ import { SectionCard } from "@/components/shared/section-card"; import { ToolbarSelect } from "@/components/shared/toolbar-select"; import { MicrocosmTargetDetail } from "@/components/microcosm/microcosm-target-detail"; import { withBasePath } from "@/lib/base-path"; +import { hasCapability, type MicrocosmCountry } from "@/lib/microcosm/countries"; import { sourceLabel } from "@/lib/microcosm/source-label"; import { releaseSelectOptions, @@ -26,7 +27,10 @@ import { const PAGE_SIZE = 50; -const COUNTRY_BROWSE_COPY: Record = { +// Legacy per-country browse prompt for releases published before a typed +// `release_manifest.presentation` block; countries without an entry get the +// generic prompt. +const COUNTRY_BROWSE_COPY: Partial> = { us: "Pick a measure like EITC, population, or AGI and see how each breakdown is calibrated.", uk: @@ -34,6 +38,7 @@ const COUNTRY_BROWSE_COPY: Record = { be: "Pick a measure like population, income tax, or pension recipients and see how each breakdown is calibrated.", }; +const GENERIC_BROWSE_COPY = "Pick a measure and see how each breakdown is calibrated."; interface SortState { by: string; @@ -571,8 +576,9 @@ export function MicrocosmTargetsView({ const releaseOptions = useMemo( () => [ ...releaseSelectOptions(releaseData), - // Candidate staging runs (US-only), reviewable like a release pre-publish. - ...(country === "us" ? (stagingData?.runs ?? []) : []).map((r) => ({ + // Candidate staging runs (countries with the staging capability), + // reviewable like a release pre-publish. + ...(hasCapability(country, "staging") ? (stagingData?.runs ?? []) : []).map((r) => ({ value: `staging:${r.run_id}`, label: `candidate · ${releaseLabel(r.run_id, r.updated_at)}${ r.status && r.status !== "completed" ? ` (${r.status})` : "" @@ -688,7 +694,8 @@ export function MicrocosmTargetsView({ ], ); - const { data, isLoading, isFetching, error } = useMicrocosmTargetDiagnostics(params); + const { data, isLoading, isFetching, error, isPlaceholderData } = + useMicrocosmTargetDiagnostics(params); const variables = data?.variables ?? []; const sources = data?.sources ?? []; @@ -702,6 +709,11 @@ export function MicrocosmTargetsView({ ); const filteredTotal = data?.filtered_total ?? 0; const allTargets = data?.total_targets ?? null; + // The healthcare focus is offered when the release has healthcare targets. + // Kept-previous placeholder data belongs to the prior country/release, so it + // must not decide the card. + const hasHealthcareTargets = + !isPlaceholderData && (data?.scope_counts?.healthcare ?? 0) > 0; const pageCount = Math.max(Math.ceil(filteredTotal / PAGE_SIZE), 1); const activeVariable = variables.find((v) => v.variable_key === variable); @@ -1072,12 +1084,12 @@ export function MicrocosmTargetsView({ - {country === "us" ? ( + {hasHealthcareTargets ? ( ; } +// Typed `release_manifest.country` merged over the country registration +// (server: releaseCountry in lib/microcosm/latest-artifact.ts). +export interface MicrocosmArtifactCountry { + code: Country; + label: string; + geography_id: string | null; + geography_label: string; + repository_visibility: RepositoryVisibility; + capabilities: CountryCapability[]; +} + export interface MicrocosmCalibration { available: boolean; + country?: MicrocosmArtifactCountry; description?: string | null; diagnostics_status?: MicrocosmDiagnosticsStatus; dataset_role?: string | null; @@ -299,6 +316,7 @@ export interface MicrocosmResponse { export interface MicrocosmTargetDiagnostics { available: boolean; + country?: MicrocosmArtifactCountry; description?: string | null; path?: string | null; release_id?: string | null; @@ -310,6 +328,8 @@ export interface MicrocosmTargetDiagnostics { geographies?: string[]; variables?: MicrocosmVariableRow[]; dimensions?: MicrocosmTargetDimension[]; + // Targets per named scope in the release (the `scope` query parameter). + scope_counts?: { healthcare?: number }; summary: { diagnostics_status?: MicrocosmDiagnosticsStatus; total_targets?: number | null; @@ -607,18 +627,20 @@ export function useMicrocosmCompare(a?: string, b?: string, enabled = true) { export function useMicrocosmStagingRuns() { const { country } = useCountry(); + const staging = hasCapability(country, "staging"); return useQuery({ queryKey: ["microcosm", "staging", "runs", country], queryFn: () => apiGet("/microcosm/staging/runs", { country }), - enabled: country === "us", + enabled: staging, staleTime: 15 * 1000, - refetchInterval: country === "us" ? 30 * 1000 : false, + refetchInterval: staging ? 30 * 1000 : false, }); } export function useMicrocosmStagingRun(runId?: string) { const { country } = useCountry(); + const staging = hasCapability(country, "staging"); return useQuery({ queryKey: ["microcosm", "staging", "run", country, runId], queryFn: () => @@ -626,10 +648,10 @@ export function useMicrocosmStagingRun(runId?: string) { id: runId, country, }), - enabled: country === "us" && Boolean(runId), + enabled: staging && Boolean(runId), placeholderData: keepPreviousData, staleTime: 10 * 1000, - refetchInterval: country === "us" ? 30 * 1000 : false, + refetchInterval: staging ? 30 * 1000 : false, }); } @@ -642,7 +664,7 @@ export function useMicrocosmStagingCompare(runId?: string, release = "latest") { "/microcosm/staging/compare", { run: runId, release, country }, ), - enabled: country === "us" && Boolean(runId), + enabled: hasCapability(country, "staging") && Boolean(runId), placeholderData: keepPreviousData, staleTime: 30 * 1000, }); diff --git a/frontend/lib/cross-dataset/fact-presentation.ts b/frontend/lib/cross-dataset/fact-presentation.ts index 2dda71c1..09a1f0b4 100644 --- a/frontend/lib/cross-dataset/fact-presentation.ts +++ b/frontend/lib/cross-dataset/fact-presentation.ts @@ -4,6 +4,7 @@ import type { SourceSummary, } from "./artifact"; import type { Country } from "@/components/layout/country-context"; +import { DEFAULT_COUNTRY, isCountry } from "@/lib/microcosm/countries"; import { sourceDisplayLabel } from "./presentation"; import { sourceAuthorityLabel } from "../source-labels"; @@ -87,7 +88,7 @@ export interface FactDetailView { } const DEFAULT_PARAMS: FactCatalogParams = { - country: "us", + country: DEFAULT_COUNTRY, source: "", status: "", chronicleSource: "", @@ -103,7 +104,6 @@ const DEFAULT_PARAMS: FactCatalogParams = { }; const SORTS = new Set(["fact_key", "label", "error_desc"]); -const COUNTRIES = new Set(["us", "uk", "be"]); function boundedPositiveInteger(value: string | null, fallback: number, maximum: number): number { if (!value || !/^\d+$/.test(value)) return fallback; @@ -119,10 +119,7 @@ export function parseFactCatalogParams( const countryValue = params.get("country"); return { country: - currentCountry ?? - (countryValue && COUNTRIES.has(countryValue as Country) - ? (countryValue as Country) - : DEFAULT_PARAMS.country), + currentCountry ?? (isCountry(countryValue) ? countryValue : DEFAULT_PARAMS.country), source: params.get("source")?.trim() ?? "", status: params.get("status")?.trim() ?? "", chronicleSource: params.get("ledger_source")?.trim() ?? "", diff --git a/frontend/lib/cross-dataset/source.ts b/frontend/lib/cross-dataset/source.ts index ea00f59f..a027c738 100644 --- a/frontend/lib/cross-dataset/source.ts +++ b/frontend/lib/cross-dataset/source.ts @@ -2,7 +2,11 @@ import { readFile } from "node:fs/promises"; import path from "node:path"; import { ArtifactError, CrossDatasetArtifactReader } from "./artifact"; -import type { MicrocosmCountry } from "../microcosm/latest-artifact"; +import { + DEFAULT_COUNTRY, + countryRegistration, + type MicrocosmCountry, +} from "../microcosm/countries"; interface CountryArtifactConfig { countryCode: string; @@ -19,15 +23,18 @@ interface CachedReader { const cachedReaders = new Map(); +// The unsuffixed variables remain the default country's configuration so +// existing deployments keep working; every other country uses its code as a +// suffix. function countryArtifactConfig(country: MicrocosmCountry): CountryArtifactConfig { const countryCode = country.toUpperCase(); - const suffix = country === "us" ? "" : `_${countryCode}`; + const suffix = country === DEFAULT_COUNTRY ? "" : `_${countryCode}`; return { countryCode, directoryEnv: `CROSS_DATASET_ARTIFACT_DIR${suffix}`, baseUrlEnv: `CROSS_DATASET_ARTIFACT_BASE_URL${suffix}`, expectedRunIdEnv: `CROSS_DATASET_EXPECTED_RUN_ID${suffix}`, - jurisdictionAliases: country === "uk" ? ["GB"] : [], + jurisdictionAliases: countryRegistration(country).jurisdiction_aliases ?? [], }; } diff --git a/frontend/lib/microcosm/countries.test.ts b/frontend/lib/microcosm/countries.test.ts new file mode 100644 index 00000000..b1a21e7f --- /dev/null +++ b/frontend/lib/microcosm/countries.test.ts @@ -0,0 +1,113 @@ +import { expect, test } from "bun:test"; + +import { + COUNTRY_CAPABILITIES, + COUNTRY_REGISTRY, + DEFAULT_COUNTRY, + countryCapabilities, + countryRegistration, + hasCapability, + isCountry, + isCountryCapability, + parseCountry, + selectableCountries, + type MicrocosmCountry, +} from "./countries"; + +const COUNTRIES = Object.keys(COUNTRY_REGISTRY) as MicrocosmCountry[]; + +test("every registration carries the full shape", () => { + for (const country of COUNTRIES) { + const registration = countryRegistration(country); + expect(registration.repo).toMatch(/^[a-z0-9-]+\/[a-z0-9-]+$/); + expect(registration.revision.length).toBeGreaterThan(0); + expect(registration.label.length).toBeGreaterThan(0); + expect(registration.dataset_label.length).toBeGreaterThan(0); + expect(registration.geography.length).toBeGreaterThan(0); + expect( + registration.geography_id === null || registration.geography_id.length > 0, + ).toBe(true); + expect(["public", "private"]).toContain(registration.visibility); + expect(registration.capabilities.length).toBeGreaterThan(0); + expect(registration.capabilities.every(isCountryCapability)).toBe(true); + expect(new Set(registration.capabilities).size).toBe(registration.capabilities.length); + if (registration.repo_env != null || registration.revision_env != null) { + expect(registration.repo_env).toMatch(/^[A-Z][A-Z0-9_]*$/); + expect(registration.revision_env).toMatch(/^[A-Z][A-Z0-9_]*$/); + } + } +}); + +test("keeps the live registrations on their published repositories and labels", () => { + expect(countryRegistration("us")).toMatchObject({ + repo: "policyengine/populace-us", + repo_env: "POPULACE_HF_REPO", + revision_env: "POPULACE_HF_REVISION", + label: "United States", + dataset_label: "Microcosm US", + geography: "United States", + geography_id: "0100000US", + visibility: "public", + capabilities: COUNTRY_CAPABILITIES, + }); + expect(countryRegistration("uk")).toMatchObject({ + repo: "policyengine/populace-uk-private", + repo_env: "POPULACE_UK_HF_REPO", + label: "United Kingdom", + dataset_label: "Microcosm UK", + visibility: "private", + jurisdiction_aliases: ["GB"], + }); + expect(countryRegistration("be")).toMatchObject({ + repo: "policyengine/populace-be-private", + repo_env: "POPULACE_BE_HF_REPO", + label: "Belgium", + dataset_label: "Microcosm Belgium", + geography: "Belgium", + visibility: "private", + }); +}); + +test("selectable countries follow registry order and exclude fixtures", () => { + expect(selectableCountries()).toEqual(["us", "uk", "be"]); + expect(countryRegistration("zz").fixture).toBe(true); + expect(selectableCountries()).not.toContain("zz"); +}); + +test("fixture registrations are valid countries without being selectable", () => { + expect(isCountry("zz")).toBe(true); + expect(parseCountry("zz")).toBe("zz"); +}); + +test("country parsing is exact and defaults to the registry default", () => { + expect(DEFAULT_COUNTRY).toBe("us"); + expect(parseCountry("be")).toBe("be"); + expect(parseCountry("uk")).toBe("uk"); + expect(parseCountry("us")).toBe("us"); + expect(parseCountry("BE")).toBe(DEFAULT_COUNTRY); + expect(parseCountry("fr")).toBe(DEFAULT_COUNTRY); + expect(parseCountry("")).toBe(DEFAULT_COUNTRY); + expect(parseCountry(null)).toBe(DEFAULT_COUNTRY); + expect(parseCountry(undefined)).toBe(DEFAULT_COUNTRY); + expect(isCountry("constructor")).toBe(false); + expect(isCountry("__proto__")).toBe(false); + expect(isCountry(null)).toBe(false); +}); + +test("capability gates read the registration", () => { + expect(hasCapability("us", "staging")).toBe(true); + expect(hasCapability("us", "model_coverage")).toBe(true); + expect(hasCapability("uk", "staging")).toBe(false); + expect(hasCapability("be", "pipeline")).toBe(false); + expect(hasCapability("be", "cross_dataset")).toBe(true); + expect(countryCapabilities("be")).toEqual([ + "calibration", + "targets", + "compare", + "cross_dataset", + ]); + expect(countryCapabilities("zz")).toEqual(["calibration", "targets", "compare"]); + expect(isCountryCapability("staging")).toBe(true); + expect(isCountryCapability("dashboard_admin")).toBe(false); + expect(isCountryCapability(7)).toBe(false); +}); diff --git a/frontend/lib/microcosm/countries.ts b/frontend/lib/microcosm/countries.ts new file mode 100644 index 00000000..2b1de1e5 --- /dev/null +++ b/frontend/lib/microcosm/countries.ts @@ -0,0 +1,144 @@ +// Country registry for the Microcosm dashboard: the one place a country is +// registered. Adding a country is one entry here; everything else (selectors, +// navigation, repository resolution, national geography, link behaviour) reads +// the registration or the release artifact's typed `country` block. +// +// Isomorphic: imported by client components and API routes alike, so it must +// not import server-only modules or read `process.env`. Deployment overrides +// for repositories are resolved server side in latest-artifact.ts. + +export const COUNTRY_CAPABILITIES = [ + "calibration", + "targets", + "compare", + "cross_dataset", + "staging", + "model_coverage", + "pipeline", + "variables", + "external_checks", +] as const; +export type CountryCapability = (typeof COUNTRY_CAPABILITIES)[number]; + +export type RepositoryVisibility = "public" | "private"; + +export interface CountryRegistration { + // Default HF dataset repository and revision (the server may override both + // through the deployment variables named in repo_env / revision_env). + repo: string; + revision: string; + repo_env?: string; + revision_env?: string; + // Country display name ("United States"). + label: string; + // Sidebar dataset line ("Microcosm US"). + dataset_label: string; + // National geography label and id (null when the id is unknown). + geography: string; + geography_id: string | null; + visibility: RepositoryVisibility; + capabilities: readonly CountryCapability[]; + // Other jurisdiction codes a cross-dataset bundle may use for this country. + jurisdiction_aliases?: readonly string[]; + // Conformance-only registration: a valid country that is never listed in + // selectors or alert allowlists. + fixture?: true; +} + +const ALL_CAPABILITIES: readonly CountryCapability[] = COUNTRY_CAPABILITIES; + +const CALIBRATION_CAPABILITIES: readonly CountryCapability[] = [ + "calibration", + "targets", + "compare", + "cross_dataset", +]; + +// Deprecated upstream identifiers: Microcosm's published HF repositories and +// deployment variables still use the former Populace names. +export const COUNTRY_REGISTRY = { + us: { + repo: "policyengine/populace-us", + revision: "main", + repo_env: "POPULACE_HF_REPO", + revision_env: "POPULACE_HF_REVISION", + label: "United States", + dataset_label: "Microcosm US", + geography: "United States", + geography_id: "0100000US", + visibility: "public", + capabilities: ALL_CAPABILITIES, + }, + uk: { + repo: "policyengine/populace-uk-private", + revision: "main", + repo_env: "POPULACE_UK_HF_REPO", + revision_env: "POPULACE_UK_HF_REVISION", + label: "United Kingdom", + dataset_label: "Microcosm UK", + geography: "United Kingdom", + geography_id: null, + visibility: "private", + capabilities: CALIBRATION_CAPABILITIES, + jurisdiction_aliases: ["GB"], + }, + be: { + repo: "policyengine/populace-be-private", + revision: "main", + repo_env: "POPULACE_BE_HF_REPO", + revision_env: "POPULACE_BE_HF_REVISION", + label: "Belgium", + dataset_label: "Microcosm Belgium", + geography: "Belgium", + geography_id: null, + visibility: "private", + capabilities: CALIBRATION_CAPABILITIES, + }, + // Synthetic repository used only by the third-country conformance fixture. + zz: { + repo: "policyengine/microcosm-zz-fixture", + revision: "main", + label: "Zedland", + dataset_label: "Microcosm ZZ", + geography: "Zedland", + geography_id: null, + visibility: "private", + capabilities: ["calibration", "targets", "compare"], + fixture: true, + }, +} satisfies Record; + +export type MicrocosmCountry = keyof typeof COUNTRY_REGISTRY; + +export const DEFAULT_COUNTRY: MicrocosmCountry = "us"; + +const COUNTRY_CODES = Object.keys(COUNTRY_REGISTRY) as MicrocosmCountry[]; + +export function isCountry(value: string | null | undefined): value is MicrocosmCountry { + return value != null && Object.hasOwn(COUNTRY_REGISTRY, value); +} + +export function parseCountry(value: string | null | undefined): MicrocosmCountry { + return isCountry(value) ? value : DEFAULT_COUNTRY; +} + +export function countryRegistration(country: MicrocosmCountry): CountryRegistration { + return COUNTRY_REGISTRY[country]; +} + +// Countries offered in selectors, in registry order; fixtures are excluded. +export function selectableCountries(): MicrocosmCountry[] { + return COUNTRY_CODES.filter((country) => !countryRegistration(country).fixture); +} + +export function countryCapabilities(country: MicrocosmCountry): readonly CountryCapability[] { + return COUNTRY_REGISTRY[country].capabilities; +} + +export function hasCapability(country: MicrocosmCountry, capability: CountryCapability): boolean { + return countryCapabilities(country).includes(capability); +} + +export function isCountryCapability(value: unknown): value is CountryCapability { + return typeof value === "string" && (COUNTRY_CAPABILITIES as readonly string[]).includes(value); +} diff --git a/frontend/lib/microcosm/latest-artifact.test.ts b/frontend/lib/microcosm/latest-artifact.test.ts index 68df241d..f184a912 100644 --- a/frontend/lib/microcosm/latest-artifact.test.ts +++ b/frontend/lib/microcosm/latest-artifact.test.ts @@ -22,8 +22,10 @@ import { microcosmRepo, microcosmRevision, parseCountry, + releaseCountry, releasePublishedAtFromTree, releaseRole, + type ArtifactCountry, type Calibration, } from "./latest-artifact"; import { buildCalibrationTree } from "./calibration-tree"; @@ -55,6 +57,11 @@ test("coerces supported country parameters and defaults unknown values to US", ( expect(parseCountry(null)).toBe("us"); }); +test("accepts the conformance fixture country only as a registered code", () => { + expect(parseCountry("zz")).toBe("zz"); + expect(parseCountry("ZZ")).toBe("us"); +}); + test("uses the private Belgium repository and country revision", () => { expect(microcosmRepo("be")).toBe("policyengine/populace-be-private"); expect(microcosmRevision("be")).toBe("main"); @@ -734,6 +741,11 @@ test("healthcare scope includes ACA, Medicaid, Medicare, and PTC targets", () => expect(result.total_targets).toBe(4); expect(result.filtered_total).toBe(4); + expect(result.scope_counts).toEqual({ healthcare: 4 }); + expect( + latestMicrocosmTargetDiagnosticsPage("http://x/api/microcosm/target-diagnostics", cal) + .scope_counts, + ).toEqual({ healthcare: 4 }); expect(result.summary.fraction_within_10pct).toBe(0.5); expect(result.targets.map((row) => row.name)).not.toContain( "nation/irs/adjusted gross income/total/AGI in 30k-40k/taxable/All@2024", @@ -1248,3 +1260,146 @@ test("releaseRole classifies national default vs non-default local-area", () => is_local_area: true, }); }); + +const BE_COUNTRY_DEFAULTS: ArtifactCountry = { + code: "be", + label: "Belgium", + geography_id: null, + geography_label: "Belgium", + repository_visibility: "private", + capabilities: ["calibration", "targets", "compare", "cross_dataset"], +}; + +test("releaseCountry falls back to the registration when the manifest has no country block", () => { + expect(releaseCountry({}, "be")).toEqual(BE_COUNTRY_DEFAULTS); + expect(releaseCountry({ country: "BE" }, "be")).toEqual(BE_COUNTRY_DEFAULTS); + expect(releaseCountry({}, "us")).toEqual({ + code: "us", + label: "United States", + geography_id: "0100000US", + geography_label: "United States", + repository_visibility: "public", + capabilities: [ + "calibration", + "targets", + "compare", + "cross_dataset", + "staging", + "model_coverage", + "pipeline", + "variables", + "external_checks", + ], + }); +}); + +test("releaseCountry lets well-typed string fields override the registration", () => { + expect( + releaseCountry( + { + country: { + code: "BE", + label: "Kingdom of Belgium", + geography_id: "BE", + geography_label: "Belgium (national)", + repository_visibility: "public", + presentation: { ignored: true }, + }, + }, + "be", + ), + ).toEqual({ + ...BE_COUNTRY_DEFAULTS, + label: "Kingdom of Belgium", + geography_id: "BE", + geography_label: "Belgium (national)", + repository_visibility: "public", + }); +}); + +test("releaseCountry ignores a block whose code names another country", () => { + expect( + releaseCountry( + { country: { code: "us", label: "United States", repository_visibility: "public" } }, + "be", + ), + ).toEqual(BE_COUNTRY_DEFAULTS); + expect(releaseCountry({ country: { code: 7, label: "Seven" } }, "be")).toEqual( + BE_COUNTRY_DEFAULTS, + ); + // An explicit `code: null` is a present, malformed field — the whole block + // is ignored, exactly like a mismatched code. + expect( + releaseCountry( + { country: { code: null, label: "Wrong label", geography_label: "Wrong geography" } }, + "be", + ), + ).toEqual(BE_COUNTRY_DEFAULTS); +}); + +test("releaseCountry narrows capabilities to the registration and never widens them", () => { + expect( + releaseCountry( + { country: { code: "be", capabilities: ["targets", "staging", "calibration", "bogus", 3] } }, + "be", + ).capabilities, + ).toEqual(["calibration", "targets"]); + expect(releaseCountry({ country: { capabilities: [] } }, "be").capabilities).toEqual([]); + expect(releaseCountry({ country: { capabilities: "all" } }, "be").capabilities).toEqual( + BE_COUNTRY_DEFAULTS.capabilities, + ); +}); + +test("releaseCountry ignores malformed field values", () => { + expect( + releaseCountry( + { + country: { + label: "", + geography_label: 12, + geography_id: { id: "x" }, + repository_visibility: "open", + }, + }, + "be", + ), + ).toEqual(BE_COUNTRY_DEFAULTS); +}); + +test("the calibration summary and target page carry the typed country block", () => { + const cal = buildCalibration( + beDiagnosticsFixture, + "be-country", + null, + {}, + { ...beReleaseManifestFixture, country: { code: "be", label: "Kingdom of Belgium" } }, + {}, + "be", + ); + expect(cal.country_info).toEqual({ ...BE_COUNTRY_DEFAULTS, label: "Kingdom of Belgium" }); + expect(latestMicrocosmCalibrationSummary(cal).country).toEqual(cal.country_info); + expect( + latestMicrocosmTargetDiagnosticsPage("http://x/api/microcosm/target-diagnostics", cal).country, + ).toEqual(cal.country_info); +}); + +test("the artifact's national geography label shapes rows without a geography", () => { + const cal = buildCalibration( + beDiagnosticsFixture, + "be-geography", + null, + {}, + { ...beReleaseManifestFixture, country: { geography_label: "Belgium (national)" } }, + {}, + "be", + ); + const national = cal.rows.filter((row) => row.level === "national"); + expect(national.length).toBeGreaterThan(0); + expect(national.every((row) => row.geography === "Belgium (national)")).toBe(true); + expect(cal.rows[0]).toMatchObject({ geography: "Brussels", level: "region" }); +}); + +test("the target page reports scope target counts for the release", () => { + expect(page("").scope_counts).toEqual({ healthcare: 0 }); + expect(page("?scope=healthcare").scope_counts).toEqual({ healthcare: 0 }); +}); diff --git a/frontend/lib/microcosm/latest-artifact.ts b/frontend/lib/microcosm/latest-artifact.ts index 453b4f53..4ac4cca0 100644 --- a/frontend/lib/microcosm/latest-artifact.ts +++ b/frontend/lib/microcosm/latest-artifact.ts @@ -6,6 +6,17 @@ import { sourceAuthorityLabel } from "@/lib/source-labels"; import { normalizeChronicleMetadata } from "./chronicle-metadata"; +import { + COUNTRY_REGISTRY, + DEFAULT_COUNTRY, + countryRegistration, + isCountry, + isCountryCapability, + parseCountry, + type CountryCapability, + type MicrocosmCountry, + type RepositoryVisibility, +} from "./countries"; import { normalizeTargetLossAttribution, targetLossAttributionSummary, @@ -13,24 +24,27 @@ import { type TargetLossDiagnosticWarning, } from "./target-loss-attribution"; +// The registry is the registration point; these re-exports keep the server +// modules and routes that import country helpers from here working. +export { isCountry, parseCountry, type MicrocosmCountry }; + type JsonObject = Record; type TargetRow = JsonObject; export type CalibrationLossKind = "normalized_target_loss" | "raw_optimizer_objective"; -const DEFAULT_GEOGRAPHY = "United States"; +const DEFAULT_GEOGRAPHY = countryRegistration(DEFAULT_COUNTRY).geography; const DEFAULT_GEOGRAPHY_LEVEL = "national"; // Deprecated upstream identifiers: Microcosm's published HF repositories and -// deployment variables still use the former Populace names. -export const MICROCOSM_HF_REPO_ENV = "POPULACE_HF_REPO"; -export const MICROCOSM_HF_REVISION_ENV = "POPULACE_HF_REVISION"; -export const MICROCOSM_UK_HF_REPO_ENV = "POPULACE_UK_HF_REPO"; -export const MICROCOSM_UK_HF_REVISION_ENV = "POPULACE_UK_HF_REVISION"; -export const MICROCOSM_BE_HF_REPO_ENV = "POPULACE_BE_HF_REPO"; -export const MICROCOSM_BE_HF_REVISION_ENV = "POPULACE_BE_HF_REVISION"; -export const MICROCOSM_HF_REPO = - process.env[MICROCOSM_HF_REPO_ENV] ?? "policyengine/populace-us"; -export const MICROCOSM_HF_REVISION = process.env[MICROCOSM_HF_REVISION_ENV] ?? "main"; +// deployment variables still use the former Populace names. The names live on +// each country's registration; they are re-exported here for deployment docs +// and tests. +export const MICROCOSM_HF_REPO_ENV = COUNTRY_REGISTRY.us.repo_env; +export const MICROCOSM_HF_REVISION_ENV = COUNTRY_REGISTRY.us.revision_env; +export const MICROCOSM_UK_HF_REPO_ENV = COUNTRY_REGISTRY.uk.repo_env; +export const MICROCOSM_UK_HF_REVISION_ENV = COUNTRY_REGISTRY.uk.revision_env; +export const MICROCOSM_BE_HF_REPO_ENV = COUNTRY_REGISTRY.be.repo_env; +export const MICROCOSM_BE_HF_REVISION_ENV = COUNTRY_REGISTRY.be.revision_env; interface MicrocosmCountryRepository { repo: string; @@ -38,41 +52,33 @@ interface MicrocosmCountryRepository { geography: string; } -// A repository entry is the server-side registration point for a country. Keep -// its national geography beside the repository so downstream shaping does not -// require a second exhaustive country table. -export const COUNTRY_REPO = { - us: { - repo: MICROCOSM_HF_REPO, - revision: MICROCOSM_HF_REVISION, - geography: "United States", - }, - uk: { - repo: process.env[MICROCOSM_UK_HF_REPO_ENV] ?? "policyengine/populace-uk-private", - revision: process.env[MICROCOSM_UK_HF_REVISION_ENV] ?? "main", - geography: "United Kingdom", - }, - be: { - repo: process.env[MICROCOSM_BE_HF_REPO_ENV] ?? "policyengine/populace-be-private", - revision: process.env[MICROCOSM_BE_HF_REVISION_ENV] ?? "main", - geography: "Belgium", - }, - // Synthetic repository used only by the third-country conformance fixture. - zz: { - repo: "policyengine/microcosm-zz-fixture", - revision: "main", - geography: "Zedland", - }, -} satisfies Record; - -export type MicrocosmCountry = keyof typeof COUNTRY_REPO; +function envOverride(name: string | undefined): string | undefined { + return name == null ? undefined : process.env[name]; +} -export function parseCountry(value: string | null | undefined): MicrocosmCountry { - return value != null && Object.hasOwn(COUNTRY_REPO, value) - ? (value as MicrocosmCountry) - : "us"; +// Server-side view of a registration: the registry defaults with this +// deployment's repository/revision overrides applied. Keep the national +// geography beside the repository so downstream shaping does not require a +// second exhaustive country table. +function resolveCountryRepository(country: MicrocosmCountry): MicrocosmCountryRepository { + const registration = countryRegistration(country); + return { + repo: envOverride(registration.repo_env) ?? registration.repo, + revision: envOverride(registration.revision_env) ?? registration.revision, + geography: registration.geography, + }; } +export const COUNTRY_REPO = Object.fromEntries( + (Object.keys(COUNTRY_REGISTRY) as MicrocosmCountry[]).map((country) => [ + country, + resolveCountryRepository(country), + ]), +) as Record; + +export const MICROCOSM_HF_REPO = COUNTRY_REPO.us.repo; +export const MICROCOSM_HF_REVISION = COUNTRY_REPO.us.revision; + // Release/run ids are interpolated into HuggingFace URLs that carry the // server's HF token, so an unvalidated id ("../../..") could redirect the // authenticated request to arbitrary paths after URL normalization. Every id @@ -713,7 +719,7 @@ function artifactVariable( function parseDottedTarget( name: string, row: TargetRow, - country: MicrocosmCountry, + nationalGeography: string, ): ParsedTarget | null { if (!name.includes(".")) return null; const metadata = asObject(row.metadata); @@ -724,7 +730,7 @@ function parseDottedTarget( const geoId = stringValue(metadata.ledger_geography_id); const geography = geoLevel === "country" - ? microcosmCountryGeography(country) + ? nationalGeography : geoLevel === "congressional_district" ? districtFromGeoId(geoId) ?? "" : stateFromGeoId(geoId) ?? stringValue(metadata.state) ?? ""; @@ -756,7 +762,7 @@ function parseDottedTarget( return { geography, level, source, variable, breakdown }; } -function parseTarget(name: string, country: MicrocosmCountry): ParsedTarget { +function parseTarget(name: string, nationalGeography: string): ParsedTarget { const parts = name.split("/"); const p0 = parts[0] ?? ""; const fips = /^US(\d{2})$/.exec(p0); @@ -791,7 +797,7 @@ function parseTarget(name: string, country: MicrocosmCountry): ParsedTarget { } if (p0 === "nation" || p0 === "national" || p0 === "us") { return { - geography: microcosmCountryGeography(country), level: "national", source: parts[1] ?? "", + geography: nationalGeography, level: "national", source: parts[1] ?? "", variable: parts[2] ?? "", breakdown: parts.slice(3).join(" · "), }; } @@ -1064,10 +1070,11 @@ function calibrationStatus( // published metadata alongside. v1 rows simply lack those extra fields. function enrichTargetRow( rawRow: TargetRow, - skippedByName: Map = new Map(), - droppedTargetNames: Set = new Set(), - country: MicrocosmCountry = "us", + skippedByName: Map, + droppedTargetNames: Set, + artifactCountry: ArtifactCountry, ): TargetRow { + const nationalGeography = artifactCountry.geography_label; const metadata = normalizeChronicleMetadata(rawRow.metadata); const row: TargetRow = { ...rawRow, metadata }; const fullName = String(row.name ?? ""); @@ -1108,7 +1115,8 @@ function enrichTargetRow( const filterDecomposition = decomposeTargetFilter(row.filter); const publisher = chroniclePublisherFromMetadata(metadata); const parsedFromName = - parseDottedTarget(baseName, row, country) ?? parseTarget(baseName, country); + parseDottedTarget(baseName, row, nationalGeography) ?? + parseTarget(baseName, nationalGeography); const parsed: ParsedTarget = { ...parsedFromName, geography: filterDecomposition?.geography ?? parsedFromName.geography, @@ -1122,9 +1130,7 @@ function enrichTargetRow( : parsedFromName.breakdown, }; const hasGeography = Boolean(parsed.geography.trim()); - const geography = hasGeography - ? parsed.geography - : microcosmCountryGeography(country); + const geography = hasGeography ? parsed.geography : nationalGeography; const level = hasGeography ? parsed.level : DEFAULT_GEOGRAPHY_LEVEL; const measureCol = asObject(row.measure); const metadataTargetDimensions = @@ -1714,6 +1720,8 @@ export function microcosmTargetTreemap( export interface Calibration { source: "huggingface_live"; country: MicrocosmCountry; + // Typed `release_manifest.country` merged over the registration. + country_info: ArtifactCountry; description: string | null; diagnostics_status: DiagnosticsStatus; release_id: string; @@ -1757,6 +1765,9 @@ interface TargetDiagnosticsMetadata { levels: string[]; geographies: string[]; variables: ReturnType; + // Targets per named scope (the `scope` query parameter); a scope with no + // targets in the release has nothing to focus on. + scope_counts: { healthcare: number }; } interface InvestigationSignal { @@ -1850,6 +1861,63 @@ export function releaseRole(releaseManifest: JsonObject): ReleaseRole { }; } +// The country a release presents as, typed: registry defaults, overridden by +// the string fields of `release_manifest.country` when present and well-typed. +// The dashboard is selected by registry, so the artifact cannot re-route it: a +// block whose `code` names another country is ignored whole. Capabilities can +// only narrow what a deployment serves (intersection with the registration), +// never widen it. Unknown keys are ignored. +export interface ArtifactCountry { + code: MicrocosmCountry; + label: string; + geography_id: string | null; + geography_label: string; + repository_visibility: RepositoryVisibility; + capabilities: CountryCapability[]; +} + +function visibilityValue(value: unknown): RepositoryVisibility | null { + return value === "public" || value === "private" ? value : null; +} + +export function releaseCountry( + releaseManifest: JsonObject, + country: MicrocosmCountry, +): ArtifactCountry { + const registration = countryRegistration(country); + const defaults: ArtifactCountry = { + code: country, + label: registration.label, + geography_id: registration.geography_id, + geography_label: registration.geography, + repository_visibility: registration.visibility, + capabilities: [...registration.capabilities], + }; + const block = asObject(releaseManifest.country); + // Presence, not nullishness: an explicit `code: null` is a malformed block, + // not an absent field, and must not slip past the whole-block rejection. + if ( + Object.hasOwn(block, "code") && + (typeof block.code !== "string" || block.code.trim().toLowerCase() !== country) + ) { + return defaults; + } + const capabilities = Array.isArray(block.capabilities) + ? new Set(block.capabilities.filter(isCountryCapability)) + : null; + return { + code: country, + label: stringValue(block.label)?.trim() ?? defaults.label, + geography_id: stringValue(block.geography_id)?.trim() ?? defaults.geography_id, + geography_label: stringValue(block.geography_label)?.trim() ?? defaults.geography_label, + repository_visibility: + visibilityValue(block.repository_visibility) ?? defaults.repository_visibility, + capabilities: capabilities + ? defaults.capabilities.filter((capability) => capabilities.has(capability)) + : defaults.capabilities, + }; +} + export function buildCalibration( diag: JsonObject, releaseId: string, @@ -1871,8 +1939,9 @@ export function buildCalibration( : []; const skippedByName = skippedTargetReasons(skipped); const dropped = new Set(droppedTargetNames); + const artifactCountry = releaseCountry(releaseManifest, country); const enrichedRows = addEstimateScopeWarnings( - targets.map((row) => enrichTargetRow(row, skippedByName, dropped, country)), + targets.map((row) => enrichTargetRow(row, skippedByName, dropped, artifactCountry)), ); const role = releaseRole(releaseManifest); const normalizedAttribution = normalizeTargetLossAttribution({ @@ -1887,6 +1956,7 @@ export function buildCalibration( return { source: "huggingface_live", country, + country_info: artifactCountry, description: stringValue(diag.description) ?? stringValue(releaseManifest.description) ?? @@ -2141,6 +2211,7 @@ function targetDiagnosticsMetadata(rows: TargetRow[]): TargetDiagnosticsMetadata levels: microcosmTargetLevels(rows), geographies: microcosmTargetGeographies(rows), variables: microcosmVariableSummary(rows), + scope_counts: { healthcare: rows.filter(isHealthcareTarget).length }, }; targetDiagnosticsMetadataCache.set(rows, metadata); return metadata; @@ -2368,6 +2439,7 @@ function targetInvestigationPacket(row: TargetRow, cal: Calibration) { export function latestMicrocosmCalibrationSummary(cal: Calibration) { return { available: true, + country: cal.country_info, description: cal.description, diagnostics_status: cal.diagnostics_status, ...releaseRole(cal.release_manifest), @@ -2547,6 +2619,7 @@ export function latestMicrocosmTargetDiagnosticsPage(requestUrl: string, cal: Ca return { available: true, + country: cal.country_info, description: cal.description, diagnostics_status: cal.diagnostics_status, ...releaseRole(cal.release_manifest), @@ -2560,6 +2633,7 @@ export function latestMicrocosmTargetDiagnosticsPage(requestUrl: string, cal: Ca geographies: metadata.geographies, variables: metadata.variables, dimensions, + scope_counts: metadata.scope_counts, summary: { diagnostics_status: cal.diagnostics_status, total_targets: scopedRows.length, diff --git a/frontend/lib/microcosm/source-attribution.test.ts b/frontend/lib/microcosm/source-attribution.test.ts index 8ca33b3b..1f6fc037 100644 --- a/frontend/lib/microcosm/source-attribution.test.ts +++ b/frontend/lib/microcosm/source-attribution.test.ts @@ -36,3 +36,22 @@ test("does not expose the private Belgium Hugging Face dataset", () => { href: null, }); }); + +test("the release artifact's repository visibility overrides the registration", () => { + expect( + microcosmSourceAttribution("be", "policyengine/populace-be", "public"), + ).toEqual({ + label: "Microcosm", + href: "https://huggingface.co/datasets/policyengine/populace-be", + }); + expect( + microcosmSourceAttribution("us", "policyengine/populace-us", "private"), + ).toEqual({ + label: "Microcosm", + href: null, + }); + expect(microcosmSourceAttribution("us", "policyengine/populace-us", undefined)).toEqual({ + label: "Microcosm", + href: "https://huggingface.co/datasets/policyengine/populace-us", + }); +}); diff --git a/frontend/lib/microcosm/source-attribution.ts b/frontend/lib/microcosm/source-attribution.ts index 0efbff3f..3fae4dad 100644 --- a/frontend/lib/microcosm/source-attribution.ts +++ b/frontend/lib/microcosm/source-attribution.ts @@ -1,4 +1,8 @@ -import type { MicrocosmCountry } from "./latest-artifact"; +import { + countryRegistration, + type MicrocosmCountry, + type RepositoryVisibility, +} from "./countries"; export interface MicrocosmSourceAttribution { label: string; @@ -16,14 +20,17 @@ export function microcosmPublicationUrl( return `https://huggingface.co/datasets/${repoPath}/tree/${encodeURIComponent(releaseId)}`; } +// Link the dataset only when its repository is public: the registration's +// visibility, or the release artifact's `repository_visibility` when supplied. export function microcosmSourceAttribution( country: MicrocosmCountry, sourceRepo: string, + visibility: RepositoryVisibility = countryRegistration(country).visibility, ): MicrocosmSourceAttribution { return { label: "Microcosm", href: - country === "us" + visibility === "public" ? `https://huggingface.co/datasets/${sourceRepo}` : null, }; diff --git a/frontend/lib/microcosm/staging-artifact.test.ts b/frontend/lib/microcosm/staging-artifact.test.ts index 23a34414..85ce9501 100644 --- a/frontend/lib/microcosm/staging-artifact.test.ts +++ b/frontend/lib/microcosm/staging-artifact.test.ts @@ -16,6 +16,9 @@ test("names the country when staging is unavailable", () => { expect(stagingUnavailableReason("be")).toBe( "Belgium has no staging repository.", ); + expect(stagingUnavailableReason("zz")).toBe( + "Zedland has no staging repository.", + ); }); test("Belgium staging loaders return an empty state before resolving artifacts", async () => { diff --git a/frontend/lib/microcosm/staging-artifact.ts b/frontend/lib/microcosm/staging-artifact.ts index 937a063b..b53faddb 100644 --- a/frontend/lib/microcosm/staging-artifact.ts +++ b/frontend/lib/microcosm/staging-artifact.ts @@ -10,6 +10,7 @@ import { loadRelease, microcosmCountryGeography, } from "@/lib/microcosm/latest-artifact"; +import { hasCapability } from "@/lib/microcosm/countries"; import { type ReformValidation, buildReformValidation, @@ -26,8 +27,10 @@ export const MICROCOSM_STAGING_HF_REPO = export const MICROCOSM_STAGING_HF_REVISION = process.env[MICROCOSM_STAGING_HF_REVISION_ENV] ?? "main"; +// Staging telemetry is served for countries registered with the `staging` +// capability; the single staging repository above is the one they read. export function stagingUnavailableReason(country: MicrocosmCountry): string | null { - return country === "us" + return hasCapability(country, "staging") ? null : `${microcosmCountryGeography(country)} has no staging repository.`; } diff --git a/frontend/lib/microcosm/third-country-conformance.test.ts b/frontend/lib/microcosm/third-country-conformance.test.ts index a0264c86..c84b2917 100644 --- a/frontend/lib/microcosm/third-country-conformance.test.ts +++ b/frontend/lib/microcosm/third-country-conformance.test.ts @@ -1,6 +1,8 @@ import { describe, expect, test } from "bun:test"; import { isCountry as isClientCountry } from "@/components/layout/country-context"; +import { navGroupsForCountry } from "@/components/layout/nav-items"; +import { countryCapabilities, selectableCountries } from "@/lib/microcosm/countries"; import { sourceAuthorityLabel } from "@/lib/source-labels"; import diagnosticsFixture from "./fixtures/zz-release/calibration_diagnostics.json"; @@ -36,7 +38,7 @@ const calibration = buildCalibration( ); describe("synthetic third-country conformance", () => { - test("one COUNTRY_REPO registration supplies repository and national geography behavior", () => { + test("one country registration supplies repository, national geography, and country block behavior", () => { expect(parseCountry(COUNTRY)).toBe(COUNTRY); expect(microcosmRepo(COUNTRY)).toBe("policyengine/microcosm-zz-fixture"); expect(microcosmRevision(COUNTRY)).toBe("main"); @@ -51,6 +53,56 @@ describe("synthetic third-country conformance", () => { geography: "Zedland", level: "national", }); + + // The summary's typed country block comes from the registration alone. + expect(latestMicrocosmCalibrationSummary(calibration).country).toEqual({ + code: COUNTRY, + label: "Zedland", + geography_id: null, + geography_label: "Zedland", + repository_visibility: "private", + capabilities: [...countryCapabilities(COUNTRY)], + }); + + // A release_manifest.country block overrides the registration's labels and + // flows through buildCalibration into the summary and the row geography. + const overridden = buildCalibration( + diagnosticsFixture, + RELEASE_ID, + "2026-08-23T18:00:00Z", + {}, + { + ...releaseManifestFixture, + country: { + code: "zz", + label: "Republic of Zedland", + geography_label: "Zedland (national)", + }, + }, + {}, + COUNTRY, + ); + expect(latestMicrocosmCalibrationSummary(overridden).country).toMatchObject({ + code: COUNTRY, + label: "Republic of Zedland", + geography_label: "Zedland (national)", + capabilities: [...countryCapabilities(COUNTRY)], + }); + const overriddenPage = latestMicrocosmTargetDiagnosticsPage( + "http://x/api/microcosm/target-diagnostics?level=national", + overridden, + ); + expect(overriddenPage.country?.label).toBe("Republic of Zedland"); + expect(overriddenPage.targets.map((row) => row.geography)).toEqual(["Zedland (national)"]); + expect( + overridden.rows.find((row) => row.base_name === "fixture_revenue_personal_income_tax"), + ).toMatchObject({ geography: "Zedland (national)", level: "national" }); + expect( + overridden.rows + .filter((row) => row.level === "region") + .map((row) => row.geography) + .sort(), + ).toEqual(["North", "North", "South"]); }); test("release artifacts produce the common overview response shape", () => { @@ -142,12 +194,16 @@ describe("synthetic third-country conformance", () => { ); }); - test.todo( - "zz can enter the shared client views: the client country parser still uses a closed country table", - () => { - expect(isClientCountry(COUNTRY)).toBe(true); - }, - ); + test("zz can enter the shared client views through the registry without being selectable", () => { + expect(isClientCountry(COUNTRY)).toBe(true); + expect(selectableCountries()).not.toContain(COUNTRY); + // Navigation is gated by the registration's capabilities, not a country code. + expect( + navGroupsForCountry(COUNTRY) + .flatMap((group) => group.items) + .map((item) => item.href), + ).toEqual(["/microcosm", "/microcosm/targets", "/microcosm/compare"]); + }); test.todo( "zz overview data can carry artifact-provided intro copy: the summary omits a typed presentation contract", () => {