Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions .gitattributes
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
# Preserve hash-pinned publisher bytes, including original line endings/spacing.
db/data/stats_nz/subnational_population_estimates_2025/subnational-population-estimates-at-30-june-2025-provisional.xlsx binary
db/data/stats_nz/regional_council_2025_codes/regional-council-2025-codes.json binary
db/data/stats_nz/national_population_estimates_2025/national-population-estimates-at-30-june-2025-landing-page.html binary
5 changes: 5 additions & 0 deletions changelog.d/nz-partial-population-2025.added.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
Add 85 source-backed, provisional Stats NZ population counts for 30 June 2025,
covering the 16 regions and the separately published national total across all
ages and four broad age groups. Pin publisher artifacts and REGC 2025 codes.
Keep five-year-age-by-sex regional coverage and national single-year-age coverage
explicitly incomplete; no calibrated population or completed target set is added.
3 changes: 3 additions & 0 deletions chronicle/source_package.py
Original file line number Diff line number Diff line change
Expand Up @@ -74,6 +74,9 @@

SOURCE_PACKAGE_RESOURCE_PACKAGE = "packages"
SOURCE_PACKAGE_ALIASES = {
"stats-nz-subnational-population-estimates-2025": Path(
"stats_nz/subnational_population_estimates_2025"
),
"bea-nipa-personal-income-components": Path("bea/nipa_personal_income_components"),
"bea-nipa-personal-income-disposition": Path(
"bea/nipa_personal_income_disposition"
Expand Down
19 changes: 19 additions & 0 deletions db/data/stats_nz/national_population_estimates_2025/manifest.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
source_id: stats_nz
package_id: national_population_estimates_2025
dataset: stats_nz_national_population_estimates_2025
source_page: https://www.stats.govt.nz/information-releases/national-population-estimates-at-30-june-2025/
table: 'National population estimates: At 30 June 2025 (landing-page evidence only;
no single-year-age artifact)'
files:
2025:
filename: national-population-estimates-at-30-june-2025-landing-page.html
source_url: https://www.stats.govt.nz/information-releases/national-population-estimates-at-30-june-2025/
sha256: b2d2751800857895b1498d87df6f03ac3511ec8c7a9b17cc3150c8ab56533053
size_bytes: 61286
fetched_at: '2026-08-30T01:43:59+00:00'
storage:
r2:
provider: r2
bucket: ledger-raw
key: raw/nz/stats_nz/national_population_estimates_2025/2025/b2d2751800857895b1498d87df6f03ac3511ec8c7a9b17cc3150c8ab56533053/national-population-estimates-at-30-june-2025-landing-page.html
uri: r2://ledger-raw/raw/nz/stats_nz/national_population_estimates_2025/2025/b2d2751800857895b1498d87df6f03ac3511ec8c7a9b17cc3150c8ab56533053/national-population-estimates-at-30-june-2025-landing-page.html

Large diffs are not rendered by default.

18 changes: 18 additions & 0 deletions db/data/stats_nz/regional_council_2025_codes/manifest.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
source_id: stats_nz
package_id: regional_council_2025_codes
dataset: stats_nz_regional_council_2025_codes
source_page: https://services2.arcgis.com/vKb0s8tBIA3bdocZ/arcgis/rest/services/Regional_Council_2025/FeatureServer
table: Regional council 2025 code and name attributes (1 January 2025 boundaries)
files:
2025:
filename: regional-council-2025-codes.json
source_url: https://services2.arcgis.com/vKb0s8tBIA3bdocZ/arcgis/rest/services/Regional_Council_2025/FeatureServer/0/query?where=1%3D1&outFields=REGC2025_V1_00%2CREGC2025_V1_00_NAME&returnGeometry=false&orderByFields=REGC2025_V1_00&f=pjson
sha256: e5d53fa6abf742121d123bff970c77658a16e8c9b405b83d1a978cc22920d660
size_bytes: 3381
fetched_at: '2026-08-30T01:43:16+00:00'
storage:
r2:
provider: r2
bucket: ledger-raw
key: raw/nz/stats_nz/regional_council_2025_codes/2025/e5d53fa6abf742121d123bff970c77658a16e8c9b405b83d1a978cc22920d660/regional-council-2025-codes.json
uri: r2://ledger-raw/raw/nz/stats_nz/regional_council_2025_codes/2025/e5d53fa6abf742121d123bff970c77658a16e8c9b405b83d1a978cc22920d660/regional-council-2025-codes.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,140 @@
{
"objectIdFieldName" : "OBJECTID",
"uniqueIdField" :
{
"name" : "OBJECTID",
"isSystemMaintained" : true
},
"globalIdFieldName" : "",
"geometryType" : "esriGeometryPolygon",
"spatialReference" : {
"wkid" : 2193,
"latestWkid" : 2193
},
"fields" : [
{
"name" : "REGC2025_V1_00",
"type" : "esriFieldTypeString",
"alias" : "Regional Council Code",
"sqlType" : "sqlTypeOther",
"length" : 2,
"domain" : null,
"defaultValue" : null,
"description" : "{\"value\":\"Regional Council unique identifier\",\"fieldValueType\":\"uniqueIdentifier\"}"
},
{
"name" : "REGC2025_V1_00_NAME",
"type" : "esriFieldTypeString",
"alias" : "Regional Council Name",
"sqlType" : "sqlTypeOther",
"length" : 254,
"domain" : null,
"defaultValue" : null,
"description" : "{\"value\":\"Regional Council name (supports macrons)\",\"fieldValueType\":\"nameOrTitle\"}"
}
],
"features" : [
{
"attributes" : {
"REGC2025_V1_00" : "01",
"REGC2025_V1_00_NAME" : "Northland Region"
}
},
{
"attributes" : {
"REGC2025_V1_00" : "02",
"REGC2025_V1_00_NAME" : "Auckland"
}
},
{
"attributes" : {
"REGC2025_V1_00" : "03",
"REGC2025_V1_00_NAME" : "Waikato Region"
}
},
{
"attributes" : {
"REGC2025_V1_00" : "04",
"REGC2025_V1_00_NAME" : "Bay of Plenty Region"
}
},
{
"attributes" : {
"REGC2025_V1_00" : "05",
"REGC2025_V1_00_NAME" : "Gisborne Region"
}
},
{
"attributes" : {
"REGC2025_V1_00" : "06",
"REGC2025_V1_00_NAME" : "Hawke's Bay Region"
}
},
{
"attributes" : {
"REGC2025_V1_00" : "07",
"REGC2025_V1_00_NAME" : "Taranaki Region"
}
},
{
"attributes" : {
"REGC2025_V1_00" : "08",
"REGC2025_V1_00_NAME" : "Manawatū-Whanganui Region"
}
},
{
"attributes" : {
"REGC2025_V1_00" : "09",
"REGC2025_V1_00_NAME" : "Wellington Region"
}
},
{
"attributes" : {
"REGC2025_V1_00" : "12",
"REGC2025_V1_00_NAME" : "West Coast Region"
}
},
{
"attributes" : {
"REGC2025_V1_00" : "13",
"REGC2025_V1_00_NAME" : "Canterbury Region"
}
},
{
"attributes" : {
"REGC2025_V1_00" : "14",
"REGC2025_V1_00_NAME" : "Otago Region"
}
},
{
"attributes" : {
"REGC2025_V1_00" : "15",
"REGC2025_V1_00_NAME" : "Southland Region"
}
},
{
"attributes" : {
"REGC2025_V1_00" : "16",
"REGC2025_V1_00_NAME" : "Tasman Region"
}
},
{
"attributes" : {
"REGC2025_V1_00" : "17",
"REGC2025_V1_00_NAME" : "Nelson Region"
}
},
{
"attributes" : {
"REGC2025_V1_00" : "18",
"REGC2025_V1_00_NAME" : "Marlborough Region"
}
},
{
"attributes" : {
"REGC2025_V1_00" : "99",
"REGC2025_V1_00_NAME" : "Area Outside Region"
}
}
]
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
source_id: stats_nz
package_id: subnational_population_estimates_2025
dataset: stats_nz_subnational_population_estimates_2025
source_page: https://www.stats.govt.nz/information-releases/subnational-population-estimates-at-30-june-2025/
table: 'Subnational population estimates: At 30 June 2025 (provisional)'
files:
2025:
filename: subnational-population-estimates-at-30-june-2025-provisional.xlsx
source_url: https://www.stats.govt.nz/assets/Uploads/Subnational-population-estimates/Subnational-population-estimates-At-30-June-2025/Download-data/subnational-population-estimates-at-30-june-2025-provisional.xlsx
sha256: 001e8a896cfb50f5ed17836dc815b235e3bcca55ee91c9869a2afaeb054b50a6
size_bytes: 97990
fetched_at: '2026-08-30T01:38:43+00:00'
storage:
r2:
provider: r2
bucket: ledger-raw
key: raw/nz/stats_nz/subnational_population_estimates_2025/2025/001e8a896cfb50f5ed17836dc815b235e3bcca55ee91c9869a2afaeb054b50a6/subnational-population-estimates-at-30-june-2025-provisional.xlsx
uri: r2://ledger-raw/raw/nz/stats_nz/subnational_population_estimates_2025/2025/001e8a896cfb50f5ed17836dc815b235e3bcca55ee91c9869a2afaeb054b50a6/subnational-population-estimates-at-30-june-2025-provisional.xlsx
Binary file not shown.
98 changes: 98 additions & 0 deletions docs/pe-nz-population-source-notes.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,98 @@
# NZ population source notes

This records the bounded source-ingestion work for demographic packages 1 and 2
of [issue #176](https://github.com/PolicyEngine/chronicle/issues/176), checked on
30 August 2026 UTC. It does not mark the demographic target surface complete.

## Pinned publisher evidence

| Artifact | Publisher reference | SHA-256 | Bytes |
|---|---|---|---:|
| Subnational population workbook | At 30 June 2025, published 29 October 2025 | `001e8a896cfb50f5ed17836dc815b235e3bcca55ee91c9869a2afaeb054b50a6` | 97,990 |
| Regional council code/name attributes | Boundaries at 1 January 2025 | `e5d53fa6abf742121d123bff970c77658a16e8c9b405b83d1a978cc22920d660` | 3,381 |
| National population release landing page only | At 30 June 2025, published 19 August 2025 | `b2d2751800857895b1498d87df6f03ac3511ec8c7a9b17cc3150c8ab56533053` | 61,286 |

All artifacts came directly from Stats NZ's website or Stats NZ's official
geographic service. Their manifests record exact publisher URLs, hashes,
retrieval timestamps, sizes, and immutable country-scoped R2 keys:

- [Subnational workbook manifest](../db/data/stats_nz/subnational_population_estimates_2025/manifest.yaml).
- [REGC 2025 classification manifest](../db/data/stats_nz/regional_council_2025_codes/manifest.yaml).
- [National landing-page evidence manifest](../db/data/stats_nz/national_population_estimates_2025/manifest.yaml).

The objects use
`r2://ledger-raw/raw/nz/stats_nz/<package>/2025/<sha256>/<filename>`.
After upload, all three remote objects were downloaded through the authorized
Chronicle/Wrangler path and their byte counts and SHA-256 hashes matched the
local publisher artifacts.
The two population references share a 30 June 2025 reference date, not a
publication date. The national landing page is evidence of the missing download
surface, **not** a single-year-age data artifact or a national source package.
The classification is geography evidence only and emits no population facts.

## S7: partial workbook package

The [source package](../packages/stats_nz/subnational_population_estimates_2025/README.md)
admits 85 cells from Tables 1 and 3: 17 geographic rows × all ages/four broad
age bands. Its precise scope, semantics, and guards are documented there.

The workbook does not contain the requested five-year-age × sex × region cut.
Stats NZ's public metadata catalog lists `POPES_SUB_004`:

> Subnational population estimates (RC, SA2), by age and sex, at 30 June 1996-2025 (2025 boundaries)

This identifies the needed population/age/sex/boundary family, not a claim that
the query's original publication vintage has been verified. The documented API
request for its structure returned **HTTP 401** without a subscription:

```text
https://api.data.stats.govt.nz/rest/dataflow/STATSNZ/POPES_SUB_004/1.0?references=all&detail=referencepartial
```

The [official API guide](https://www.stats.govt.nz/tools/aotearoa-data-explorer/ade-api-user-guide/)
describes API subscription access. No existing subscription was available to
this ingestion run. No account was created and no authentication bypass was
attempted. An authorized export still needs verification of the exact five-year
age categories, sex categories, 2025 REGC boundaries, reference date, and source
revision vintage before it can extend this package. In particular,
`POPES_SUB_003` advertises **2026 boundaries** and is not a substitute.

## S8: national single-year ages remain missing

The pinned [19 August 2025 national release](https://www.stats.govt.nz/information-releases/national-population-estimates-at-30-june-2025/)
contains headline counts and directs users to Infoshare's Population / Population
Estimates – DPE tables for annual and quarterly single-year-age data. Its page
payload contains no downloadable XLSX/CSV document block. This corrects the
inventory's assumption that the release itself provided a single-year-age XLSX.

No reproducible publisher single-year-age artifact for this original reference
date/publication vintage was acquired. The public ADE catalog query did not
identify a corresponding national 2025 age table. The missing artifact remains
an access/discovery gap, not a zero population count. A subscribed API request
may assist discovery, but access alone does not establish that the required
original-vintage table is available.

The release explicitly says that Stats NZ revises the preceding six quarters.
A current 2025 observation may therefore differ from the 19 August 2025
publication. An eventual publisher export must carry its actual revision
vintage; it must not be relabeled as the original release. No latest-2026
population, projection, mirrored third-party bytes, or inferred age allocation
has been substituted. No national single-year-age source-package alias is
registered, and the national facts count added by S8 is **zero**.

## Validation and boundary

The partial S7 package passes source-package validation, exact source-cell
comparisons, geography mapping checks, source-semantic drift tests, consumer
contract validation, and an explicit one-package build. The bounded source
build records 85/85 lineaged facts and 2,265 parsed cells.

The source build warns that Axiom CLI canonical-concept validation did not run.
All mappings are source-label relations; this work claims neither Axiom
canonical certification nor calibration quality.

Microcosm retains responsibility for choosing constraints, reconciling source
universes, aging, imputing, constructing solver targets, and activating a
population. The 85 counts are available source facts, not proof that these
downstream steps have run. Independent source-fidelity and boundary review is
still required before landing.
4 changes: 2 additions & 2 deletions docs/pe-nz-source-checklist.md
Original file line number Diff line number Diff line change
Expand Up @@ -53,8 +53,8 @@ tests, and record a verified `raw/nz/...` R2 URI.

| Package from #176 | Artifact pinned | Package valid | `raw/nz` verified | Notes |
|---|---:|---:|---:|---|
| `stats_nz/subnational_population_estimates_2025` | [ ] | [ ] | [ ] | |
| `stats_nz/national_population_estimates_2025` | [ ] | [ ] | [ ] | |
| `stats_nz/subnational_population_estimates_2025` | [x] | [ ] | [x] | **Partial:** 85 validated workbook counts (16 regions + published NZ total; all ages/four broad age bands). Five-year-age × sex coverage is missing; its ADE structure request returned HTTP 401. Package-completion box stays unchecked. [Evidence and access notes](pe-nz-population-source-notes.md). |
| `stats_nz/national_population_estimates_2025` | [ ] | [ ] | [ ] | **Blocked:** no original-vintage single-year-age artifact acquired. The 19 August 2025 release points to Infoshare, not an XLSX. Landing-page evidence only is hash-pinned in `raw/nz`; it emits zero facts. [Evidence and access notes](pe-nz-population-source-notes.md). |
| `stats_nz/census_2023_households_by_region` | [ ] | [ ] | [ ] | |
| `stats_nz/census_2023_family_type` | [ ] | [ ] | [ ] | |
| `stats_nz/census_2023_ethnicity_age_region` | [ ] | [ ] | [ ] | |
Expand Down
Loading
Loading