chore(zarr-metadata): build 0.4.0 changelog

packages/zarr-metadata/CHANGELOG.md

@@ -2,6 +2,159 @@
 
 <!-- towncrier release notes start -->
 
+## 0.4.0 (2026-07-29)
+
+### Features
+
+- Added `zarr_metadata.model`: frozen-dataclass models (`ZarrV2ArrayMetadata`,
+  `ZarrV3ArrayMetadata`, `ZarrV2GroupMetadata`, `ZarrV3GroupMetadata`,
+  `ZarrV2ConsolidatedMetadata`, `ZarrV3ConsolidatedMetadata`, `ZarrV3NamedConfig`)
+  that are canonical, semantically lossless representations of Zarr metadata
+  documents, plus structural validators (`validate_*` / `is_*` / `parse_*`).
+  Every v3 extension point (data type, chunk grid, chunk key encoding, codecs,
+  storage transformers) is held as `ZarrV3NamedConfig`: a name, configuration,
+  and `must_understand` obligation; nothing is interpreted. On the wire, an
+  empty configuration with the default obligation uses the spec's plain-string
+  shorthand. Model fields are annotated with the role alias
+  `ZarrV3MetadataField` (today exactly `ZarrV3NamedConfig`), so annotations
+  convey the logical meaning and stay put if the spec adds another field form.
+
+  Validation is strict about what the types declare: v2 `dtype` / `order` /
+  `compressor` / `filters` / `dimension_separator` shapes and the fixed
+  `zarr_format` / `node_type` literals are all enforced. Every
+  `ValidationProblem` carries a machine-readable `kind`
+  (`missing_key` / `invalid_type` / `invalid_value` / `invalid_json`) so
+  consumers can dispatch on the failure mode without matching message strings,
+  and every ingestion failure — including missing store keys and undecodable
+  bytes in `from_key_value` — surfaces as `MetadataValidationError`. An
+  adversarial review added further structural checks: JSON booleans are not
+  accepted as dimension lengths, dimensions are non-negative,
+  `dimension_names` must have one entry per dimension of `shape`, `attributes`
+  and `configuration` values are JSON-checked recursively (like `fill_value`),
+  non-finite floats and non-standard JSON constants are rejected, abstract
+  mappings and sequences normalize to encoder-safe canonical containers,
+  v2 `shape` and `chunks` must have the same rank, non-null v2 filter pipelines
+  contain at least one filter, document `TypeIs` guards only narrow values that
+  already use the declared canonical containers,
+  and the inline consolidated-metadata envelope and entries are deep-validated
+  so the group validator's verdict always agrees with the model constructor.
+
+  The v3 models expose `must_understand_fields`: the subset of `extra_fields`
+  not explicitly waived with `must_understand: false` (fields are implicitly
+  must-understand per the spec). Readers discharge the spec's fail-to-open
+  duty by subtracting the extension names they recognize; the model only
+  partitions by obligation, since recognition is reader-specific.
+
+  Optional pydantic integration ships as `zarr_metadata.pydantic` (importing it
+  requires pydantic 2.13 or newer; the core package does not depend on it): one
+  `Annotated`
+  field type per model, validating raw documents through `from_json`, passing
+  core-model instances through unchanged, serializing via `to_json`, and
+  publishing JSON Schemas derived from private constrained document types that
+  mirror the independently expressible runtime rules. Cross-field cardinality
+  relations still require runtime validation. The instances are the core model
+  classes, so values interoperate freely with non-pydantic code.
+
+  `create_default` keeps its output self-consistent: overriding `shape` without
+  a chunk grid derives one regular chunk covering the array (v3
+  `chunk_shape == shape`; v2 `chunks == shape`) instead of silently keeping the
+  scalar default's 0-d grid.
+
+  A v2 `.zarray` that omits `dimension_separator` is interpreted with the v2
+  convention's default `"."` (the model previously normalized absence to `"/"`,
+  which would misaddress the chunks of real-world default-separator arrays).
+  The value is never null: absent, `"."`, or `"/"` are the only spellings.
+
+  Optional document keys use `UNSET` — a PEP 661 sentinel
+  (`typing_extensions.Sentinel`), usable directly in type expressions — never
+  `None`: in a model, `None` always corresponds to a JSON `null` in the
+  document (a v2 `compressor`, an unnamed dimension inside `dimension_names`),
+  and `UNSET` always means the key is absent. Checker note: ty types the
+  sentinel exactly; pyright needs `<= 1.1.404` until microsoft/pyright#11115
+  is fixed (this package's CI pins it); mypy users need a `cast` or
+  `type: ignore` at narrowing sites until python/mypy#21647 merges. This keeps semantically distinct spellings
+  distinct — an absent `dimension_names` ("there are no dimension names") and
+  an explicit `[null, null]` ("every dimension has a name, which is null") are
+  different documents and round-trip as such. The `consolidated_metadata: null`
+  written by a historical zarr-python bug is the one deliberate exception to
+  faithful round-tripping: those stores remain readable, but the bug spelling
+  is repaired to absence on read and never written back.
+
+  The v2 models treat the `.zattrs` file's presence as part of the store:
+  `attributes` is `UNSET` when no `.zattrs` file exists (and `to_key_value`
+  emits none), while an explicit empty `.zattrs` is `{}` and round-trips as a
+  file. Previously `to_key_value` always emitted `.zattrs`, silently adding a
+  file to stores that never had one.
+
+  The store-key `Literal` aliases (`ZarrV2ArrayMetadataStoreKey`,
+  `ZarrV2AttributesStoreKey`, ...) are exported from `zarr_metadata.model`
+  alongside their constants, and each `to_key_value` return type is keyed by
+  them, so the set of store keys a model can emit is visible in its signature.
+  `from_key_value` deliberately keeps `Mapping[str, bytes]` input: it accepts
+  any string-keyed store mapping and ignores unrelated keys.
+
+  `to_json` returns a document that shares no mutable state with the model:
+  every value that can hold a mutable container (attributes, configurations,
+  extra fields, v2 codec configurations, fill values, consolidated entries) is
+  deep-copied on the way out, so editing a serialized document can never
+  silently mutate the frozen model that produced it. ([#4119](https://github.com/zarr-developers/zarr-python/issues/4119))
+
+### Improved Documentation
+
+- `zarr-metadata` now has a standalone documentation site at
+  <https://zarr-metadata.readthedocs.io/>, with a comprehensive API reference
+  covering every public module, versioned by this package's release tags. The
+  package also gained a `justfile` collecting its development commands
+  (`test`, `lint`, `typecheck`, `docs-check`, `docs-serve`, `changelog-draft`),
+  which the package CI workflow now delegates to. ([#4208](https://github.com/zarr-developers/zarr-python/issues/4208))
+
+### Deprecations and Removals
+
+- The document (TypedDict) types are renamed to put the format version at the
+  front of the name and to mark the JSON-document form with a `JSON` suffix,
+  so a format version can never be misread as a class revision and the bare
+  entity names are reserved for the `zarr_metadata.model` dataclasses:
+
+  - `ArrayMetadataV2` → `ZarrV2ArrayMetadataJSON` (and `...Partial` accordingly)
+  - `ArrayMetadataV3` → `ZarrV3ArrayMetadataJSON` (and `...Partial` accordingly)
+  - `GroupMetadataV2` → `ZarrV2GroupMetadataJSON` (and `...Partial` accordingly)
+  - `GroupMetadataV3` → `ZarrV3GroupMetadataJSON` (and `...Partial` accordingly)
+  - `ConsolidatedMetadataV2` → `ZarrV2ConsolidatedMetadataJSON`
+  - `ConsolidatedMetadataV3` → `ZarrV3ConsolidatedMetadataJSON`
+  - `NamedConfigV3` → `ZarrV3NamedConfigJSON`
+  - `MetadataV3` → `ZarrV3MetadataFieldJSON` (the union of the bare-name and
+    named-configuration spellings of one metadata field)
+  - `ExtensionFieldV3` → `ZarrV3ExtensionField`
+  - `CodecMetadataV2` → `ZarrV2CodecMetadata`
+  - `DataTypeMetadataV2` → `ZarrV2DataTypeMetadata`
+  - `ArrayOrderV2` → `ZarrV2ArrayOrder`
+  - `ArrayDimensionSeparatorV2` → `ZarrV2ArrayDimensionSeparator`
+  - `ZArrayMetadata` → `ZarrV2ZArrayJSON` (the strict on-disk `.zarray` document)
+  - `ZGroupMetadata` → `ZarrV2ZGroupJSON` (the strict on-disk `.zgroup` document)
+  - `ZAttrsMetadata` → `ZarrV2ZAttrsJSON` (the `.zattrs` document)
+
+  The old names are removed, not aliased. The `zarr_metadata.pydantic` field
+  types take the bare entity names (`ZarrV3ArrayMetadata`, ...), matching the
+  model classes they validate into.
+
+  The conventions, stated once for future additions: CamelCase type names put
+  the format version first (`ZarrV2ArrayMetadataJSON`,
+  `ZarrV3ArrayMetadataStoreKey`), while SCREAMING_SNAKE constants and
+  snake_case functions put it last (`ARRAY_METADATA_STORE_KEY_V2`,
+  `validate_array_metadata_v3`). The `JSON` suffix marks a raw-document type
+  whose bare name is taken by (or reserved for) a `zarr_metadata.model`
+  dataclass; raw field-level types the models hold verbatim
+  (`ZarrV2CodecMetadata`, `ZarrV3ExtensionField`) keep their bare names.
+  Extension-entity types put the registered entity name first and end in
+  exactly one role suffix (`BloscCodecMetadata`, `Uint8DataTypeName`) — the
+  `V2` in `V2ChunkKeyEncodingMetadata` is that encoding's entity name, not a
+  format version, which is always spelled `ZarrV2`/`ZarrV3`. Every public
+  type name is checked against this grammar by
+  `tests/test_public_api.py::test_public_type_names_comply_with_naming_grammar`.
+
+  ([#4119](https://github.com/zarr-developers/zarr-python/issues/4119))
+
+
 ## 0.3.0 (2026-06-19)
 
 ### Deprecations and Removals