Skip to content

fix: minor correctness and hygiene fixes from the sync-pipeline audit - #4205

Open
d-v-b wants to merge 1 commit into
zarr-developers:mainfrom
d-v-b:fix/perf-pr-minor-cleanups
Open

fix: minor correctness and hygiene fixes from the sync-pipeline audit#4205
d-v-b wants to merge 1 commit into
zarr-developers:mainfrom
d-v-b:fix/perf-pr-minor-cleanups

Conversation

@d-v-b

@d-v-b d-v-b commented Jul 29, 2026

Copy link
Copy Markdown
Contributor

A variety of fixes exposed after claude did an audit of #3885

🤖 AI text below 🤖

  • Codec construction warnings (e.g. sharding's "disables partial reads") fired twice per array open, and on every decode/encode through the fused pipeline's async fallback. Re-constructions of an already-validated codec chain now go through codecs_from_list_unchecked, which validates structure without repeating first-construction advisory warnings; each warning fires exactly once per open under both pipelines.
  • concurrent_iter returned a lazy generator while its docstring promised eagerly scheduled tasks; it now materializes the task list so awaiting one at a time cannot serialize the batch.
  • A garbage codec_pipeline.max_workers value (e.g. from the environment) raised ValueError mid-read; it now warns and falls back to the default, consistent with tolerant handling of config input.
  • The as-completed pipeline helpers abandoned in-flight tasks when one failed, leaving stray background writes and "Task exception was never retrieved" warnings; failures now cancel and drain outstanding tasks.
  • Benchmarks: seed the data generator for reproducibility; fix a copy-pasted docstring.
  • Remove dead commented-out test blocks referencing the removed set_range API.

Assisted-by: ClaudeCode:claude-sonnet-5

Based on this claude-authored PR: d-v-b#259

Author attestation

  • I am a human, these are my changes, and I have reviewed and understood every change and can explain why each is correct.

TODO

  • Add unit tests and/or doctests in docstrings
  • Add docstrings and API docs for any new/modified user-facing classes and functions
  • New/modified features documented in docs/user-guide/*.md
  • Changes documented as a new file in changes/
  • GitHub Actions have all passed
  • Test coverage is 100% (Codecov passes)

@codecov

codecov Bot commented Jul 29, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 97.77778% with 1 line in your changes missing coverage. Please review.
✅ Project coverage is 93.91%. Comparing base (123268a) to head (6d30b53).

Files with missing lines Patch % Lines
src/zarr/core/codec_pipeline.py 97.56% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #4205      +/-   ##
==========================================
+ Coverage   93.90%   93.91%   +0.01%     
==========================================
  Files          91       91              
  Lines       12672    12693      +21     
==========================================
+ Hits        11900    11921      +21     
  Misses        772      772              
Files with missing lines Coverage Δ
src/zarr/core/array.py 97.85% <ø> (ø)
src/zarr/core/chunk_utils.py 98.27% <100.00%> (ø)
src/zarr/core/common.py 88.75% <100.00%> (ø)
src/zarr/core/codec_pipeline.py 96.39% <97.56%> (+0.16%) ⬆️
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

- Codec construction warnings (e.g. sharding's "disables partial reads")
  fired twice per array open, and on every decode/encode through the fused
  pipeline's async fallback. Re-constructions of an already-validated codec
  chain now go through codecs_from_list_unchecked, which validates
  structure without repeating first-construction advisory warnings; each
  warning fires exactly once per open under both pipelines.
- concurrent_iter returned a lazy generator while its docstring promised
  eagerly scheduled tasks; it now materializes the task list so awaiting
  one at a time cannot serialize the batch.
- A garbage codec_pipeline.max_workers value (e.g. from the environment)
  raised ValueError mid-read; it now warns and falls back to the default,
  consistent with tolerant handling of config input.
- The as-completed pipeline helpers abandoned in-flight tasks when one
  failed, leaving stray background writes and "Task exception was never
  retrieved" warnings; failures now cancel and drain outstanding tasks.
- Benchmarks: seed the data generator for reproducibility; fix a
  copy-pasted docstring.
- Remove dead commented-out test blocks referencing the removed set_range
  API.

Assisted-by: ClaudeCode:claude-sonnet-5
@d-v-b
d-v-b force-pushed the fix/perf-pr-minor-cleanups branch from ef0c196 to 6d30b53 Compare July 29, 2026 14:32
@d-v-b
d-v-b marked this pull request as ready for review July 29, 2026 14:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant