Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 12 additions & 1 deletion .github/workflows/docs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,12 @@
#
# Scope: Runs on push to main (deploys dev alias) and version tags (deploys stable).
# Uses the pixi docs environment for reproducible builds.
# Jobs: deploy (mike + mkdocs)
# Jobs: deploy (mike + zensical)
#
# Note: The docs are built by Zensical, and versioned with the Zensical maintainers'
# fork of mike (installed from git via the pixi docs feature, since it is not
# published to PyPI). The mike CLI and the gh-pages layout are unchanged, so
# already-published versions stay browsable.
name: Docs

on:
Expand Down Expand Up @@ -37,6 +42,12 @@ jobs:
with:
environments: docs

# `mike deploy` builds the site itself, but its internal build is not strict:
# `zensical build` exits 0 even when a markdown-exec code block raises, which
# would publish tracebacks. Build once up front with --strict to fail fast.
- name: 🔍 Verify docs build
run: pixi run -e docs build_docs

- name: Deploy docs
env:
GH_TOKEN: ${{ github.token }}
Expand Down
8 changes: 8 additions & 0 deletions .pre-commit-config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -7,13 +7,21 @@ ci:
autofix_commit_msg: "style(pre-commit.ci): auto fixes [...]"
autoupdate_commit_msg: "ci(pre-commit.ci): autoupdate"

# Vendored third-party files, kept byte-identical to what upstream ships. The SIL OFL
# text in particular should not be rewritten by the whitespace hooks.
exclude: ^brand/fonts/

repos:
- repo: https://github.com/pre-commit/pre-commit-hooks
rev: v5.0.0
hooks:
- id: trailing-whitespace
- id: end-of-file-fixer
- id: check-yaml
# mkdocs.yml uses `!!python/name:` tags for the pymdownx.emoji generator/index,
# which the safe YAML loader cannot construct. Excluded here rather than passing
# --unsafe, so every other YAML file (CI workflows included) stays fully checked.
exclude: ^mkdocs\.yml$
- id: check-toml
- id: check-merge-conflict
- id: debug-statements
Expand Down
5 changes: 5 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -41,6 +41,11 @@
- Apply the `io_retry` policy to the IO paths that bypass the zarr store: the pyarrow table backend's dataset load/write, the AnnData backend's direct local/fsspec writes, and the `fractal_fsspec_store` metadata probe (where 401 auth failures are translated to `NgioValueError` inside the retried call, so they are never retried — even in blanket mode).

### Documentation
- Rebuild the docs on [Zensical](https://zensical.org) instead of MkDocs + Material, and move every executed code block out of the Markdown into standalone scripts under `docs/snippets/`, included via `pymdownx.snippets` and run at build time. The five tutorial notebooks become Markdown pages, and CI builds with `--strict`.
- Apply a new design system to the docs and the README. The full theme in `docs/stylesheets/ngio.css`.
- Rewrite the copy to the design system's content conventions across every page, the nav and `llms.txt`, and add a landing page, a glossary, and API cross-references.
- Correct what the docs claim against what ngio actually does, most substantially on the table specification pages.
- Add `CODE_OF_CONDUCT.md` and `CITATION.cff`, and move `CONTRIBUTING.md` to the repository root, single-sourced into the docs so GitHub's community widgets pick them up.
- Add a "Configuration" getting-started page documenting the config file location (`~/.ngio/ngio_config.json` / `NGIO_CONFIG_PATH`), the `io_retry` policy (fields, backoff strategies, marker matching, snapshot-at-open vs read-at-call semantics), and its relationship to the lower-level `s3fs.custom_retry_markers` mechanism. `NgioConfig`, `RetryConfig`, and `get_config` are now listed in the top-level API reference.

### Fix
Expand Down
27 changes: 27 additions & 0 deletions CITATION.cff
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
cff-version: 1.2.0
title: "ngio: a Python library for OME-Zarr bioimage analysis"
message: "If you use this software, please cite it using these metadata."
type: software
authors:
- given-names: Lorenzo
family-names: Cerrone
affiliation: "BioVisionCenter, University of Zurich"
alias: lorenzocerrone
- given-names: Joel
family-names: Luethi
affiliation: "BioVisionCenter, University of Zurich"
alias: jluethi
repository-code: "https://github.com/BioVisionCenter/ngio"
url: "https://biovisioncenter.github.io/ngio/"
abstract: >-
ngio is a Python library that simplifies bioimage analysis workflows on OME-Zarr data.
It provides an object-based API for images, labels, tables, regions of interest and
high-content screening plates.
keywords:
- OME-Zarr
- OME-NGFF
- bioimage analysis
- microscopy
- Python
- zarr
license: BSD-3-Clause
21 changes: 10 additions & 11 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,9 +18,16 @@ test13 pytest # 3.13
dev lint # lint/format (pre-commit hooks)
dev ty check # type check (Ruff ty)
docs serve_docs # docs preview
test_nb # run notebooks
docs build_docs # build the site
docs test_snippets # run every docs snippet script standalone
docs clean_docs_data # drop generated ./data/*.zarr stores
```

## Docs
Snippet mechanics and build traps: see `docs/CLAUDE.md`. Snippet scripts repeat their
imports so each rendered block stands alone (hence the `docs/snippets/**` ruff
per-file-ignores).

## Config
- Python: 3.11–3.14
- Versioning: VCS via `hatch-vcs` (git tags, no hardcoded versions)
Expand All @@ -34,7 +41,6 @@ test_nb # run notebooks
- Don't restate types in prose — they live in the signature (`channel: The channel to load.`, not `channel (int): ...`)
- Sections: `Args`, `Returns`, `Raises`, `Example`, `Note`
- One-line summary, blank line, then body
- Code examples in fenced ` ```python ` blocks, not `>>>` doctests
- Terse: behavior and edge cases only, don't restate the signature
- Type checking via `ty`
- Internal modules prefixed with `_`
Expand All @@ -43,12 +49,5 @@ test_nb # run notebooks

## Changelog

- Follow the format in `CHANGELOG.md`
- **Always** update `CHANGELOG.md` when making code changes — add entries under the current `## [vX.Y.Z]` section (or create one if missing).
- Use these subsections (omit empty ones):
- `### Features` — new user-visible behaviour
- `### Fix` — bug fixes
- `### API Breaking Changes` — anything that breaks existing call sites (include before/after example)
- `### Chores` — internal refactors, dependency bumps, CI changes
- `### Documentation` — doc-only changes
- One bullet per logical change; use backticks for identifiers.
- Follow the format in `CHANGELOG.md`.
- **Always** update `CHANGELOG.md` when making code changes — add entries under the current `## [Unreleased]` section (or create one if missing).
59 changes: 59 additions & 0 deletions CODE_OF_CONDUCT.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,59 @@
# Code of Conduct

## Our Pledge

We as members, contributors, and leaders pledge to make participation in the ngio
community a harassment-free experience for everyone, regardless of age, body size,
visible or invisible disability, ethnicity, sex characteristics, gender identity and
expression, level of experience, education, socio-economic status, nationality, personal
appearance, race, religion, or sexual identity and orientation.

We pledge to act and interact in ways that contribute to an open, welcoming, diverse,
inclusive, and healthy community.

## Our Standards

Examples of behaviour that contributes to a positive environment:

- Demonstrating empathy and kindness toward other people
- Being respectful of differing opinions, viewpoints, and experiences
- Giving and gracefully accepting constructive feedback
- Accepting responsibility, apologising to those affected by our mistakes, and learning
from the experience
- Focusing on what is best for the overall community

Examples of unacceptable behaviour:

- The use of sexualised language or imagery, and sexual attention or advances of any kind
- Trolling, insulting or derogatory comments, and personal or political attacks
- Public or private harassment
- Publishing others' private information, such as a physical or email address, without
their explicit permission
- Other conduct which could reasonably be considered inappropriate in a professional
setting

## Enforcement Responsibilities

Community leaders are responsible for clarifying and enforcing these standards, and will
take appropriate and fair corrective action in response to any behaviour that they deem
inappropriate, threatening, offensive, or harmful.

## Scope

This Code of Conduct applies within all community spaces — the issue tracker, pull
requests, discussions, and any other project channel — and also applies when an
individual is officially representing the community in public spaces.

## Enforcement

Instances of abusive, harassing, or otherwise unacceptable behaviour may be reported to
the maintainers at the [BioVisionCenter](https://www.biovisioncenter.uzh.ch/en.html),
University of Zurich, or by opening a confidential report with a maintainer directly. All
complaints will be reviewed and investigated promptly and fairly. Community leaders are
obligated to respect the privacy and security of the reporter of any incident.

## Attribution

This Code of Conduct is adapted from the [Contributor
Covenant](https://www.contributor-covenant.org/), version 2.1, available at
<https://www.contributor-covenant.org/version/2/1/code_of_conduct.html>.
77 changes: 77 additions & 0 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,77 @@
# Contributing

Contributions are welcome! Please open an issue to discuss significant changes before opening a PR.

## Prerequisites

- [Pixi](https://pixi.sh) — manages all environments and dependencies
- Git

Install Pixi, then clone and set up:

```bash
git clone https://github.com/BioVisionCenter/ngio
cd ngio
pixi install
```

## Development

Work in the `dev` environment, which includes linters, type checker, and test dependencies:

```bash
pixi shell -e dev # activate shell
# or prefix individual commands:
pixi run -e dev <cmd>
```

## Running tests

```bash
pixi run -e test pytest # single run (Python 3.11)
pixi run -e test13 pytest # a specific Python version: test11, test12, test13, test14
```

Coverage is reported automatically. Pull requests run a reduced matrix — Linux on Python 3.11 and 3.13 — while `main` and tags run the full one: `test11`–`test14` across Linux, macOS, and Windows.

## Linting and formatting

```bash
pixi run -e dev lint # run all hooks on all files
```

This runs Ruff (lint + format), `typos` (spell check), YAML/TOML validation, and notebook output stripping. Hooks also run automatically on `git commit`.

## Commit conventions

Please follow [Conventional Commits](https://www.conventionalcommits.org/) — this is not enforced by a hook (yet), but helps maintain a clean history and enables automated changelog generation.

Examples:

```
feat: add support for multiscale labels
fix: correct axis order in NgffImage
docs: update contributing guide
```

## Opening a pull request

1. Fork the repo and create a branch from `main`.
2. Make your changes with tests where relevant.
3. Run `pixi run -e dev lint` and ensure all checks pass.
4. Open a PR against `main` with a clear description of what and why.

CI runs the linters and the test matrix automatically.

## Releasing *(maintainers only)*

Versions are derived from git tags via `hatch-vcs`. Use the Pixi bump tasks in the `dev` environment:

```bash
pixi run -e dev bump-patch # 0.5.7 → 0.5.8
pixi run -e dev bump-minor # 0.5.7 → 0.6.0
pixi run -e dev bump-major # 0.5.7 → 1.0.0
pixi run -e dev bump-alpha # → 0.6.0a1 (pre-release)
```

Append `-- --dry-run` to preview without creating a tag. Once tagged, CI builds and publishes to PyPI automatically.
100 changes: 51 additions & 49 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,74 +1,76 @@
# ngio - Next Generation file format IO
<p>
<picture>
<source media="(prefers-color-scheme: dark)" srcset="https://raw.githubusercontent.com/BioVisionCenter/ngio/main/docs/assets/logo-lockup-dark.svg">
<img alt="" width="320" src="https://raw.githubusercontent.com/BioVisionCenter/ngio/main/docs/assets/logo-lockup.svg">
</picture>
</p>

[![License](https://img.shields.io/pypi/l/ngio.svg?color=green)](https://github.com/BioVisionCenter/ngio/raw/main/LICENSE)
[![PyPI](https://img.shields.io/pypi/v/ngio.svg?color=green)](https://pypi.org/project/ngio)
[![Python Version](https://img.shields.io/pypi/pyversions/ngio.svg?color=green)](https://python.org)
[![CI](https://github.com/BioVisionCenter/ngio/actions/workflows/ci.yml/badge.svg)](https://github.com/BioVisionCenter/ngio/actions/workflows/ci.yml)
[![codecov](https://codecov.io/gh/BioVisionCenter/ngio/graph/badge.svg?token=FkmF26FZki)](https://codecov.io/gh/BioVisionCenter/ngio)

ngio is a Python library designed to simplify bioimage analysis workflows, offering an intuitive interface for working with OME-Zarr files.
**Next generation file format IO — a Python library for OME-Zarr bioimage analysis.**

## What is ngio?

ngio is built for the [OME-Zarr](https://ngff.openmicroscopy.org/) file format, a modern, cloud-optimized format for biological imaging data. OME-Zarr stores large, multi-dimensional microscopy images and metadata in an efficient and scalable way.

ngio's mission is to streamline working with OME-Zarr files by providing a simple, object-based API for opening, exploring, and manipulating OME-Zarr images and high-content screening (HCS) plates. It also offers comprehensive support for labels, tables and regions of interest (ROIs), making it easy to extract and analyze specific regions in your data.

## Key Features

### 🔍 Simple Object-Based API

- Easily open, explore, and manipulate OME-Zarr images and HCS plates
- Create and derive new images and labels with minimal boilerplate code

### 📊 Rich Tables and Regions of Interest (ROI) Support

- Tight integration with [tabular data](https://biovisioncenter.github.io/ngio/stable/table_specs/overview/)
- Extract and analyze specific regions of interest
- Store measurements and other metadata in the OME-Zarr container
- Extensible & modular allowing users to define custom table schemas and on disk serialization

### 🔄 Scalable Data Processing

- Powerful iterators for building scalable and generalizable image processing pipelines
- Extensible mapping mechanism for custom parallelization strategies
ngio is built for [OME-Zarr](https://ngff.openmicroscopy.org/), a cloud-optimised format
that stores large, multi-dimensional microscopy images and their metadata in an efficient,
scalable way. It provides an object-based API for opening, exploring and manipulating
OME-Zarr images and high-content screening (HCS) plates, along with labels, tables and
regions of interest (ROIs) for extracting and analysing specific regions of your data.

## Installation

You can install ngio via pip:
To install ngio, use whichever package manager you already work with — it is published on
both PyPI and conda-forge.

```bash
pip install ngio
pip install ngio # pip
uv add ngio # uv project (or: uv pip install ngio)
pixi add ngio # pixi, from conda-forge (--pypi for PyPI)
mamba install -c conda-forge ngio # mamba/conda
```

To get started check out the [Quickstart Guide](https://BioVisionCenter.github.io/ngio/stable/getting_started/0_quickstart/).

## Supported OME-Zarr versions

ngio supports OME-Zarr v0.4/v0.5. Support for version 0.6 and higher is planned for future releases.
Then work through the
[quickstart](https://biovisioncenter.github.io/ngio/stable/getting_started/0_quickstart/).

## Development Status
## Key features

ngio is under active development and is not yet stable. The API is subject to change, and bugs and breaking changes are expected.
We follow [Semantic Versioning](https://semver.org/). Which means for 0.x releases potentially breaking changes can be introduced in minor releases.
- **Object-based API** — open, explore and manipulate OME-Zarr images and HCS plates;
derive new images and labels with minimal boilerplate.
- **Tables and ROIs** — tight integration with [tabular
data](https://biovisioncenter.github.io/ngio/stable/table_specs/overview/), extensible
table schemas, and measurements stored alongside the image.
- **Scalable processing** — iterators for building pipelines that generalise from a single
ROI to a full plate, with a pluggable mapping mechanism for parallelisation.
- **Remote stores** — stream from S3 and other fsspec-backed sources, with a configurable
IO retry policy.
- **Supported OME-Zarr versions** — ngio supports OME-Zarr v0.4 and v0.5, backed by either Zarr v2 or v3 storage. Support for
v0.6 and later is planned.

### Available Features
## Versioning

- ✅ OME-Zarr metadata handling and validation
- ✅ Image and label access across pyramid levels
- ✅ ROI and table support
- ✅ Image processing iterators
- ✅ Streaming from remote sources
- ✅ Documentation and examples
ngio follows [semantic versioning](https://semver.org/): from 1.0 onwards the public API
is stable, and breaking changes are reserved for major releases.

### Upcoming Features
## Documentation

- Enhanced performance optimizations (parallel iterators, optimized io strategies)
Full documentation, including guides, tutorials and the API reference, is at
[biovisioncenter.github.io/ngio](https://biovisioncenter.github.io/ngio/). The worked
examples are executed when the site is built, so the code and the output you read are what
actually ran.

## Contributors
## Citing ngio

ngio is developed at the [BioVisionCenter](https://www.biovisioncenter.uzh.ch/en.html), University of Zurich, by [@lorenzocerrone](https://github.com/lorenzocerrone) and [@jluethi](https://github.com/jluethi).
If ngio contributes to work you publish, please cite it. See
[`CITATION.cff`](https://github.com/BioVisionCenter/ngio/blob/main/CITATION.cff) for the
current citation metadata.

## License
## Project

ngio is released under the BSD-3-Clause License. See [LICENSE](https://github.com/BioVisionCenter/ngio/blob/main/LICENSE) for details.
ngio is developed at the [BioVisionCenter](https://www.biovisioncenter.uzh.ch/en.html),
University of Zurich, by [@lorenzocerrone](https://github.com/lorenzocerrone) and
[@jluethi](https://github.com/jluethi). It is released under the BSD-3-Clause
[licence](https://github.com/BioVisionCenter/ngio/blob/main/LICENSE), and developed in the
open on [GitHub](https://github.com/BioVisionCenter/ngio) — issues and contributions
welcome.
Loading
Loading