seqforge

Compile (arbitrary FASTQ files) + (unstructured human/DB metadata) into a validated, machine-independent dataset manifest, then into a runnable Snakemake config — for headless reprocessing of large collections of public sequencing datasets into a genomic-AI training corpus.

seqforge is a compiler, not a chatbot. Deterministic code owns every decision. The LLM has exactly two jobs: parse prose into span-verified assertions, and arbitrate ambiguity the deterministic layer has already flagged. Everything else is a verifier.

It produces two artifacts, and only the second is plural:

• manifest.yaml — what the data is. One per dataset, immutable, content-addressed.
• processing.yaml — what to do with it: genome, aligner, introns or not. Many per dataset.

Install

pip install seqforge

That gives you the compiler and the seqforge CLI. The two lab-only stages — compose against a real genome and kb e2e — additionally need the lab's liulab-genome and liulab-data, which are not on PyPI:

pip install "liulab-genome @ git+https://github.com/liuhlab/liulab-genome.git" \
            "liulab-data   @ git+https://github.com/liuhlab/liulab-data.git"

Inside the lab, pixi install already pulls both.

Develop

Everything runs through pixi (not pip/conda/venv):

pixi install                     # build environments
pixi run check                   # lint + fmt-check + typecheck + the three test lanes
pixi run test                    # the unit lane only
pixi run test-corpus             # the hermetic corpus only
pixi run test-external           # the lane needing binaries we do not own
pixi run -- pre-commit install   # once per clone — the fast hooks, not the suite

Most of the non-negotiable rules are enforced by tests, so pixi run check is the mechanism rather than a formality — CI runs it on every push and PR.

Where to look

For	Read
Using it — the tour, the tutorials, the concepts	https://liuhlab.github.io/seqforge/
Working on it — the rules, and where to read next	AGENTS.md (CLAUDE.md symlinks to it)
One decision per file, one paragraph	docs/adr/, and beside the code each governs
What a term means	CONTEXT-MAP.md, and one CONTEXT.md per context
What changed, and when	CHANGELOG.md
What is not yet built	the open issues