Fix Nextflow I/O wiring: channels, lib imports, script execution

[victor-fdz]

Summary

Fixes 6 critical issues in the Nextflow pipeline scaffold that prevented execution.

Problems Fixed

1. Script execution via python ${projectDir}/bin/... → direct execution

Nextflow adds bin/ to $PATH automatically. Calling scripts directly (e.g., label.py) is the correct pattern — ${projectDir} breaks inside containers.

2. Scripts not executable

All bin/*.py scripts now have +x permission (required by Nextflow).

3. lib/ imports broken in Nextflow work dirs

Scripts use from lib import ... with a sys.path hack based on __file__. In Nextflow's symlinked work dirs, the parent path no longer resolves to the project root. Fix: Stage lib/ as a path input and set PYTHONPATH explicitly.

4. Channel type mismatch (queue vs value)

FASTA and gene-list channels were Channel.fromPath(...) (queue — consumed once). Combined with the labeling output queue channel, Nextflow can't pair them. Fix: Channel.value(file(...)) for single-file inputs reused across all items.

5. Output glob too specific

LABEL_GENE_PAIRS output path was hardcoded to a pattern that didn't match actual filenames. Switched to *.tsv wildcard.

6. publishDir closure syntax

Simplified to standard publishDir "...", mode: ... form.

Housekeeping

• Added .nextflow/, .nextflow.log*, and work/ to .gitignore
• Removed accidentally committed Nextflow cache files

How to Run

nextflow run main.nf \
  --input data/merged_kinases_tpm_humanandmouse.tsv \
  --name test_kinases \
  --human_fasta data/promoter_kinases_human.fasta \
  --mouse_fasta data/promoter_kinases_mouse.fasta \
  -profile docker

[Copilot]

Pull request overview

This PR fixes several execution-blocking issues in the Nextflow pipeline scaffold by correcting channel semantics, staging lib/ into work dirs (and wiring it through processes), and switching process scripts to Nextflow’s bin/ execution model. It also adds missing Python entrypoint scripts and updates ignores to avoid committing Nextflow runtime artifacts.

Changes:

• Reworked workflow inputs to use value channels for single-use files and added a staged lib/ directory channel passed into all processes.
• Updated Nextflow modules to set PYTHONPATH from the staged lib/ and to execute bin/*.py scripts directly.
• Added multiple bin/ scripts (profiling/normalize/label/encode_split/model_processing/preprocess/compare_same_species_models) and updated .gitignore for Nextflow runtime paths.

Reviewed changes

Copilot reviewed 5 out of 13 changed files in this pull request and generated no comments.

Show a summary per file

File	Description
workflows/main.nf	Fixes channel wiring and stages lib/ for downstream Python imports.
modules/local/label_gene_pairs.nf	Adds project_lib input, sets PYTHONPATH, loosens output glob.
modules/local/encode_and_split_pairs.nf	Adds project_lib input, sets PYTHONPATH, adds optional --random-seqs.
modules/local/train_and_evaluate_model.nf	Adds project_lib input, sets PYTHONPATH, switches to direct script exec.
main.nf	Removes inline DSL directive (kept in nextflow.config).
bin/profiling.py	Adds profiling CLI script (but currently has CRLF line endings + minor help formatting issue).
bin/preprocess.py	Adds a “chain existing scripts” preprocessing entrypoint.
bin/normalize.py	Adds normalization benchmarking CLI script.
bin/model_processing.py	Adds model training/evaluation CLI script used by Nextflow module.
bin/label.py	Adds labeling CLI script used by Nextflow module.
bin/encode_split.py	Adds encoding/splitting CLI script used by Nextflow module (but currently exposes unsupported split modes + misleading defaults).
bin/compare_same_species_models.py	Adds a large standalone analysis script for same-species model comparison.
.gitignore	Ignores Nextflow runtime directories/logs and work directory.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.