(fix)fix ppdl init from noise and atom filtering. - #370
Conversation
## What breaks
Every protpardelle job in `run_experiments protpardelle` dies instantly:
```
ERROR | run_grid_search:301 - Job failed with exception: [Errno 2] No such file or directory: '.../wjq_139929586767296.results.pkl'
```
`get_pixi_env()` still had a hardcoded boltz/protenix/rf3 branch, so
`protpardelle` fell through to `raise ValueError("Unknown model:
protpardelle")`. That call is the first statement of
`run_guidance_queue_script`, so no worker ever reached `subprocess.run`
— which is why there are no `wjq_*.log` files and no `Running worker N:
...` line in the output.
`StructurePredictor.PROTPARDELLE`, the `protpardelle` pixi environment
in `pyproject.toml`, and `env = "protpardelle"` in
`experiments/protpardelle.toml` were all already in place. Only this
function was missed.
## Why the error was unreadable
The `ValueError` is raised inside a `ProcessPoolExecutor` child, so it
lands on the future. The collection loop never inspected the future — it
went straight to opening `wjq_*.results.pkl`, which the dead worker
never wrote, and the surrounding `except Exception` reported *that*
`FileNotFoundError`. The real exception was discarded.
## Changes
- `get_pixi_env()` now resolves through a `MODEL_PIXI_ENVS` mapping
keyed on `StructurePredictor`, so a newly added predictor cannot
silently miss an environment.
- The result collector checks `completed.exception()` first and logs it
with `log.opt(exception=...)`, reports a nonzero subprocess exit code,
and points at the `wjq_*.log` path when the results pickle is genuinely
missing.
## Tests
Two added to `tests/test_run_grid_search.py`: every `StructurePredictor`
resolves to a name in `VALID_PIXI_ENVS`, and an unknown model still
raises with the valid options listed.
**These have not been run** — `sampleworks` is not installable on the
machine this was written on. Please run `pixi run -e protpardelle-dev
tests -- -k grid_search` before merging.
## Follow-up, not in this PR
Once this lands, the `Running worker N: [...]` line will print the real
command. If it shows `pixi run -e protpardelle ...` instead of a direct
`.pixi/envs/protpardelle/bin/python`, the image has no baked
protpardelle env and `get_pixi_env_python()` is falling back to a
runtime solve on shared storage — works, but slow. That function
hardcodes `/app` and does not route through
`runner._pixi_project_dir()`, which is the one taught to honor
`RUNTIME_PIXI` on `michaelanzuoni/fix-runtime-pixi-project-dir`.
---------
Co-authored-by: xraymemory <me.anzuoni@gmail.com>
…that identify valid pixi environments
Review fixes for #352, targeted at its branch so they land together. ## 1. `_load_valid_pixi_envs()` can crash at import, or bind to the wrong manifest The walk-up from `schema.py` has two failure modes. **A wheel install makes `sampleworks.runs` unimportable.** The wheel target is `packages = ["src/sampleworks"]` with `analyses`/`experiments` force-included — `pyproject.toml` is not among them. The walk then reaches the filesystem root and raises at module scope, so `VALID_PIXI_ENVS = _load_valid_pixi_envs()` takes the whole package down with it. Simulated from a site-packages-shaped path: ``` wheel-style install -> FileNotFoundError: Could not find pyproject.toml while searching from schema.py ``` This does not bite today because the pixi environments install sampleworks editable, which leaves `schema.py` in the source tree. It bites the moment anything consumes a built wheel. **A `pyproject.toml` above the environment wins silently.** That is exactly the ACTL image layout — envs at `/app/.pixi/envs/<env>/lib/python3.12/site-packages/`, manifest at `/app/pyproject.toml`. Same simulation with a manifest above the env: ``` stale-manifest-above-env -> ('FOUND', '/…/app/pyproject.toml', ('stale_env',)) ``` So the environments validated here need not be the ones the runner resolves — and `runner._pixi_project_dir()` already treats `/app` versus a synced checkout as a real distinction. **Fix:** resolve `SAMPLEWORKS_PIXI_PROJECT_DIR` first, matching `_pixi_project_dir()`, then fall back to the walk for editable installs and plain checkouts. Return `()` when nothing is reachable and skip `Job.env` validation in that case, rather than making the module unimportable — the runner already fails with a clear message when an environment is genuinely missing (`_missing_prebuilt_env_message`). I did not import `_pixi_project_dir` directly because `runner` imports `schema`, so that would be circular. Happy to hoist it into a shared module instead if you prefer one implementation. Five tests added in `tests/runs/test_schema.py` covering the override, the missing-manifest fallback, and validation on both sides. Verified passing. ## 2. Stray warning-level debug line `log.warning(f"Running guidance job queue, job_queue_path: {job_queue_path}")` fires once per worker at WARNING, and the `log.info(f"Running worker {worker_num}: {cmd} …")` a few lines below already contains the same path inside `cmd`. Removed. ## 3. `BUNDLED` is a set, and it parametrizes tests `@pytest.mark.parametrize("name", BUNDLED)` over a set gives ordering that varies with `PYTHONHASHSEED`, so test IDs differ run to run: ``` seed=1 -> ['rf3_partial_chiral_off', 'boltz2_xrd', 'protpardelle', 'rf3_protenix'] seed=2 -> ['boltz2', 'full_8gpu', 'protpardelle', 'protenix_dual'] ``` It was a list before the move to `conftest`. Made it a tuple and used `set(BUNDLED)` at the two equality comparisons. ## Verified, no change needed `--guidance-start` defaulting to `-1` is consistent with `GuidanceConfig.guidance_start` and both consumers guard on `> 0` (`guidance_script_utils.py:584,613`). Wiring it into `build_args_for_process_pool` is a real fix on its own — without it the preset's `guidance-start = 400` was silently dropped. ## Left alone, flagging instead `PPDL_CC89_CHECKPOINT = "/mnt/diffuse-shared/marcus/model_params/weights/cc89_epoch415.pth"` is a personal path in a bundled preset. `rf3.toml` uses `/checkpoints/rf3_foundry_01_24_latest.ckpt`, matching what the image bakes, and #332 tracks doing the same for protpardelle. This preset will not resolve for anyone else or on a pod without that mount. Not changed since it is presumably what you are running against right now. ## Checks `ruff check` and `ruff format --check` clean on all changed files. I could not run the full suite locally — sampleworks is not installable on macOS for the model envs — so the new tests were run in isolation against `src/` on the path; CI covers the rest. Co-authored-by: xraymemory <me.anzuoni@gmail.com>
…pardelleWrapper.{featurize,init_from_noise}
📝 WalkthroughWalkthroughThe Pixi configuration adds development tools and pins Boltz dependencies. A shared atom validity mask now checks occupancy and coordinates. Protpardelle applies this mask consistently to atoms and atom37 mapping tensors. ChangesPixi dependency configuration
Atom validity filtering
Estimated code review effort: 3 (Moderate) | ~20 minutes Sequence Diagram(s)sequenceDiagram
participant featurize
participant atom37_mapping
participant get_valid_atom_mask
featurize->>atom37_mapping: derive atom37 indices from protein-chain atoms
featurize->>get_valid_atom_mask: compute validity mask
get_valid_atom_mask-->>featurize: return mask
featurize->>featurize: filter atoms and mapping tensors
Possibly related PRs
Suggested reviewers: 🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches 💡 1🛠️ Fix failing CI checks 💡
📝 Generate docstrings
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
There was a problem hiding this comment.
Actionable comments posted: 2
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@pyproject.toml`:
- Around line 21-22: Update the [tool.pixi.feature.dev] platforms configuration
to support the osx-arm64 platform after verifying its dependencies; if macOS
development is not supported, instead remove dev from boltz-osx or split the
feature so boltz-osx has a valid platform intersection and pixi.lock no longer
records an empty package set.
In `@src/sampleworks/eval/structure_utils.py`:
- Around line 246-249: Update get_valid_atom_mask() to use
np.isfinite(atom_array.coord).all(axis=-1) so both NaN and infinite coordinates
are rejected before returning the occupancy mask. Add a NumPy-style docstring
documenting the function’s purpose, accepted atom_array input, and returned
boolean mask.
🪄 Autofix
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro Plus
Run ID: a9b03f6d-b42c-4110-8081-a2abb0b495f4
⛔ Files ignored due to path filters (1)
pixi.lockis excluded by!**/*.lock
📒 Files selected for processing (3)
pyproject.tomlsrc/sampleworks/eval/structure_utils.pysrc/sampleworks/models/protpardelle/wrapper.py
| [tool.pixi.feature.dev] | ||
| platforms = ["linux-64"] |
There was a problem hiding this comment.
🩺 Stability & Availability | 🟠 Major | ⚡ Quick win
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
set -euo pipefail
python3 - <<'PY'
from pathlib import Path
import tomllib
manifest = tomllib.loads(Path("pyproject.toml").read_text())
pixi = manifest["tool"]["pixi"]
workspace_platforms = set(pixi["workspace"]["platforms"])
features = pixi["feature"]
failed = False
for name, definition in pixi["environments"].items():
selected = definition["features"] if isinstance(definition, dict) else definition
platforms = set(workspace_platforms)
for feature in selected:
platforms &= set(features.get(feature, {}).get("platforms", workspace_platforms))
if not platforms:
print(f"{name}: empty platform intersection from {selected}")
failed = True
raise SystemExit(failed)
PYRepository: diff-use/sampleworks
Length of output: 278
🏁 Script executed:
#!/bin/bash
set -euo pipefail
printf '%s\n' '--- pyproject.toml relevant sections ---'
nl -ba pyproject.toml | sed -n '1,55p;165,195p'
printf '%s\n' '--- lockfile candidates and boltz/gemmi references ---'
git ls-files | grep -E '(^|/)(pixi\.lock|.*lock.*)$' || true
rg -n -C 3 'boltz-osx|analysis-dev|protpardelle-dev|gemmi|boltz' . \
-g 'pixi.lock' -g 'pyproject.toml' -g '*.lock' || true
printf '%s\n' '--- available TOML parsers/runtimes ---'
python3 --version
command -v ruby || true
ruby -e 'begin; require "tomlrb"; puts "tomlrb"; rescue LoadError; end' 2>/dev/null || true
command -v pixi || trueRepository: diff-use/sampleworks
Length of output: 239
🌐 Web query:
Pixi manifest feature platforms environment platform intersection documentation
💡 Result:
In Pixi, the relationship between environments, features, and platforms is centered on how the workspace resolves dependencies for various architectures and system requirements. Workspace Platforms Every Pixi project defines its supported platforms in the [workspace] table [1][2]. These are the platforms for which Pixi will solve dependencies and create lockfile entries [3][2]. You can define them as simple conda subdirs (e.g., "linux-64") or as rich-platform entries that include virtual packages (like CUDA or specific glibc versions) [3][4]. Feature Platforms Features allow you to define modular sets of dependencies, tasks, and configurations that can be composed into environments [5][6]. A feature can optionally define its own platforms list [1][4]. - If a feature specifies platforms, it restricts where that feature is compatible [7][8]. - If a feature does not specify platforms, it inherits the platforms defined at the workspace level [4][7]. Environment Platforms (Intersection) An environment is a collection of features. When Pixi creates an environment, it determines the supported platforms by calculating the intersection of the platforms supported by all features included in that environment [7][6]. - For example, if an environment includes a 'default' feature (supporting all workspace platforms) and a 'cuda' feature (restricted to 'linux-64'), the resulting environment will only be compatible with 'linux-64' [8]. - This intersection mechanism ensures that the environment is only valid for platforms where all included features are compatible [7][6]. Target Specifiers Beyond features, you can use target specifiers ([target.]) in your manifest to override configuration specifically for a single platform [3][8]. These targets must always be a subset of the platforms defined in [workspace.platforms]; attempting to target an undefined platform will result in an error [3][8].
Citations:
- 1: https://pixi.prefix.dev/latest/reference/pixi_manifest/
- 2: https://pixi.prefix.dev/latest/conda_ecosystem/
- 3: https://pixi.prefix.dev/dev/workspace/multi_platform_configuration/
- 4: https://github.com/prefix-dev/pixi/blob/main/docs/reference/pixi_manifest.md
- 5: https://pixi.prefix.dev/latest/workspace/multi_environment/
- 6: https://prefix.dev/blog/introducing_multi_env_pixi
- 7: https://github.com/prefix-dev/pixi/blob/dac71154/crates/pixi_manifest/src/features_ext.rs
- 8: https://prefix-dev-pixi.mintlify.app/concepts/multi-platform
🏁 Script executed:
#!/bin/bash
set -euo pipefail
printf '%s\n' '--- pyproject.toml relevant sections ---'
awk 'NR<=55 || (NR>=165 && NR<=195) {printf "%6d %s\n", NR, $0}' pyproject.toml
printf '%s\n' '--- tracked lockfiles ---'
git ls-files | grep -E '(^|/)(pixi\.lock|.*lock.*)$' || true
printf '%s\n' '--- relevant manifest and lock references ---'
rg -n -C 4 'boltz-osx|analysis-dev|protpardelle-dev|gemmi|boltz' . \
-g 'pixi.lock' -g 'pyproject.toml' -g '*.lock' || true
printf '%s\n' '--- workspace and feature platform declarations ---'
rg -n -C 3 'platforms|^\[tool\.pixi\.(feature|environment|workspace)' pyproject.tomlRepository: diff-use/sampleworks
Length of output: 50376
🌐 Web query:
site:pixi.prefix.dev feature platforms environment intersection pixi
💡 Result:
In the context of the Pixi package manager, the environment's supported platforms are determined by the intersection of the platforms defined across all features that comprise that environment [1][2]. When you define an environment in a Pixi workspace, it is composed of one or more features (including the default feature) [3]. Each feature can explicitly declare a list of platforms it supports [1][4]. Because an environment must be compatible with all of its constituent features, Pixi calculates the final set of supported platforms for that environment by taking the intersection of the platforms defined by every individual feature included in it [1][2]. Key points regarding this behavior: - Default Platform Behavior: If a feature does not explicitly override the platforms, it inherits the platforms defined at the workspace level [1][2]. - Workspace Best Practice: It is generally recommended to define the workspace platforms to include all platforms that you intend to support across any of your environments [1][2]. - Binding Features: By explicitly defining platforms within a feature (e.g., binding a feature to a specific "cuda" variant), you can control the environment's final platform compatibility [1][5]. - Lock File Impact: Pixi uses these platform definitions to solve dependencies and store them in the pixi.lock file for each environment [1][5]. If the intersection of platforms across the selected features results in an empty set, the environment will not be supportable on any platform [1][2].
Citations:
- 1: https://pixi.prefix.dev/latest/reference/pixi_manifest/
- 2: http://pixi.prefix.dev/v0.62.0/reference/pixi_manifest/
- 3: https://pixi.prefix.dev/latest/tutorials/multi%5Fenvironment/
- 4: https://pixi.prefix.dev/latest/workspace/multi%5Fenvironment/
- 5: https://pixi.prefix.dev/latest/workspace/multi_platform_configuration/
🏁 Script executed:
#!/bin/bash
set -euo pipefail
printf '%s\n' '--- workspace platforms and all environments ---'
awk 'NR>=50 && NR<=75 {printf "%6d %s\n", NR, $0}'
awk 'NR>=189 && NR<=202 {printf "%6d %s\n", NR, $0}'
printf '%s\n' '--- lock metadata for affected environments ---'
awk '
/^ (analysis-dev|boltz-osx|protpardelle-dev):$/ {show=1; count=0}
show {printf "%6d %s\n", NR, $0; count++}
show && count>=14 {show=0}
' pixi.lock
printf '%s\n' '--- lock platform keys near affected environments ---'
rg -n -A 8 -B 2 '^( (analysis-dev|boltz-osx|protpardelle-dev):|platforms:|packages: \{\})' pixi.lock | head -120Repository: diff-use/sampleworks
Length of output: 207
🏁 Script executed:
#!/bin/bash
set -euo pipefail
python3 - <<'PY'
from pathlib import Path
lines = Path("pyproject.toml").read_text().splitlines()
workspace = []
features = {}
environments = {}
section = None
for raw in lines:
line = raw.strip()
if not line or line.startswith("#"):
continue
if line.startswith("[") and line.endswith("]"):
section = line[1:-1]
continue
if "=" not in line:
continue
key, value = (part.strip() for part in line.split("=", 1))
if section == "tool.pixi.workspace" and key == "platforms":
workspace = value
elif section == "tool.pixi.feature.dev" and key == "platforms":
features["dev"] = value
elif section == "tool.pixi.feature.boltz-osx" and key == "platforms":
features["boltz-osx"] = value
elif section == "tool.pixi.environments" and key in {"analysis-dev", "boltz-osx", "protpardelle-dev"}:
environments[key] = value
print("workspace_platforms =", workspace)
print("dev_platforms =", features.get("dev"))
print("boltz-osx_platforms =", features.get("boltz-osx"))
for name, definition in environments.items():
print(name, "features =", definition)
PYRepository: diff-use/sampleworks
Length of output: 478
Allow dev on its consumer platforms.
dev supports only linux-64, but boltz-osx includes it and requires osx-arm64. Pixi therefore gives boltz-osx an empty platform intersection, reflected by packages: {} in pixi.lock. If macOS development is supported, add osx-arm64 to dev after checking its dependencies. Otherwise, remove dev from boltz-osx or split the feature.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@pyproject.toml` around lines 21 - 22, Update the [tool.pixi.feature.dev]
platforms configuration to support the osx-arm64 platform after verifying its
dependencies; if macOS development is not supported, instead remove dev from
boltz-osx or split the feature so boltz-osx has a valid platform intersection
and pixi.lock no longer records an empty package set.
| def get_valid_atom_mask(atom_array: AtomArray | AtomArrayStack | Any) -> Any: | ||
| valid_atom_mask = atom_array.occupancy > 0 | ||
| valid_atom_mask &= ~np.any(np.isnan(atom_array.coord), axis=-1) | ||
| return valid_atom_mask |
There was a problem hiding this comment.
🎯 Functional Correctness | 🟠 Major | ⚡ Quick win
🧩 Analysis chain
🌐 Web query:
In NumPy 2.0, does numpy.isnan return False for positive and negative infinity, and does numpy.isfinite return False for those values?
💡 Result:
Yes, in NumPy 2.0, both functions behave as you described: 1. numpy.isnan returns False for both positive and negative infinity [1][2]. It is designed specifically to test for NaN (Not a Number) values [1][3]. 2. numpy.isfinite returns False for both positive and negative infinity [4][5]. It tests for finiteness, meaning it returns True only for numbers that are neither NaN nor infinity (positive or negative) [3][5].
Citations:
- 1: https://numpy.org/doc/2.0/reference/generated/numpy.isnan.html
- 2: https://numpy.org/doc/stable/reference/generated/numpy.isnan.html
- 3: https://numpy.org/doc/2.0/reference/routines.logic.html
- 4: https://numpy.org/doc/2.0/reference/generated/numpy.isfinite.html
- 5: https://numpy.org/doc/2.1/reference/generated/numpy.isfinite.html
🏁 Script executed:
#!/bin/bash
set -e
printf '%s\n' '--- target function and nearby helpers ---'
sed -n '220,265p' src/sampleworks/eval/structure_utils.py
printf '%s\n' '--- usages ---'
rg -n -C 4 'get_valid_atom_mask|valid_atom_mask|isfinite|isnan' src/sampleworks
printf '%s\n' '--- relevant file outline ---'
ast-grep outline src/sampleworks/eval/structure_utils.py --match 'get_valid_atom_mask' --view expandedRepository: diff-use/sampleworks
Length of output: 26380
🏁 Script executed:
#!/bin/bash
set -e
printf '%s\n' '--- preprocessing contract and call path ---'
sed -n '145,190p' src/sampleworks/eval/structure_utils.py
sed -n '520,555p' src/sampleworks/models/protpardelle/wrapper.py
printf '%s\n' '--- repository references to finite-coordinate requirements ---'
rg -n -C 3 'finite coordin|non-finite coordin|valid atom mask|valid_atom_mask' README.md docs src tests 2>/dev/null || true
printf '%s\n' '--- standalone NumPy behavior probe ---'
python3 - <<'PY'
import numpy as np
values = np.array([np.nan, np.inf, -np.inf, 0.0, 1.0])
print("values:", values.tolist())
print("isnan:", np.isnan(values).tolist())
print("isfinite:", np.isfinite(values).tolist())
coords = np.array([
[0.0, 1.0, 2.0],
[np.nan, 1.0, 2.0],
[np.inf, 1.0, 2.0],
[-np.inf, 1.0, 2.0],
])
occupancy = np.ones(4)
old_mask = (occupancy > 0) & ~np.any(np.isnan(coords), axis=-1)
finite_mask = (occupancy > 0) & np.isfinite(coords).all(axis=-1)
print("old_mask:", old_mask.tolist())
print("finite_mask:", finite_mask.tolist())
PYRepository: diff-use/sampleworks
Length of output: 12872
🏁 Script executed:
#!/bin/bash
set -e
printf '%s\n' '--- integration test expectations ---'
sed -n '380,430p' tests/integration/test_mismatch_integration.py
printf '%s\n' '--- full structure preprocessing coordinate path ---'
sed -n '190,230p' src/sampleworks/eval/structure_utils.py
printf '%s\n' '--- reward-input contract ---'
sed -n '35,85p' src/sampleworks/core/rewards/protocol.py
printf '%s\n' '--- relevant finite-coordinate tests ---'
sed -n '750,805p' tests/utils/test_atom_array_utils.py
printf '%s\n' '--- pure-Python edge-case sanity check (NumPy unavailable in sandbox) ---'
python3 - <<'PY'
import math
values = [float("nan"), float("inf"), float("-inf"), 0.0]
print("isnan:", [math.isnan(value) for value in values])
print("isfinite:", [math.isfinite(value) for value in values])
PYRepository: diff-use/sampleworks
Length of output: 9008
Reject non-finite coordinates and document get_valid_atom_mask().
Replace the NaN-only check with np.isfinite(atom_array.coord).all(axis=-1). Otherwise, infinite coordinates can reach reconciliation and model processing. Add the required NumPy-style docstring.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@src/sampleworks/eval/structure_utils.py` around lines 246 - 249, Update
get_valid_atom_mask() to use np.isfinite(atom_array.coord).all(axis=-1) so both
NaN and infinite coordinates are rejected before returning the occupancy mask.
Add a NumPy-style docstring documenting the function’s purpose, accepted
atom_array input, and returned boolean mask.
Source: Coding guidelines
Summary by CodeRabbit
Bug Fixes
Chores