fix(dpmodel): keep output reductions on active backend - #5789
Conversation
Resolve the configured energy precision through the active array namespace before casting dense or graph fitting outputs. Use namespace casts and reductions throughout so Torch, JAX, and other array-API inputs do not call NumPy-only methods or leave their device. Add focused Torch regressions for dense extensive, dense masked-intensive, and ragged graph reductions. Fixes deepmodeling#5640. Coding-Agent: Codex Codex-Version: codex-cli 0.144.1 Model: gpt-5.6-sol Reasoning-Effort: xhigh
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Repository UI Review profile: CHILL Plan: Pro Plus Run ID: 📒 Files selected for processing (1)
🚧 Files skipped from review as they are similar to previous changes (1)
📝 WalkthroughWalkthroughThe output transforms resolve energy precision through the active array namespace before reducing values and masks. New Torch tests cover dense and graph extensive and intensive reductions, including masked and count-based denominator paths. ChangesBackend-aware output reductions
Estimated code review effort: 3 (Moderate) | ~20 minutes Suggested reviewers: 🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
There was a problem hiding this comment.
🧹 Nitpick comments (1)
source/tests/common/dpmodel/test_transform_output.py (1)
35-37: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low valueGracefully skip tests if Torch is not installed.
If
torchis an optional dependency for local testing, consider usingpytest.importorskipto gracefully skip the test instead of failing with aModuleNotFoundError.💡 Proposed refactor
Apply this change to the current test and the other tests in this file:
- import torch + torch = pytest.importorskip("torch")Also, remember to add
import pytestat the top of the file:import pytest🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@source/tests/common/dpmodel/test_transform_output.py` around lines 35 - 37, Update the Torch-dependent tests in this file, including test_dense_torch_extensive_reduction_stays_on_backend, to use pytest.importorskip for the torch import so they are skipped when Torch is unavailable; add the pytest import at module scope and preserve the existing test behavior when Torch is installed.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Nitpick comments:
In `@source/tests/common/dpmodel/test_transform_output.py`:
- Around line 35-37: Update the Torch-dependent tests in this file, including
test_dense_torch_extensive_reduction_stays_on_backend, to use
pytest.importorskip for the torch import so they are skipped when Torch is
unavailable; add the pytest import at module scope and preserve the existing
test behavior when Torch is installed.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Repository UI
Review profile: CHILL
Plan: Pro
Run ID: 7cb7c430-ddcb-4944-b3c9-d1b50ff26b2c
📒 Files selected for processing (3)
deepmd/dpmodel/model/edge_transform_output.pydeepmd/dpmodel/model/transform_output.pysource/tests/common/dpmodel/test_transform_output.py
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## master #5789 +/- ##
==========================================
- Coverage 78.88% 78.63% -0.25%
==========================================
Files 1054 1054
Lines 121756 121757 +1
Branches 4402 4406 +4
==========================================
- Hits 96043 95741 -302
- Misses 24147 24446 +299
- Partials 1566 1570 +4 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
Collect the backend-preservation regressions only when the optional Torch dependency is available. CI installs Torch, while lightweight source-test environments can now skip the file cleanly instead of failing during test execution. Coding-Agent: Codex Codex-Version: codex-cli 0.144.1 Model: gpt-5.6-sol Reasoning-Effort: xhigh
wanghan-iapcm
left a comment
There was a problem hiding this comment.
Correct fix, well aligned with the array-api discipline. The generic dpmodel reduction helpers were hard-coding NumPy ops (Tensor.astype, np.sum) and passing a NumPy dtype class into the active namespace, so they crashed/mis-typed on Torch -- exactly the pt_expt path via make_model. Resolving the energy precision through get_xp_precision and using xp.astype/xp.sum fixes it, and NumPy numerics are preserved (the int-vs-float64 mask divisor is value-identical). The three Torch regressions genuinely fail pre-fix (raw torch.Tensor has no .astype; the graph path rejects the NumPy dtype) and assert a float64 torch.Tensor result. Minor, non-blocking: only Torch is committed-tested here, but it's the representative namespace that exposes both bugs and the NumPy path stays covered by existing tests. LGTM.
Dismissing to re-review through the /code-review skill per process.
wanghan-iapcm
left a comment
There was a problem hiding this comment.
The fix itself is correct and complete (verified: get_xp_precision resolves the namespace-native dtype, NumPy numerics are preserved bit-identically, and the three new tests genuinely fail pre-fix). One blocking gap before merge: the mask is None reduction branches are untested in both files. Mutation testing confirms it -- reverting the fix on those exact lines while keeping the rest still passes all three tests, so a regression there would go undetected. Per the repo convention of covering every reachable branch (including both sides of a boolean), please add coverage for the no-mask intensive path (and ideally the graph extensive path). Requesting changes only for the test coverage; the code is good.
Exercise dense and graph intensive reductions without masks and the graph extensive path on Torch, covering the backend-native dtype branches requested in review. Coding-Agent: Codex Codex-Version: codex-cli 0.144.4 Model: gpt-5.6-sol Reasoning-Effort: xhigh
|
Possible reviewers based on changed lines, exact file history, and exact-file review history:
No review request was made automatically. Coding agent: Codex |
wanghan-iapcm
left a comment
There was a problem hiding this comment.
Both review comments are addressed with genuine regression tests, verified against HEAD. The dense intensive/no-mask branch (transform_output.py) and the graph n_node-fallback + extensive branches (edge_transform_output.py) are each now exercised by a Torch test that asserts the result stays a torch tensor in energy precision. Confirmed empirically that all three pre-fix idioms raise on a Torch tensor (Tensor.astype -> AttributeError, xp.astype with a NumPy dtype class -> TypeError, np.sum on a tensor -> TypeError), so every new test fails pre-fix and passes after -- closing the mutation-survival gap the original comments flagged. No residual numpy idioms remain in either transform; the fix is complete.
|
Addressed the CodeRabbit nitpick in Revalidated the current PR head: Coding agent: Codex |
Resolve the graph output conflict by combining backend-native precision casts with owned-node masking from master. Coding-Agent: Codex Codex-Version: codex-cli 0.144.6 Model: gpt-5.6-sol Reasoning-Effort: xhigh
Summary
Why existing tests missed this
There were no direct tests exercising the backend-agnostic dpmodel transform helpers with Torch arrays. Normal PyTorch and Paddle model tests call their backend-specific transform implementations instead, while the existing dpmodel graph tests use NumPy. As a result, neither Tensor.astype nor the NumPy dtype passed into the Torch namespace was reached.
The new tests invoke the generic dpmodel helpers directly and verify that results remain Torch tensors in the configured energy precision.
Validation
Closes #5640.
Coding agent: Codex
Codex version: codex-cli 0.144.1
Model: gpt-5.6-sol
Reasoning effort: xhigh
Summary by CodeRabbit
Bug Fixes
Tests