Skip to content

Guard the Dask scatter in run_TPI so serial (client=None) runs work - #1212

Open
vahid-ahmadi wants to merge 1 commit into
PSLmodels:masterfrom
vahid-ahmadi:fix/tpi-serial-client-guard
Open

Guard the Dask scatter in run_TPI so serial (client=None) runs work#1212
vahid-ahmadi wants to merge 1 commit into
PSLmodels:masterfrom
vahid-ahmadi:fix/tpi-serial-client-guard

Conversation

@vahid-ahmadi

@vahid-ahmadi vahid-ahmadi commented Aug 28, 2026

Copy link
Copy Markdown
Contributor

Why

run_TPI(p, client=None) crashes. client.scatter is called unconditionally, so passing no Dask client raises AttributeError before the TPI loop is entered — which makes the serial fallback at TPI.py:1039 unreachable in exactly the case it exists for. SS.inner_loop already guards the equivalent block with if client:; TPI doesn't.

What changes after merging

Serial TPI runs work. Anyone calling run_TPI without a Dask cluster — scripts, notebooks, CI, small debugging runs — goes from a hard crash to a working solve. No behaviour changes when a client is supplied.

Change

Wraps the schema-backup / scatter / restore block in if client:. scattered_p_future is only read inside the if client: submit branch, so this is a guard plus an indent — no logic moves.

Evidence

New tests/test_TPI.py::test_run_TPI_serial_no_client seeds SS from the cached test_io_data pickles and monkeypatches inner_loop to raise a sentinel, asserting execution reaches the first serial household solve. No solve is run; ~1s, no Dask, so it is unmarked and guards this path in normal CI.

with the fix:      1 passed in 1.15s
fix stashed:       FAILED - AttributeError: 'NoneType' object has no attribute 'scatter'

Fixes the first half of #1211. The second half (SS re-scattering per residual evaluation) is #1214.

run_TPI called client.scatter(p, broadcast=True) unconditionally, so
run_TPI(p, client=None) raised AttributeError before the TPI loop was
entered -- making the serial fallback inside the loop unreachable in
exactly the case it exists for. SS.inner_loop already guards the
equivalent block with `if client:`; this matches that.

Adds a fast regression test that seeds baseline SS results from the
cached test_io_data pickles and monkeypatches TPI.inner_loop to raise a
sentinel, asserting only that execution reaches the first serial
household solve. No SS or TPI solve is performed, so it runs in ~1s and
needs no Dask cluster.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XCKMb1aicxYaeUC1us2nvF
@codecov-commenter

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 8.33333% with 11 lines in your changes missing coverage. Please review.
✅ Project coverage is 75.71%. Comparing base (a30defc) to head (9a2f823).

Files with missing lines Patch % Lines
ogcore/TPI.py 8.33% 11 Missing ⚠️
Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##           master    #1212      +/-   ##
==========================================
+ Coverage   74.07%   75.71%   +1.64%     
==========================================
  Files          22       22              
  Lines        5920     5921       +1     
==========================================
+ Hits         4385     4483      +98     
+ Misses       1535     1438      -97     
Flag Coverage Δ
unittests 75.71% <8.33%> (+1.64%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
ogcore/TPI.py 52.42% <8.33%> (+17.17%) ⬆️

... and 1 file with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@vahid-ahmadi

Copy link
Copy Markdown
Contributor Author

On the Codecov flag (8.33% patch coverage, 11 lines missing) — worth explaining, since it reads worse than it is.

The diff is a guard plus a re-indent, so Codecov counts the 11 re-indented lines of the schema-backup/scatter/restore block as new. Those lines only execute with a Dask client, and the non-local CI suite runs without one — so they show as uncovered even though they are byte-identical to what was already there and already ran in the local-marked tests.

The same report shows ogcore/TPI.py coverage going up 17.17% (35.25% → 52.42%), which is the real effect: before this change run_TPI(p, client=None) raised AttributeError before the loop, so the entire serial path was unreachable and uncovered. It now executes.

I have separately verified the serial path is not just reachable but numerically correct: with ENFORCE_SOLUTION_CHECKS = False, serial output compared against a real LocalCluster run across all 72 keys of the returned dict gives 0 mismatches at 1e-8. The regression test in this PR asserts reachability only, which is the narrow thing that was broken — I did not want to add a multi-minute solve to CI for it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants