Slamming: preference-driven in-hub variable fixing#765
Open
DLWoodruff wants to merge 8 commits into
Open
Conversation
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #765 +/- ##
==========================================
+ Coverage 73.73% 73.93% +0.20%
==========================================
Files 165 166 +1
Lines 21121 21346 +225
==========================================
+ Hits 15574 15783 +209
- Misses 5547 5563 +16 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
Design for PySP-style, preference-driven slamming as a new in-hub Slammer extension: a per-nonant directives file (by-name with wildcards, --slamming-directives-file) supplies can_slam/directions/priority, with a pluggable trigger (Phase 1: iteration-count; Phase 2: stall detection). Decoupled trigger/preferences/action layers, multistage, fully backward compatible (inactive unless the file is given). Distinct from the existing SlamMin/Max incumbent-finder spokes. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…e, sizes example Preference-driven in-hub slamming: a new Slammer extension that fixes non-converged nonants to user-chosen values to break stalls. Iteration-count trigger, one slam per event, sticky, multistage-ready, fully backward compatible (activated only by --slamming-directives-file). - mpisppy/extensions/slammer.py: Slammer extension + directives parser/matcher. Three decoupled layers (trigger/preferences/action) per the design. Directions lb/ub/nearest/anywhere/min/max; min/max reduced over comms[ndn]. - Name matching uses a simple glob (*/? wildcards, [] literal), NOT fnmatch: fnmatch reads Pyomo index brackets as character classes, so DoBuild[*] would not match DoBuild[Seattle]. Design doc reconciled. Multi-index names contain a comma and so must be CSV-quoted. - config.py: slamming_args() + checker() enforcing backward compatibility (a slam option without the file is a hard error). - cfg_vanilla.add_slammer(), wired from generic/extensions.py (activated only when the file is given); args registered in generic/parsing.py. - examples/sizes/config/slamming_directives.csv: translated from the historical PySP wwph.suffixes in the same dir. - doc/src/extensions.rst: slammer section + flag. - test_slammer.py wired into run_coverage.bash and test_pr_and_main.yml. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
User-facing extensions.rst now describes the slammer's directions (with a value-per-direction table), the iteration-count trigger, eligibility, and stickiness as behavior, instead of pointing at the design doc. Reword "user-chosen value" to "according to pre-specified user preferences" and remove phase-counter references from the slammer.py docstrings and comments (rationale stated directly). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…d check The docstring and extensions.rst said slamming "forces non-converged variables", which reads as a per-variable precondition. There is no convergence gate: eligibility is matched+can_slam, not-already-fixed, not-surrogate, has an applicable direction. Reword to describe the not-settling regime as the intended use and state explicitly that convergence is not tested per variable. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…swap should_slam's docstring claimed a future stall detector could replace the predicate "without touching the rest". Real stall/cycle detection is a significant redesign, not a clean predicate swap, so the docstring now just describes what the iteration-count trigger does. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…ow an error - Rename should_slam -> iteration_for_slam (it is specifically the iteration-count check, not a generic stall-detector seam). - A directive pattern matching no nonanticipative variable is now a hard error (was a rank-0 warning), and the message names the directives file. The per-pattern match is reduced across the hub (ROOT) before the check, so a pattern matched only on another rank is not falsely flagged and all ranks raise together (matters for multistage). - Tests, design doc §5, and extensions.rst reconciled; live-verified on sizes (valid file runs clean; bad pattern errors naming the file). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…onsumer project There is no Phase 2 of slamming. This design delivers the slamming mechanism; deciding *when* to slam (stall/cycle detection) is a separate future project that invokes the mechanism as a consumer (the action layer), not a stall detector swapped into the trigger predicate. Reword the status line (the mechanism is implemented, not "nothing yet"), §0 goals/non-goals, §4, §6, §7, §11 (now "Scope and future work"), and §13 accordingly; drop the "one-line swap / drop into the seam" framing. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Pyomo#759 removed reduced_costs_rho from the rho-setter mutual-exclusion sum in Config.checker(), so the test helper no longer needs to seed it (the deprecation guard treats an unseeded None value as falsy). Seed exactly the flags the sum still reads. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Adds slamming to mpi-sppy: forcing (fixing) selected nonanticipative
variables according to pre-specified user preferences while a decomposition hub
is running, to drive toward a feasible/converged incumbent when ordinary
convergence is slow. It is an
Extension(in the in-hub-fixing family withfixer.py,reduced_costs_fixer.py,relaxed_ph_fixer.py) and runs under anyPHBase-derived hub (PH, APH, Subgradient, FWPH).This is distinct from the existing
SlamMin/SlamMaxspokes, which arenon-destructive incumbent finders; the slammer alters the hub search per user
preference and is multistage-capable (the spokes are two-stage only).
Design:
doc/designs/slamming_design.md.How it's used
Activated only when a directives file is supplied, so a run with no slamming
options is byte-for-byte unchanged:
--slamming-directives-file <file>— activates the extension--slam-start-iter K— first iteration at which slamming may occur (default 1)--iters-between-slams M— cadence once started (default 1)Supplying the other options without the file is a hard error.
The directives file is a CSV keyed by nonant name with shell-style wildcards
(
*/?; index brackets[ ]are matched literally — deliberately notfnmatch, whose bracket semantics would makeDoBuild[*]fail to matchDoBuild[Seattle]). Each row gives a name pattern, an optionalcan_slam(
1/0), an ordered|-separated list ofdirectionsfrom{lb, ub, nearest, anywhere, min, max}(first applicable is used), and apriority(largest is slammed first; ties by name). Matching islast-match-wins; only variables matched by a
can_slamrule are ever slammed;a pattern that matches nothing is an error that names the file.
A worked example translated from PySP's
wwph.suffixesships atexamples/sizes/config/slamming_directives.csv.Behavior
choice is identical on every rank). Eligibility: matched +
can_slam, notalready fixed, not a surrogate, has an applicable direction. There is no
convergence gate (slamming an already-settled variable just pins it).
lb/ub/nearest/anywhereuse only rank-identical data (nocommunication);
min/maxdo one small reduction over the per-nodecommunicator, and only when such a slam actually fires.
Scope
This is the slamming mechanism. Deciding when to slam beyond the built-in
iteration-count trigger (genuine stall/cycle detection) is a separate future
project that would invoke this mechanism as a consumer; it is intentionally not
part of this PR.
Tests / docs
mpisppy/tests/test_slammer.py— 43 solver-free unit tests (parser, matcher,every direction, selection / eligibility / stickiness, trigger,
backward-compatibility, zero-match error). Wired into
run_coverage.bashand
test_pr_and_main.yml.doc/src/extensions.rst(slammer section + flag) and the design doc.mpiexec -np 2): slams fire oncadence, the cross-rank
maxreduction stays coherent, persistent-solverupdates work, and a bad pattern errors naming the file.
🤖 Generated with Claude Code