Fix NaN handling and single-class fit in MovesManagementClassifier and PlannedGivingIntentScorer - #253
Merged
Conversation
…d PlannedGivingIntentScorer MovesManagementClassifier rejected NaN even though its HistGradientBoostingClassifier backend handles missing values natively, and fitting on a single-class y silently produced a classifier whose predict_proba returned only 1 column. It now allows NaN through fit/predict/predict_proba/action_priority and raises a clear ValueError when fewer than 2 classes are present. Also dropped a dead, already-redundant feature_names_in_ assignment and documented that action_priority's confidence is an uncalibrated max probability, not a calibrated likelihood. PlannedGivingIntentScorer's fixed cv=2 calibration surfaced scikit-learn's raw "Requesting 2-fold cross-validation..." error whenever a class had fewer than 2 examples; it now raises a clear ValueError before calibration runs. Also removed an unreachable branch in predict_intent_score and documented that NaN features are rejected.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
MovesManagementClassifier was rejecting NaN input even though its
HistGradientBoostingClassifier backend handles missing values natively.
Fitting it on a single-class y also silently produced a classifier whose
predict_proba returned only 1 column instead of failing loudly. This PR:
allow-nanhandling to fit/predict/predict_proba/action_priority andthe
allow_nanestimator tagfeature_names_in_assignment(
validate_dataalready sets it)action_priority's confidence is an uncalibrated maxprobability rather than a calibrated likelihood, and that class_weight
defaults to "balanced"
PlannedGivingIntentScorer calibrates with a fixed
cv=2, so any class withfewer than 2 examples made scikit-learn raise its raw "Requesting 2-fold
cross-validation..." error, which doesn't tell the caller what to do about
it. This PR:
classes or any class has fewer than 2 examples
predict_intent_score(predict_proba isnow guaranteed to return 2 columns, since fit rejects fewer than 2 classes)
backend doesn't support them
I also looked at whether
PlannedGivingIntentScorer.fitshould take anoptional
sample_weight, since GradientBoostingClassifier supports it. Itried adding it, but it breaks
check_estimator'scheck_sample_weight_equivalence_on_dense_datacheck: CalibratedClassifierCV'sinternal cv=2 folding isn't equivalent between a weighted sample and a
repeated one, so passing sample_weight through doesn't behave the way that
check requires. Rather than half-support it, I left
fit's signature as(X, y)only.How tested
Added regression tests for each behavior fix, confirmed they fail against
the pre-fix code and pass against the fix:
test_fit_accepts_nan_featuresandtest_fit_raises_clear_error_on_single_classin tests/test_moves.py
test_fit_raises_clear_error_when_a_class_has_too_few_examplesandtest_fit_raises_clear_error_on_single_classin tests/test_planned_giving.pyAlso removed
test_predict_intent_score_single_class_returns_zeros, whichonly exercised the now-deleted dead branch by mocking predict_proba's
return shape.
Ran locally with the project's editable dev install:
make ci: 2197 passed, 30 skipped, coverage 98.40% (floor 92%)make riskcov: risk-tier coverage 98% (floor 93%)pytest tests/test_sklearn_compliance.py -k "Moves or Planned": allpassing, including after confirming the
poor_scoreclassifier tag isn'tneeded for MovesManagementClassifier (removed it once tests passed without
it)
Changelog
Added an entry under
## [Unreleased]/### Fixedin CHANGELOG.md.