Matching silently substitutes the training mean on failure, and that score is fed to Optuna

[vahid-ahmadi]

Matching's hyperparameter tuning catches every exception and substitutes the training mean, with no log, no warning and no counter. That score is then handed to Optuna, so a parameter set under which matching fails on every record is scored as if it were a mean-predictor — and can be selected as best. autoimpute can then report Matching as the winning method on a score matching never produced.

The code

# microimpute/models/matching.py:642-646
except Exception:
    # If chunk fails, use mean of training data as prediction
    mean_val = X_train_fold[var].mean()
    y_pred = np.full(len(X_val_var), mean_val)

and again at matching.py:663-667. Neither handler calls self.logger. y_pred flows straight into compute_loss and thence into the Optuna objective.

A mean-predictor is not a neutral score — on a low-signal target it can beat a genuine matching fit on quantile loss, so the failure is not merely hidden but actively rewarded.

Related, in the predict path

matching.py:228-236 fills a failed chunk with NaN. That one does log (self.logger.warning("Chunk N failed: ... Filling with NaN values.")), but _process_matching_results returns a frame with silent NaN blocks and no summary of how many records were affected, so a caller has no way to know what fraction of the output is missing without checking themselves.

Suggested fix

• In the tuning path, let the exception propagate so Optuna prunes the trial. A failed trial should be pruned, not scored.
• If a fallback must stay, log at warning with the exception, count the affected records, and attach the count to the result.
• In the predict path, attach n_failed_records to the returned result so the caller can act on it.

Found during a pre-JOSS-submission audit (#201), confirmed verbatim by an independent reviewer.