fix: LagsAdder.features_added() over-reports lags outside the horizon window - #1023
Conversation
… window ## What LagsAdder.transform() only materializes lags that fall within a horizon's valid window (>= the horizon and <= history_available); a configured lag below the shortest horizon or beyond history_available is never added. But features_added() returned a name for every configured lag, so it advertised columns that transform() never creates -- a contract mismatch that can mislead any consumer relying on it (feature selection / column validation). ## Fix Return only the lags transform() actually produces: the union of the per-horizon valid lags (self._horizon_lags), matching the columns built in _transform_single_horizon / _transform_versioned. Same shape of fix as the merged OpenSTEF#965 (RollingAggregatesAdder) and OpenSTEF#980 (DatetimeFeaturesAdder). ## Tests Added regression tests (fail on the previous code, pass with the fix): single-horizon and multi-horizon cases assert features_added() equals the columns transform() actually adds, with out-of-window custom_lags dropped. ruff, format, ty clean; models transforms (64) and meta presets (29) pass. Signed-off-by: RAJVEER42 <irajveer.bishnoi2310@gmail.com>
|
Dear @Valyrian-Code, Thank you for your contribution. We'd like to clarify compliance with our contribution policy (https://github.com/OpenSTEF/.github/blob/main/CONTRIBUTING.md#ai-assisted-contributions). This project requires contributors to disclose any use of AI-assisted tools during the development of a pull request. Could you please confirm whether AI tools were involved in producing any part of this change set? This is a routine policy check and not a judgment of the contribution itself. |
|
Hi @LeandervdBijl, yes, same as I described on #1017: #1017 (comment). Same workflow here. I found the contract mismatch, used AI assistance while drafting the fix and tests, reviewed everything, and verified it locally (fail before, pass after, plus ruff, ty and the full transform suites). I take full responsibility for the change. |
bartpleiter
left a comment
There was a problem hiding this comment.
Nice improvement! Thank you for your contribution.
What
LagsAdder.transform()only materializes lags that fall within a horizon's valid window (>=the horizon and<=history_available) — a configured lag below the shortest horizon, or beyond the available history, is never added. Butfeatures_added()returned a name for every configured lag (self._lags), so it advertised columns thattransform()never creates.That is a contract mismatch: any consumer relying on
features_added()(feature selection, column validation) is told about columns that never materialize. Same defect shape as the recently merged #965 (RollingAggregatesAdder) and #980 (DatetimeFeaturesAdder).Concretely, with
history_available=10d,horizons=[2D],custom_lags=[1D, 3D, 30D],add_trivial_lags=False:transform()adds onlyload_lag_P3D(1D is below the 2D horizon, 30D exceeds the 10D history);features_added()previously returnedload_lag_P30D,load_lag_P3D,load_lag_P1D.Fix
Return only the lags
transform()actually produces — the union of the per-horizon valid lags (self._horizon_lags), which mirrors the columns built in_transform_single_horizon/_transform_versioned:No change to
transform(), thelags/horizon_lagsproperties, or any public field.Tests
Added regression tests (both fail on the previous code, pass with the fix):
features_added()equals the columnstransform()adds, with the out-of-windowcustom_lagsdropped;features_added()equals the union of columns created across horizons.ruff check,ruff format --check, andty checkare clean on the changed files. Theopenstef-modelstime-domain transforms suite (64) and theopenstef-metapresets suite (29 — the one internal consumer,forecasting_workflow.py, uses this as an exclude set where the change is behaviour-preserving) both pass.