Skip to content
This repository was archived by the owner on Aug 25, 2026. It is now read-only.

Support Optax extra arguments in StatefulTrainer - #143

Open
sylvesterkaczmarek wants to merge 2 commits into
google-deepmind:mainfrom
sylvesterkaczmarek:feat/trainer-optimizer-extra-args
Open

Support Optax extra arguments in StatefulTrainer#143
sylvesterkaczmarek wants to merge 2 commits into
google-deepmind:mainfrom
sylvesterkaczmarek:feat/trainer-optimizer-extra-args

Conversation

@sylvesterkaczmarek

Copy link
Copy Markdown

Summary

Fixes #126.

StatefulTrainer currently calls optimizer_def.update(grads, state, params) without any way to pass keyword-only extra arguments, which prevents using Optax GradientTransformationExtraArgs transformations that require additional signals.

This change:

  • wraps standard Optax transformations with optax.with_extra_args_support() so existing optimizers keep their current behavior;
  • adds a keyword-only optimizer_extra_args mapping to StatefulTrainer.step();
  • forwards that mapping only to optimizer_def.update(), while the existing **kwargs continue to go only to the loss function.

Testing

Added a regression test using a GradientTransformationExtraArgs update that requires a scale keyword argument and verifies a training step succeeds when the extra argument is supplied.

The existing deterministic MLP training test continues to exercise a normal optax.adam optimizer through the default path.

Signed-off-by: Sylvester Kaczmarek <assistant@SylvesterKaczmarek.com>
Signed-off-by: Sylvester Kaczmarek <assistant@SylvesterKaczmarek.com>
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Passing extra arguments to gradient transformations

1 participant