Skip to content

Add Muon court training recipe - #159

Draft
abcamiletto wants to merge 1 commit into
Intellindust-AI-Lab:mainfrom
abcamiletto:agent/court-training-defaults
Draft

Add Muon court training recipe#159
abcamiletto wants to merge 1 commit into
Intellindust-AI-Lab:mainfrom
abcamiletto:agent/court-training-defaults

Conversation

@abcamiletto

Copy link
Copy Markdown

What changed

  • add a hybrid MuonAdamW optimizer that applies Muon to hidden matrix parameters and AdamW to embeddings, prediction heads, and non-matrix parameters
  • preserve parameter names when YAML optimizer groups are constructed
  • update Torch and Torchvision and migrate transforms and AMP to their current public APIs
  • add the final 768px, 14-epoch DEIMv2-X court-detection training recipe

Why

The court detector tuning showed that the hybrid optimizer and shorter 768px schedule were the best simple training recipe. The previous Torchvision transform hooks were private APIs and no longer run under the Torch version required for native Muon.

Impact

Existing AdamW configurations continue to work. The new court recipe selects MuonAdamW; optimizer checkpoints preserve both native optimizer states and can be resumed normally.

Validation

  • byte-compiled engine and train.py
  • Ruff error checks on all changed Python files
  • exercised ConvertPILImage, ConvertBoxes, and dynamic image/box padding on Torchvision 0.28
  • constructed the full DEIMv2-X court model and partitioned 119 parameters to Muon and 566 to AdamW
  • initialized and stepped the flat-cosine scheduler
  • exercised optimizer step, zero-grad, state save, and state restore
  • constructed an existing AdamW DEIMv2-X configuration to verify compatibility

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant