Skip to content

[MLPerf 6.1][GRPO] Update Qwen3.5 reference assets - #904

Open
jepio wants to merge 1 commit into
mlcommons:masterfrom
jepio:docs/qwen35-storage-dataset
Open

[MLPerf 6.1][GRPO] Update Qwen3.5 reference assets#904
jepio wants to merge 1 commit into
mlcommons:masterfrom
jepio:docs/qwen35-storage-dataset

Conversation

@jepio

@jepio jepio commented Sep 3, 2026

Copy link
Copy Markdown
Contributor

Summary

During dataset publishing to MLCommons storage we identified entries that had different behavior between architectures or were fundamentally broken. The result is that the number of datasets in the training and validation set slightly changed, without impacting convergence behavior of the benchmark.

We are also proposing a change to the reference configuration to reduce command and test timeouts from 300/180 to 60/60. These don't affect correctness/convergence but prevent training stalls and allow for better submissions.

Detailed changes in this PR are:

  • Documents MLCommons storage as the primary source for the qualified Qwen3.5 checkpoint, dataset, and SIF images.
  • Keeps the source preparation and build instructions as optional alternatives.
  • Updates the qualified dataset identities, task counts, runtime settings, and NeMo-RL submodule revision.
  • Links the NeMo-RL updated mlperf-training-qwen35-main branch.

Related compliance checker update:

@jepio
jepio requested review from a team as code owners September 3, 2026 11:26
@github-actions

github-actions Bot commented Sep 3, 2026

Copy link
Copy Markdown

MLCommons CLA bot All contributors have signed the MLCommons CLA ✍️ ✅

Signed-off-by: Jeremi Piotrowski <jpiotrowski@nvidia.com>
@jepio
jepio force-pushed the docs/qwen35-storage-dataset branch from d4622d4 to 49cba8f Compare September 3, 2026 14:07
Comment thread llm_post_training/RL
@RissyRan

RissyRan commented Sep 4, 2026

Copy link
Copy Markdown

Thanks for the change and proposal! Have 2 questions:

  1. Similar one about 251 samples in validation in Update Qwen3.5 validation sample count logging#475
  2. What're are actual train and validation test/command timeout in reference runs?

@ShriyaRishab

Copy link
Copy Markdown
Contributor

@jepio can you please answer the open questions and add some stats?

@jepio

jepio commented Sep 10, 2026

Copy link
Copy Markdown
Contributor Author

@ShriyaRishab

  1. i answered the 251 validation sample question here: Update Qwen3.5 validation sample count logging#475 (comment)
  2. before this PR the reference uses:
train test time 300s
validation test time 180s
train command timeout 300s
validation command timeout 300s

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants