Skip to content

Add MiniMax-M3 and MiniMax-M2.7 to benchmark model registry - #402

Open
octo-patch wants to merge 2 commits into
pinchbench:mainfrom
octo-patch:octo/20260728-model-add-recvq7Rt1ZFUKT
Open

Add MiniMax-M3 and MiniMax-M2.7 to benchmark model registry#402
octo-patch wants to merge 2 commits into
pinchbench:mainfrom
octo-patch:octo/20260728-model-add-recvq7Rt1ZFUKT

Conversation

@octo-patch

@octo-patch octo-patch commented Jul 28, 2026

Copy link
Copy Markdown

Reason: The scheduled OpenRouter benchmark registry lists only the obsolete MiniMax M2.1 and M2.5 models and is missing the current MiniMax-M3 and MiniMax-M2.7 targets. The custom endpoint path also assigned every model the same 200,000-token, text-only metadata.

This PR adds openrouter/minimax/minimax-m3 and openrouter/minimax/minimax-m2.7 to .github/benchmark-models.yml. The existing M2.1 and M2.5 entries remain so historical leaderboard runs stay reproducible.

It also updates custom OpenAI-compatible endpoint configuration for the official global and China base URLs. MiniMax-M3 and MiniMax-M2.7 now receive their model-specific context windows, input modalities, reasoning support, and pricing; unrelated custom endpoints keep the existing generic defaults.

Checks:

  • python3 tests/test_lib_agent_models.py (4 tests)
  • python3 tests/test_lib_agent_judge.py (3 tests)
  • ruff==0.15.6 on the touched and related test files
  • Parsed .github/benchmark-models.yml with PyYAML and verified 35 unique model entries
  • git diff --check

@kilo-code-bot

kilo-code-bot Bot commented Jul 28, 2026

Copy link
Copy Markdown
Contributor

Code Review Summary

Status: No Issues Found | Recommendation: Merge

Files Reviewed (3 files)
  • .github/benchmark-models.yml
  • scripts/lib_agent.py
  • tests/test_lib_agent_models.py
Previous Review Summary (commit 90a335f)

Current summary above is authoritative. Previous snapshots are kept for context only.

Previous review (commit 90a335f)

Status: No Issues Found | Recommendation: Merge

Files Reviewed (1 file)
  • .github/benchmark-models.yml

Reviewed by claude-sonnet-5 路 Input: 174 路 Output: 4.3K 路 Cached: 248.6K

@octo-patch

Copy link
Copy Markdown
Author

Updated the custom endpoint configuration for MiniMax-M3 and MiniMax-M2.7 to use the official global and China endpoint metadata, including context windows, input modalities, reasoning support, and pricing, and pushed commit 3ece9ac. I ran 7 focused and existing unit tests, Ruff 0.15.6, registry YAML validation, and git diff --check.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant