Skip to content

[Fix] Restore LongBench v2 answer prompt punctuation#2452

Open
yhzhu99 wants to merge 1 commit into
open-compass:mainfrom
yhzhu99:fix-longbenchv2-answer-format
Open

[Fix] Restore LongBench v2 answer prompt punctuation#2452
yhzhu99 wants to merge 1 commit into
open-compass:mainfrom
yhzhu99:fix-longbenchv2-answer-format

Conversation

@yhzhu99

@yhzhu99 yhzhu99 commented May 15, 2026

Copy link
Copy Markdown

Summary

  • Restore the missing closing quote and period in the LongBench v2 answer-format instruction.
  • Add a lightweight AST-based regression test for the prompt string.

Verified against the upstream LongBench prompt in THUDM/LongBench.

Fixes #2129

Tests

  • python3 -m pytest tests/datasets/test_longbenchv2_config.py -q
  • python3 -m flake8 tests/datasets/test_longbenchv2_config.py
  • python3 -m py_compile opencompass/configs/datasets/longbenchv2/longbenchv2_gen_75fbba.py tests/datasets/test_longbenchv2_config.py
  • git diff --check

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug] LongBench v2 dataset config is incorrect

2 participants