Skip to content

Add challenge evaluator calibration guard#403

Open
zergzorg wants to merge 1 commit into
SCIBASE-AI:mainfrom
zergzorg:codex/challenge-evaluator-calibration-18
Open

Add challenge evaluator calibration guard#403
zergzorg wants to merge 1 commit into
SCIBASE-AI:mainfrom
zergzorg:codex/challenge-evaluator-calibration-18

Conversation

@zergzorg
Copy link
Copy Markdown

Summary

  • Adds a self-contained challenge-evaluator-calibration-guard module for Scientific Bounty System issue Scientific Bounty System #18.
  • Validates synthetic seed coverage, evaluator role/conflict exclusions, severity drift, criterion drift, inter-rater spread, and score-normalization anchors before live scoring opens.
  • Emits deterministic JSON/Markdown/SVG outputs plus a short MP4 demo artifact for reviewer verification.

Validation

  • npm run check
  • npm test
  • npm run demo
  • ffprobe -v error -show_entries format=duration,size -show_entries stream=codec_name,width,height -of default=noprint_wrappers=1 challenge-evaluator-calibration-guard/reports/demo.mp4 -> H.264, 960x540, 5s
  • git diff --check
  • rg -n "(password|secret|wallet|paypal|bank|passport|tax|kyc|private key|api key|token)" challenge-evaluator-calibration-guard || true -> no matches

Non-overlap

This is a focused calibration seed readiness guard before live sponsor/reviewer scoring. It avoids the existing intake, rubric-readiness, scoring/arbitration, reviewer consensus, review-integrity, appeal, cancellation/no-award, benchmark leakage, workspace privacy, data-room access, license/dependency, human-subjects, withdrawal, and embargo-release slices.

Synthetic data only; no credentials, private identity data, payout details, or private dashboard data are included.

@algora-pbc /claim #18

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant