Skip to content

Add review calibration bench#145

Open
FeeeeelixWong wants to merge 1 commit into
SCIBASE-AI:mainfrom
FeeeeelixWong:codex/review-calibration-15
Open

Add review calibration bench#145
FeeeeelixWong wants to merge 1 commit into
SCIBASE-AI:mainfrom
FeeeeelixWong:codex/review-calibration-15

Conversation

@FeeeeelixWong
Copy link
Copy Markdown

/claim #15

Summary

  • add a self-contained peer-review calibration bench for the Community & User Reputation System bounty
  • compare reviewer scores against consensus panels to detect leniency/severity drift, reproducibility blind spots, and calibration gaps
  • produce role/status labels, coaching actions, quarantine recommendations, digestable evidence, and a small demo video

Why this is scoped to issue #15

This focuses on the review-quality side of community reputation: calibrated reviewers, trust signals, coaching tasks, and reproducible evidence for reputation changes. It avoids the broader account reputation/appeals work already covered by other PRs.

Demo

  • review-calibration-bench/docs/demo.mp4
  • review-calibration-bench/docs/demo.svg

Verification

  • npm run check
  • npm test
  • npm run demo
  • ffprobe review-calibration-bench/docs/demo.mp4
  • git diff --check

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant