Rubric-graded benchmarks

5 tracked · updated August 16, 2026

These benchmarks grade free-text answers against rubrics that physicians or domain experts wrote, criterion by criterion, instead of scoring a picked letter. They reward completeness, accuracy, and safe framing at once, which is why frontier scores on them run low and move slowly. All of them descend from the idea that a health answer is a judgment to be audited, not a fact to be matched.

The other categories sit on the index: agentic and workflow benchmarks, documentation and coding benchmarks, safety benchmarks, knowledge and exam benchmarks, composite indices.