Method
Where each number comes from, how it is checked, and how the index is built.
On this page
Where numbers come from
Every result traces to one of three kinds of document. Bar colour on the charts is the lab; who ran each result is written beside it.
- Official leaderboard. The benchmark's owner ran the model and published the score on their own board or in their own paper.
- Independent run. A party that neither built the model nor owns the benchmark ran it and published the result with its configuration.
- Vendor-reported. The lab that built the model printed the score in its system card, model card or launch post. The lab chose the settings, and sometimes the grader.
Right now the 472 headline results split as 361 official leaderboard, 49 independent run and 62 vendor-reported. Each board chart can be filtered by who reported the number.
How a row is checked
A row is published only when all four of these are recorded:
- the document, preferring the first-party original over any page that repeats it;
- the number exactly as the document prints it, with its own rounding and its own percent sign or lack of one;
- a verbatim quote from the document that contains the number and shows which model and which measure it belongs to;
- a locator: page and table, section, figure, or the board's row and the date the board showed it.
The configuration the source states goes into the row as well: reasoning effort, harness, grader, whether a length adjustment was applied. It appears under the model name on every chart and in full in the row's sources. Two rows for the same model on one board almost always differ in configuration, and both are kept.
Printed form and chart form can differ. A card that prints 66.0 on a board scored 0 to 1 is charted at 0.660; the row's sources show both. Nothing is rounded beyond what the source printed.
When documents disagree
A first-party document outranks a page that mirrors it, and a benchmark owner's board outranks a secondary write-up of that board. The row keeps the number from its document of record.
Other documents that print the same result are listed in the row's sources, each with the number it printed: “also printed in” when it matches, “a different number in” when it does not. When the disagreement cannot be settled from the documents, the row carries the disputed marker.
Reading a board
Rank means position among the rows on that board and that measure only. Some boards list many more models than this site carries; where the board's full field size is known, pages say so as “N of M models on the board”. A rank here can be higher than the model's rank on the official board for that reason.
Vendor-reported rows on the same board may use different graders or settings from each other. The configuration line under each row is there so a reader can decide whether two rows are comparable for their purpose.
The index
The Clinical Benchmarks Index is this site's own calculation, made from the board results and nothing else. It gives one ordering for readers who want one; it is not a benchmark, and the boards remain the evidence.
- Per board. A model's best headline result counts on the board's own 0 to 100 scale: a board scored 0 to 1 is read times 100, a percentage as printed. A board's leader keeps its actual score instead of being stretched to 100.
- Board weight. Boards differ in how directly they measure clinical work, and a board whose leader is close to 100 has little room left to separate models. So boards that measure clinical work most directly count most (core 1, supporting 0.6, narrow 0.35), and boards near their ceiling count for less (headroom: 100 minus the leader's score, over 40, between 0.25 and 1); a board's weight is the two multiplied. Core: HealthBench Professional (physician-chosen workplace tasks graded on physician rubrics); First, Do NOHARM (v2) (how often and how badly consult recommendations could harm a patient); PhysicianBench (long physician workflows inside real EHR systems, checked by execution); Bedside Bench (physician-written cases on clinical reasoning and safety); APEX-1 GP (primary-care consults graded by practising GPs). Supporting: MedScribe (Vals AI) (visit notes, one documentation task); MAST (Medical AI Superintelligence Test) (a composite that overlaps boards already in the index); DAYJOB Health (long agent assignments mixing clinical, pharmacy and payer work); MLCR-AA (reasoning over long medical records, one documentation skill). Narrow: MedCode (Vals AI) (billing codes, an administrative task); MedXpertQA (MM) (multiple-choice exam questions over images); RCM-Bench (revenue cycle and billing, an administrative task). Near their ceiling now: Bedside Bench: leader 94.6, weight 0.25; RCM-Bench: leader 92.7, weight 0.088; MedScribe (Vals AI): leader 91.4, weight 0.15; MedXpertQA (MM): leader 81.5, weight 0.162.
- Easy and hard boards. Some boards run high for every model and some run low. Each board's offset is how far its scores run above or below the same models' scores on the other boards, fitted over every ranked model at once, and the offsets balance out to 0 across the boards. A result counts as its board score minus its board's offset, its adjusted result: a result on an easy board does not lift a model, and a result on a hard board does not sink it. Offsets now: Bedside Bench +39.2; RCM-Bench +36.5; MedScribe (Vals AI) +33.8; MedXpertQA (MM) +22.0; First, Do NOHARM (v2) +18.2; APEX-1 GP +11.9; MAST (Medical AI Superintelligence Test) +8.8; HealthBench Professional +5.0; MedCode (Vals AI) −3.3; PhysicianBench −16.4; MLCR-AA −26.5; DAYJOB Health −29.9.
- Across boards. The adjusted average is the weighted mean of the model's adjusted results, each counting by its board's weight, over the boards the model has results on. A board with no result is left out; it never counts as zero, so a model is not marked down for a board it has not been run on.
- Field average. A score from few boards rests on less evidence, so it is pulled toward the field average, 44.2, the average level of every ranked model. The field average counts as if it were 1.5 more boards, so its share shrinks as a model's boards grow; the table below gives it for each board count in use.
- Score. The adjusted average and the field average, each times its share, added and given to one decimal. Every figure in a working is the published one, so the working adds up.
| Boards | 1 | 2 | 3 | 4 | 5 | 6 |
|---|---|---|---|---|---|---|
| Field average share | 60% | 43% | 33% | 27% | 23% | 20% |
- Who is ranked. Every model with a result on at least one index board. Left out: harness plus model pairings, products, research systems, human and baseline references.
- Which boards. A board counts when it has results from at least 5 such models. The table below gives the reason for each board that does not count.
- Coverage. The pips beside each entry mark the index boards its score rests on, and the index views can be filtered to models with results on 2, 3 or 5 or more boards. A filter hides entries; it changes no score or rank.
Worked example
Claude Sonnet 5.5, 63.2, from 7 of 12 boards
Claude Sonnet 5.5 has results on 7 of 12 index boards. On RCM-Bench it scores 91.1, but that board runs +36.5 (easy board), so the result counts as 54.6. On DAYJOB Health it scores 34.0, and that board runs −29.9 (hard board), so the result counts as 63.9. The weighted average of its adjusted results is 67.2. With seven boards the field average of 44.2 takes 17.6% of the score, so the index is 63.2, rank 1.
- HealthBench Professional0.69269.2+5.0 easy board64.20.743 weightAnthropic; max effort; Opus 4.8 grader; safety classifiers enabled; length-adjusted; raw 77.1%
- MedScribe (Vals AI)91.1091.1+33.8 easy board57.30.15 weight, near its ceilingNear its ceiling: leader 91.4, weight 0.15model ID anthropic/claude-sonnet-5-5; temperature=1; max_output_tokens=128000; compute_effort=max
- MedCode (Vals AI)52.9252.9−3.3 hard board56.20.319 weightmodel ID anthropic/claude-sonnet-5-5; compute_effort=max; temperature=1; max_output_tokens=128000
- PhysicianBench63.263.2−16.4 hard board79.60.79 weightAnthropic harness; 100 tasks; pass@1; max effort; Opus 5 rubric grader; safety classifiers enabled
- RCM-Bench91.0591.1+36.5 easy board54.60.088 weight, near its ceilingNear its ceiling: leader 92.7, weight 0.088Score (avg@3), provider-default reasoning
- DAYJOB Health3434.0−29.9 hard board63.90.6 weightScore (pass@1 at 95%), reasoning adaptive/max
- MLCR-AA41.741.7−26.5 hard board68.20.6 weightoverall pass rate, Max, Default Fallback
- Adjusted average of 7 boards, by weight
- 67.2
- Field average, counts for 17.6% with 7 of 12 boards
- 44.2
- Index
- = 63.2
Score: the result on the board’s own 0 to 100 scale (dot; tick: the board’s leader). Board runs: how far the board’s scores sit above or below the same models’ scores on the other boards. Adjusted: the score minus that offset. The index is the adjusted average with the field average mixed in, more of it the fewer boards a model has. Each result links to its quote and page; the line under it is the setting it was measured with.
The method in use took effect on 31 October 2026. The version before it took the plain weighted average of a model's board scores and multiplied it down when the model had results on fewer boards. That marked new models down for boards they had not yet been run on, and let a model whose boards happened to be easy look stronger than it was.
| Board | Models | Leader, 0 to 100 | Weight | Offset | In the index |
|---|---|---|---|---|---|
| HealthBench Professional | 37 | 70.3 | 0.743 | +5.0 | Yes |
| MedScribe (Vals AI) | 95 | 91.4 | 0.15 | +33.8 | Yes, near its ceiling |
| MedCode (Vals AI) | 95 | 63.6 | 0.319 | −3.3 | Yes |
| MedXpertQA (MM) | 22 | 81.5 | 0.162 | +22.0 | Yes, near its ceiling |
| MAST (Medical AI Superintelligence Test) | 8 | 60.2 | 0.597 | +8.8 | Yes |
| First, Do NOHARM (v2) | 13 | 79.7 | 0.507 | +18.2 | Yes |
| PhysicianBench | 19 | 68.4 | 0.79 | −16.4 | Yes |
| HealthAgentBench | 0 | – | – | – | No: fewer than 5 models |
| CHI-Bench | 0 | – | – | – | No: fewer than 5 models |
| Bedside Bench | 15 | 94.6 | 0.25 | +39.2 | Yes, near its ceiling |
| RCM-Bench | 20 | 92.7 | 0.088 | +36.5 | Yes, near its ceiling |
| DAYJOB Health | 15 | 48.3 | 0.6 | −29.9 | Yes |
| APEX-1 GP | 12 | 73.0 | 0.674 | +11.9 | Yes |
| MLCR-AA | 27 | 41.7 | 0.6 | −26.5 | Yes |
128 models are ranked, 28 of them from one board.
Flags on single rows
Rows that have been matched against their document carry no marker. The other two states are always visible, on charts, tables and model pages:
- Unverified the number has been recorded but not yet matched against its source document.
- Disputed documents disagree about the number; the row's sources show each one.
What we never do
- Combine, average or rank scores from different benchmarks anywhere except the index, which is labelled as this site's own calculation wherever it appears. Each board ranks only its own rows.
- Add precision a source did not print, or convert a score to another scale and present it as the source's.
- Publish a number without its document. A row whose document is not yet recorded shows “source pending”.
What gets a board
A benchmark is tracked when it tests clinical work (reasoning, documentation, coding, EHR tasks, administrative work or patient safety) and has recent results for current models. Benchmarks that saturated or stopped reporting current models are listed with the reason at the foot of the boards page. Benchmarks about AI that people use for their own health are covered by the sister site linked in the footer.