Clinical Benchmarks

HealthAgentBench

Agent harnesses completing realistic terminal-based healthcare tasks built from real clinical artifacts.

More about this boardLess

Agentic task success in realistic terminal-based healthcare environments built from real clinical artifacts; evaluates agent harnesses (Claude Code, Codex, Copilot) end to end, not bare models.

Paper: arXiv 2606.31179. The rows are agent harnesses rather than bare models, and the board is run by Microsoft Research; Copilot, Microsoft's own harness, does not top it.

Published by Microsoft Research, released Jul 2026. 54 tasks across 7 environments; 162 trials (3 attempts per task). Mean task success rate, 0-100%, higher better; cost per task also reported.

Headline metric
pooled task success rate across 3 attempts x 54 tasks (162 trials)
Rows on the board
12
Board last updated
27 Jul 2026
Publisher
Microsoft Research
Rows
12
Models
10 of 12 on the board
Released
Jul 2026
Measured to
Jul 2026
Unit
54 agentic tasks
Documents cited
1

Every result on HealthAgentBench, on the board's own scale

12 results, one column each, highest first. Mean task success rate, 0-100%, higher better; cost per task also reported.

Column colour is the model's lab. The table below says who ran each result.

Every published result, ranked on this board alone

12 rows. Higher is better. Bars run from 0 to 100; whiskers show a published interval. Select a row for its quote, page and document.

1Claude Code (Opus 5)Anthropic · $3.3/task55Board publisherJul 2026[1]
1 Claude Code (Opus 5) 55% $3.3
As printed
55%
Where
Leaderboard table (homepage), rank 1, Success Rate column
Setting
$3.3/task; harness+model evaluated jointly
Reported by
official leaderboard
Document
[1] HealthAgentBench leaderboard
2Codex (GPT-5.6-sol)OpenAI · $5.2/task45Board publisherJul 2026[1]
2 Codex (GPT-5.6-sol) 45% $5.2
As printed
45%
Where
Leaderboard table (homepage), rank 2, Success Rate column
Setting
$5.2/task
Reported by
official leaderboard
Document
[1] HealthAgentBench leaderboard
3Codex (GPT 5.5)OpenAI · $2.8/task42Board publisherJul 2026[1]
3 Codex (GPT 5.5) 42% $2.8
As printed
42%
Where
Leaderboard table (homepage), rank 3, Success Rate column
Setting
$2.8/task
Reported by
official leaderboard
Document
[1] HealthAgentBench leaderboard
4Copilot (Opus 4.8)Microsoft/Anthropic · $3.1/task36Board publisherJul 2026[1]
4 Copilot (Opus 4.8) 36% $3.1
As printed
36%
Where
Leaderboard table (homepage), rank 4, Success Rate column
Setting
$3.1/task
Reported by
official leaderboard
Document
[1] HealthAgentBench leaderboard
5Copilot (GPT 5.5)Microsoft/OpenAI · $2.6/task35Board publisherJul 2026[1]
5 Copilot (GPT 5.5) 35% $2.6
As printed
35%
Where
Leaderboard table (homepage), rank 5, Success Rate column
Setting
$2.6/task
Reported by
official leaderboard
Document
[1] HealthAgentBench leaderboard
6Claude Code (Opus 4.8)Anthropic · $4.0/task32Board publisherJul 2026[1]
6 Claude Code (Opus 4.8) 32% $4.0
As printed
32%
Where
Leaderboard table (homepage), rank 6, Success Rate column
Setting
$4.0/task
Reported by
official leaderboard
Document
[1] HealthAgentBench leaderboard
7Codex (GPT 5.4)OpenAI · $1.3/task28Board publisherJul 2026[1]
7 Codex (GPT 5.4) 28% $1.3
As printed
28%
Where
Leaderboard table (homepage), rank 7, Success Rate column
Setting
$1.3/task
Reported by
official leaderboard
Document
[1] HealthAgentBench leaderboard
8Claude Code (Opus 4.7)Anthropic · $4.8/task27Board publisherJul 2026[1]
8 Claude Code (Opus 4.7) 27% $4.8
As printed
27%
Where
Leaderboard table (homepage), rank 8, Success Rate column
Setting
$4.8/task
Reported by
official leaderboard
Document
[1] HealthAgentBench leaderboard
9Codex (GPT 5.3)OpenAI · $1.0/task22Board publisherJul 2026[1]
Agent | Success Rate | cost/task Codex (GPT 5.3) 22% $1.0
As printed
22%
Where
Homepage leaderboard, rank 9, Success Rate and cost/task; displayed 2026-09-30.
Setting
$1.0/task; Codex harness; mean success across 3 attempts × 54 tasks
Reported by
official leaderboard
Document
[1] HealthAgentBench leaderboard
10Claude Code (Opus 4.6)Anthropic · $4.1/task19Board publisherJul 2026[1]
10 Claude Code (Opus 4.6) 19% $4.1
As printed
19%
Where
Leaderboard table (homepage), rank 10, Success Rate column
Setting
$4.1/task
Reported by
official leaderboard
Document
[1] HealthAgentBench leaderboard
11Claude Code (Sonnet 4.6)Anthropic · $2.9/task17Board publisherJul 2026[1]
11 Claude Code (Sonnet 4.6) 17% $2.9
As printed
17%
Where
Leaderboard table (homepage), rank 11, Success Rate column
Setting
$2.9/task
Reported by
official leaderboard
Document
[1] HealthAgentBench leaderboard
12Codex (GPT 5.4 Mini)OpenAI · $0.6/task16Board publisherJul 2026[1]
12 Codex (GPT 5.4 Mini) 16% $0.6
As printed
16%
Where
Leaderboard table (homepage), rank 12, Success Rate column
Setting
$0.6/task
Reported by
official leaderboard
Document
[1] HealthAgentBench leaderboard
References

1 document supports this board

  1. 1.HealthAgentBench leaderboardofficial leaderboard, Microsoft Research (HealthAgentBench), 27 Jul 2026. Every result it supports