HealthAgentBench
Agent harnesses completing realistic terminal-based healthcare tasks built from real clinical artifacts.
Microsoft ResearchOfficial pagePaper
More about this boardLess
Agentic task success in realistic terminal-based healthcare environments built from real clinical artifacts; evaluates agent harnesses (Claude Code, Codex, Copilot) end to end, not bare models.
Paper: arXiv 2606.31179. The rows are agent harnesses rather than bare models, and the board is run by Microsoft Research; Copilot, Microsoft's own harness, does not top it.
Published by Microsoft Research, released Jul 2026. 54 tasks across 7 environments; 162 trials (3 attempts per task). Mean task success rate, 0-100%, higher better; cost per task also reported.
- Headline metric
- pooled task success rate across 3 attempts x 54 tasks (162 trials)
- Rows on the board
- 12
- Board last updated
- 27 Jul 2026
- Rows
- 12
- Models
- 10 of 12on the board
- Labs
- 4
- Last measured
- Jul 2026
- Leader
- 55Claude Code (Opus 5)
Ranking
Mean task success rate, 0-100%, higher better; cost per task also reported
- Anthropic
- OpenAI
All official leaderboard
12 of 12 rows
Rows and sources
Open a row for the quote, the page and the document.
1Claude Code (Opus 5) $3.3/task; harness+model evaluated jointly 55
Printed as 55%Official leaderboard, measured Jul 2026Configuration: $3.3/task; harness+model evaluated jointlyHealthAgentBench leaderboard official leaderboard, Microsoft Research (HealthAgentBench), 27 Jul 2026. Leaderboard table (homepage), rank 1, Success Rate column1 Claude Code (Opus 5) 55% $3.3
Every result from this document- Also printed in HealthAgentBench detailed results: 55%, Detailed results table, rank 1
2Codex (GPT-5.6-sol) $5.2/task 45
Printed as 45%Official leaderboard, measured Jul 2026Configuration: $5.2/taskHealthAgentBench leaderboard official leaderboard, Microsoft Research (HealthAgentBench), 27 Jul 2026. Leaderboard table (homepage), rank 2, Success Rate column2 Codex (GPT-5.6-sol) 45% $5.2
Every result from this document- Also printed in HealthAgentBench detailed results: 45%, Detailed results table, rank 2
3Codex (GPT 5.5) $2.8/task 42
Printed as 42%Official leaderboard, measured Jul 2026Configuration: $2.8/taskHealthAgentBench leaderboard official leaderboard, Microsoft Research (HealthAgentBench), 27 Jul 2026. Leaderboard table (homepage), rank 3, Success Rate column3 Codex (GPT 5.5) 42% $2.8
Every result from this document- Also printed in HealthAgentBench detailed results: 42%, Detailed results table, rank 3
- Also printed in HealthAgentBench: A Unified Benchmark Suite of Realistic Agentic Healthcare Environments for Challenging Frontier AI Agents (arXiv 2606.31179v1): 42%, p. 11, Figure 4 (pooled task success rate, ten agents)
4Copilot (Opus 4.8) $3.1/task 36
Printed as 36%Official leaderboard, measured Jul 2026Configuration: $3.1/taskHealthAgentBench leaderboard official leaderboard, Microsoft Research (HealthAgentBench), 27 Jul 2026. Leaderboard table (homepage), rank 4, Success Rate column4 Copilot (Opus 4.8) 36% $3.1
Every result from this document- Also printed in HealthAgentBench detailed results: 36%, Detailed results table, rank 4
- Also printed in HealthAgentBench: A Unified Benchmark Suite of Realistic Agentic Healthcare Environments for Challenging Frontier AI Agents (arXiv 2606.31179v1): 36%, p. 11, Figure 4 (pooled task success rate, ten agents)
5Copilot (GPT 5.5) $2.6/task 35
Printed as 35%Official leaderboard, measured Jul 2026Configuration: $2.6/taskHealthAgentBench leaderboard official leaderboard, Microsoft Research (HealthAgentBench), 27 Jul 2026. Leaderboard table (homepage), rank 5, Success Rate column5 Copilot (GPT 5.5) 35% $2.6
Every result from this document- Also printed in HealthAgentBench detailed results: 35%, Detailed results table, rank 5
- Also printed in HealthAgentBench: A Unified Benchmark Suite of Realistic Agentic Healthcare Environments for Challenging Frontier AI Agents (arXiv 2606.31179v1): 35%, p. 11, Figure 4 (pooled task success rate, ten agents)
6Claude Code (Opus 4.8) $4.0/task 32
Printed as 32%Official leaderboard, measured Jul 2026Configuration: $4.0/taskHealthAgentBench leaderboard official leaderboard, Microsoft Research (HealthAgentBench), 27 Jul 2026. Leaderboard table (homepage), rank 6, Success Rate column6 Claude Code (Opus 4.8) 32% $4.0
Every result from this document- Also printed in HealthAgentBench detailed results: 32%, Detailed results table, rank 6
- Also printed in HealthAgentBench: A Unified Benchmark Suite of Realistic Agentic Healthcare Environments for Challenging Frontier AI Agents (arXiv 2606.31179v1): 32%, p. 11, Figure 4 (pooled task success rate, ten agents)
7Codex (GPT 5.4) $1.3/task 28
Printed as 28%Official leaderboard, measured Jul 2026Configuration: $1.3/taskHealthAgentBench leaderboard official leaderboard, Microsoft Research (HealthAgentBench), 27 Jul 2026. Leaderboard table (homepage), rank 7, Success Rate column7 Codex (GPT 5.4) 28% $1.3
Every result from this document- Also printed in HealthAgentBench detailed results: 28%, Detailed results table, rank 7
- Also printed in HealthAgentBench: A Unified Benchmark Suite of Realistic Agentic Healthcare Environments for Challenging Frontier AI Agents (arXiv 2606.31179v1): 28%, p. 11, Figure 4 (pooled task success rate, ten agents)
8Claude Code (Opus 4.7) $4.8/task 27
Printed as 27%Official leaderboard, measured Jul 2026Configuration: $4.8/taskHealthAgentBench leaderboard official leaderboard, Microsoft Research (HealthAgentBench), 27 Jul 2026. Leaderboard table (homepage), rank 8, Success Rate column8 Claude Code (Opus 4.7) 27% $4.8
Every result from this document- Also printed in HealthAgentBench detailed results: 27%, Detailed results table, rank 8
- Also printed in HealthAgentBench: A Unified Benchmark Suite of Realistic Agentic Healthcare Environments for Challenging Frontier AI Agents (arXiv 2606.31179v1): 27%, p. 11, Figure 4 (pooled task success rate, ten agents)
9Codex (GPT 5.3) $1.0/task; Codex harness; mean success across 3 attempts × 54 tasks 22
Printed as 22%Official leaderboard, measured Jul 2026Configuration: $1.0/task; Codex harness; mean success across 3 attempts × 54 tasksHealthAgentBench leaderboard official leaderboard, Microsoft Research (HealthAgentBench), 27 Jul 2026. Homepage leaderboard, rank 9, Success Rate and cost/task; displayed 2026-09-30.Agent | Success Rate | cost/task Codex (GPT 5.3) 22% $1.0
Every result from this document10Claude Code (Opus 4.6) $4.1/task 19
Printed as 19%Official leaderboard, measured Jul 2026Configuration: $4.1/taskHealthAgentBench leaderboard official leaderboard, Microsoft Research (HealthAgentBench), 27 Jul 2026. Leaderboard table (homepage), rank 10, Success Rate column10 Claude Code (Opus 4.6) 19% $4.1
Every result from this document- Also printed in HealthAgentBench detailed results: 19%, Detailed results table, rank 10
- Also printed in HealthAgentBench: A Unified Benchmark Suite of Realistic Agentic Healthcare Environments for Challenging Frontier AI Agents (arXiv 2606.31179v1): 19%, p. 11, Figure 4 (pooled task success rate, ten agents)
11Claude Code (Sonnet 4.6) $2.9/task 17
Printed as 17%Official leaderboard, measured Jul 2026Configuration: $2.9/taskHealthAgentBench leaderboard official leaderboard, Microsoft Research (HealthAgentBench), 27 Jul 2026. Leaderboard table (homepage), rank 11, Success Rate column11 Claude Code (Sonnet 4.6) 17% $2.9
Every result from this document- Also printed in HealthAgentBench detailed results: 17%, Detailed results table, rank 11
- Also printed in HealthAgentBench: A Unified Benchmark Suite of Realistic Agentic Healthcare Environments for Challenging Frontier AI Agents (arXiv 2606.31179v1): 17%, p. 11, Figure 4 (pooled task success rate, ten agents)
12Codex (GPT 5.4 Mini) $0.6/task 16
Printed as 16%Official leaderboard, measured Jul 2026Configuration: $0.6/taskHealthAgentBench leaderboard official leaderboard, Microsoft Research (HealthAgentBench), 27 Jul 2026. Leaderboard table (homepage), rank 12, Success Rate column12 Codex (GPT 5.4 Mini) 16% $0.6
Every result from this document- Also printed in HealthAgentBench detailed results: 16%, Detailed results table, rank 12
- Also printed in HealthAgentBench: A Unified Benchmark Suite of Realistic Agentic Healthcare Environments for Challenging Frontier AI Agents (arXiv 2606.31179v1): 16%, p. 11, Figure 4 (pooled task success rate, ten agents)
Documents
3
- HealthAgentBench detailed resultsofficial leaderboard, Microsoft Research (HealthAgentBench), 27 Jul 2026Results it supports
- HealthAgentBench leaderboardofficial leaderboard, Microsoft Research (HealthAgentBench), 27 Jul 2026Results it supports
- HealthAgentBench: A Unified Benchmark Suite of Realistic Agentic Healthcare Environments for Challenging Frontier AI Agents (arXiv 2606.31179v1)paper, Microsoft Research (Liu, Zhang, Qin et al.), 30 Jun 2026Results it supports