Clinical Benchmarks

First, Do NOHARM (v2)

How often, and how severely, model consultation recommendations contain potentially harmful errors.

More about this boardLess

Frequency and severity of potentially harmful errors in LLM-generated medical consultation recommendations (Numerous Options Harm Assessment for Risk in Medicine); primary-care-to-specialist consults.

Paper: arXiv 2512.01241. The v1 study found potential for severe harm in up to 24.6 percent of directly applied recommendations, with errors of omission behind more than 80 percent of the severe cases.

Published by Stanford/Harvard-led consortium (50+ researchers incl. 29 board-certified physicians); hosted by ARISE, released Dec 2025. 1,100 consultation cases, 10 specialties, 12,747 expert annotations on 4,249 management options. Percentage safety score, higher better.

Headline metric
F1 (Weighted)
Board marked as preview
Yes
Board last updated
15 Aug 2026
Models in the full dataset
43
Models in the board's default view
19
Publisher
Stanford/Harvard-led consortium (50+ researchers incl. 29 board-certified physicians); hosted by ARISE
Rows
17
Models
17 of 19 on the board
Released
Dec 2025
Measured to
Sep 2026
Unit
1,100 consultation cases
Documents cited
2

Every result on First, Do NOHARM (v2), on the board's own scale

17 results, one column each, highest first. Percentage safety score, higher better.

Column colour is the model's lab. The table below says who ran each result.

Every published result, ranked on this board alone

17 rows. Higher is better. Bars run from 0 to 100; whiskers show a published interval. Select a row for its quote, page and document.

1LiSA 2.5AMBOSS · First Do NOHARM v2 overall86.2Board publisherSep 2026[1]
First Do NOHARM v2 overall metric across 19 models 1LiSA 2.5RAGAMBOSS 86.2%
As printed
86.2%
Where
First Do NOHARM v2, overall leaderboard; LiSA 2.5 row; displayed 2026-09-30.
Setting
First Do NOHARM v2 overall; preview; RAG clinical product; run date unpublished
Reported by
official leaderboard
Document
[1] ARISE MAST technical leaderboard
2Doximity Ask 6.1Doximity · First Do NOHARM v2 overall84.5Board publisherSep 2026[1]
First Do NOHARM v2 overall metric across 19 models 2Doximity Ask 6.1RAGDoximity 84.5%
As printed
84.5%
Where
First Do NOHARM v2, overall leaderboard; Doximity Ask 6.1 row; displayed 2026-09-30.
Setting
First Do NOHARM v2 overall; preview; RAG clinical product; run date unpublished
Reported by
official leaderboard
Document
[1] ARISE MAST technical leaderboard
3OpenEvidenceOpenEvidence · First Do NOHARM v2 overall80.0Board publisherSep 2026[1]
First Do NOHARM v2 overall metric across 19 models 3OpenEvidenceRAGOpenEvidence 80.0%
As printed
80.0%
Where
First Do NOHARM v2, overall leaderboard; OpenEvidence row; displayed 2026-09-30.
Setting
First Do NOHARM v2 overall; preview; RAG clinical product; run date unpublished
Reported by
official leaderboard
Document
[1] ARISE MAST technical leaderboard
4Glass 5.6 MaxGlass Health · First Do NOHARM v2 overall79.7Board publisherSep 2026[1]
First Do NOHARM v2 overall metric across 19 models 4Glass 5.6 MaxRAGGlass Health 79.7%
As printed
79.7%
Where
First Do NOHARM v2, overall leaderboard; Glass 5.6 Max row; displayed 2026-09-30.
Setting
First Do NOHARM v2 overall; preview; RAG clinical product; run date unpublished
Reported by
official leaderboard
Document
[1] ARISE MAST technical leaderboard
4Muse Spark 1.1Meta79.7Board publisherAug 2026[2]
5Muse Spark 1.1Meta 79.7%
As printed
79.7%
Where
arise-ai.org/mast/technical, 'First Do NOHARM v2 overall metric across 19 models' ranking (Latest Flagships view)
Reported by
official leaderboard
Document
[2] MAST technical leaderboard (First Do NOHARM v2 and per-benchmark results)
6Claude Opus 5Anthropic · v2 run on ARISE74.6Board publisherAug 2026[2]
6Claude Opus 5Anthropic 74.6%
As printed
74.6%
Where
arise-ai.org/mast/technical, 'First Do NOHARM v2 overall metric across 19 models' ranking (Latest Flagships view), plus Model Leaderboard SAFETY column
Setting
v2 run on ARISE; 19 models on the board
Reported by
official leaderboard
Document
[2] MAST technical leaderboard (First Do NOHARM v2 and per-benchmark results)
7Kimi K3Moonshot AI74.0Board publisherAug 2026[2]
7Kimi K3OSSMoonshot AI 74.0%
As printed
74.0%
Where
arise-ai.org/mast/technical, 'First Do NOHARM v2 overall metric across 19 models' ranking (Latest Flagships view), plus Model Leaderboard SAFETY column
Reported by
official leaderboard
Document
[2] MAST technical leaderboard (First Do NOHARM v2 and per-benchmark results)
8GPT-5.6 SolOpenAI70.1Board publisherAug 2026[2]
8GPT-5.6 SolOpenAI 70.1%
As printed
70.1%
Where
arise-ai.org/mast/technical, 'First Do NOHARM v2 overall metric across 19 models' ranking (Latest Flagships view), plus Model Leaderboard SAFETY column
Reported by
official leaderboard
Document
[2] MAST technical leaderboard (First Do NOHARM v2 and per-benchmark results)
9GPT-5.5OpenAI70.0Board publisherAug 2026[2]
9GPT-5.5OpenAI 70.0%
As printed
70.0%
Where
arise-ai.org/mast/technical, 'First Do NOHARM v2 overall metric across 19 models' ranking (Latest Flagships view), plus Model Leaderboard SAFETY column
Reported by
official leaderboard
Document
[2] MAST technical leaderboard (First Do NOHARM v2 and per-benchmark results)
10GPT-5OpenAI · from the Model Leaderboard…68.6Board publisherAug 2026[2]
10 GPT-5 72.6%±2.3 68.6%±4.5 80.3%±4.4 30.1%±9.0 44.2%±2.5 45.4%±1.3 73.1%±3.9 73.7%±2.7 46.6%±0.0
As printed
68.6%
Where
arise-ai.org/mast/technical, Model Leaderboard (Top 10 shown), row 10, SAFETY column
Setting
from the Model Leaderboard SAFETY column (NOHARM v2 F1 weighted, shown with CI); not in the Latest Flagships ranking
Reported by
official leaderboard
Document
[2] MAST technical leaderboard (First Do NOHARM v2 and per-benchmark results)
11Claude Fable 5Anthropic65.0Board publisherAug 2026[2]
10Claude Fable 5Anthropic 65.0%
As printed
65.0%
Where
arise-ai.org/mast/technical, 'First Do NOHARM v2 overall metric across 19 models' ranking (Latest Flagships view)
Reported by
official leaderboard
Document
[2] MAST technical leaderboard (First Do NOHARM v2 and per-benchmark results)
12Gemini 3.1 ProGoogle62.6Board publisherAug 2026[2]
11Gemini 3.1 ProGoogle 62.6%
As printed
62.6%
Where
arise-ai.org/mast/technical, 'First Do NOHARM v2 overall metric across 19 models' ranking (Latest Flagships view), plus Model Leaderboard SAFETY column
Reported by
official leaderboard
Document
[2] MAST technical leaderboard (First Do NOHARM v2 and per-benchmark results)
13Gemini 2.5 ProGoogle61.9Board publisherAug 2026[2]
12Gemini 2.5 ProGoogle 61.9%
As printed
61.9%
Where
arise-ai.org/mast/technical, 'First Do NOHARM v2 overall metric across 19 models' ranking (Latest Flagships view)
Reported by
official leaderboard
Document
[2] MAST technical leaderboard (First Do NOHARM v2 and per-benchmark results)
14Qwen3.5 397B A17BAlibaba61.1Board publisherAug 2026[2]
13Qwen3.5 397B A17BOSSAlibaba 61.1%
As printed
61.1%
Where
arise-ai.org/mast/technical, 'First Do NOHARM v2 overall metric across 19 models' ranking (Latest Flagships view)
Reported by
official leaderboard
Document
[2] MAST technical leaderboard (First Do NOHARM v2 and per-benchmark results)
15Kimi K2.6Moonshot AI59.1Board publisherAug 2026[2]
14Kimi K2.6OSSMoonshot AI 59.1%
As printed
59.1%
Where
arise-ai.org/mast/technical, 'First Do NOHARM v2 overall metric across 19 models' ranking (Latest Flagships view)
Reported by
official leaderboard
Document
[2] MAST technical leaderboard (First Do NOHARM v2 and per-benchmark results)
16GLM 5.1Zhipu · First Do NOHARM v2 overall57.9Board publisherSep 2026[1]
First Do NOHARM v2 overall metric across 19 models 15GLM 5.1OSSZ.ai 57.9%
As printed
57.9%
Where
First Do NOHARM v2, overall leaderboard; GLM 5.1 row; displayed 2026-09-30.
Setting
First Do NOHARM v2 overall; preview; open-weight model; run date unpublished
Reported by
official leaderboard
Document
[1] ARISE MAST technical leaderboard
17DeepSeek R1DeepSeek55.8Board publisherAug 2026[2]
16DeepSeek R1OSSDeepSeek 55.8%
As printed
55.8%
Where
arise-ai.org/mast/technical, 'First Do NOHARM v2 overall metric across 19 models' ranking (Latest Flagships view)
Reported by
official leaderboard
Document
[2] MAST technical leaderboard (First Do NOHARM v2 and per-benchmark results)
References

2 documents support this board

  1. 1.ARISE MAST technical leaderboardofficial leaderboard, ARISE. Every result it supports
  2. 2.MAST technical leaderboard (First Do NOHARM v2 and per-benchmark results)official leaderboard, ARISE AI Research Network, 15 Aug 2026. Every result it supports