First, Do NOHARM (v2)
How often, and how severely, model consultation recommendations contain potentially harmful errors.
More about this boardLess
Frequency and severity of potentially harmful errors in LLM-generated medical consultation recommendations (Numerous Options Harm Assessment for Risk in Medicine); primary-care-to-specialist consults.
Paper: arXiv 2512.01241. The v1 study found potential for severe harm in up to 24.6 percent of directly applied recommendations, with errors of omission behind more than 80 percent of the severe cases.
Published by Stanford/Harvard-led consortium (50+ researchers incl. 29 board-certified physicians); hosted by ARISE, released Dec 2025. 1,100 consultation cases, 10 specialties, 12,747 expert annotations on 4,249 management options. Percentage safety score, higher better.
- Headline metric
- F1 (Weighted)
- Board marked as preview
- Yes
- Board last updated
- 15 Aug 2026
- Models in the full dataset
- 43
- Models in the board's default view
- 19
- Publisher
- Stanford/Harvard-led consortium (50+ researchers incl. 29 board-certified physicians); hosted by ARISE
- Rows
- 17
- Models
- 17 of 19 on the board
- Released
- Dec 2025
- Measured to
- Sep 2026
- Unit
- 1,100 consultation cases
- Documents cited
- 2
Every result on First, Do NOHARM (v2), on the board's own scale
17 results, one column each, highest first. Percentage safety score, higher better.
Column colour is the model's lab. The table below says who ran each result.
Every published result, ranked on this board alone
17 rows. Higher is better. Bars run from 0 to 100; whiskers show a published interval. Select a row for its quote, page and document.
1LiSA 2.5AMBOSS · First Do NOHARM v2 overall86.2Board publisherSep 2026[1]
First Do NOHARM v2 overall metric across 19 models 1LiSA 2.5RAGAMBOSS 86.2%
2Doximity Ask 6.1Doximity · First Do NOHARM v2 overall84.5Board publisherSep 2026[1]
First Do NOHARM v2 overall metric across 19 models 2Doximity Ask 6.1RAGDoximity 84.5%
3OpenEvidenceOpenEvidence · First Do NOHARM v2 overall80.0Board publisherSep 2026[1]
First Do NOHARM v2 overall metric across 19 models 3OpenEvidenceRAGOpenEvidence 80.0%
4Glass 5.6 MaxGlass Health · First Do NOHARM v2 overall79.7Board publisherSep 2026[1]
First Do NOHARM v2 overall metric across 19 models 4Glass 5.6 MaxRAGGlass Health 79.7%
4Muse Spark 1.1Meta79.7Board publisherAug 2026[2]
5Muse Spark 1.1Meta 79.7%
6Claude Opus 5Anthropic · v2 run on ARISE74.6Board publisherAug 2026[2]
6Claude Opus 5Anthropic 74.6%
7Kimi K3Moonshot AI74.0Board publisherAug 2026[2]
7Kimi K3OSSMoonshot AI 74.0%
8GPT-5.6 SolOpenAI70.1Board publisherAug 2026[2]
8GPT-5.6 SolOpenAI 70.1%
9GPT-5.5OpenAI70.0Board publisherAug 2026[2]
9GPT-5.5OpenAI 70.0%
10GPT-5OpenAI · from the Model Leaderboard…68.6Board publisherAug 2026[2]
10 GPT-5 72.6%±2.3 68.6%±4.5 80.3%±4.4 30.1%±9.0 44.2%±2.5 45.4%±1.3 73.1%±3.9 73.7%±2.7 46.6%±0.0
11Claude Fable 5Anthropic65.0Board publisherAug 2026[2]
10Claude Fable 5Anthropic 65.0%
12Gemini 3.1 ProGoogle62.6Board publisherAug 2026[2]
11Gemini 3.1 ProGoogle 62.6%
13Gemini 2.5 ProGoogle61.9Board publisherAug 2026[2]
12Gemini 2.5 ProGoogle 61.9%
14Qwen3.5 397B A17BAlibaba61.1Board publisherAug 2026[2]
13Qwen3.5 397B A17BOSSAlibaba 61.1%
15Kimi K2.6Moonshot AI59.1Board publisherAug 2026[2]
14Kimi K2.6OSSMoonshot AI 59.1%
16GLM 5.1Zhipu · First Do NOHARM v2 overall57.9Board publisherSep 2026[1]
First Do NOHARM v2 overall metric across 19 models 15GLM 5.1OSSZ.ai 57.9%
17DeepSeek R1DeepSeek55.8Board publisherAug 2026[2]
16DeepSeek R1OSSDeepSeek 55.8%
2 documents support this board
- 1.ARISE MAST technical leaderboardofficial leaderboard, ARISE. Every result it supports
- 2.MAST technical leaderboard (First Do NOHARM v2 and per-benchmark results)official leaderboard, ARISE AI Research Network, 15 Aug 2026. Every result it supports