MedXpertQA (MM)
Expert-level multiple-choice questions over clinical images across 17 specialties.
More about this boardLess
Expert-level multimodal medical multiple-choice QA covering clinical images (X-ray, histology, dermatology, charts) across 17 specialties; MM subset of the 4,460-question MedXpertQA benchmark.
Assembled table, not a single board. Rows come from several vendors' own launch documents: Meta's Muse Spark evaluation (whose values live only in a rendered table image and cover Meta's own model plus four competitors it re-ran or quoted), Alibaba's Qwen3.5, 3.7 and 3.8 posts (Alibaba's own runs of its models and of competitors), and Google's Gemma 4 model card. Protocols are not known to match across rows; each row's source line says which document it came from and who ran it. benchlm.ai presents a subset as one mirrored view while citing only Meta's methodology.
Published by TsinghuaC3I (Tsinghua University), released Jan 2025. 2,000 multimodal questions (MM subset). Percentage accuracy 0-100, higher better.
- Accuracy. Same 2025 paper; multimodal test under its original prompt and answer-extraction protocol. Unit: %.
- Board last updated
- 8 Apr 2026
- Paper version
- ICML 2025 / arXiv v3
- Publisher
- TsinghuaC3I (Tsinghua University)
- Rows
- 22
- Models
- 22 on the board
- Released
- Jan 2025
- Measured to
- Aug 2026
- Unit
- 2,000 multimodal questions
- Documents cited
- 5
Every result on MedXpertQA (MM), on the board's own scale
22 results, one column each, highest first. Percentage accuracy 0-100, higher better.
Column colour is the model's lab. The table below says who ran each result.
Every published result, ranked on this board alone
22 rows. Higher is better. Bars run from 0 to 100; whiskers show a published interval. Select a row for its quote, page and document.
1GPT-5.6 SolOpenAI · Qwen-run comparison…81.5Independent run[1]
| MedXpertQA-MM | 71.7 | 80.0 | 80.7 | 81.5 | 71.0 | 80.4 |
2Gemini 3.1 ProGoogle · Meta's Muse Spark launch…81.3Independent runApr 2026[2]
MedXpertQA (MM) | Medical Multiple Choice | 78.4 | 64.8 | 81.3 | 77.1 | 65.8 (image table row; columns Muse Spark Thinking, Opus 4.6 Max, Gemini 3.1 Pro High, GPT 5.4 Xhigh, Grok 4.2 Reasoning)
3Qwen3.8 MaxAlibaba · Alibaba's own Qwen3.8 launch…80.4Model's own labAug 2026[1]
| MedXpertQA-MM | 71.7 | 80.0 | 80.7 | 81.5 | 71.0 | 80.4 |
4Claude Fable 5Anthropic · Qwen-run comparison…80.0Independent run[1]
| MedXpertQA-MM | 71.7 | 80.0 | 80.7 | 81.5 | 71.0 | 80.4 |
5Muse SparkMeta · Meta's Muse Spark launch…78.4Model's own labApr 2026[2]
MedXpertQA (MM) | Medical Multiple Choice | 78.4 | 64.8 | 81.3 | 77.1 | 65.8 (image table row; columns Muse Spark Thinking, Opus 4.6 Max, Gemini 3.1 Pro High, GPT 5.4 Xhigh, Grok 4.2 Reasoning)
6GPT-5.4OpenAI · Meta's Muse Spark launch…77.1Independent runApr 2026[2]
MedXpertQA (MM) | Medical Multiple Choice | 78.4 | 64.8 | 81.3 | 77.1 | 65.8 (image table row; columns Muse Spark Thinking, Opus 4.6 Max, Gemini 3.1 Pro High, GPT 5.4 Xhigh, Grok 4.2 Reasoning)
7Gemini-3 ProGoogle · Qwen3.5 model card…76.0Independent runFeb 2026[3]
| GPT5.2 | Claude 4.5 Opus | Gemini-3 Pro | Qwen3-VL-235B-A22B | K2.5-1T-A32B | Qwen3.5-397B-A17B MedXpertQA-MM | 73.3 | 63.6 | 76.0 | 47.6 | 65.3 | 70.0
8GPT-5.2OpenAI · Qwen-run comparison…73.3Independent run[3]
MedXpertQA-MM 73.3 63.6 76.0 47.6 65.3 70.0
9Claude Opus 4.8Anthropic · Qwen-run comparison…71.7Independent run[1]
| MedXpertQA-MM | 71.7 | 80.0 | 80.7 | 81.5 | 71.0 | 80.4 |
10Qwen3.7 PlusAlibaba · Alibaba's own Qwen3.7 Plus…71.0Model's own labMay 2026[4]
| MedXpertQA-MM | 77.3 | 64.4 | 80.7 | 68.7 | 71.0 |
11Qwen3.5 397B A17BAlibaba · self-reported…70.0Model's own lab[3]
MedXpertQA-MM 73.3 63.6 76.0 47.6 65.3 70.0
12Qwen3.6 PlusAlibaba · Qwen-run comparison…68.7Model's own lab[4]
| MedXpertQA-MM | 77.3 | 64.4 | 80.7 | 68.7 | 71.0 |
13Grok 4.20SpaceX AI · Meta's Muse Spark launch…65.8Independent runApr 2026[2]
MedXpertQA (MM) | Medical Multiple Choice | 78.4 | 64.8 | 81.3 | 77.1 | 65.8 (image table row; columns Muse Spark Thinking, Opus 4.6 Max, Gemini 3.1 Pro High, GPT 5.4 Xhigh, Grok 4.2 Reasoning)
14Kimi K2.5Moonshot AI · Qwen-run comparison…65.3Independent run[3]
MedXpertQA-MM 73.3 63.6 76.0 47.6 65.3 70.0
15Claude Opus 4.6Anthropic · Meta's Muse Spark launch…64.8Independent runApr 2026[2]
MedXpertQA (MM) | Medical Multiple Choice | 78.4 | 64.8 | 81.3 | 77.1 | 65.8 (image table row; columns Muse Spark Thinking, Opus 4.6 Max, Gemini 3.1 Pro High, GPT 5.4 Xhigh, Grok 4.2 Reasoning)
16Claude 4.5 OpusAnthropic · Qwen3.5 model card…63.6Independent runFeb 2026[3]
| GPT5.2 | Claude 4.5 Opus | Gemini-3 Pro | Qwen3-VL-235B-A22B | K2.5-1T-A32B | Qwen3.5-397B-A17B MedXpertQA-MM | 73.3 | 63.6 | 76.0 | 47.6 | 65.3 | 70.0
17Gemma 4 31BGoogle · Google model card61.3Model's own labApr 2026[5]
| Gemma 4 31B | Gemma 4 26B A4B | Gemma 4 12B Unified | Gemma 4 E4B | Gemma 4 E2B | Gemma 3 27B (no think) MedXPertQA MM | 61.3% | 58.1% | 48.7% | 28.7% | 23.5% | -
18Gemma 4 26B A4BGoogle · Google model card58.1Model's own labApr 2026[5]
| Gemma 4 31B | Gemma 4 26B A4B | Gemma 4 12B Unified | Gemma 4 E4B | Gemma 4 E2B | Gemma 3 27B (no think) MedXPertQA MM | 61.3% | 58.1% | 48.7% | 28.7% | 23.5% | -
19Gemma 4 12BGoogle · Google's Gemma 4 model card48.7Model's own labApr 2026[5]
| MedXPertQA MM | 61.3% | 58.1% | 48.7% | 28.7% | 23.5% | \- |
20Qwen3-VL-235B-A22BAlibaba · Qwen3.5 model card…47.6Model's own labFeb 2026[3]
| GPT5.2 | Claude 4.5 Opus | Gemini-3 Pro | Qwen3-VL-235B-A22B | K2.5-1T-A32B | Qwen3.5-397B-A17B MedXpertQA-MM | 73.3 | 63.6 | 76.0 | 47.6 | 65.3 | 70.0
21Gemma 4 E4BGoogle · Google model card28.7Model's own labApr 2026[5]
| Gemma 4 31B | Gemma 4 26B A4B | Gemma 4 12B Unified | Gemma 4 E4B | Gemma 4 E2B | Gemma 3 27B (no think) MedXPertQA MM | 61.3% | 58.1% | 48.7% | 28.7% | 23.5% | -
22Gemma 4 E2BGoogle · Google model card23.5Model's own labApr 2026[5]
| Gemma 4 31B | Gemma 4 26B A4B | Gemma 4 12B Unified | Gemma 4 E4B | Gemma 4 E2B | Gemma 3 27B (no think) MedXPertQA MM | 61.3% | 58.1% | 48.7% | 28.7% | 23.5% | -
5 documents support this board
- 1.Qwen3.8-Max: A New Bar for Coding and Coworklaunch post, Alibaba, 2 Aug 2026. Every result it supports
- 2.Introducing Muse Spark: Scaling Towards Personal Superintelligencelaunch post, Meta, 8 Apr 2026. Every result it supports
- 3.Qwen/Qwen3.5-397B-A17B model cardmodel card, Alibaba / Qwen, 16 Feb 2026. Every result it supports
- 4.Qwen3.7-Plus: Multimodal Agent Intelligencelaunch post, Alibaba, 31 May 2026. Every result it supports
- 5.Gemma 4 model cardmodel card, Google, 2 Apr 2026. Every result it supports