Clinical Benchmarks

MedXpertQA (MM)

Expert-level multiple-choice questions over clinical images across 17 specialties.

More about this boardLess

Expert-level multimodal medical multiple-choice QA covering clinical images (X-ray, histology, dermatology, charts) across 17 specialties; MM subset of the 4,460-question MedXpertQA benchmark.

Assembled table, not a single board. Rows come from several vendors' own launch documents: Meta's Muse Spark evaluation (whose values live only in a rendered table image and cover Meta's own model plus four competitors it re-ran or quoted), Alibaba's Qwen3.5, 3.7 and 3.8 posts (Alibaba's own runs of its models and of competitors), and Google's Gemma 4 model card. Protocols are not known to match across rows; each row's source line says which document it came from and who ran it. benchlm.ai presents a subset as one mirrored view while citing only Meta's methodology.

Published by TsinghuaC3I (Tsinghua University), released Jan 2025. 2,000 multimodal questions (MM subset). Percentage accuracy 0-100, higher better.

  • Accuracy. Same 2025 paper; multimodal test under its original prompt and answer-extraction protocol. Unit: %.
Board last updated
8 Apr 2026
Paper version
ICML 2025 / arXiv v3
Publisher
TsinghuaC3I (Tsinghua University)
Rows
22
Models
22 on the board
Released
Jan 2025
Measured to
Aug 2026
Unit
2,000 multimodal questions
Documents cited
5

Every result on MedXpertQA (MM), on the board's own scale

22 results, one column each, highest first. Percentage accuracy 0-100, higher better.

Column colour is the model's lab. The table below says who ran each result.

Every published result, ranked on this board alone

22 rows. Higher is better. Bars run from 0 to 100; whiskers show a published interval. Select a row for its quote, page and document.

1GPT-5.6 SolOpenAI · Qwen-run comparison…81.5Independent run[1]
| MedXpertQA-MM | 71.7 | 80.0 | 80.7 | 81.5 | 71.0 | 80.4 |
As printed
81.5
Where
Full Benchmark Table, second (multimodal) table, row MedXpertQA-MM; columns Opus4.8 / Fable5 / Gemini3.1-Pro / GPT5.6-Sol / Qwen3.7-Plus / Qwen3.8-Max
Setting
Qwen-run comparison in the Qwen3.8-Max launch post
Reported by
independent run
Document
[1] Qwen3.8-Max: A New Bar for Coding and Cowork
2Gemini 3.1 ProGoogle · Meta's Muse Spark launch…81.3Independent runApr 2026[2]
MedXpertQA (MM) | Medical Multiple Choice | 78.4 | 64.8 | 81.3 | 77.1 | 65.8 (image table row; columns Muse Spark Thinking, Opus 4.6 Max, Gemini 3.1 Pro High, GPT 5.4 Xhigh, Grok 4.2 Reasoning)
As printed
81.3%
Where
Launch-post benchmark table image, HEALTH section, row MedXpertQA (MM), column Gemini 3.1 Pro High; the same table is page 5 of the Eval Methodology PDF; protocol p. 2: 'MedXpertQA Text/Multimodal: ... The multimodal variant contains 2,000 multimodal medical questions with clinical images (X-rays, histology, dermatology, etc.) and 5 answer choices (A-E). For grading, we use gpt-oss-120b to parse the predicted answer letter from free-form text.'
Setting
Meta's Muse Spark launch table (Meta reports the better of vendor self-reports and its own reproduction)
Reported by
independent run
Document
[2] Introducing Muse Spark: Scaling Towards Personal Superintelligence
3Qwen3.8 MaxAlibaba · Alibaba's own Qwen3.8 launch…80.4Model's own labAug 2026[1]
| MedXpertQA-MM | 71.7 | 80.0 | 80.7 | 81.5 | 71.0 | 80.4 |
As printed
80.4%
Where
Multimodal Benchmarks table, Multimodal Reasoning section, row MedXpertQA-MM, column Qwen3.8-Max; header row: | | Opus4.8 | Fable5 | Gemini3.1-Pro | GPT5.6-Sol | Qwen3.7-Plus | Qwen3.8-Max |
Setting
Alibaba's own Qwen3.8 launch table; protocol not stated
Reported by
vendor-reported
Document
[1] Qwen3.8-Max: A New Bar for Coding and Cowork
4Claude Fable 5Anthropic · Qwen-run comparison…80.0Independent run[1]
| MedXpertQA-MM | 71.7 | 80.0 | 80.7 | 81.5 | 71.0 | 80.4 |
As printed
80.0
Where
Full Benchmark Table, second (multimodal) table, row MedXpertQA-MM; columns Opus4.8 / Fable5 / Gemini3.1-Pro / GPT5.6-Sol / Qwen3.7-Plus / Qwen3.8-Max
Setting
Qwen-run comparison in the Qwen3.8-Max launch post
Reported by
independent run
Document
[1] Qwen3.8-Max: A New Bar for Coding and Cowork
5Muse SparkMeta · Meta's Muse Spark launch…78.4Model's own labApr 2026[2]
MedXpertQA (MM) | Medical Multiple Choice | 78.4 | 64.8 | 81.3 | 77.1 | 65.8 (image table row; columns Muse Spark Thinking, Opus 4.6 Max, Gemini 3.1 Pro High, GPT 5.4 Xhigh, Grok 4.2 Reasoning)
As printed
78.4%
Where
Launch-post benchmark table image, HEALTH section, row MedXpertQA (MM), column Muse Spark Thinking; the same table is page 5 of the Eval Methodology PDF; protocol p. 2: 'MedXpertQA Text/Multimodal: ... The multimodal variant contains 2,000 multimodal medical questions with clinical images (X-rays, histology, dermatology, etc.) and 5 answer choices (A-E). For grading, we use gpt-oss-120b to parse the predicted answer letter from free-form text.'
Setting
Meta's Muse Spark launch table (Meta reports the better of vendor self-reports and its own reproduction)
Reported by
vendor-reported
Document
[2] Introducing Muse Spark: Scaling Towards Personal Superintelligence
6GPT-5.4OpenAI · Meta's Muse Spark launch…77.1Independent runApr 2026[2]
MedXpertQA (MM) | Medical Multiple Choice | 78.4 | 64.8 | 81.3 | 77.1 | 65.8 (image table row; columns Muse Spark Thinking, Opus 4.6 Max, Gemini 3.1 Pro High, GPT 5.4 Xhigh, Grok 4.2 Reasoning)
As printed
77.1%
Where
Launch-post benchmark table image, HEALTH section, row MedXpertQA (MM), column GPT 5.4 Xhigh; the same table is page 5 of the Eval Methodology PDF; protocol p. 2: 'MedXpertQA Text/Multimodal: ... The multimodal variant contains 2,000 multimodal medical questions with clinical images (X-rays, histology, dermatology, etc.) and 5 answer choices (A-E). For grading, we use gpt-oss-120b to parse the predicted answer letter from free-form text.'
Setting
Meta's Muse Spark launch table (Meta reports the better of vendor self-reports and its own reproduction)
Reported by
independent run
Document
[2] Introducing Muse Spark: Scaling Towards Personal Superintelligence
7Gemini-3 ProGoogle · Qwen3.5 model card…76.0Independent runFeb 2026[3]
| GPT5.2 | Claude 4.5 Opus | Gemini-3 Pro | Qwen3-VL-235B-A22B | K2.5-1T-A32B | Qwen3.5-397B-A17B MedXpertQA-MM | 73.3 | 63.6 | 76.0 | 47.6 | 65.3 | 70.0
As printed
76.0
Where
Benchmark Results > Vision Language > Medical VQA; MedXpertQA-MM row, Gemini-3 Pro column.
Setting
Qwen3.5 model card comparison; source-specific evaluation, not harmonized across vendors
Reported by
independent run
Document
[3] Qwen/Qwen3.5-397B-A17B model card
8GPT-5.2OpenAI · Qwen-run comparison…73.3Independent run[3]
MedXpertQA-MM 73.3 63.6 76.0 47.6 65.3 70.0
As printed
73.3
Where
Benchmark Results > Vision Language > Medical VQA, row MedXpertQA-MM; columns GPT5.2 / Claude 4.5 Opus / Gemini-3 Pro / Qwen3-VL-235B-A22B / K2.5-1T-A32B / Qwen3.5-397B-A17B
Setting
Qwen-run comparison in the Qwen3.5-397B-A17B model card
Reported by
independent run
Document
[3] Qwen/Qwen3.5-397B-A17B model card
9Claude Opus 4.8Anthropic · Qwen-run comparison…71.7Independent run[1]
| MedXpertQA-MM | 71.7 | 80.0 | 80.7 | 81.5 | 71.0 | 80.4 |
As printed
71.7
Where
Full Benchmark Table, second (multimodal) table, row MedXpertQA-MM; columns Opus4.8 / Fable5 / Gemini3.1-Pro / GPT5.6-Sol / Qwen3.7-Plus / Qwen3.8-Max
Setting
Qwen-run comparison in the Qwen3.8-Max launch post
Reported by
independent run
Document
[1] Qwen3.8-Max: A New Bar for Coding and Cowork
10Qwen3.7 PlusAlibaba · Alibaba's own Qwen3.7 Plus…71.0Model's own labMay 2026[4]
| MedXpertQA-MM | 77.3 | 64.4 | 80.7 | 68.7 | 71.0 |
As printed
71.0%
Where
Multimodal Benchmarks table, Multimodal Reasoning section, row MedXpertQA-MM, column Qwen3.7-Plus; header row: | | GPT-5.4 (xhigh) | Opus-4.6 Max | Gemini-3.1 Pro | Qwen3.6-Plus | Qwen3.7-Plus |
Setting
Alibaba's own Qwen3.7 Plus launch table; protocol not stated
Reported by
vendor-reported
Document
[4] Qwen3.7-Plus: Multimodal Agent Intelligence
11Qwen3.5 397B A17BAlibaba · self-reported…70.0Model's own lab[3]
MedXpertQA-MM 73.3 63.6 76.0 47.6 65.3 70.0
As printed
70.0
Where
Benchmark Results > Vision Language > Medical VQA, row MedXpertQA-MM; columns GPT5.2 / Claude 4.5 Opus / Gemini-3 Pro / Qwen3-VL-235B-A22B / K2.5-1T-A32B / Qwen3.5-397B-A17B
Setting
self-reported in the Qwen3.5-397B-A17B model card
Reported by
vendor-reported
Document
[3] Qwen/Qwen3.5-397B-A17B model card
12Qwen3.6 PlusAlibaba · Qwen-run comparison…68.7Model's own lab[4]
| MedXpertQA-MM | 77.3 | 64.4 | 80.7 | 68.7 | 71.0 |
As printed
68.7
Where
Multimodal Benchmarks table, row MedXpertQA-MM; columns GPT-5.4 (xhigh) / Opus-4.6 Max / Gemini-3.1 Pro / Qwen3.6-Plus / Qwen3.7-Plus
Setting
Qwen-run comparison in the Qwen3.7-Plus launch post
Reported by
vendor-reported
Document
[4] Qwen3.7-Plus: Multimodal Agent Intelligence
13Grok 4.20SpaceX AI · Meta's Muse Spark launch…65.8Independent runApr 2026[2]
MedXpertQA (MM) | Medical Multiple Choice | 78.4 | 64.8 | 81.3 | 77.1 | 65.8 (image table row; columns Muse Spark Thinking, Opus 4.6 Max, Gemini 3.1 Pro High, GPT 5.4 Xhigh, Grok 4.2 Reasoning)
As printed
65.8%
Where
Launch-post benchmark table image, HEALTH section, row MedXpertQA (MM), column Grok 4.2 Reasoning; the same table is page 5 of the Eval Methodology PDF; protocol p. 2: 'MedXpertQA Text/Multimodal: ... The multimodal variant contains 2,000 multimodal medical questions with clinical images (X-rays, histology, dermatology, etc.) and 5 answer choices (A-E). For grading, we use gpt-oss-120b to parse the predicted answer letter from free-form text.'
Setting
Meta's Muse Spark launch table (Meta reports the better of vendor self-reports and its own reproduction)
Reported by
independent run
Document
[2] Introducing Muse Spark: Scaling Towards Personal Superintelligence
14Kimi K2.5Moonshot AI · Qwen-run comparison…65.3Independent run[3]
MedXpertQA-MM 73.3 63.6 76.0 47.6 65.3 70.0
As printed
65.3
Where
Benchmark Results > Vision Language > Medical VQA, row MedXpertQA-MM; columns GPT5.2 / Claude 4.5 Opus / Gemini-3 Pro / Qwen3-VL-235B-A22B / K2.5-1T-A32B / Qwen3.5-397B-A17B
Setting
Qwen-run comparison in the Qwen3.5-397B-A17B model card (K2.5-1T-A32B column)
Reported by
independent run
Document
[3] Qwen/Qwen3.5-397B-A17B model card
15Claude Opus 4.6Anthropic · Meta's Muse Spark launch…64.8Independent runApr 2026[2]
MedXpertQA (MM) | Medical Multiple Choice | 78.4 | 64.8 | 81.3 | 77.1 | 65.8 (image table row; columns Muse Spark Thinking, Opus 4.6 Max, Gemini 3.1 Pro High, GPT 5.4 Xhigh, Grok 4.2 Reasoning)
As printed
64.8%
Where
Launch-post benchmark table image, HEALTH section, row MedXpertQA (MM), column Opus 4.6 Max; the same table is page 5 of the Eval Methodology PDF; protocol p. 2: 'MedXpertQA Text/Multimodal: ... The multimodal variant contains 2,000 multimodal medical questions with clinical images (X-rays, histology, dermatology, etc.) and 5 answer choices (A-E). For grading, we use gpt-oss-120b to parse the predicted answer letter from free-form text.'
Setting
Meta's Muse Spark launch table (Meta reports the better of vendor self-reports and its own reproduction)
Reported by
independent run
Document
[2] Introducing Muse Spark: Scaling Towards Personal Superintelligence
16Claude 4.5 OpusAnthropic · Qwen3.5 model card…63.6Independent runFeb 2026[3]
| GPT5.2 | Claude 4.5 Opus | Gemini-3 Pro | Qwen3-VL-235B-A22B | K2.5-1T-A32B | Qwen3.5-397B-A17B MedXpertQA-MM | 73.3 | 63.6 | 76.0 | 47.6 | 65.3 | 70.0
As printed
63.6
Where
Benchmark Results > Vision Language > Medical VQA; MedXpertQA-MM row, Claude 4.5 Opus column.
Setting
Qwen3.5 model card comparison; source-specific evaluation, not harmonized across vendors
Reported by
independent run
Document
[3] Qwen/Qwen3.5-397B-A17B model card
17Gemma 4 31BGoogle · Google model card61.3Model's own labApr 2026[5]
| Gemma 4 31B | Gemma 4 26B A4B | Gemma 4 12B Unified | Gemma 4 E4B | Gemma 4 E2B | Gemma 3 27B (no think) MedXPertQA MM | 61.3% | 58.1% | 48.7% | 28.7% | 23.5% | -
As printed
61.3%
Where
Evaluation Results, MedXPertQA MM row, Gemma 4 31B column.
Setting
Google model card; MedXPertQA MM row; vendor-reported; protocol differs from other source families
Reported by
vendor-reported
Document
[5] Gemma 4 model card
18Gemma 4 26B A4BGoogle · Google model card58.1Model's own labApr 2026[5]
| Gemma 4 31B | Gemma 4 26B A4B | Gemma 4 12B Unified | Gemma 4 E4B | Gemma 4 E2B | Gemma 3 27B (no think) MedXPertQA MM | 61.3% | 58.1% | 48.7% | 28.7% | 23.5% | -
As printed
58.1%
Where
Evaluation Results, MedXPertQA MM row, Gemma 4 26B A4B column.
Setting
Google model card; MedXPertQA MM row; vendor-reported; protocol differs from other source families
Reported by
vendor-reported
Document
[5] Gemma 4 model card
19Gemma 4 12BGoogle · Google's Gemma 4 model card48.7Model's own labApr 2026[5]
| MedXPertQA MM | 61.3% | 58.1% | 48.7% | 28.7% | 23.5% | \- |
As printed
48.7%
Where
Benchmark Results table, Vision section, row MedXPertQA MM, column Gemma 4 12B Unified; header row: | | Gemma 4 31B | Gemma 4 26B A4B | Gemma 4 12B Unified | Gemma 4 E4B | Gemma 4 E2B | Gemma 3 27B (no think) |
Setting
Google's Gemma 4 model card, Unified 12B; protocol not stated
Reported by
vendor-reported
Document
[5] Gemma 4 model card
20Qwen3-VL-235B-A22BAlibaba · Qwen3.5 model card…47.6Model's own labFeb 2026[3]
| GPT5.2 | Claude 4.5 Opus | Gemini-3 Pro | Qwen3-VL-235B-A22B | K2.5-1T-A32B | Qwen3.5-397B-A17B MedXpertQA-MM | 73.3 | 63.6 | 76.0 | 47.6 | 65.3 | 70.0
As printed
47.6
Where
Benchmark Results > Vision Language > Medical VQA; MedXpertQA-MM row, Qwen3-VL-235B-A22B column.
Setting
Qwen3.5 model card comparison; source-specific evaluation, not harmonized across vendors
Reported by
vendor-reported
Document
[3] Qwen/Qwen3.5-397B-A17B model card
21Gemma 4 E4BGoogle · Google model card28.7Model's own labApr 2026[5]
| Gemma 4 31B | Gemma 4 26B A4B | Gemma 4 12B Unified | Gemma 4 E4B | Gemma 4 E2B | Gemma 3 27B (no think) MedXPertQA MM | 61.3% | 58.1% | 48.7% | 28.7% | 23.5% | -
As printed
28.7%
Where
Evaluation Results, MedXPertQA MM row, Gemma 4 E4B column.
Setting
Google model card; MedXPertQA MM row; vendor-reported; protocol differs from other source families
Reported by
vendor-reported
Document
[5] Gemma 4 model card
22Gemma 4 E2BGoogle · Google model card23.5Model's own labApr 2026[5]
| Gemma 4 31B | Gemma 4 26B A4B | Gemma 4 12B Unified | Gemma 4 E4B | Gemma 4 E2B | Gemma 3 27B (no think) MedXPertQA MM | 61.3% | 58.1% | 48.7% | 28.7% | 23.5% | -
As printed
23.5%
Where
Evaluation Results, MedXPertQA MM row, Gemma 4 E2B column.
Setting
Google model card; MedXPertQA MM row; vendor-reported; protocol differs from other source families
Reported by
vendor-reported
Document
[5] Gemma 4 model card
References

5 documents support this board

  1. 1.Qwen3.8-Max: A New Bar for Coding and Coworklaunch post, Alibaba, 2 Aug 2026. Every result it supports
  2. 2.Introducing Muse Spark: Scaling Towards Personal Superintelligencelaunch post, Meta, 8 Apr 2026. Every result it supports
  3. 3.Qwen/Qwen3.5-397B-A17B model cardmodel card, Alibaba / Qwen, 16 Feb 2026. Every result it supports
  4. 4.Qwen3.7-Plus: Multimodal Agent Intelligencelaunch post, Alibaba, 31 May 2026. Every result it supports
  5. 5.Gemma 4 model cardmodel card, Google, 2 Apr 2026. Every result it supports