Clinical Benchmarks

MedPIC-Bench

Tests whether models apply and withdraw medication-safety rules correctly as patient information changes.

Hou et al.Official pagePaper
More about this boardLess

MedPIC-Bench evaluates whether language models use patient information to apply medication-safety rules correctly. Its 467 expert-validated multiple-choice questions combine fixed guideline-following cases with counterfactual cases in which changes to patient information alter whether a rule applies. Models must select the exact correct set of options; linked-pair accuracy requires both cases to be correct. The benchmark also distinguishes activating a safety warning from withdrawing a warning whose trigger is absent.

Published by Zhitian Hou, Yuhang Liu, Pengkai Wang, Zeyu Liu, Guanghao Zhu, Zheng Liu, Shuo Cai, Congkai Xie, Zhijie Sang, Kun Zeng, Hongxia Yang (The Hong Kong Polytechnic University; InfiX.ai; Sun Yat-sen University), released 4 Aug 2026. Overall exact-match accuracy (%). 0–100.

  • Guideline-following accuracy (GF). Exact option-set match accuracy on 284 fixed patient cases requiring application of a medication-safety recommendation. Unit: %.
  • Counterfactual accuracy (CF). Exact option-set match accuracy on 183 questions with controlled changes in patient information that alter rule applicability. Unit: %.
  • Linked counterfactual pair accuracy. Share of the 89 linked counterfactual pairs where both questions, presented separately, are answered correctly. The pairing comes from the paper; the public dataset does not include pair identifiers. Unit: %.
  • Risk activation accuracy. Exact option-set match accuracy on the 71 counterfactual questions where patient information activates a medication-safety rule. Unit: %.
  • Risk deactivation accuracy. Exact option-set match accuracy on the 68 counterfactual questions where changed patient information removes a medication-safety rule trigger. Unit: %.
  • Published GF-minus-CF accuracy gap. Guideline-following accuracy minus counterfactual accuracy, in percentage points, as printed in Table 2. It describes how much accuracy drops when patient information changes. The paper does not treat a smaller gap as better, so this board does not rank models on it. Unit: percentage points.
Authors
Zhitian Hou, Yuhang Liu, Pengkai Wang, Zeyu Liu, Guanghao Zhu, Zheng Liu, Shuo Cai, Congkai Xie, Zhijie Sang, Kun Zeng, Hongxia Yang
License
CC BY 4.0
Limitations
  • Released questions.json has no explicit pair ID, rule ID, source-citation field, raw model responses or evaluator code. Pair scores are paper-reported only.
  • No live leaderboard: these results are the August 2026 study snapshot.
  • Printed gaps can differ by 0.1 from subtracting the rounded guideline-following and counterfactual scores; the printed values are kept.
  • Drug-category labels are multi-valued and can overlap; their counts do not sum to question count.
  • Dataset questions are constructed from selected rules and disproportionately cover older adults; this is not an observed clinical cohort or patient-outcome evaluation.
  • Risk activation/deactivation exclude the other 44 counterfactual items; do not average those two scores to reconstruct total CF.
  • Model-group averages confound architecture, size and training. No claim that medical adaptation generally helps or harms follows from them.
Institutions
The Hong Kong Polytechnic University, InfiX.ai, Sun Yat-sen University
Paper version
arXiv:2608.03028v1
Dataset revision
9ef6db4f13865b14fc2e6be3f94dcfaf3a0cf983
Rows
28
Models
28
Labs
11
Last measured
Aug 2026
Leader
80.7Gemini-3.1-Pro

Headline score

0–100

  • Anthropic
  • Google
  • OpenAI
  • Alibaba
  • Meta
  • Other labs
All independent run
  1. 1Gemini-3.1-Prozero-shot80.7
  2. 2GPT-5zero-shot75.6
  3. 3Qwen3.5-Pluszero-shot72.4
  4. 4DeepSeek-V4-Prozero-shot71.7
  5. 5GPT-OSS-120Bzero-shot70.7
  6. 6Qwen3.5-27Bzero-shot69.4
  7. 7Qwen3.5-35B-A3Bzero-shot68.5
  8. 8GPT-5.2zero-shot68.1
  9. 9Claude-Sonnet-4.6zero-shot64.2
  10. 10Qwen3.5-9Bzero-shot63.8
  11. 11MedGemma-27B-Textzero-shot61.2
  12. 12Fleming-R1-32Bzero-shot60.6
  13. 13Lingshu-32Bzero-shot59.5
  14. 14GPT-OSS-20Bzero-shot59.1
  15. 15Llama-3.1-70Bzero-shot55.7
  16. 16HuatuoGPT-o1-70Bzero-shot55.2
  17. 17Gemini-2.5-Prozero-shot54.4
  18. 18Fleming-R1-7Bzero-shot50.7
  19. 19Baichuan-M2-32Bzero-shot47.8
  20. 20HuatuoGPT-o1-8Bzero-shot44.3

20 of 28 rows

Rows and sources

Open a row for the quote, the page and the document.

  1. 1Gemini-3.1-Pro zero-shot; independent questions; exact option-set match 80.7
    Printed as 80.7Independent run, measured Aug 2026Configuration: zero-shot; independent questions; exact option-set match
    Evaluating Counterfactual Sensitivity to Patient Information in Medication-Safety Reasoning paper, Zhitian Hou, Yuhang Liu, Pengkai Wang, Zeyu Liu, Guanghao Zhu, Zheng Liu, Shuo Cai, Congkai Xie, Zhijie Sang, Kun Zeng, Hongxia Yang (The Hong Kong Polytechnic University; InfiX.ai; Sun Yat-sen University), 4 Aug 2026. arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row Gemini-3.1-Pro, column Overall; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair
    Gemini-3.1-Pro | 80.7 | 87.7 | 69.9 | 17.7 | 74.6 | 77.9 | 48.3
    Every result from this document
  2. 2GPT-5 zero-shot; independent questions; exact option-set match 75.6
    Printed as 75.6Independent run, measured Aug 2026Configuration: zero-shot; independent questions; exact option-set match
    Evaluating Counterfactual Sensitivity to Patient Information in Medication-Safety Reasoning paper, Zhitian Hou, Yuhang Liu, Pengkai Wang, Zeyu Liu, Guanghao Zhu, Zheng Liu, Shuo Cai, Congkai Xie, Zhijie Sang, Kun Zeng, Hongxia Yang (The Hong Kong Polytechnic University; InfiX.ai; Sun Yat-sen University), 4 Aug 2026. arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row GPT-5, column Overall; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair
    GPT-5 | 75.6 | 83.5 | 63.4 | 20.1 | 70.4 | 61.8 | 38.2
    Every result from this document
  3. 3Qwen3.5-Plus zero-shot; independent questions; exact option-set match 72.4
    Printed as 72.4Independent run, measured Aug 2026Configuration: zero-shot; independent questions; exact option-set match
    Evaluating Counterfactual Sensitivity to Patient Information in Medication-Safety Reasoning paper, Zhitian Hou, Yuhang Liu, Pengkai Wang, Zeyu Liu, Guanghao Zhu, Zheng Liu, Shuo Cai, Congkai Xie, Zhijie Sang, Kun Zeng, Hongxia Yang (The Hong Kong Polytechnic University; InfiX.ai; Sun Yat-sen University), 4 Aug 2026. arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row Qwen3.5-Plus, column Overall; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair
    Qwen3.5-Plus | 72.4 | 81.7 | 57.9 | 23.8 | 66.2 | 55.9 | 32.6
    Every result from this document
  4. 4DeepSeek-V4-Pro zero-shot; independent questions; exact option-set match 71.7
    Printed as 71.7Independent run, measured Aug 2026Configuration: zero-shot; independent questions; exact option-set match
    Evaluating Counterfactual Sensitivity to Patient Information in Medication-Safety Reasoning paper, Zhitian Hou, Yuhang Liu, Pengkai Wang, Zeyu Liu, Guanghao Zhu, Zheng Liu, Shuo Cai, Congkai Xie, Zhijie Sang, Kun Zeng, Hongxia Yang (The Hong Kong Polytechnic University; InfiX.ai; Sun Yat-sen University), 4 Aug 2026. arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row DeepSeek-V4-Pro, column Overall; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair
    DeepSeek-V4-Pro | 71.7 | 81.3 | 56.8 | 24.5 | 67.6 | 50.0 | 32.6
    Every result from this document
  5. 5GPT-OSS-120B zero-shot; independent questions; exact option-set match 70.7
    Printed as 70.7Independent run, measured Aug 2026Configuration: zero-shot; independent questions; exact option-set match
    Evaluating Counterfactual Sensitivity to Patient Information in Medication-Safety Reasoning paper, Zhitian Hou, Yuhang Liu, Pengkai Wang, Zeyu Liu, Guanghao Zhu, Zheng Liu, Shuo Cai, Congkai Xie, Zhijie Sang, Kun Zeng, Hongxia Yang (The Hong Kong Polytechnic University; InfiX.ai; Sun Yat-sen University), 4 Aug 2026. arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row GPT-OSS-120B, column Overall; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair
    GPT-OSS-120B | 70.7 | 79.6 | 56.8 | 22.7 | 62.0 | 55.9 | 30.3
    Every result from this document
  6. 6Qwen3.5-27B zero-shot; independent questions; exact option-set match 69.4
    Printed as 69.4Independent run, measured Aug 2026Configuration: zero-shot; independent questions; exact option-set match
    Evaluating Counterfactual Sensitivity to Patient Information in Medication-Safety Reasoning paper, Zhitian Hou, Yuhang Liu, Pengkai Wang, Zeyu Liu, Guanghao Zhu, Zheng Liu, Shuo Cai, Congkai Xie, Zhijie Sang, Kun Zeng, Hongxia Yang (The Hong Kong Polytechnic University; InfiX.ai; Sun Yat-sen University), 4 Aug 2026. arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row Qwen3.5-27B, column Overall; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair
    Qwen3.5-27B | 69.4 | 78.2 | 55.7 | 22.4 | 67.6 | 51.5 | 31.5
    Every result from this document
  7. 7Qwen3.5-35B-A3B zero-shot; independent questions; exact option-set match 68.5
    Printed as 68.5Independent run, measured Aug 2026Configuration: zero-shot; independent questions; exact option-set match
    Evaluating Counterfactual Sensitivity to Patient Information in Medication-Safety Reasoning paper, Zhitian Hou, Yuhang Liu, Pengkai Wang, Zeyu Liu, Guanghao Zhu, Zheng Liu, Shuo Cai, Congkai Xie, Zhijie Sang, Kun Zeng, Hongxia Yang (The Hong Kong Polytechnic University; InfiX.ai; Sun Yat-sen University), 4 Aug 2026. arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row Qwen3.5-35B-A3B, column Overall; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair
    Qwen3.5-35B-A3B | 68.5 | 77.5 | 54.6 | 22.8 | 70.4 | 45.6 | 30.3
    Every result from this document
  8. 8GPT-5.2 zero-shot; independent questions; exact option-set match 68.1
    Printed as 68.1Independent run, measured Aug 2026Configuration: zero-shot; independent questions; exact option-set match
    Evaluating Counterfactual Sensitivity to Patient Information in Medication-Safety Reasoning paper, Zhitian Hou, Yuhang Liu, Pengkai Wang, Zeyu Liu, Guanghao Zhu, Zheng Liu, Shuo Cai, Congkai Xie, Zhijie Sang, Kun Zeng, Hongxia Yang (The Hong Kong Polytechnic University; InfiX.ai; Sun Yat-sen University), 4 Aug 2026. arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row GPT-5.2, column Overall; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair
    GPT-5.2 | 68.1 | 78.2 | 52.5 | 25.7 | 63.4 | 42.6 | 29.2
    Every result from this document
  9. 9Claude-Sonnet-4.6 zero-shot; independent questions; exact option-set match 64.2
    Printed as 64.2Independent run, measured Aug 2026Configuration: zero-shot; independent questions; exact option-set match
    Evaluating Counterfactual Sensitivity to Patient Information in Medication-Safety Reasoning paper, Zhitian Hou, Yuhang Liu, Pengkai Wang, Zeyu Liu, Guanghao Zhu, Zheng Liu, Shuo Cai, Congkai Xie, Zhijie Sang, Kun Zeng, Hongxia Yang (The Hong Kong Polytechnic University; InfiX.ai; Sun Yat-sen University), 4 Aug 2026. arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row Claude-Sonnet-4.6, column Overall; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair
    Claude-Sonnet-4.6 | 64.2 | 74.6 | 48.1 | 26.6 | 62.0 | 35.3 | 21.3
    Every result from this document
  10. 10Qwen3.5-9B zero-shot; independent questions; exact option-set match 63.8
    Printed as 63.8Independent run, measured Aug 2026Configuration: zero-shot; independent questions; exact option-set match
    Evaluating Counterfactual Sensitivity to Patient Information in Medication-Safety Reasoning paper, Zhitian Hou, Yuhang Liu, Pengkai Wang, Zeyu Liu, Guanghao Zhu, Zheng Liu, Shuo Cai, Congkai Xie, Zhijie Sang, Kun Zeng, Hongxia Yang (The Hong Kong Polytechnic University; InfiX.ai; Sun Yat-sen University), 4 Aug 2026. arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row Qwen3.5-9B, column Overall; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair
    Qwen3.5-9B | 63.8 | 73.9 | 48.1 | 25.9 | 54.9 | 47.1 | 24.7
    Every result from this document
  11. 11MedGemma-27B-Text zero-shot; independent questions; exact option-set match 61.2
    Printed as 61.2Independent run, measured Aug 2026Configuration: zero-shot; independent questions; exact option-set match
    Evaluating Counterfactual Sensitivity to Patient Information in Medication-Safety Reasoning paper, Zhitian Hou, Yuhang Liu, Pengkai Wang, Zeyu Liu, Guanghao Zhu, Zheng Liu, Shuo Cai, Congkai Xie, Zhijie Sang, Kun Zeng, Hongxia Yang (The Hong Kong Polytechnic University; InfiX.ai; Sun Yat-sen University), 4 Aug 2026. arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row MedGemma-27B-Text, column Overall; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair
    MedGemma-27B-Text | 61.2 | 67.6 | 51.4 | 16.2 | 76.1 | 32.4 | 23.6
    Every result from this document
  12. 12Fleming-R1-32B zero-shot; independent questions; exact option-set match 60.6
    Printed as 60.6Independent run, measured Aug 2026Configuration: zero-shot; independent questions; exact option-set match
    Evaluating Counterfactual Sensitivity to Patient Information in Medication-Safety Reasoning paper, Zhitian Hou, Yuhang Liu, Pengkai Wang, Zeyu Liu, Guanghao Zhu, Zheng Liu, Shuo Cai, Congkai Xie, Zhijie Sang, Kun Zeng, Hongxia Yang (The Hong Kong Polytechnic University; InfiX.ai; Sun Yat-sen University), 4 Aug 2026. arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row Fleming-R1-32B, column Overall; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair
    Fleming-R1-32B | 60.6 | 69.0 | 47.5 | 21.5 | 59.2 | 44.1 | 20.2
    Every result from this document
  13. 13Lingshu-32B zero-shot; independent questions; exact option-set match 59.5
    Printed as 59.5Independent run, measured Aug 2026Configuration: zero-shot; independent questions; exact option-set match
    Evaluating Counterfactual Sensitivity to Patient Information in Medication-Safety Reasoning paper, Zhitian Hou, Yuhang Liu, Pengkai Wang, Zeyu Liu, Guanghao Zhu, Zheng Liu, Shuo Cai, Congkai Xie, Zhijie Sang, Kun Zeng, Hongxia Yang (The Hong Kong Polytechnic University; InfiX.ai; Sun Yat-sen University), 4 Aug 2026. arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row Lingshu-32B, column Overall; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair
    Lingshu-32B | 59.5 | 63.7 | 53.0 | 10.7 | 60.6 | 54.4 | 25.8
    Every result from this document
  14. 14GPT-OSS-20B zero-shot; independent questions; exact option-set match 59.1
    Printed as 59.1Independent run, measured Aug 2026Configuration: zero-shot; independent questions; exact option-set match
    Evaluating Counterfactual Sensitivity to Patient Information in Medication-Safety Reasoning paper, Zhitian Hou, Yuhang Liu, Pengkai Wang, Zeyu Liu, Guanghao Zhu, Zheng Liu, Shuo Cai, Congkai Xie, Zhijie Sang, Kun Zeng, Hongxia Yang (The Hong Kong Polytechnic University; InfiX.ai; Sun Yat-sen University), 4 Aug 2026. arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row GPT-OSS-20B, column Overall; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair
    GPT-OSS-20B | 59.1 | 66.5 | 47.5 | 19.0 | 52.1 | 48.5 | 24.7
    Every result from this document
  15. 15Llama-3.1-70B zero-shot; independent questions; exact option-set match 55.7
    Printed as 55.7Independent run, measured Aug 2026Configuration: zero-shot; independent questions; exact option-set match
    Evaluating Counterfactual Sensitivity to Patient Information in Medication-Safety Reasoning paper, Zhitian Hou, Yuhang Liu, Pengkai Wang, Zeyu Liu, Guanghao Zhu, Zheng Liu, Shuo Cai, Congkai Xie, Zhijie Sang, Kun Zeng, Hongxia Yang (The Hong Kong Polytechnic University; InfiX.ai; Sun Yat-sen University), 4 Aug 2026. arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row Llama-3.1-70B, column Overall; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair
    Llama-3.1-70B | 55.7 | 61.3 | 47.0 | 14.3 | 63.4 | 32.4 | 16.9
    Every result from this document
  16. 16HuatuoGPT-o1-70B zero-shot; independent questions; exact option-set match 55.2
    Printed as 55.2Independent run, measured Aug 2026Configuration: zero-shot; independent questions; exact option-set match
    Evaluating Counterfactual Sensitivity to Patient Information in Medication-Safety Reasoning paper, Zhitian Hou, Yuhang Liu, Pengkai Wang, Zeyu Liu, Guanghao Zhu, Zheng Liu, Shuo Cai, Congkai Xie, Zhijie Sang, Kun Zeng, Hongxia Yang (The Hong Kong Polytechnic University; InfiX.ai; Sun Yat-sen University), 4 Aug 2026. arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row HuatuoGPT-o1-70B, column Overall; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair
    HuatuoGPT-o1-70B | 55.2 | 63.7 | 42.1 | 21.7 | 63.4 | 23.5 | 14.6
    Every result from this document
  17. 17Gemini-2.5-Pro zero-shot; independent questions; exact option-set match 54.4
    Printed as 54.4Independent run, measured Aug 2026Configuration: zero-shot; independent questions; exact option-set match
    Evaluating Counterfactual Sensitivity to Patient Information in Medication-Safety Reasoning paper, Zhitian Hou, Yuhang Liu, Pengkai Wang, Zeyu Liu, Guanghao Zhu, Zheng Liu, Shuo Cai, Congkai Xie, Zhijie Sang, Kun Zeng, Hongxia Yang (The Hong Kong Polytechnic University; InfiX.ai; Sun Yat-sen University), 4 Aug 2026. arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row Gemini-2.5-Pro, column Overall; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair
    Gemini-2.5-Pro | 54.4 | 60.6 | 44.8 | 15.8 | 63.4 | 27.9 | 20.2
    Every result from this document
  18. 18Fleming-R1-7B zero-shot; independent questions; exact option-set match 50.7
    Printed as 50.7Independent run, measured Aug 2026Configuration: zero-shot; independent questions; exact option-set match
    Evaluating Counterfactual Sensitivity to Patient Information in Medication-Safety Reasoning paper, Zhitian Hou, Yuhang Liu, Pengkai Wang, Zeyu Liu, Guanghao Zhu, Zheng Liu, Shuo Cai, Congkai Xie, Zhijie Sang, Kun Zeng, Hongxia Yang (The Hong Kong Polytechnic University; InfiX.ai; Sun Yat-sen University), 4 Aug 2026. arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row Fleming-R1-7B, column Overall; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair
    Fleming-R1-7B | 50.7 | 59.9 | 36.6 | 23.2 | 53.5 | 29.4 | 12.4
    Every result from this document
  19. 19Baichuan-M2-32B zero-shot; independent questions; exact option-set match 47.8
    Printed as 47.8Independent run, measured Aug 2026Configuration: zero-shot; independent questions; exact option-set match
    Evaluating Counterfactual Sensitivity to Patient Information in Medication-Safety Reasoning paper, Zhitian Hou, Yuhang Liu, Pengkai Wang, Zeyu Liu, Guanghao Zhu, Zheng Liu, Shuo Cai, Congkai Xie, Zhijie Sang, Kun Zeng, Hongxia Yang (The Hong Kong Polytechnic University; InfiX.ai; Sun Yat-sen University), 4 Aug 2026. arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row Baichuan-M2-32B, column Overall; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair
    Baichuan-M2-32B | 47.8 | 54.6 | 37.2 | 17.4 | 49.3 | 30.9 | 11.2
    Every result from this document
  20. 20HuatuoGPT-o1-8B zero-shot; independent questions; exact option-set match 44.3
    Printed as 44.3Independent run, measured Aug 2026Configuration: zero-shot; independent questions; exact option-set match
    Evaluating Counterfactual Sensitivity to Patient Information in Medication-Safety Reasoning paper, Zhitian Hou, Yuhang Liu, Pengkai Wang, Zeyu Liu, Guanghao Zhu, Zheng Liu, Shuo Cai, Congkai Xie, Zhijie Sang, Kun Zeng, Hongxia Yang (The Hong Kong Polytechnic University; InfiX.ai; Sun Yat-sen University), 4 Aug 2026. arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row HuatuoGPT-o1-8B, column Overall; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair
    HuatuoGPT-o1-8B | 44.3 | 50.0 | 35.5 | 14.5 | 49.3 | 20.6 | 13.5
    Every result from this document
  21. 21Lingshu-7B zero-shot; independent questions; exact option-set match 43.3
    Printed as 43.3Independent run, measured Aug 2026Configuration: zero-shot; independent questions; exact option-set match
    Evaluating Counterfactual Sensitivity to Patient Information in Medication-Safety Reasoning paper, Zhitian Hou, Yuhang Liu, Pengkai Wang, Zeyu Liu, Guanghao Zhu, Zheng Liu, Shuo Cai, Congkai Xie, Zhijie Sang, Kun Zeng, Hongxia Yang (The Hong Kong Polytechnic University; InfiX.ai; Sun Yat-sen University), 4 Aug 2026. arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row Lingshu-7B, column Overall; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair
    Lingshu-7B | 43.3 | 50.7 | 31.7 | 19.0 | 39.4 | 32.4 | 6.7
    Every result from this document
  22. 22HealthGPT-Pro-8B zero-shot; independent questions; exact option-set match 42.6
    Printed as 42.6Independent run, measured Aug 2026Configuration: zero-shot; independent questions; exact option-set match
    Evaluating Counterfactual Sensitivity to Patient Information in Medication-Safety Reasoning paper, Zhitian Hou, Yuhang Liu, Pengkai Wang, Zeyu Liu, Guanghao Zhu, Zheng Liu, Shuo Cai, Congkai Xie, Zhijie Sang, Kun Zeng, Hongxia Yang (The Hong Kong Polytechnic University; InfiX.ai; Sun Yat-sen University), 4 Aug 2026. arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row HealthGPT-Pro-8B, column Overall; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair
    HealthGPT-Pro-8B | 42.6 | 51.8 | 28.4 | 23.3 | 42.3 | 20.6 | 5.6
    Every result from this document
  23. 23Hulu-Med-7B zero-shot; independent questions; exact option-set match 42.0
    Printed as 42.0Independent run, measured Aug 2026Configuration: zero-shot; independent questions; exact option-set match
    Evaluating Counterfactual Sensitivity to Patient Information in Medication-Safety Reasoning paper, Zhitian Hou, Yuhang Liu, Pengkai Wang, Zeyu Liu, Guanghao Zhu, Zheng Liu, Shuo Cai, Congkai Xie, Zhijie Sang, Kun Zeng, Hongxia Yang (The Hong Kong Polytechnic University; InfiX.ai; Sun Yat-sen University), 4 Aug 2026. arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row Hulu-Med-7B, column Overall; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair
    Hulu-Med-7B | 42.0 | 48.2 | 32.2 | 16.0 | 36.6 | 38.2 | 14.6
    Every result from this document
  24. 24Gemma-3-27B zero-shot; independent questions; exact option-set match 40.3
    Printed as 40.3Independent run, measured Aug 2026Configuration: zero-shot; independent questions; exact option-set match
    Evaluating Counterfactual Sensitivity to Patient Information in Medication-Safety Reasoning paper, Zhitian Hou, Yuhang Liu, Pengkai Wang, Zeyu Liu, Guanghao Zhu, Zheng Liu, Shuo Cai, Congkai Xie, Zhijie Sang, Kun Zeng, Hongxia Yang (The Hong Kong Polytechnic University; InfiX.ai; Sun Yat-sen University), 4 Aug 2026. arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row Gemma-3-27B, column Overall; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair
    Gemma-3-27B | 40.3 | 44.7 | 33.3 | 11.4 | 53.5 | 10.3 | 5.6
    Every result from this document
  25. 25Llama-3.1-8B zero-shot; independent questions; exact option-set match 40.0
    Printed as 40.0Independent run, measured Aug 2026Configuration: zero-shot; independent questions; exact option-set match
    Evaluating Counterfactual Sensitivity to Patient Information in Medication-Safety Reasoning paper, Zhitian Hou, Yuhang Liu, Pengkai Wang, Zeyu Liu, Guanghao Zhu, Zheng Liu, Shuo Cai, Congkai Xie, Zhijie Sang, Kun Zeng, Hongxia Yang (The Hong Kong Polytechnic University; InfiX.ai; Sun Yat-sen University), 4 Aug 2026. arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row Llama-3.1-8B, column Overall; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair
    Llama-3.1-8B | 40.0 | 44.4 | 33.3 | 11.0 | 54.9 | 10.3 | 5.6
    Every result from this document
  26. 26Gemma-3-12B zero-shot; independent questions; exact option-set match 39.6
    Printed as 39.6Independent run, measured Aug 2026Configuration: zero-shot; independent questions; exact option-set match
    Evaluating Counterfactual Sensitivity to Patient Information in Medication-Safety Reasoning paper, Zhitian Hou, Yuhang Liu, Pengkai Wang, Zeyu Liu, Guanghao Zhu, Zheng Liu, Shuo Cai, Congkai Xie, Zhijie Sang, Kun Zeng, Hongxia Yang (The Hong Kong Polytechnic University; InfiX.ai; Sun Yat-sen University), 4 Aug 2026. arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row Gemma-3-12B, column Overall; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair
    Gemma-3-12B | 39.6 | 44.0 | 32.8 | 11.2 | 57.7 | 7.4 | 3.4
    Every result from this document
  27. 26MedGemma-4B zero-shot; independent questions; exact option-set match 39.6
    Printed as 39.6Independent run, measured Aug 2026Configuration: zero-shot; independent questions; exact option-set match
    Evaluating Counterfactual Sensitivity to Patient Information in Medication-Safety Reasoning paper, Zhitian Hou, Yuhang Liu, Pengkai Wang, Zeyu Liu, Guanghao Zhu, Zheng Liu, Shuo Cai, Congkai Xie, Zhijie Sang, Kun Zeng, Hongxia Yang (The Hong Kong Polytechnic University; InfiX.ai; Sun Yat-sen University), 4 Aug 2026. arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row MedGemma-4B, column Overall; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair
    MedGemma-4B | 39.6 | 45.4 | 30.6 | 14.8 | 46.5 | 10.3 | 5.6
    Every result from this document
  28. 28HealthGPT-Pro-4B zero-shot; independent questions; exact option-set match 36.0
    Printed as 36.0Independent run, measured Aug 2026Configuration: zero-shot; independent questions; exact option-set match
    Evaluating Counterfactual Sensitivity to Patient Information in Medication-Safety Reasoning paper, Zhitian Hou, Yuhang Liu, Pengkai Wang, Zeyu Liu, Guanghao Zhu, Zheng Liu, Shuo Cai, Congkai Xie, Zhijie Sang, Kun Zeng, Hongxia Yang (The Hong Kong Polytechnic University; InfiX.ai; Sun Yat-sen University), 4 Aug 2026. arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row HealthGPT-Pro-4B, column Overall; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair
    HealthGPT-Pro-4B | 36.0 | 37.7 | 33.3 | 4.3 | 36.6 | 41.2 | 11.2
    Every result from this document
Guideline-following accuracy (GF), 28 rows
  1. 1Gemini-3.1-Pro zero-shot; independent questions; exact option-set match 87.7
    Printed as 87.7Independent run, measured Aug 2026Configuration: zero-shot; independent questions; exact option-set match
    Evaluating Counterfactual Sensitivity to Patient Information in Medication-Safety Reasoning paper, Zhitian Hou, Yuhang Liu, Pengkai Wang, Zeyu Liu, Guanghao Zhu, Zheng Liu, Shuo Cai, Congkai Xie, Zhijie Sang, Kun Zeng, Hongxia Yang (The Hong Kong Polytechnic University; InfiX.ai; Sun Yat-sen University), 4 Aug 2026. arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row Gemini-3.1-Pro, column GF; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair
    Gemini-3.1-Pro | 80.7 | 87.7 | 69.9 | 17.7 | 74.6 | 77.9 | 48.3
    Every result from this document
  2. 2GPT-5 zero-shot; independent questions; exact option-set match 83.5
    Printed as 83.5Independent run, measured Aug 2026Configuration: zero-shot; independent questions; exact option-set match
    Evaluating Counterfactual Sensitivity to Patient Information in Medication-Safety Reasoning paper, Zhitian Hou, Yuhang Liu, Pengkai Wang, Zeyu Liu, Guanghao Zhu, Zheng Liu, Shuo Cai, Congkai Xie, Zhijie Sang, Kun Zeng, Hongxia Yang (The Hong Kong Polytechnic University; InfiX.ai; Sun Yat-sen University), 4 Aug 2026. arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row GPT-5, column GF; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair
    GPT-5 | 75.6 | 83.5 | 63.4 | 20.1 | 70.4 | 61.8 | 38.2
    Every result from this document
  3. 3Qwen3.5-Plus zero-shot; independent questions; exact option-set match 81.7
    Printed as 81.7Independent run, measured Aug 2026Configuration: zero-shot; independent questions; exact option-set match
    Evaluating Counterfactual Sensitivity to Patient Information in Medication-Safety Reasoning paper, Zhitian Hou, Yuhang Liu, Pengkai Wang, Zeyu Liu, Guanghao Zhu, Zheng Liu, Shuo Cai, Congkai Xie, Zhijie Sang, Kun Zeng, Hongxia Yang (The Hong Kong Polytechnic University; InfiX.ai; Sun Yat-sen University), 4 Aug 2026. arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row Qwen3.5-Plus, column GF; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair
    Qwen3.5-Plus | 72.4 | 81.7 | 57.9 | 23.8 | 66.2 | 55.9 | 32.6
    Every result from this document
  4. 4DeepSeek-V4-Pro zero-shot; independent questions; exact option-set match 81.3
    Printed as 81.3Independent run, measured Aug 2026Configuration: zero-shot; independent questions; exact option-set match
    Evaluating Counterfactual Sensitivity to Patient Information in Medication-Safety Reasoning paper, Zhitian Hou, Yuhang Liu, Pengkai Wang, Zeyu Liu, Guanghao Zhu, Zheng Liu, Shuo Cai, Congkai Xie, Zhijie Sang, Kun Zeng, Hongxia Yang (The Hong Kong Polytechnic University; InfiX.ai; Sun Yat-sen University), 4 Aug 2026. arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row DeepSeek-V4-Pro, column GF; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair
    DeepSeek-V4-Pro | 71.7 | 81.3 | 56.8 | 24.5 | 67.6 | 50.0 | 32.6
    Every result from this document
  5. 5GPT-OSS-120B zero-shot; independent questions; exact option-set match 79.6
    Printed as 79.6Independent run, measured Aug 2026Configuration: zero-shot; independent questions; exact option-set match
    Evaluating Counterfactual Sensitivity to Patient Information in Medication-Safety Reasoning paper, Zhitian Hou, Yuhang Liu, Pengkai Wang, Zeyu Liu, Guanghao Zhu, Zheng Liu, Shuo Cai, Congkai Xie, Zhijie Sang, Kun Zeng, Hongxia Yang (The Hong Kong Polytechnic University; InfiX.ai; Sun Yat-sen University), 4 Aug 2026. arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row GPT-OSS-120B, column GF; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair
    GPT-OSS-120B | 70.7 | 79.6 | 56.8 | 22.7 | 62.0 | 55.9 | 30.3
    Every result from this document
  6. 6GPT-5.2 zero-shot; independent questions; exact option-set match 78.2
    Printed as 78.2Independent run, measured Aug 2026Configuration: zero-shot; independent questions; exact option-set match
    Evaluating Counterfactual Sensitivity to Patient Information in Medication-Safety Reasoning paper, Zhitian Hou, Yuhang Liu, Pengkai Wang, Zeyu Liu, Guanghao Zhu, Zheng Liu, Shuo Cai, Congkai Xie, Zhijie Sang, Kun Zeng, Hongxia Yang (The Hong Kong Polytechnic University; InfiX.ai; Sun Yat-sen University), 4 Aug 2026. arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row GPT-5.2, column GF; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair
    GPT-5.2 | 68.1 | 78.2 | 52.5 | 25.7 | 63.4 | 42.6 | 29.2
    Every result from this document
  7. 6Qwen3.5-27B zero-shot; independent questions; exact option-set match 78.2
    Printed as 78.2Independent run, measured Aug 2026Configuration: zero-shot; independent questions; exact option-set match
    Evaluating Counterfactual Sensitivity to Patient Information in Medication-Safety Reasoning paper, Zhitian Hou, Yuhang Liu, Pengkai Wang, Zeyu Liu, Guanghao Zhu, Zheng Liu, Shuo Cai, Congkai Xie, Zhijie Sang, Kun Zeng, Hongxia Yang (The Hong Kong Polytechnic University; InfiX.ai; Sun Yat-sen University), 4 Aug 2026. arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row Qwen3.5-27B, column GF; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair
    Qwen3.5-27B | 69.4 | 78.2 | 55.7 | 22.4 | 67.6 | 51.5 | 31.5
    Every result from this document
  8. 8Qwen3.5-35B-A3B zero-shot; independent questions; exact option-set match 77.5
    Printed as 77.5Independent run, measured Aug 2026Configuration: zero-shot; independent questions; exact option-set match
    Evaluating Counterfactual Sensitivity to Patient Information in Medication-Safety Reasoning paper, Zhitian Hou, Yuhang Liu, Pengkai Wang, Zeyu Liu, Guanghao Zhu, Zheng Liu, Shuo Cai, Congkai Xie, Zhijie Sang, Kun Zeng, Hongxia Yang (The Hong Kong Polytechnic University; InfiX.ai; Sun Yat-sen University), 4 Aug 2026. arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row Qwen3.5-35B-A3B, column GF; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair
    Qwen3.5-35B-A3B | 68.5 | 77.5 | 54.6 | 22.8 | 70.4 | 45.6 | 30.3
    Every result from this document
  9. 9Claude-Sonnet-4.6 zero-shot; independent questions; exact option-set match 74.6
    Printed as 74.6Independent run, measured Aug 2026Configuration: zero-shot; independent questions; exact option-set match
    Evaluating Counterfactual Sensitivity to Patient Information in Medication-Safety Reasoning paper, Zhitian Hou, Yuhang Liu, Pengkai Wang, Zeyu Liu, Guanghao Zhu, Zheng Liu, Shuo Cai, Congkai Xie, Zhijie Sang, Kun Zeng, Hongxia Yang (The Hong Kong Polytechnic University; InfiX.ai; Sun Yat-sen University), 4 Aug 2026. arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row Claude-Sonnet-4.6, column GF; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair
    Claude-Sonnet-4.6 | 64.2 | 74.6 | 48.1 | 26.6 | 62.0 | 35.3 | 21.3
    Every result from this document
  10. 10Qwen3.5-9B zero-shot; independent questions; exact option-set match 73.9
    Printed as 73.9Independent run, measured Aug 2026Configuration: zero-shot; independent questions; exact option-set match
    Evaluating Counterfactual Sensitivity to Patient Information in Medication-Safety Reasoning paper, Zhitian Hou, Yuhang Liu, Pengkai Wang, Zeyu Liu, Guanghao Zhu, Zheng Liu, Shuo Cai, Congkai Xie, Zhijie Sang, Kun Zeng, Hongxia Yang (The Hong Kong Polytechnic University; InfiX.ai; Sun Yat-sen University), 4 Aug 2026. arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row Qwen3.5-9B, column GF; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair
    Qwen3.5-9B | 63.8 | 73.9 | 48.1 | 25.9 | 54.9 | 47.1 | 24.7
    Every result from this document
  11. 11Fleming-R1-32B zero-shot; independent questions; exact option-set match 69.0
    Printed as 69.0Independent run, measured Aug 2026Configuration: zero-shot; independent questions; exact option-set match
    Evaluating Counterfactual Sensitivity to Patient Information in Medication-Safety Reasoning paper, Zhitian Hou, Yuhang Liu, Pengkai Wang, Zeyu Liu, Guanghao Zhu, Zheng Liu, Shuo Cai, Congkai Xie, Zhijie Sang, Kun Zeng, Hongxia Yang (The Hong Kong Polytechnic University; InfiX.ai; Sun Yat-sen University), 4 Aug 2026. arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row Fleming-R1-32B, column GF; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair
    Fleming-R1-32B | 60.6 | 69.0 | 47.5 | 21.5 | 59.2 | 44.1 | 20.2
    Every result from this document
  12. 12MedGemma-27B-Text zero-shot; independent questions; exact option-set match 67.6
    Printed as 67.6Independent run, measured Aug 2026Configuration: zero-shot; independent questions; exact option-set match
    Evaluating Counterfactual Sensitivity to Patient Information in Medication-Safety Reasoning paper, Zhitian Hou, Yuhang Liu, Pengkai Wang, Zeyu Liu, Guanghao Zhu, Zheng Liu, Shuo Cai, Congkai Xie, Zhijie Sang, Kun Zeng, Hongxia Yang (The Hong Kong Polytechnic University; InfiX.ai; Sun Yat-sen University), 4 Aug 2026. arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row MedGemma-27B-Text, column GF; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair
    MedGemma-27B-Text | 61.2 | 67.6 | 51.4 | 16.2 | 76.1 | 32.4 | 23.6
    Every result from this document
  13. 13GPT-OSS-20B zero-shot; independent questions; exact option-set match 66.5
    Printed as 66.5Independent run, measured Aug 2026Configuration: zero-shot; independent questions; exact option-set match
    Evaluating Counterfactual Sensitivity to Patient Information in Medication-Safety Reasoning paper, Zhitian Hou, Yuhang Liu, Pengkai Wang, Zeyu Liu, Guanghao Zhu, Zheng Liu, Shuo Cai, Congkai Xie, Zhijie Sang, Kun Zeng, Hongxia Yang (The Hong Kong Polytechnic University; InfiX.ai; Sun Yat-sen University), 4 Aug 2026. arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row GPT-OSS-20B, column GF; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair
    GPT-OSS-20B | 59.1 | 66.5 | 47.5 | 19.0 | 52.1 | 48.5 | 24.7
    Every result from this document
  14. 14HuatuoGPT-o1-70B zero-shot; independent questions; exact option-set match 63.7
    Printed as 63.7Independent run, measured Aug 2026Configuration: zero-shot; independent questions; exact option-set match
    Evaluating Counterfactual Sensitivity to Patient Information in Medication-Safety Reasoning paper, Zhitian Hou, Yuhang Liu, Pengkai Wang, Zeyu Liu, Guanghao Zhu, Zheng Liu, Shuo Cai, Congkai Xie, Zhijie Sang, Kun Zeng, Hongxia Yang (The Hong Kong Polytechnic University; InfiX.ai; Sun Yat-sen University), 4 Aug 2026. arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row HuatuoGPT-o1-70B, column GF; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair
    HuatuoGPT-o1-70B | 55.2 | 63.7 | 42.1 | 21.7 | 63.4 | 23.5 | 14.6
    Every result from this document
  15. 14Lingshu-32B zero-shot; independent questions; exact option-set match 63.7
    Printed as 63.7Independent run, measured Aug 2026Configuration: zero-shot; independent questions; exact option-set match
    Evaluating Counterfactual Sensitivity to Patient Information in Medication-Safety Reasoning paper, Zhitian Hou, Yuhang Liu, Pengkai Wang, Zeyu Liu, Guanghao Zhu, Zheng Liu, Shuo Cai, Congkai Xie, Zhijie Sang, Kun Zeng, Hongxia Yang (The Hong Kong Polytechnic University; InfiX.ai; Sun Yat-sen University), 4 Aug 2026. arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row Lingshu-32B, column GF; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair
    Lingshu-32B | 59.5 | 63.7 | 53.0 | 10.7 | 60.6 | 54.4 | 25.8
    Every result from this document
  16. 16Llama-3.1-70B zero-shot; independent questions; exact option-set match 61.3
    Printed as 61.3Independent run, measured Aug 2026Configuration: zero-shot; independent questions; exact option-set match
    Evaluating Counterfactual Sensitivity to Patient Information in Medication-Safety Reasoning paper, Zhitian Hou, Yuhang Liu, Pengkai Wang, Zeyu Liu, Guanghao Zhu, Zheng Liu, Shuo Cai, Congkai Xie, Zhijie Sang, Kun Zeng, Hongxia Yang (The Hong Kong Polytechnic University; InfiX.ai; Sun Yat-sen University), 4 Aug 2026. arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row Llama-3.1-70B, column GF; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair
    Llama-3.1-70B | 55.7 | 61.3 | 47.0 | 14.3 | 63.4 | 32.4 | 16.9
    Every result from this document
  17. 17Gemini-2.5-Pro zero-shot; independent questions; exact option-set match 60.6
    Printed as 60.6Independent run, measured Aug 2026Configuration: zero-shot; independent questions; exact option-set match
    Evaluating Counterfactual Sensitivity to Patient Information in Medication-Safety Reasoning paper, Zhitian Hou, Yuhang Liu, Pengkai Wang, Zeyu Liu, Guanghao Zhu, Zheng Liu, Shuo Cai, Congkai Xie, Zhijie Sang, Kun Zeng, Hongxia Yang (The Hong Kong Polytechnic University; InfiX.ai; Sun Yat-sen University), 4 Aug 2026. arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row Gemini-2.5-Pro, column GF; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair
    Gemini-2.5-Pro | 54.4 | 60.6 | 44.8 | 15.8 | 63.4 | 27.9 | 20.2
    Every result from this document
  18. 18Fleming-R1-7B zero-shot; independent questions; exact option-set match 59.9
    Printed as 59.9Independent run, measured Aug 2026Configuration: zero-shot; independent questions; exact option-set match
    Evaluating Counterfactual Sensitivity to Patient Information in Medication-Safety Reasoning paper, Zhitian Hou, Yuhang Liu, Pengkai Wang, Zeyu Liu, Guanghao Zhu, Zheng Liu, Shuo Cai, Congkai Xie, Zhijie Sang, Kun Zeng, Hongxia Yang (The Hong Kong Polytechnic University; InfiX.ai; Sun Yat-sen University), 4 Aug 2026. arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row Fleming-R1-7B, column GF; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair
    Fleming-R1-7B | 50.7 | 59.9 | 36.6 | 23.2 | 53.5 | 29.4 | 12.4
    Every result from this document
  19. 19Baichuan-M2-32B zero-shot; independent questions; exact option-set match 54.6
    Printed as 54.6Independent run, measured Aug 2026Configuration: zero-shot; independent questions; exact option-set match
    Evaluating Counterfactual Sensitivity to Patient Information in Medication-Safety Reasoning paper, Zhitian Hou, Yuhang Liu, Pengkai Wang, Zeyu Liu, Guanghao Zhu, Zheng Liu, Shuo Cai, Congkai Xie, Zhijie Sang, Kun Zeng, Hongxia Yang (The Hong Kong Polytechnic University; InfiX.ai; Sun Yat-sen University), 4 Aug 2026. arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row Baichuan-M2-32B, column GF; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair
    Baichuan-M2-32B | 47.8 | 54.6 | 37.2 | 17.4 | 49.3 | 30.9 | 11.2
    Every result from this document
  20. 20HealthGPT-Pro-8B zero-shot; independent questions; exact option-set match 51.8
    Printed as 51.8Independent run, measured Aug 2026Configuration: zero-shot; independent questions; exact option-set match
    Evaluating Counterfactual Sensitivity to Patient Information in Medication-Safety Reasoning paper, Zhitian Hou, Yuhang Liu, Pengkai Wang, Zeyu Liu, Guanghao Zhu, Zheng Liu, Shuo Cai, Congkai Xie, Zhijie Sang, Kun Zeng, Hongxia Yang (The Hong Kong Polytechnic University; InfiX.ai; Sun Yat-sen University), 4 Aug 2026. arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row HealthGPT-Pro-8B, column GF; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair
    HealthGPT-Pro-8B | 42.6 | 51.8 | 28.4 | 23.3 | 42.3 | 20.6 | 5.6
    Every result from this document
  21. 21Lingshu-7B zero-shot; independent questions; exact option-set match 50.7
    Printed as 50.7Independent run, measured Aug 2026Configuration: zero-shot; independent questions; exact option-set match
    Evaluating Counterfactual Sensitivity to Patient Information in Medication-Safety Reasoning paper, Zhitian Hou, Yuhang Liu, Pengkai Wang, Zeyu Liu, Guanghao Zhu, Zheng Liu, Shuo Cai, Congkai Xie, Zhijie Sang, Kun Zeng, Hongxia Yang (The Hong Kong Polytechnic University; InfiX.ai; Sun Yat-sen University), 4 Aug 2026. arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row Lingshu-7B, column GF; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair
    Lingshu-7B | 43.3 | 50.7 | 31.7 | 19.0 | 39.4 | 32.4 | 6.7
    Every result from this document
  22. 22HuatuoGPT-o1-8B zero-shot; independent questions; exact option-set match 50.0
    Printed as 50.0Independent run, measured Aug 2026Configuration: zero-shot; independent questions; exact option-set match
    Evaluating Counterfactual Sensitivity to Patient Information in Medication-Safety Reasoning paper, Zhitian Hou, Yuhang Liu, Pengkai Wang, Zeyu Liu, Guanghao Zhu, Zheng Liu, Shuo Cai, Congkai Xie, Zhijie Sang, Kun Zeng, Hongxia Yang (The Hong Kong Polytechnic University; InfiX.ai; Sun Yat-sen University), 4 Aug 2026. arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row HuatuoGPT-o1-8B, column GF; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair
    HuatuoGPT-o1-8B | 44.3 | 50.0 | 35.5 | 14.5 | 49.3 | 20.6 | 13.5
    Every result from this document
  23. 23Hulu-Med-7B zero-shot; independent questions; exact option-set match 48.2
    Printed as 48.2Independent run, measured Aug 2026Configuration: zero-shot; independent questions; exact option-set match
    Evaluating Counterfactual Sensitivity to Patient Information in Medication-Safety Reasoning paper, Zhitian Hou, Yuhang Liu, Pengkai Wang, Zeyu Liu, Guanghao Zhu, Zheng Liu, Shuo Cai, Congkai Xie, Zhijie Sang, Kun Zeng, Hongxia Yang (The Hong Kong Polytechnic University; InfiX.ai; Sun Yat-sen University), 4 Aug 2026. arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row Hulu-Med-7B, column GF; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair
    Hulu-Med-7B | 42.0 | 48.2 | 32.2 | 16.0 | 36.6 | 38.2 | 14.6
    Every result from this document
  24. 24MedGemma-4B zero-shot; independent questions; exact option-set match 45.4
    Printed as 45.4Independent run, measured Aug 2026Configuration: zero-shot; independent questions; exact option-set match
    Evaluating Counterfactual Sensitivity to Patient Information in Medication-Safety Reasoning paper, Zhitian Hou, Yuhang Liu, Pengkai Wang, Zeyu Liu, Guanghao Zhu, Zheng Liu, Shuo Cai, Congkai Xie, Zhijie Sang, Kun Zeng, Hongxia Yang (The Hong Kong Polytechnic University; InfiX.ai; Sun Yat-sen University), 4 Aug 2026. arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row MedGemma-4B, column GF; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair
    MedGemma-4B | 39.6 | 45.4 | 30.6 | 14.8 | 46.5 | 10.3 | 5.6
    Every result from this document
  25. 25Gemma-3-27B zero-shot; independent questions; exact option-set match 44.7
    Printed as 44.7Independent run, measured Aug 2026Configuration: zero-shot; independent questions; exact option-set match
    Evaluating Counterfactual Sensitivity to Patient Information in Medication-Safety Reasoning paper, Zhitian Hou, Yuhang Liu, Pengkai Wang, Zeyu Liu, Guanghao Zhu, Zheng Liu, Shuo Cai, Congkai Xie, Zhijie Sang, Kun Zeng, Hongxia Yang (The Hong Kong Polytechnic University; InfiX.ai; Sun Yat-sen University), 4 Aug 2026. arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row Gemma-3-27B, column GF; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair
    Gemma-3-27B | 40.3 | 44.7 | 33.3 | 11.4 | 53.5 | 10.3 | 5.6
    Every result from this document
  26. 26Llama-3.1-8B zero-shot; independent questions; exact option-set match 44.4
    Printed as 44.4Independent run, measured Aug 2026Configuration: zero-shot; independent questions; exact option-set match
    Evaluating Counterfactual Sensitivity to Patient Information in Medication-Safety Reasoning paper, Zhitian Hou, Yuhang Liu, Pengkai Wang, Zeyu Liu, Guanghao Zhu, Zheng Liu, Shuo Cai, Congkai Xie, Zhijie Sang, Kun Zeng, Hongxia Yang (The Hong Kong Polytechnic University; InfiX.ai; Sun Yat-sen University), 4 Aug 2026. arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row Llama-3.1-8B, column GF; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair
    Llama-3.1-8B | 40.0 | 44.4 | 33.3 | 11.0 | 54.9 | 10.3 | 5.6
    Every result from this document
  27. 27Gemma-3-12B zero-shot; independent questions; exact option-set match 44.0
    Printed as 44.0Independent run, measured Aug 2026Configuration: zero-shot; independent questions; exact option-set match
    Evaluating Counterfactual Sensitivity to Patient Information in Medication-Safety Reasoning paper, Zhitian Hou, Yuhang Liu, Pengkai Wang, Zeyu Liu, Guanghao Zhu, Zheng Liu, Shuo Cai, Congkai Xie, Zhijie Sang, Kun Zeng, Hongxia Yang (The Hong Kong Polytechnic University; InfiX.ai; Sun Yat-sen University), 4 Aug 2026. arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row Gemma-3-12B, column GF; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair
    Gemma-3-12B | 39.6 | 44.0 | 32.8 | 11.2 | 57.7 | 7.4 | 3.4
    Every result from this document
  28. 28HealthGPT-Pro-4B zero-shot; independent questions; exact option-set match 37.7
    Printed as 37.7Independent run, measured Aug 2026Configuration: zero-shot; independent questions; exact option-set match
    Evaluating Counterfactual Sensitivity to Patient Information in Medication-Safety Reasoning paper, Zhitian Hou, Yuhang Liu, Pengkai Wang, Zeyu Liu, Guanghao Zhu, Zheng Liu, Shuo Cai, Congkai Xie, Zhijie Sang, Kun Zeng, Hongxia Yang (The Hong Kong Polytechnic University; InfiX.ai; Sun Yat-sen University), 4 Aug 2026. arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row HealthGPT-Pro-4B, column GF; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair
    HealthGPT-Pro-4B | 36.0 | 37.7 | 33.3 | 4.3 | 36.6 | 41.2 | 11.2
    Every result from this document
Counterfactual accuracy (CF), 28 rows
  1. 1Gemini-3.1-Pro zero-shot; independent questions; exact option-set match 69.9
    Printed as 69.9Independent run, measured Aug 2026Configuration: zero-shot; independent questions; exact option-set match
    Evaluating Counterfactual Sensitivity to Patient Information in Medication-Safety Reasoning paper, Zhitian Hou, Yuhang Liu, Pengkai Wang, Zeyu Liu, Guanghao Zhu, Zheng Liu, Shuo Cai, Congkai Xie, Zhijie Sang, Kun Zeng, Hongxia Yang (The Hong Kong Polytechnic University; InfiX.ai; Sun Yat-sen University), 4 Aug 2026. arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row Gemini-3.1-Pro, column CF; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair
    Gemini-3.1-Pro | 80.7 | 87.7 | 69.9 | 17.7 | 74.6 | 77.9 | 48.3
    Every result from this document
  2. 2GPT-5 zero-shot; independent questions; exact option-set match 63.4
    Printed as 63.4Independent run, measured Aug 2026Configuration: zero-shot; independent questions; exact option-set match
    Evaluating Counterfactual Sensitivity to Patient Information in Medication-Safety Reasoning paper, Zhitian Hou, Yuhang Liu, Pengkai Wang, Zeyu Liu, Guanghao Zhu, Zheng Liu, Shuo Cai, Congkai Xie, Zhijie Sang, Kun Zeng, Hongxia Yang (The Hong Kong Polytechnic University; InfiX.ai; Sun Yat-sen University), 4 Aug 2026. arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row GPT-5, column CF; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair
    GPT-5 | 75.6 | 83.5 | 63.4 | 20.1 | 70.4 | 61.8 | 38.2
    Every result from this document
  3. 3Qwen3.5-Plus zero-shot; independent questions; exact option-set match 57.9
    Printed as 57.9Independent run, measured Aug 2026Configuration: zero-shot; independent questions; exact option-set match
    Evaluating Counterfactual Sensitivity to Patient Information in Medication-Safety Reasoning paper, Zhitian Hou, Yuhang Liu, Pengkai Wang, Zeyu Liu, Guanghao Zhu, Zheng Liu, Shuo Cai, Congkai Xie, Zhijie Sang, Kun Zeng, Hongxia Yang (The Hong Kong Polytechnic University; InfiX.ai; Sun Yat-sen University), 4 Aug 2026. arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row Qwen3.5-Plus, column CF; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair
    Qwen3.5-Plus | 72.4 | 81.7 | 57.9 | 23.8 | 66.2 | 55.9 | 32.6
    Every result from this document
  4. 4DeepSeek-V4-Pro zero-shot; independent questions; exact option-set match 56.8
    Printed as 56.8Independent run, measured Aug 2026Configuration: zero-shot; independent questions; exact option-set match
    Evaluating Counterfactual Sensitivity to Patient Information in Medication-Safety Reasoning paper, Zhitian Hou, Yuhang Liu, Pengkai Wang, Zeyu Liu, Guanghao Zhu, Zheng Liu, Shuo Cai, Congkai Xie, Zhijie Sang, Kun Zeng, Hongxia Yang (The Hong Kong Polytechnic University; InfiX.ai; Sun Yat-sen University), 4 Aug 2026. arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row DeepSeek-V4-Pro, column CF; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair
    DeepSeek-V4-Pro | 71.7 | 81.3 | 56.8 | 24.5 | 67.6 | 50.0 | 32.6
    Every result from this document
  5. 4GPT-OSS-120B zero-shot; independent questions; exact option-set match 56.8
    Printed as 56.8Independent run, measured Aug 2026Configuration: zero-shot; independent questions; exact option-set match
    Evaluating Counterfactual Sensitivity to Patient Information in Medication-Safety Reasoning paper, Zhitian Hou, Yuhang Liu, Pengkai Wang, Zeyu Liu, Guanghao Zhu, Zheng Liu, Shuo Cai, Congkai Xie, Zhijie Sang, Kun Zeng, Hongxia Yang (The Hong Kong Polytechnic University; InfiX.ai; Sun Yat-sen University), 4 Aug 2026. arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row GPT-OSS-120B, column CF; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair
    GPT-OSS-120B | 70.7 | 79.6 | 56.8 | 22.7 | 62.0 | 55.9 | 30.3
    Every result from this document
  6. 6Qwen3.5-27B zero-shot; independent questions; exact option-set match 55.7
    Printed as 55.7Independent run, measured Aug 2026Configuration: zero-shot; independent questions; exact option-set match
    Evaluating Counterfactual Sensitivity to Patient Information in Medication-Safety Reasoning paper, Zhitian Hou, Yuhang Liu, Pengkai Wang, Zeyu Liu, Guanghao Zhu, Zheng Liu, Shuo Cai, Congkai Xie, Zhijie Sang, Kun Zeng, Hongxia Yang (The Hong Kong Polytechnic University; InfiX.ai; Sun Yat-sen University), 4 Aug 2026. arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row Qwen3.5-27B, column CF; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair
    Qwen3.5-27B | 69.4 | 78.2 | 55.7 | 22.4 | 67.6 | 51.5 | 31.5
    Every result from this document
  7. 7Qwen3.5-35B-A3B zero-shot; independent questions; exact option-set match 54.6
    Printed as 54.6Independent run, measured Aug 2026Configuration: zero-shot; independent questions; exact option-set match
    Evaluating Counterfactual Sensitivity to Patient Information in Medication-Safety Reasoning paper, Zhitian Hou, Yuhang Liu, Pengkai Wang, Zeyu Liu, Guanghao Zhu, Zheng Liu, Shuo Cai, Congkai Xie, Zhijie Sang, Kun Zeng, Hongxia Yang (The Hong Kong Polytechnic University; InfiX.ai; Sun Yat-sen University), 4 Aug 2026. arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row Qwen3.5-35B-A3B, column CF; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair
    Qwen3.5-35B-A3B | 68.5 | 77.5 | 54.6 | 22.8 | 70.4 | 45.6 | 30.3
    Every result from this document
  8. 8Lingshu-32B zero-shot; independent questions; exact option-set match 53.0
    Printed as 53.0Independent run, measured Aug 2026Configuration: zero-shot; independent questions; exact option-set match
    Evaluating Counterfactual Sensitivity to Patient Information in Medication-Safety Reasoning paper, Zhitian Hou, Yuhang Liu, Pengkai Wang, Zeyu Liu, Guanghao Zhu, Zheng Liu, Shuo Cai, Congkai Xie, Zhijie Sang, Kun Zeng, Hongxia Yang (The Hong Kong Polytechnic University; InfiX.ai; Sun Yat-sen University), 4 Aug 2026. arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row Lingshu-32B, column CF; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair
    Lingshu-32B | 59.5 | 63.7 | 53.0 | 10.7 | 60.6 | 54.4 | 25.8
    Every result from this document
  9. 9GPT-5.2 zero-shot; independent questions; exact option-set match 52.5
    Printed as 52.5Independent run, measured Aug 2026Configuration: zero-shot; independent questions; exact option-set match
    Evaluating Counterfactual Sensitivity to Patient Information in Medication-Safety Reasoning paper, Zhitian Hou, Yuhang Liu, Pengkai Wang, Zeyu Liu, Guanghao Zhu, Zheng Liu, Shuo Cai, Congkai Xie, Zhijie Sang, Kun Zeng, Hongxia Yang (The Hong Kong Polytechnic University; InfiX.ai; Sun Yat-sen University), 4 Aug 2026. arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row GPT-5.2, column CF; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair
    GPT-5.2 | 68.1 | 78.2 | 52.5 | 25.7 | 63.4 | 42.6 | 29.2
    Every result from this document
  10. 10MedGemma-27B-Text zero-shot; independent questions; exact option-set match 51.4
    Printed as 51.4Independent run, measured Aug 2026Configuration: zero-shot; independent questions; exact option-set match
    Evaluating Counterfactual Sensitivity to Patient Information in Medication-Safety Reasoning paper, Zhitian Hou, Yuhang Liu, Pengkai Wang, Zeyu Liu, Guanghao Zhu, Zheng Liu, Shuo Cai, Congkai Xie, Zhijie Sang, Kun Zeng, Hongxia Yang (The Hong Kong Polytechnic University; InfiX.ai; Sun Yat-sen University), 4 Aug 2026. arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row MedGemma-27B-Text, column CF; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair
    MedGemma-27B-Text | 61.2 | 67.6 | 51.4 | 16.2 | 76.1 | 32.4 | 23.6
    Every result from this document
  11. 11Claude-Sonnet-4.6 zero-shot; independent questions; exact option-set match 48.1
    Printed as 48.1Independent run, measured Aug 2026Configuration: zero-shot; independent questions; exact option-set match
    Evaluating Counterfactual Sensitivity to Patient Information in Medication-Safety Reasoning paper, Zhitian Hou, Yuhang Liu, Pengkai Wang, Zeyu Liu, Guanghao Zhu, Zheng Liu, Shuo Cai, Congkai Xie, Zhijie Sang, Kun Zeng, Hongxia Yang (The Hong Kong Polytechnic University; InfiX.ai; Sun Yat-sen University), 4 Aug 2026. arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row Claude-Sonnet-4.6, column CF; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair
    Claude-Sonnet-4.6 | 64.2 | 74.6 | 48.1 | 26.6 | 62.0 | 35.3 | 21.3
    Every result from this document
  12. 11Qwen3.5-9B zero-shot; independent questions; exact option-set match 48.1
    Printed as 48.1Independent run, measured Aug 2026Configuration: zero-shot; independent questions; exact option-set match
    Evaluating Counterfactual Sensitivity to Patient Information in Medication-Safety Reasoning paper, Zhitian Hou, Yuhang Liu, Pengkai Wang, Zeyu Liu, Guanghao Zhu, Zheng Liu, Shuo Cai, Congkai Xie, Zhijie Sang, Kun Zeng, Hongxia Yang (The Hong Kong Polytechnic University; InfiX.ai; Sun Yat-sen University), 4 Aug 2026. arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row Qwen3.5-9B, column CF; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair
    Qwen3.5-9B | 63.8 | 73.9 | 48.1 | 25.9 | 54.9 | 47.1 | 24.7
    Every result from this document
  13. 13Fleming-R1-32B zero-shot; independent questions; exact option-set match 47.5
    Printed as 47.5Independent run, measured Aug 2026Configuration: zero-shot; independent questions; exact option-set match
    Evaluating Counterfactual Sensitivity to Patient Information in Medication-Safety Reasoning paper, Zhitian Hou, Yuhang Liu, Pengkai Wang, Zeyu Liu, Guanghao Zhu, Zheng Liu, Shuo Cai, Congkai Xie, Zhijie Sang, Kun Zeng, Hongxia Yang (The Hong Kong Polytechnic University; InfiX.ai; Sun Yat-sen University), 4 Aug 2026. arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row Fleming-R1-32B, column CF; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair
    Fleming-R1-32B | 60.6 | 69.0 | 47.5 | 21.5 | 59.2 | 44.1 | 20.2
    Every result from this document
  14. 13GPT-OSS-20B zero-shot; independent questions; exact option-set match 47.5
    Printed as 47.5Independent run, measured Aug 2026Configuration: zero-shot; independent questions; exact option-set match
    Evaluating Counterfactual Sensitivity to Patient Information in Medication-Safety Reasoning paper, Zhitian Hou, Yuhang Liu, Pengkai Wang, Zeyu Liu, Guanghao Zhu, Zheng Liu, Shuo Cai, Congkai Xie, Zhijie Sang, Kun Zeng, Hongxia Yang (The Hong Kong Polytechnic University; InfiX.ai; Sun Yat-sen University), 4 Aug 2026. arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row GPT-OSS-20B, column CF; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair
    GPT-OSS-20B | 59.1 | 66.5 | 47.5 | 19.0 | 52.1 | 48.5 | 24.7
    Every result from this document
  15. 15Llama-3.1-70B zero-shot; independent questions; exact option-set match 47.0
    Printed as 47.0Independent run, measured Aug 2026Configuration: zero-shot; independent questions; exact option-set match
    Evaluating Counterfactual Sensitivity to Patient Information in Medication-Safety Reasoning paper, Zhitian Hou, Yuhang Liu, Pengkai Wang, Zeyu Liu, Guanghao Zhu, Zheng Liu, Shuo Cai, Congkai Xie, Zhijie Sang, Kun Zeng, Hongxia Yang (The Hong Kong Polytechnic University; InfiX.ai; Sun Yat-sen University), 4 Aug 2026. arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row Llama-3.1-70B, column CF; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair
    Llama-3.1-70B | 55.7 | 61.3 | 47.0 | 14.3 | 63.4 | 32.4 | 16.9
    Every result from this document
  16. 16Gemini-2.5-Pro zero-shot; independent questions; exact option-set match 44.8
    Printed as 44.8Independent run, measured Aug 2026Configuration: zero-shot; independent questions; exact option-set match
    Evaluating Counterfactual Sensitivity to Patient Information in Medication-Safety Reasoning paper, Zhitian Hou, Yuhang Liu, Pengkai Wang, Zeyu Liu, Guanghao Zhu, Zheng Liu, Shuo Cai, Congkai Xie, Zhijie Sang, Kun Zeng, Hongxia Yang (The Hong Kong Polytechnic University; InfiX.ai; Sun Yat-sen University), 4 Aug 2026. arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row Gemini-2.5-Pro, column CF; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair
    Gemini-2.5-Pro | 54.4 | 60.6 | 44.8 | 15.8 | 63.4 | 27.9 | 20.2
    Every result from this document
  17. 17HuatuoGPT-o1-70B zero-shot; independent questions; exact option-set match 42.1
    Printed as 42.1Independent run, measured Aug 2026Configuration: zero-shot; independent questions; exact option-set match
    Evaluating Counterfactual Sensitivity to Patient Information in Medication-Safety Reasoning paper, Zhitian Hou, Yuhang Liu, Pengkai Wang, Zeyu Liu, Guanghao Zhu, Zheng Liu, Shuo Cai, Congkai Xie, Zhijie Sang, Kun Zeng, Hongxia Yang (The Hong Kong Polytechnic University; InfiX.ai; Sun Yat-sen University), 4 Aug 2026. arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row HuatuoGPT-o1-70B, column CF; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair
    HuatuoGPT-o1-70B | 55.2 | 63.7 | 42.1 | 21.7 | 63.4 | 23.5 | 14.6
    Every result from this document
  18. 18Baichuan-M2-32B zero-shot; independent questions; exact option-set match 37.2
    Printed as 37.2Independent run, measured Aug 2026Configuration: zero-shot; independent questions; exact option-set match
    Evaluating Counterfactual Sensitivity to Patient Information in Medication-Safety Reasoning paper, Zhitian Hou, Yuhang Liu, Pengkai Wang, Zeyu Liu, Guanghao Zhu, Zheng Liu, Shuo Cai, Congkai Xie, Zhijie Sang, Kun Zeng, Hongxia Yang (The Hong Kong Polytechnic University; InfiX.ai; Sun Yat-sen University), 4 Aug 2026. arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row Baichuan-M2-32B, column CF; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair
    Baichuan-M2-32B | 47.8 | 54.6 | 37.2 | 17.4 | 49.3 | 30.9 | 11.2
    Every result from this document
  19. 19Fleming-R1-7B zero-shot; independent questions; exact option-set match 36.6
    Printed as 36.6Independent run, measured Aug 2026Configuration: zero-shot; independent questions; exact option-set match
    Evaluating Counterfactual Sensitivity to Patient Information in Medication-Safety Reasoning paper, Zhitian Hou, Yuhang Liu, Pengkai Wang, Zeyu Liu, Guanghao Zhu, Zheng Liu, Shuo Cai, Congkai Xie, Zhijie Sang, Kun Zeng, Hongxia Yang (The Hong Kong Polytechnic University; InfiX.ai; Sun Yat-sen University), 4 Aug 2026. arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row Fleming-R1-7B, column CF; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair
    Fleming-R1-7B | 50.7 | 59.9 | 36.6 | 23.2 | 53.5 | 29.4 | 12.4
    Every result from this document
  20. 20HuatuoGPT-o1-8B zero-shot; independent questions; exact option-set match 35.5
    Printed as 35.5Independent run, measured Aug 2026Configuration: zero-shot; independent questions; exact option-set match
    Evaluating Counterfactual Sensitivity to Patient Information in Medication-Safety Reasoning paper, Zhitian Hou, Yuhang Liu, Pengkai Wang, Zeyu Liu, Guanghao Zhu, Zheng Liu, Shuo Cai, Congkai Xie, Zhijie Sang, Kun Zeng, Hongxia Yang (The Hong Kong Polytechnic University; InfiX.ai; Sun Yat-sen University), 4 Aug 2026. arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row HuatuoGPT-o1-8B, column CF; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair
    HuatuoGPT-o1-8B | 44.3 | 50.0 | 35.5 | 14.5 | 49.3 | 20.6 | 13.5
    Every result from this document
  21. 21Gemma-3-27B zero-shot; independent questions; exact option-set match 33.3
    Printed as 33.3Independent run, measured Aug 2026Configuration: zero-shot; independent questions; exact option-set match
    Evaluating Counterfactual Sensitivity to Patient Information in Medication-Safety Reasoning paper, Zhitian Hou, Yuhang Liu, Pengkai Wang, Zeyu Liu, Guanghao Zhu, Zheng Liu, Shuo Cai, Congkai Xie, Zhijie Sang, Kun Zeng, Hongxia Yang (The Hong Kong Polytechnic University; InfiX.ai; Sun Yat-sen University), 4 Aug 2026. arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row Gemma-3-27B, column CF; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair
    Gemma-3-27B | 40.3 | 44.7 | 33.3 | 11.4 | 53.5 | 10.3 | 5.6
    Every result from this document
  22. 21HealthGPT-Pro-4B zero-shot; independent questions; exact option-set match 33.3
    Printed as 33.3Independent run, measured Aug 2026Configuration: zero-shot; independent questions; exact option-set match
    Evaluating Counterfactual Sensitivity to Patient Information in Medication-Safety Reasoning paper, Zhitian Hou, Yuhang Liu, Pengkai Wang, Zeyu Liu, Guanghao Zhu, Zheng Liu, Shuo Cai, Congkai Xie, Zhijie Sang, Kun Zeng, Hongxia Yang (The Hong Kong Polytechnic University; InfiX.ai; Sun Yat-sen University), 4 Aug 2026. arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row HealthGPT-Pro-4B, column CF; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair
    HealthGPT-Pro-4B | 36.0 | 37.7 | 33.3 | 4.3 | 36.6 | 41.2 | 11.2
    Every result from this document
  23. 21Llama-3.1-8B zero-shot; independent questions; exact option-set match 33.3
    Printed as 33.3Independent run, measured Aug 2026Configuration: zero-shot; independent questions; exact option-set match
    Evaluating Counterfactual Sensitivity to Patient Information in Medication-Safety Reasoning paper, Zhitian Hou, Yuhang Liu, Pengkai Wang, Zeyu Liu, Guanghao Zhu, Zheng Liu, Shuo Cai, Congkai Xie, Zhijie Sang, Kun Zeng, Hongxia Yang (The Hong Kong Polytechnic University; InfiX.ai; Sun Yat-sen University), 4 Aug 2026. arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row Llama-3.1-8B, column CF; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair
    Llama-3.1-8B | 40.0 | 44.4 | 33.3 | 11.0 | 54.9 | 10.3 | 5.6
    Every result from this document
  24. 24Gemma-3-12B zero-shot; independent questions; exact option-set match 32.8
    Printed as 32.8Independent run, measured Aug 2026Configuration: zero-shot; independent questions; exact option-set match
    Evaluating Counterfactual Sensitivity to Patient Information in Medication-Safety Reasoning paper, Zhitian Hou, Yuhang Liu, Pengkai Wang, Zeyu Liu, Guanghao Zhu, Zheng Liu, Shuo Cai, Congkai Xie, Zhijie Sang, Kun Zeng, Hongxia Yang (The Hong Kong Polytechnic University; InfiX.ai; Sun Yat-sen University), 4 Aug 2026. arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row Gemma-3-12B, column CF; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair
    Gemma-3-12B | 39.6 | 44.0 | 32.8 | 11.2 | 57.7 | 7.4 | 3.4
    Every result from this document
  25. 25Hulu-Med-7B zero-shot; independent questions; exact option-set match 32.2
    Printed as 32.2Independent run, measured Aug 2026Configuration: zero-shot; independent questions; exact option-set match
    Evaluating Counterfactual Sensitivity to Patient Information in Medication-Safety Reasoning paper, Zhitian Hou, Yuhang Liu, Pengkai Wang, Zeyu Liu, Guanghao Zhu, Zheng Liu, Shuo Cai, Congkai Xie, Zhijie Sang, Kun Zeng, Hongxia Yang (The Hong Kong Polytechnic University; InfiX.ai; Sun Yat-sen University), 4 Aug 2026. arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row Hulu-Med-7B, column CF; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair
    Hulu-Med-7B | 42.0 | 48.2 | 32.2 | 16.0 | 36.6 | 38.2 | 14.6
    Every result from this document
  26. 26Lingshu-7B zero-shot; independent questions; exact option-set match 31.7
    Printed as 31.7Independent run, measured Aug 2026Configuration: zero-shot; independent questions; exact option-set match
    Evaluating Counterfactual Sensitivity to Patient Information in Medication-Safety Reasoning paper, Zhitian Hou, Yuhang Liu, Pengkai Wang, Zeyu Liu, Guanghao Zhu, Zheng Liu, Shuo Cai, Congkai Xie, Zhijie Sang, Kun Zeng, Hongxia Yang (The Hong Kong Polytechnic University; InfiX.ai; Sun Yat-sen University), 4 Aug 2026. arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row Lingshu-7B, column CF; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair
    Lingshu-7B | 43.3 | 50.7 | 31.7 | 19.0 | 39.4 | 32.4 | 6.7
    Every result from this document
  27. 27MedGemma-4B zero-shot; independent questions; exact option-set match 30.6
    Printed as 30.6Independent run, measured Aug 2026Configuration: zero-shot; independent questions; exact option-set match
    Evaluating Counterfactual Sensitivity to Patient Information in Medication-Safety Reasoning paper, Zhitian Hou, Yuhang Liu, Pengkai Wang, Zeyu Liu, Guanghao Zhu, Zheng Liu, Shuo Cai, Congkai Xie, Zhijie Sang, Kun Zeng, Hongxia Yang (The Hong Kong Polytechnic University; InfiX.ai; Sun Yat-sen University), 4 Aug 2026. arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row MedGemma-4B, column CF; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair
    MedGemma-4B | 39.6 | 45.4 | 30.6 | 14.8 | 46.5 | 10.3 | 5.6
    Every result from this document
  28. 28HealthGPT-Pro-8B zero-shot; independent questions; exact option-set match 28.4
    Printed as 28.4Independent run, measured Aug 2026Configuration: zero-shot; independent questions; exact option-set match
    Evaluating Counterfactual Sensitivity to Patient Information in Medication-Safety Reasoning paper, Zhitian Hou, Yuhang Liu, Pengkai Wang, Zeyu Liu, Guanghao Zhu, Zheng Liu, Shuo Cai, Congkai Xie, Zhijie Sang, Kun Zeng, Hongxia Yang (The Hong Kong Polytechnic University; InfiX.ai; Sun Yat-sen University), 4 Aug 2026. arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row HealthGPT-Pro-8B, column CF; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair
    HealthGPT-Pro-8B | 42.6 | 51.8 | 28.4 | 23.3 | 42.3 | 20.6 | 5.6
    Every result from this document
Linked counterfactual pair accuracy, 28 rows
  1. 1Gemini-3.1-Pro zero-shot; independent questions; exact option-set match 48.3
    Printed as 48.3Independent run, measured Aug 2026Configuration: zero-shot; independent questions; exact option-set match
    Evaluating Counterfactual Sensitivity to Patient Information in Medication-Safety Reasoning paper, Zhitian Hou, Yuhang Liu, Pengkai Wang, Zeyu Liu, Guanghao Zhu, Zheng Liu, Shuo Cai, Congkai Xie, Zhijie Sang, Kun Zeng, Hongxia Yang (The Hong Kong Polytechnic University; InfiX.ai; Sun Yat-sen University), 4 Aug 2026. arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row Gemini-3.1-Pro, column Pair; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair
    Gemini-3.1-Pro | 80.7 | 87.7 | 69.9 | 17.7 | 74.6 | 77.9 | 48.3
    Every result from this document
  2. 2GPT-5 zero-shot; independent questions; exact option-set match 38.2
    Printed as 38.2Independent run, measured Aug 2026Configuration: zero-shot; independent questions; exact option-set match
    Evaluating Counterfactual Sensitivity to Patient Information in Medication-Safety Reasoning paper, Zhitian Hou, Yuhang Liu, Pengkai Wang, Zeyu Liu, Guanghao Zhu, Zheng Liu, Shuo Cai, Congkai Xie, Zhijie Sang, Kun Zeng, Hongxia Yang (The Hong Kong Polytechnic University; InfiX.ai; Sun Yat-sen University), 4 Aug 2026. arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row GPT-5, column Pair; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair
    GPT-5 | 75.6 | 83.5 | 63.4 | 20.1 | 70.4 | 61.8 | 38.2
    Every result from this document
  3. 3DeepSeek-V4-Pro zero-shot; independent questions; exact option-set match 32.6
    Printed as 32.6Independent run, measured Aug 2026Configuration: zero-shot; independent questions; exact option-set match
    Evaluating Counterfactual Sensitivity to Patient Information in Medication-Safety Reasoning paper, Zhitian Hou, Yuhang Liu, Pengkai Wang, Zeyu Liu, Guanghao Zhu, Zheng Liu, Shuo Cai, Congkai Xie, Zhijie Sang, Kun Zeng, Hongxia Yang (The Hong Kong Polytechnic University; InfiX.ai; Sun Yat-sen University), 4 Aug 2026. arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row DeepSeek-V4-Pro, column Pair; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair
    DeepSeek-V4-Pro | 71.7 | 81.3 | 56.8 | 24.5 | 67.6 | 50.0 | 32.6
    Every result from this document
  4. 3Qwen3.5-Plus zero-shot; independent questions; exact option-set match 32.6
    Printed as 32.6Independent run, measured Aug 2026Configuration: zero-shot; independent questions; exact option-set match
    Evaluating Counterfactual Sensitivity to Patient Information in Medication-Safety Reasoning paper, Zhitian Hou, Yuhang Liu, Pengkai Wang, Zeyu Liu, Guanghao Zhu, Zheng Liu, Shuo Cai, Congkai Xie, Zhijie Sang, Kun Zeng, Hongxia Yang (The Hong Kong Polytechnic University; InfiX.ai; Sun Yat-sen University), 4 Aug 2026. arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row Qwen3.5-Plus, column Pair; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair
    Qwen3.5-Plus | 72.4 | 81.7 | 57.9 | 23.8 | 66.2 | 55.9 | 32.6
    Every result from this document
  5. 5Qwen3.5-27B zero-shot; independent questions; exact option-set match 31.5
    Printed as 31.5Independent run, measured Aug 2026Configuration: zero-shot; independent questions; exact option-set match
    Evaluating Counterfactual Sensitivity to Patient Information in Medication-Safety Reasoning paper, Zhitian Hou, Yuhang Liu, Pengkai Wang, Zeyu Liu, Guanghao Zhu, Zheng Liu, Shuo Cai, Congkai Xie, Zhijie Sang, Kun Zeng, Hongxia Yang (The Hong Kong Polytechnic University; InfiX.ai; Sun Yat-sen University), 4 Aug 2026. arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row Qwen3.5-27B, column Pair; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair
    Qwen3.5-27B | 69.4 | 78.2 | 55.7 | 22.4 | 67.6 | 51.5 | 31.5
    Every result from this document
  6. 6GPT-OSS-120B zero-shot; independent questions; exact option-set match 30.3
    Printed as 30.3Independent run, measured Aug 2026Configuration: zero-shot; independent questions; exact option-set match
    Evaluating Counterfactual Sensitivity to Patient Information in Medication-Safety Reasoning paper, Zhitian Hou, Yuhang Liu, Pengkai Wang, Zeyu Liu, Guanghao Zhu, Zheng Liu, Shuo Cai, Congkai Xie, Zhijie Sang, Kun Zeng, Hongxia Yang (The Hong Kong Polytechnic University; InfiX.ai; Sun Yat-sen University), 4 Aug 2026. arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row GPT-OSS-120B, column Pair; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair
    GPT-OSS-120B | 70.7 | 79.6 | 56.8 | 22.7 | 62.0 | 55.9 | 30.3
    Every result from this document
  7. 6Qwen3.5-35B-A3B zero-shot; independent questions; exact option-set match 30.3
    Printed as 30.3Independent run, measured Aug 2026Configuration: zero-shot; independent questions; exact option-set match
    Evaluating Counterfactual Sensitivity to Patient Information in Medication-Safety Reasoning paper, Zhitian Hou, Yuhang Liu, Pengkai Wang, Zeyu Liu, Guanghao Zhu, Zheng Liu, Shuo Cai, Congkai Xie, Zhijie Sang, Kun Zeng, Hongxia Yang (The Hong Kong Polytechnic University; InfiX.ai; Sun Yat-sen University), 4 Aug 2026. arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row Qwen3.5-35B-A3B, column Pair; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair
    Qwen3.5-35B-A3B | 68.5 | 77.5 | 54.6 | 22.8 | 70.4 | 45.6 | 30.3
    Every result from this document
  8. 8GPT-5.2 zero-shot; independent questions; exact option-set match 29.2
    Printed as 29.2Independent run, measured Aug 2026Configuration: zero-shot; independent questions; exact option-set match
    Evaluating Counterfactual Sensitivity to Patient Information in Medication-Safety Reasoning paper, Zhitian Hou, Yuhang Liu, Pengkai Wang, Zeyu Liu, Guanghao Zhu, Zheng Liu, Shuo Cai, Congkai Xie, Zhijie Sang, Kun Zeng, Hongxia Yang (The Hong Kong Polytechnic University; InfiX.ai; Sun Yat-sen University), 4 Aug 2026. arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row GPT-5.2, column Pair; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair
    GPT-5.2 | 68.1 | 78.2 | 52.5 | 25.7 | 63.4 | 42.6 | 29.2
    Every result from this document
  9. 9Lingshu-32B zero-shot; independent questions; exact option-set match 25.8
    Printed as 25.8Independent run, measured Aug 2026Configuration: zero-shot; independent questions; exact option-set match
    Evaluating Counterfactual Sensitivity to Patient Information in Medication-Safety Reasoning paper, Zhitian Hou, Yuhang Liu, Pengkai Wang, Zeyu Liu, Guanghao Zhu, Zheng Liu, Shuo Cai, Congkai Xie, Zhijie Sang, Kun Zeng, Hongxia Yang (The Hong Kong Polytechnic University; InfiX.ai; Sun Yat-sen University), 4 Aug 2026. arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row Lingshu-32B, column Pair; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair
    Lingshu-32B | 59.5 | 63.7 | 53.0 | 10.7 | 60.6 | 54.4 | 25.8
    Every result from this document
  10. 10GPT-OSS-20B zero-shot; independent questions; exact option-set match 24.7
    Printed as 24.7Independent run, measured Aug 2026Configuration: zero-shot; independent questions; exact option-set match
    Evaluating Counterfactual Sensitivity to Patient Information in Medication-Safety Reasoning paper, Zhitian Hou, Yuhang Liu, Pengkai Wang, Zeyu Liu, Guanghao Zhu, Zheng Liu, Shuo Cai, Congkai Xie, Zhijie Sang, Kun Zeng, Hongxia Yang (The Hong Kong Polytechnic University; InfiX.ai; Sun Yat-sen University), 4 Aug 2026. arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row GPT-OSS-20B, column Pair; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair
    GPT-OSS-20B | 59.1 | 66.5 | 47.5 | 19.0 | 52.1 | 48.5 | 24.7
    Every result from this document
  11. 10Qwen3.5-9B zero-shot; independent questions; exact option-set match 24.7
    Printed as 24.7Independent run, measured Aug 2026Configuration: zero-shot; independent questions; exact option-set match
    Evaluating Counterfactual Sensitivity to Patient Information in Medication-Safety Reasoning paper, Zhitian Hou, Yuhang Liu, Pengkai Wang, Zeyu Liu, Guanghao Zhu, Zheng Liu, Shuo Cai, Congkai Xie, Zhijie Sang, Kun Zeng, Hongxia Yang (The Hong Kong Polytechnic University; InfiX.ai; Sun Yat-sen University), 4 Aug 2026. arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row Qwen3.5-9B, column Pair; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair
    Qwen3.5-9B | 63.8 | 73.9 | 48.1 | 25.9 | 54.9 | 47.1 | 24.7
    Every result from this document
  12. 12MedGemma-27B-Text zero-shot; independent questions; exact option-set match 23.6
    Printed as 23.6Independent run, measured Aug 2026Configuration: zero-shot; independent questions; exact option-set match
    Evaluating Counterfactual Sensitivity to Patient Information in Medication-Safety Reasoning paper, Zhitian Hou, Yuhang Liu, Pengkai Wang, Zeyu Liu, Guanghao Zhu, Zheng Liu, Shuo Cai, Congkai Xie, Zhijie Sang, Kun Zeng, Hongxia Yang (The Hong Kong Polytechnic University; InfiX.ai; Sun Yat-sen University), 4 Aug 2026. arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row MedGemma-27B-Text, column Pair; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair
    MedGemma-27B-Text | 61.2 | 67.6 | 51.4 | 16.2 | 76.1 | 32.4 | 23.6
    Every result from this document
  13. 13Claude-Sonnet-4.6 zero-shot; independent questions; exact option-set match 21.3
    Printed as 21.3Independent run, measured Aug 2026Configuration: zero-shot; independent questions; exact option-set match
    Evaluating Counterfactual Sensitivity to Patient Information in Medication-Safety Reasoning paper, Zhitian Hou, Yuhang Liu, Pengkai Wang, Zeyu Liu, Guanghao Zhu, Zheng Liu, Shuo Cai, Congkai Xie, Zhijie Sang, Kun Zeng, Hongxia Yang (The Hong Kong Polytechnic University; InfiX.ai; Sun Yat-sen University), 4 Aug 2026. arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row Claude-Sonnet-4.6, column Pair; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair
    Claude-Sonnet-4.6 | 64.2 | 74.6 | 48.1 | 26.6 | 62.0 | 35.3 | 21.3
    Every result from this document
  14. 14Fleming-R1-32B zero-shot; independent questions; exact option-set match 20.2
    Printed as 20.2Independent run, measured Aug 2026Configuration: zero-shot; independent questions; exact option-set match
    Evaluating Counterfactual Sensitivity to Patient Information in Medication-Safety Reasoning paper, Zhitian Hou, Yuhang Liu, Pengkai Wang, Zeyu Liu, Guanghao Zhu, Zheng Liu, Shuo Cai, Congkai Xie, Zhijie Sang, Kun Zeng, Hongxia Yang (The Hong Kong Polytechnic University; InfiX.ai; Sun Yat-sen University), 4 Aug 2026. arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row Fleming-R1-32B, column Pair; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair
    Fleming-R1-32B | 60.6 | 69.0 | 47.5 | 21.5 | 59.2 | 44.1 | 20.2
    Every result from this document
  15. 14Gemini-2.5-Pro zero-shot; independent questions; exact option-set match 20.2
    Printed as 20.2Independent run, measured Aug 2026Configuration: zero-shot; independent questions; exact option-set match
    Evaluating Counterfactual Sensitivity to Patient Information in Medication-Safety Reasoning paper, Zhitian Hou, Yuhang Liu, Pengkai Wang, Zeyu Liu, Guanghao Zhu, Zheng Liu, Shuo Cai, Congkai Xie, Zhijie Sang, Kun Zeng, Hongxia Yang (The Hong Kong Polytechnic University; InfiX.ai; Sun Yat-sen University), 4 Aug 2026. arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row Gemini-2.5-Pro, column Pair; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair
    Gemini-2.5-Pro | 54.4 | 60.6 | 44.8 | 15.8 | 63.4 | 27.9 | 20.2
    Every result from this document
  16. 16Llama-3.1-70B zero-shot; independent questions; exact option-set match 16.9
    Printed as 16.9Independent run, measured Aug 2026Configuration: zero-shot; independent questions; exact option-set match
    Evaluating Counterfactual Sensitivity to Patient Information in Medication-Safety Reasoning paper, Zhitian Hou, Yuhang Liu, Pengkai Wang, Zeyu Liu, Guanghao Zhu, Zheng Liu, Shuo Cai, Congkai Xie, Zhijie Sang, Kun Zeng, Hongxia Yang (The Hong Kong Polytechnic University; InfiX.ai; Sun Yat-sen University), 4 Aug 2026. arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row Llama-3.1-70B, column Pair; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair
    Llama-3.1-70B | 55.7 | 61.3 | 47.0 | 14.3 | 63.4 | 32.4 | 16.9
    Every result from this document
  17. 17HuatuoGPT-o1-70B zero-shot; independent questions; exact option-set match 14.6
    Printed as 14.6Independent run, measured Aug 2026Configuration: zero-shot; independent questions; exact option-set match
    Evaluating Counterfactual Sensitivity to Patient Information in Medication-Safety Reasoning paper, Zhitian Hou, Yuhang Liu, Pengkai Wang, Zeyu Liu, Guanghao Zhu, Zheng Liu, Shuo Cai, Congkai Xie, Zhijie Sang, Kun Zeng, Hongxia Yang (The Hong Kong Polytechnic University; InfiX.ai; Sun Yat-sen University), 4 Aug 2026. arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row HuatuoGPT-o1-70B, column Pair; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair
    HuatuoGPT-o1-70B | 55.2 | 63.7 | 42.1 | 21.7 | 63.4 | 23.5 | 14.6
    Every result from this document
  18. 17Hulu-Med-7B zero-shot; independent questions; exact option-set match 14.6
    Printed as 14.6Independent run, measured Aug 2026Configuration: zero-shot; independent questions; exact option-set match
    Evaluating Counterfactual Sensitivity to Patient Information in Medication-Safety Reasoning paper, Zhitian Hou, Yuhang Liu, Pengkai Wang, Zeyu Liu, Guanghao Zhu, Zheng Liu, Shuo Cai, Congkai Xie, Zhijie Sang, Kun Zeng, Hongxia Yang (The Hong Kong Polytechnic University; InfiX.ai; Sun Yat-sen University), 4 Aug 2026. arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row Hulu-Med-7B, column Pair; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair
    Hulu-Med-7B | 42.0 | 48.2 | 32.2 | 16.0 | 36.6 | 38.2 | 14.6
    Every result from this document
  19. 19HuatuoGPT-o1-8B zero-shot; independent questions; exact option-set match 13.5
    Printed as 13.5Independent run, measured Aug 2026Configuration: zero-shot; independent questions; exact option-set match
    Evaluating Counterfactual Sensitivity to Patient Information in Medication-Safety Reasoning paper, Zhitian Hou, Yuhang Liu, Pengkai Wang, Zeyu Liu, Guanghao Zhu, Zheng Liu, Shuo Cai, Congkai Xie, Zhijie Sang, Kun Zeng, Hongxia Yang (The Hong Kong Polytechnic University; InfiX.ai; Sun Yat-sen University), 4 Aug 2026. arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row HuatuoGPT-o1-8B, column Pair; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair
    HuatuoGPT-o1-8B | 44.3 | 50.0 | 35.5 | 14.5 | 49.3 | 20.6 | 13.5
    Every result from this document
  20. 20Fleming-R1-7B zero-shot; independent questions; exact option-set match 12.4
    Printed as 12.4Independent run, measured Aug 2026Configuration: zero-shot; independent questions; exact option-set match
    Evaluating Counterfactual Sensitivity to Patient Information in Medication-Safety Reasoning paper, Zhitian Hou, Yuhang Liu, Pengkai Wang, Zeyu Liu, Guanghao Zhu, Zheng Liu, Shuo Cai, Congkai Xie, Zhijie Sang, Kun Zeng, Hongxia Yang (The Hong Kong Polytechnic University; InfiX.ai; Sun Yat-sen University), 4 Aug 2026. arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row Fleming-R1-7B, column Pair; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair
    Fleming-R1-7B | 50.7 | 59.9 | 36.6 | 23.2 | 53.5 | 29.4 | 12.4
    Every result from this document
  21. 21Baichuan-M2-32B zero-shot; independent questions; exact option-set match 11.2
    Printed as 11.2Independent run, measured Aug 2026Configuration: zero-shot; independent questions; exact option-set match
    Evaluating Counterfactual Sensitivity to Patient Information in Medication-Safety Reasoning paper, Zhitian Hou, Yuhang Liu, Pengkai Wang, Zeyu Liu, Guanghao Zhu, Zheng Liu, Shuo Cai, Congkai Xie, Zhijie Sang, Kun Zeng, Hongxia Yang (The Hong Kong Polytechnic University; InfiX.ai; Sun Yat-sen University), 4 Aug 2026. arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row Baichuan-M2-32B, column Pair; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair
    Baichuan-M2-32B | 47.8 | 54.6 | 37.2 | 17.4 | 49.3 | 30.9 | 11.2
    Every result from this document
  22. 21HealthGPT-Pro-4B zero-shot; independent questions; exact option-set match 11.2
    Printed as 11.2Independent run, measured Aug 2026Configuration: zero-shot; independent questions; exact option-set match
    Evaluating Counterfactual Sensitivity to Patient Information in Medication-Safety Reasoning paper, Zhitian Hou, Yuhang Liu, Pengkai Wang, Zeyu Liu, Guanghao Zhu, Zheng Liu, Shuo Cai, Congkai Xie, Zhijie Sang, Kun Zeng, Hongxia Yang (The Hong Kong Polytechnic University; InfiX.ai; Sun Yat-sen University), 4 Aug 2026. arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row HealthGPT-Pro-4B, column Pair; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair
    HealthGPT-Pro-4B | 36.0 | 37.7 | 33.3 | 4.3 | 36.6 | 41.2 | 11.2
    Every result from this document
  23. 23Lingshu-7B zero-shot; independent questions; exact option-set match 6.7
    Printed as 6.7Independent run, measured Aug 2026Configuration: zero-shot; independent questions; exact option-set match
    Evaluating Counterfactual Sensitivity to Patient Information in Medication-Safety Reasoning paper, Zhitian Hou, Yuhang Liu, Pengkai Wang, Zeyu Liu, Guanghao Zhu, Zheng Liu, Shuo Cai, Congkai Xie, Zhijie Sang, Kun Zeng, Hongxia Yang (The Hong Kong Polytechnic University; InfiX.ai; Sun Yat-sen University), 4 Aug 2026. arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row Lingshu-7B, column Pair; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair
    Lingshu-7B | 43.3 | 50.7 | 31.7 | 19.0 | 39.4 | 32.4 | 6.7
    Every result from this document
  24. 24Gemma-3-27B zero-shot; independent questions; exact option-set match 5.6
    Printed as 5.6Independent run, measured Aug 2026Configuration: zero-shot; independent questions; exact option-set match
    Evaluating Counterfactual Sensitivity to Patient Information in Medication-Safety Reasoning paper, Zhitian Hou, Yuhang Liu, Pengkai Wang, Zeyu Liu, Guanghao Zhu, Zheng Liu, Shuo Cai, Congkai Xie, Zhijie Sang, Kun Zeng, Hongxia Yang (The Hong Kong Polytechnic University; InfiX.ai; Sun Yat-sen University), 4 Aug 2026. arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row Gemma-3-27B, column Pair; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair
    Gemma-3-27B | 40.3 | 44.7 | 33.3 | 11.4 | 53.5 | 10.3 | 5.6
    Every result from this document
  25. 24HealthGPT-Pro-8B zero-shot; independent questions; exact option-set match 5.6
    Printed as 5.6Independent run, measured Aug 2026Configuration: zero-shot; independent questions; exact option-set match
    Evaluating Counterfactual Sensitivity to Patient Information in Medication-Safety Reasoning paper, Zhitian Hou, Yuhang Liu, Pengkai Wang, Zeyu Liu, Guanghao Zhu, Zheng Liu, Shuo Cai, Congkai Xie, Zhijie Sang, Kun Zeng, Hongxia Yang (The Hong Kong Polytechnic University; InfiX.ai; Sun Yat-sen University), 4 Aug 2026. arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row HealthGPT-Pro-8B, column Pair; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair
    HealthGPT-Pro-8B | 42.6 | 51.8 | 28.4 | 23.3 | 42.3 | 20.6 | 5.6
    Every result from this document
  26. 24Llama-3.1-8B zero-shot; independent questions; exact option-set match 5.6
    Printed as 5.6Independent run, measured Aug 2026Configuration: zero-shot; independent questions; exact option-set match
    Evaluating Counterfactual Sensitivity to Patient Information in Medication-Safety Reasoning paper, Zhitian Hou, Yuhang Liu, Pengkai Wang, Zeyu Liu, Guanghao Zhu, Zheng Liu, Shuo Cai, Congkai Xie, Zhijie Sang, Kun Zeng, Hongxia Yang (The Hong Kong Polytechnic University; InfiX.ai; Sun Yat-sen University), 4 Aug 2026. arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row Llama-3.1-8B, column Pair; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair
    Llama-3.1-8B | 40.0 | 44.4 | 33.3 | 11.0 | 54.9 | 10.3 | 5.6
    Every result from this document
  27. 24MedGemma-4B zero-shot; independent questions; exact option-set match 5.6
    Printed as 5.6Independent run, measured Aug 2026Configuration: zero-shot; independent questions; exact option-set match
    Evaluating Counterfactual Sensitivity to Patient Information in Medication-Safety Reasoning paper, Zhitian Hou, Yuhang Liu, Pengkai Wang, Zeyu Liu, Guanghao Zhu, Zheng Liu, Shuo Cai, Congkai Xie, Zhijie Sang, Kun Zeng, Hongxia Yang (The Hong Kong Polytechnic University; InfiX.ai; Sun Yat-sen University), 4 Aug 2026. arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row MedGemma-4B, column Pair; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair
    MedGemma-4B | 39.6 | 45.4 | 30.6 | 14.8 | 46.5 | 10.3 | 5.6
    Every result from this document
  28. 28Gemma-3-12B zero-shot; independent questions; exact option-set match 3.4
    Printed as 3.4Independent run, measured Aug 2026Configuration: zero-shot; independent questions; exact option-set match
    Evaluating Counterfactual Sensitivity to Patient Information in Medication-Safety Reasoning paper, Zhitian Hou, Yuhang Liu, Pengkai Wang, Zeyu Liu, Guanghao Zhu, Zheng Liu, Shuo Cai, Congkai Xie, Zhijie Sang, Kun Zeng, Hongxia Yang (The Hong Kong Polytechnic University; InfiX.ai; Sun Yat-sen University), 4 Aug 2026. arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row Gemma-3-12B, column Pair; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair
    Gemma-3-12B | 39.6 | 44.0 | 32.8 | 11.2 | 57.7 | 7.4 | 3.4
    Every result from this document
Risk activation accuracy, 28 rows
  1. 1MedGemma-27B-Text zero-shot; independent questions; exact option-set match 76.1
    Printed as 76.1Independent run, measured Aug 2026Configuration: zero-shot; independent questions; exact option-set match
    Evaluating Counterfactual Sensitivity to Patient Information in Medication-Safety Reasoning paper, Zhitian Hou, Yuhang Liu, Pengkai Wang, Zeyu Liu, Guanghao Zhu, Zheng Liu, Shuo Cai, Congkai Xie, Zhijie Sang, Kun Zeng, Hongxia Yang (The Hong Kong Polytechnic University; InfiX.ai; Sun Yat-sen University), 4 Aug 2026. arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row MedGemma-27B-Text, column Activation; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair
    MedGemma-27B-Text | 61.2 | 67.6 | 51.4 | 16.2 | 76.1 | 32.4 | 23.6
    Every result from this document
  2. 2Gemini-3.1-Pro zero-shot; independent questions; exact option-set match 74.6
    Printed as 74.6Independent run, measured Aug 2026Configuration: zero-shot; independent questions; exact option-set match
    Evaluating Counterfactual Sensitivity to Patient Information in Medication-Safety Reasoning paper, Zhitian Hou, Yuhang Liu, Pengkai Wang, Zeyu Liu, Guanghao Zhu, Zheng Liu, Shuo Cai, Congkai Xie, Zhijie Sang, Kun Zeng, Hongxia Yang (The Hong Kong Polytechnic University; InfiX.ai; Sun Yat-sen University), 4 Aug 2026. arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row Gemini-3.1-Pro, column Activation; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair
    Gemini-3.1-Pro | 80.7 | 87.7 | 69.9 | 17.7 | 74.6 | 77.9 | 48.3
    Every result from this document
  3. 3GPT-5 zero-shot; independent questions; exact option-set match 70.4
    Printed as 70.4Independent run, measured Aug 2026Configuration: zero-shot; independent questions; exact option-set match
    Evaluating Counterfactual Sensitivity to Patient Information in Medication-Safety Reasoning paper, Zhitian Hou, Yuhang Liu, Pengkai Wang, Zeyu Liu, Guanghao Zhu, Zheng Liu, Shuo Cai, Congkai Xie, Zhijie Sang, Kun Zeng, Hongxia Yang (The Hong Kong Polytechnic University; InfiX.ai; Sun Yat-sen University), 4 Aug 2026. arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row GPT-5, column Activation; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair
    GPT-5 | 75.6 | 83.5 | 63.4 | 20.1 | 70.4 | 61.8 | 38.2
    Every result from this document
  4. 3Qwen3.5-35B-A3B zero-shot; independent questions; exact option-set match 70.4
    Printed as 70.4Independent run, measured Aug 2026Configuration: zero-shot; independent questions; exact option-set match
    Evaluating Counterfactual Sensitivity to Patient Information in Medication-Safety Reasoning paper, Zhitian Hou, Yuhang Liu, Pengkai Wang, Zeyu Liu, Guanghao Zhu, Zheng Liu, Shuo Cai, Congkai Xie, Zhijie Sang, Kun Zeng, Hongxia Yang (The Hong Kong Polytechnic University; InfiX.ai; Sun Yat-sen University), 4 Aug 2026. arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row Qwen3.5-35B-A3B, column Activation; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair
    Qwen3.5-35B-A3B | 68.5 | 77.5 | 54.6 | 22.8 | 70.4 | 45.6 | 30.3
    Every result from this document
  5. 5DeepSeek-V4-Pro zero-shot; independent questions; exact option-set match 67.6
    Printed as 67.6Independent run, measured Aug 2026Configuration: zero-shot; independent questions; exact option-set match
    Evaluating Counterfactual Sensitivity to Patient Information in Medication-Safety Reasoning paper, Zhitian Hou, Yuhang Liu, Pengkai Wang, Zeyu Liu, Guanghao Zhu, Zheng Liu, Shuo Cai, Congkai Xie, Zhijie Sang, Kun Zeng, Hongxia Yang (The Hong Kong Polytechnic University; InfiX.ai; Sun Yat-sen University), 4 Aug 2026. arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row DeepSeek-V4-Pro, column Activation; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair
    DeepSeek-V4-Pro | 71.7 | 81.3 | 56.8 | 24.5 | 67.6 | 50.0 | 32.6
    Every result from this document
  6. 5Qwen3.5-27B zero-shot; independent questions; exact option-set match 67.6
    Printed as 67.6Independent run, measured Aug 2026Configuration: zero-shot; independent questions; exact option-set match
    Evaluating Counterfactual Sensitivity to Patient Information in Medication-Safety Reasoning paper, Zhitian Hou, Yuhang Liu, Pengkai Wang, Zeyu Liu, Guanghao Zhu, Zheng Liu, Shuo Cai, Congkai Xie, Zhijie Sang, Kun Zeng, Hongxia Yang (The Hong Kong Polytechnic University; InfiX.ai; Sun Yat-sen University), 4 Aug 2026. arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row Qwen3.5-27B, column Activation; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair
    Qwen3.5-27B | 69.4 | 78.2 | 55.7 | 22.4 | 67.6 | 51.5 | 31.5
    Every result from this document
  7. 7Qwen3.5-Plus zero-shot; independent questions; exact option-set match 66.2
    Printed as 66.2Independent run, measured Aug 2026Configuration: zero-shot; independent questions; exact option-set match
    Evaluating Counterfactual Sensitivity to Patient Information in Medication-Safety Reasoning paper, Zhitian Hou, Yuhang Liu, Pengkai Wang, Zeyu Liu, Guanghao Zhu, Zheng Liu, Shuo Cai, Congkai Xie, Zhijie Sang, Kun Zeng, Hongxia Yang (The Hong Kong Polytechnic University; InfiX.ai; Sun Yat-sen University), 4 Aug 2026. arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row Qwen3.5-Plus, column Activation; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair
    Qwen3.5-Plus | 72.4 | 81.7 | 57.9 | 23.8 | 66.2 | 55.9 | 32.6
    Every result from this document
  8. 8Gemini-2.5-Pro zero-shot; independent questions; exact option-set match 63.4
    Printed as 63.4Independent run, measured Aug 2026Configuration: zero-shot; independent questions; exact option-set match
    Evaluating Counterfactual Sensitivity to Patient Information in Medication-Safety Reasoning paper, Zhitian Hou, Yuhang Liu, Pengkai Wang, Zeyu Liu, Guanghao Zhu, Zheng Liu, Shuo Cai, Congkai Xie, Zhijie Sang, Kun Zeng, Hongxia Yang (The Hong Kong Polytechnic University; InfiX.ai; Sun Yat-sen University), 4 Aug 2026. arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row Gemini-2.5-Pro, column Activation; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair
    Gemini-2.5-Pro | 54.4 | 60.6 | 44.8 | 15.8 | 63.4 | 27.9 | 20.2
    Every result from this document
  9. 8GPT-5.2 zero-shot; independent questions; exact option-set match 63.4
    Printed as 63.4Independent run, measured Aug 2026Configuration: zero-shot; independent questions; exact option-set match
    Evaluating Counterfactual Sensitivity to Patient Information in Medication-Safety Reasoning paper, Zhitian Hou, Yuhang Liu, Pengkai Wang, Zeyu Liu, Guanghao Zhu, Zheng Liu, Shuo Cai, Congkai Xie, Zhijie Sang, Kun Zeng, Hongxia Yang (The Hong Kong Polytechnic University; InfiX.ai; Sun Yat-sen University), 4 Aug 2026. arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row GPT-5.2, column Activation; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair
    GPT-5.2 | 68.1 | 78.2 | 52.5 | 25.7 | 63.4 | 42.6 | 29.2
    Every result from this document
  10. 8HuatuoGPT-o1-70B zero-shot; independent questions; exact option-set match 63.4
    Printed as 63.4Independent run, measured Aug 2026Configuration: zero-shot; independent questions; exact option-set match
    Evaluating Counterfactual Sensitivity to Patient Information in Medication-Safety Reasoning paper, Zhitian Hou, Yuhang Liu, Pengkai Wang, Zeyu Liu, Guanghao Zhu, Zheng Liu, Shuo Cai, Congkai Xie, Zhijie Sang, Kun Zeng, Hongxia Yang (The Hong Kong Polytechnic University; InfiX.ai; Sun Yat-sen University), 4 Aug 2026. arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row HuatuoGPT-o1-70B, column Activation; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair
    HuatuoGPT-o1-70B | 55.2 | 63.7 | 42.1 | 21.7 | 63.4 | 23.5 | 14.6
    Every result from this document
  11. 8Llama-3.1-70B zero-shot; independent questions; exact option-set match 63.4
    Printed as 63.4Independent run, measured Aug 2026Configuration: zero-shot; independent questions; exact option-set match
    Evaluating Counterfactual Sensitivity to Patient Information in Medication-Safety Reasoning paper, Zhitian Hou, Yuhang Liu, Pengkai Wang, Zeyu Liu, Guanghao Zhu, Zheng Liu, Shuo Cai, Congkai Xie, Zhijie Sang, Kun Zeng, Hongxia Yang (The Hong Kong Polytechnic University; InfiX.ai; Sun Yat-sen University), 4 Aug 2026. arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row Llama-3.1-70B, column Activation; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair
    Llama-3.1-70B | 55.7 | 61.3 | 47.0 | 14.3 | 63.4 | 32.4 | 16.9
    Every result from this document
  12. 12Claude-Sonnet-4.6 zero-shot; independent questions; exact option-set match 62.0
    Printed as 62.0Independent run, measured Aug 2026Configuration: zero-shot; independent questions; exact option-set match
    Evaluating Counterfactual Sensitivity to Patient Information in Medication-Safety Reasoning paper, Zhitian Hou, Yuhang Liu, Pengkai Wang, Zeyu Liu, Guanghao Zhu, Zheng Liu, Shuo Cai, Congkai Xie, Zhijie Sang, Kun Zeng, Hongxia Yang (The Hong Kong Polytechnic University; InfiX.ai; Sun Yat-sen University), 4 Aug 2026. arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row Claude-Sonnet-4.6, column Activation; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair
    Claude-Sonnet-4.6 | 64.2 | 74.6 | 48.1 | 26.6 | 62.0 | 35.3 | 21.3
    Every result from this document
  13. 12GPT-OSS-120B zero-shot; independent questions; exact option-set match 62.0
    Printed as 62.0Independent run, measured Aug 2026Configuration: zero-shot; independent questions; exact option-set match
    Evaluating Counterfactual Sensitivity to Patient Information in Medication-Safety Reasoning paper, Zhitian Hou, Yuhang Liu, Pengkai Wang, Zeyu Liu, Guanghao Zhu, Zheng Liu, Shuo Cai, Congkai Xie, Zhijie Sang, Kun Zeng, Hongxia Yang (The Hong Kong Polytechnic University; InfiX.ai; Sun Yat-sen University), 4 Aug 2026. arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row GPT-OSS-120B, column Activation; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair
    GPT-OSS-120B | 70.7 | 79.6 | 56.8 | 22.7 | 62.0 | 55.9 | 30.3
    Every result from this document
  14. 14Lingshu-32B zero-shot; independent questions; exact option-set match 60.6
    Printed as 60.6Independent run, measured Aug 2026Configuration: zero-shot; independent questions; exact option-set match
    Evaluating Counterfactual Sensitivity to Patient Information in Medication-Safety Reasoning paper, Zhitian Hou, Yuhang Liu, Pengkai Wang, Zeyu Liu, Guanghao Zhu, Zheng Liu, Shuo Cai, Congkai Xie, Zhijie Sang, Kun Zeng, Hongxia Yang (The Hong Kong Polytechnic University; InfiX.ai; Sun Yat-sen University), 4 Aug 2026. arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row Lingshu-32B, column Activation; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair
    Lingshu-32B | 59.5 | 63.7 | 53.0 | 10.7 | 60.6 | 54.4 | 25.8
    Every result from this document
  15. 15Fleming-R1-32B zero-shot; independent questions; exact option-set match 59.2
    Printed as 59.2Independent run, measured Aug 2026Configuration: zero-shot; independent questions; exact option-set match
    Evaluating Counterfactual Sensitivity to Patient Information in Medication-Safety Reasoning paper, Zhitian Hou, Yuhang Liu, Pengkai Wang, Zeyu Liu, Guanghao Zhu, Zheng Liu, Shuo Cai, Congkai Xie, Zhijie Sang, Kun Zeng, Hongxia Yang (The Hong Kong Polytechnic University; InfiX.ai; Sun Yat-sen University), 4 Aug 2026. arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row Fleming-R1-32B, column Activation; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair
    Fleming-R1-32B | 60.6 | 69.0 | 47.5 | 21.5 | 59.2 | 44.1 | 20.2
    Every result from this document
  16. 16Gemma-3-12B zero-shot; independent questions; exact option-set match 57.7
    Printed as 57.7Independent run, measured Aug 2026Configuration: zero-shot; independent questions; exact option-set match
    Evaluating Counterfactual Sensitivity to Patient Information in Medication-Safety Reasoning paper, Zhitian Hou, Yuhang Liu, Pengkai Wang, Zeyu Liu, Guanghao Zhu, Zheng Liu, Shuo Cai, Congkai Xie, Zhijie Sang, Kun Zeng, Hongxia Yang (The Hong Kong Polytechnic University; InfiX.ai; Sun Yat-sen University), 4 Aug 2026. arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row Gemma-3-12B, column Activation; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair
    Gemma-3-12B | 39.6 | 44.0 | 32.8 | 11.2 | 57.7 | 7.4 | 3.4
    Every result from this document
  17. 17Llama-3.1-8B zero-shot; independent questions; exact option-set match 54.9
    Printed as 54.9Independent run, measured Aug 2026Configuration: zero-shot; independent questions; exact option-set match
    Evaluating Counterfactual Sensitivity to Patient Information in Medication-Safety Reasoning paper, Zhitian Hou, Yuhang Liu, Pengkai Wang, Zeyu Liu, Guanghao Zhu, Zheng Liu, Shuo Cai, Congkai Xie, Zhijie Sang, Kun Zeng, Hongxia Yang (The Hong Kong Polytechnic University; InfiX.ai; Sun Yat-sen University), 4 Aug 2026. arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row Llama-3.1-8B, column Activation; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair
    Llama-3.1-8B | 40.0 | 44.4 | 33.3 | 11.0 | 54.9 | 10.3 | 5.6
    Every result from this document
  18. 17Qwen3.5-9B zero-shot; independent questions; exact option-set match 54.9
    Printed as 54.9Independent run, measured Aug 2026Configuration: zero-shot; independent questions; exact option-set match
    Evaluating Counterfactual Sensitivity to Patient Information in Medication-Safety Reasoning paper, Zhitian Hou, Yuhang Liu, Pengkai Wang, Zeyu Liu, Guanghao Zhu, Zheng Liu, Shuo Cai, Congkai Xie, Zhijie Sang, Kun Zeng, Hongxia Yang (The Hong Kong Polytechnic University; InfiX.ai; Sun Yat-sen University), 4 Aug 2026. arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row Qwen3.5-9B, column Activation; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair
    Qwen3.5-9B | 63.8 | 73.9 | 48.1 | 25.9 | 54.9 | 47.1 | 24.7
    Every result from this document
  19. 19Fleming-R1-7B zero-shot; independent questions; exact option-set match 53.5
    Printed as 53.5Independent run, measured Aug 2026Configuration: zero-shot; independent questions; exact option-set match
    Evaluating Counterfactual Sensitivity to Patient Information in Medication-Safety Reasoning paper, Zhitian Hou, Yuhang Liu, Pengkai Wang, Zeyu Liu, Guanghao Zhu, Zheng Liu, Shuo Cai, Congkai Xie, Zhijie Sang, Kun Zeng, Hongxia Yang (The Hong Kong Polytechnic University; InfiX.ai; Sun Yat-sen University), 4 Aug 2026. arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row Fleming-R1-7B, column Activation; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair
    Fleming-R1-7B | 50.7 | 59.9 | 36.6 | 23.2 | 53.5 | 29.4 | 12.4
    Every result from this document
  20. 19Gemma-3-27B zero-shot; independent questions; exact option-set match 53.5
    Printed as 53.5Independent run, measured Aug 2026Configuration: zero-shot; independent questions; exact option-set match
    Evaluating Counterfactual Sensitivity to Patient Information in Medication-Safety Reasoning paper, Zhitian Hou, Yuhang Liu, Pengkai Wang, Zeyu Liu, Guanghao Zhu, Zheng Liu, Shuo Cai, Congkai Xie, Zhijie Sang, Kun Zeng, Hongxia Yang (The Hong Kong Polytechnic University; InfiX.ai; Sun Yat-sen University), 4 Aug 2026. arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row Gemma-3-27B, column Activation; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair
    Gemma-3-27B | 40.3 | 44.7 | 33.3 | 11.4 | 53.5 | 10.3 | 5.6
    Every result from this document
  21. 21GPT-OSS-20B zero-shot; independent questions; exact option-set match 52.1
    Printed as 52.1Independent run, measured Aug 2026Configuration: zero-shot; independent questions; exact option-set match
    Evaluating Counterfactual Sensitivity to Patient Information in Medication-Safety Reasoning paper, Zhitian Hou, Yuhang Liu, Pengkai Wang, Zeyu Liu, Guanghao Zhu, Zheng Liu, Shuo Cai, Congkai Xie, Zhijie Sang, Kun Zeng, Hongxia Yang (The Hong Kong Polytechnic University; InfiX.ai; Sun Yat-sen University), 4 Aug 2026. arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row GPT-OSS-20B, column Activation; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair
    GPT-OSS-20B | 59.1 | 66.5 | 47.5 | 19.0 | 52.1 | 48.5 | 24.7
    Every result from this document
  22. 22Baichuan-M2-32B zero-shot; independent questions; exact option-set match 49.3
    Printed as 49.3Independent run, measured Aug 2026Configuration: zero-shot; independent questions; exact option-set match
    Evaluating Counterfactual Sensitivity to Patient Information in Medication-Safety Reasoning paper, Zhitian Hou, Yuhang Liu, Pengkai Wang, Zeyu Liu, Guanghao Zhu, Zheng Liu, Shuo Cai, Congkai Xie, Zhijie Sang, Kun Zeng, Hongxia Yang (The Hong Kong Polytechnic University; InfiX.ai; Sun Yat-sen University), 4 Aug 2026. arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row Baichuan-M2-32B, column Activation; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair
    Baichuan-M2-32B | 47.8 | 54.6 | 37.2 | 17.4 | 49.3 | 30.9 | 11.2
    Every result from this document
  23. 22HuatuoGPT-o1-8B zero-shot; independent questions; exact option-set match 49.3
    Printed as 49.3Independent run, measured Aug 2026Configuration: zero-shot; independent questions; exact option-set match
    Evaluating Counterfactual Sensitivity to Patient Information in Medication-Safety Reasoning paper, Zhitian Hou, Yuhang Liu, Pengkai Wang, Zeyu Liu, Guanghao Zhu, Zheng Liu, Shuo Cai, Congkai Xie, Zhijie Sang, Kun Zeng, Hongxia Yang (The Hong Kong Polytechnic University; InfiX.ai; Sun Yat-sen University), 4 Aug 2026. arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row HuatuoGPT-o1-8B, column Activation; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair
    HuatuoGPT-o1-8B | 44.3 | 50.0 | 35.5 | 14.5 | 49.3 | 20.6 | 13.5
    Every result from this document
  24. 24MedGemma-4B zero-shot; independent questions; exact option-set match 46.5
    Printed as 46.5Independent run, measured Aug 2026Configuration: zero-shot; independent questions; exact option-set match
    Evaluating Counterfactual Sensitivity to Patient Information in Medication-Safety Reasoning paper, Zhitian Hou, Yuhang Liu, Pengkai Wang, Zeyu Liu, Guanghao Zhu, Zheng Liu, Shuo Cai, Congkai Xie, Zhijie Sang, Kun Zeng, Hongxia Yang (The Hong Kong Polytechnic University; InfiX.ai; Sun Yat-sen University), 4 Aug 2026. arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row MedGemma-4B, column Activation; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair
    MedGemma-4B | 39.6 | 45.4 | 30.6 | 14.8 | 46.5 | 10.3 | 5.6
    Every result from this document
  25. 25HealthGPT-Pro-8B zero-shot; independent questions; exact option-set match 42.3
    Printed as 42.3Independent run, measured Aug 2026Configuration: zero-shot; independent questions; exact option-set match
    Evaluating Counterfactual Sensitivity to Patient Information in Medication-Safety Reasoning paper, Zhitian Hou, Yuhang Liu, Pengkai Wang, Zeyu Liu, Guanghao Zhu, Zheng Liu, Shuo Cai, Congkai Xie, Zhijie Sang, Kun Zeng, Hongxia Yang (The Hong Kong Polytechnic University; InfiX.ai; Sun Yat-sen University), 4 Aug 2026. arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row HealthGPT-Pro-8B, column Activation; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair
    HealthGPT-Pro-8B | 42.6 | 51.8 | 28.4 | 23.3 | 42.3 | 20.6 | 5.6
    Every result from this document
  26. 26Lingshu-7B zero-shot; independent questions; exact option-set match 39.4
    Printed as 39.4Independent run, measured Aug 2026Configuration: zero-shot; independent questions; exact option-set match
    Evaluating Counterfactual Sensitivity to Patient Information in Medication-Safety Reasoning paper, Zhitian Hou, Yuhang Liu, Pengkai Wang, Zeyu Liu, Guanghao Zhu, Zheng Liu, Shuo Cai, Congkai Xie, Zhijie Sang, Kun Zeng, Hongxia Yang (The Hong Kong Polytechnic University; InfiX.ai; Sun Yat-sen University), 4 Aug 2026. arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row Lingshu-7B, column Activation; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair
    Lingshu-7B | 43.3 | 50.7 | 31.7 | 19.0 | 39.4 | 32.4 | 6.7
    Every result from this document
  27. 27HealthGPT-Pro-4B zero-shot; independent questions; exact option-set match 36.6
    Printed as 36.6Independent run, measured Aug 2026Configuration: zero-shot; independent questions; exact option-set match
    Evaluating Counterfactual Sensitivity to Patient Information in Medication-Safety Reasoning paper, Zhitian Hou, Yuhang Liu, Pengkai Wang, Zeyu Liu, Guanghao Zhu, Zheng Liu, Shuo Cai, Congkai Xie, Zhijie Sang, Kun Zeng, Hongxia Yang (The Hong Kong Polytechnic University; InfiX.ai; Sun Yat-sen University), 4 Aug 2026. arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row HealthGPT-Pro-4B, column Activation; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair
    HealthGPT-Pro-4B | 36.0 | 37.7 | 33.3 | 4.3 | 36.6 | 41.2 | 11.2
    Every result from this document
  28. 27Hulu-Med-7B zero-shot; independent questions; exact option-set match 36.6
    Printed as 36.6Independent run, measured Aug 2026Configuration: zero-shot; independent questions; exact option-set match
    Evaluating Counterfactual Sensitivity to Patient Information in Medication-Safety Reasoning paper, Zhitian Hou, Yuhang Liu, Pengkai Wang, Zeyu Liu, Guanghao Zhu, Zheng Liu, Shuo Cai, Congkai Xie, Zhijie Sang, Kun Zeng, Hongxia Yang (The Hong Kong Polytechnic University; InfiX.ai; Sun Yat-sen University), 4 Aug 2026. arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row Hulu-Med-7B, column Activation; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair
    Hulu-Med-7B | 42.0 | 48.2 | 32.2 | 16.0 | 36.6 | 38.2 | 14.6
    Every result from this document
Risk deactivation accuracy, 28 rows
  1. 1Gemini-3.1-Pro zero-shot; independent questions; exact option-set match 77.9
    Printed as 77.9Independent run, measured Aug 2026Configuration: zero-shot; independent questions; exact option-set match
    Evaluating Counterfactual Sensitivity to Patient Information in Medication-Safety Reasoning paper, Zhitian Hou, Yuhang Liu, Pengkai Wang, Zeyu Liu, Guanghao Zhu, Zheng Liu, Shuo Cai, Congkai Xie, Zhijie Sang, Kun Zeng, Hongxia Yang (The Hong Kong Polytechnic University; InfiX.ai; Sun Yat-sen University), 4 Aug 2026. arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row Gemini-3.1-Pro, column Deactivation; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair
    Gemini-3.1-Pro | 80.7 | 87.7 | 69.9 | 17.7 | 74.6 | 77.9 | 48.3
    Every result from this document
  2. 2GPT-5 zero-shot; independent questions; exact option-set match 61.8
    Printed as 61.8Independent run, measured Aug 2026Configuration: zero-shot; independent questions; exact option-set match
    Evaluating Counterfactual Sensitivity to Patient Information in Medication-Safety Reasoning paper, Zhitian Hou, Yuhang Liu, Pengkai Wang, Zeyu Liu, Guanghao Zhu, Zheng Liu, Shuo Cai, Congkai Xie, Zhijie Sang, Kun Zeng, Hongxia Yang (The Hong Kong Polytechnic University; InfiX.ai; Sun Yat-sen University), 4 Aug 2026. arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row GPT-5, column Deactivation; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair
    GPT-5 | 75.6 | 83.5 | 63.4 | 20.1 | 70.4 | 61.8 | 38.2
    Every result from this document
  3. 3GPT-OSS-120B zero-shot; independent questions; exact option-set match 55.9
    Printed as 55.9Independent run, measured Aug 2026Configuration: zero-shot; independent questions; exact option-set match
    Evaluating Counterfactual Sensitivity to Patient Information in Medication-Safety Reasoning paper, Zhitian Hou, Yuhang Liu, Pengkai Wang, Zeyu Liu, Guanghao Zhu, Zheng Liu, Shuo Cai, Congkai Xie, Zhijie Sang, Kun Zeng, Hongxia Yang (The Hong Kong Polytechnic University; InfiX.ai; Sun Yat-sen University), 4 Aug 2026. arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row GPT-OSS-120B, column Deactivation; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair
    GPT-OSS-120B | 70.7 | 79.6 | 56.8 | 22.7 | 62.0 | 55.9 | 30.3
    Every result from this document
  4. 3Qwen3.5-Plus zero-shot; independent questions; exact option-set match 55.9
    Printed as 55.9Independent run, measured Aug 2026Configuration: zero-shot; independent questions; exact option-set match
    Evaluating Counterfactual Sensitivity to Patient Information in Medication-Safety Reasoning paper, Zhitian Hou, Yuhang Liu, Pengkai Wang, Zeyu Liu, Guanghao Zhu, Zheng Liu, Shuo Cai, Congkai Xie, Zhijie Sang, Kun Zeng, Hongxia Yang (The Hong Kong Polytechnic University; InfiX.ai; Sun Yat-sen University), 4 Aug 2026. arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row Qwen3.5-Plus, column Deactivation; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair
    Qwen3.5-Plus | 72.4 | 81.7 | 57.9 | 23.8 | 66.2 | 55.9 | 32.6
    Every result from this document
  5. 5Lingshu-32B zero-shot; independent questions; exact option-set match 54.4
    Printed as 54.4Independent run, measured Aug 2026Configuration: zero-shot; independent questions; exact option-set match
    Evaluating Counterfactual Sensitivity to Patient Information in Medication-Safety Reasoning paper, Zhitian Hou, Yuhang Liu, Pengkai Wang, Zeyu Liu, Guanghao Zhu, Zheng Liu, Shuo Cai, Congkai Xie, Zhijie Sang, Kun Zeng, Hongxia Yang (The Hong Kong Polytechnic University; InfiX.ai; Sun Yat-sen University), 4 Aug 2026. arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row Lingshu-32B, column Deactivation; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair
    Lingshu-32B | 59.5 | 63.7 | 53.0 | 10.7 | 60.6 | 54.4 | 25.8
    Every result from this document
  6. 6Qwen3.5-27B zero-shot; independent questions; exact option-set match 51.5
    Printed as 51.5Independent run, measured Aug 2026Configuration: zero-shot; independent questions; exact option-set match
    Evaluating Counterfactual Sensitivity to Patient Information in Medication-Safety Reasoning paper, Zhitian Hou, Yuhang Liu, Pengkai Wang, Zeyu Liu, Guanghao Zhu, Zheng Liu, Shuo Cai, Congkai Xie, Zhijie Sang, Kun Zeng, Hongxia Yang (The Hong Kong Polytechnic University; InfiX.ai; Sun Yat-sen University), 4 Aug 2026. arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row Qwen3.5-27B, column Deactivation; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair
    Qwen3.5-27B | 69.4 | 78.2 | 55.7 | 22.4 | 67.6 | 51.5 | 31.5
    Every result from this document
  7. 7DeepSeek-V4-Pro zero-shot; independent questions; exact option-set match 50.0
    Printed as 50.0Independent run, measured Aug 2026Configuration: zero-shot; independent questions; exact option-set match
    Evaluating Counterfactual Sensitivity to Patient Information in Medication-Safety Reasoning paper, Zhitian Hou, Yuhang Liu, Pengkai Wang, Zeyu Liu, Guanghao Zhu, Zheng Liu, Shuo Cai, Congkai Xie, Zhijie Sang, Kun Zeng, Hongxia Yang (The Hong Kong Polytechnic University; InfiX.ai; Sun Yat-sen University), 4 Aug 2026. arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row DeepSeek-V4-Pro, column Deactivation; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair
    DeepSeek-V4-Pro | 71.7 | 81.3 | 56.8 | 24.5 | 67.6 | 50.0 | 32.6
    Every result from this document
  8. 8GPT-OSS-20B zero-shot; independent questions; exact option-set match 48.5
    Printed as 48.5Independent run, measured Aug 2026Configuration: zero-shot; independent questions; exact option-set match
    Evaluating Counterfactual Sensitivity to Patient Information in Medication-Safety Reasoning paper, Zhitian Hou, Yuhang Liu, Pengkai Wang, Zeyu Liu, Guanghao Zhu, Zheng Liu, Shuo Cai, Congkai Xie, Zhijie Sang, Kun Zeng, Hongxia Yang (The Hong Kong Polytechnic University; InfiX.ai; Sun Yat-sen University), 4 Aug 2026. arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row GPT-OSS-20B, column Deactivation; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair
    GPT-OSS-20B | 59.1 | 66.5 | 47.5 | 19.0 | 52.1 | 48.5 | 24.7
    Every result from this document
  9. 9Qwen3.5-9B zero-shot; independent questions; exact option-set match 47.1
    Printed as 47.1Independent run, measured Aug 2026Configuration: zero-shot; independent questions; exact option-set match
    Evaluating Counterfactual Sensitivity to Patient Information in Medication-Safety Reasoning paper, Zhitian Hou, Yuhang Liu, Pengkai Wang, Zeyu Liu, Guanghao Zhu, Zheng Liu, Shuo Cai, Congkai Xie, Zhijie Sang, Kun Zeng, Hongxia Yang (The Hong Kong Polytechnic University; InfiX.ai; Sun Yat-sen University), 4 Aug 2026. arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row Qwen3.5-9B, column Deactivation; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair
    Qwen3.5-9B | 63.8 | 73.9 | 48.1 | 25.9 | 54.9 | 47.1 | 24.7
    Every result from this document
  10. 10Qwen3.5-35B-A3B zero-shot; independent questions; exact option-set match 45.6
    Printed as 45.6Independent run, measured Aug 2026Configuration: zero-shot; independent questions; exact option-set match
    Evaluating Counterfactual Sensitivity to Patient Information in Medication-Safety Reasoning paper, Zhitian Hou, Yuhang Liu, Pengkai Wang, Zeyu Liu, Guanghao Zhu, Zheng Liu, Shuo Cai, Congkai Xie, Zhijie Sang, Kun Zeng, Hongxia Yang (The Hong Kong Polytechnic University; InfiX.ai; Sun Yat-sen University), 4 Aug 2026. arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row Qwen3.5-35B-A3B, column Deactivation; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair
    Qwen3.5-35B-A3B | 68.5 | 77.5 | 54.6 | 22.8 | 70.4 | 45.6 | 30.3
    Every result from this document
  11. 11Fleming-R1-32B zero-shot; independent questions; exact option-set match 44.1
    Printed as 44.1Independent run, measured Aug 2026Configuration: zero-shot; independent questions; exact option-set match
    Evaluating Counterfactual Sensitivity to Patient Information in Medication-Safety Reasoning paper, Zhitian Hou, Yuhang Liu, Pengkai Wang, Zeyu Liu, Guanghao Zhu, Zheng Liu, Shuo Cai, Congkai Xie, Zhijie Sang, Kun Zeng, Hongxia Yang (The Hong Kong Polytechnic University; InfiX.ai; Sun Yat-sen University), 4 Aug 2026. arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row Fleming-R1-32B, column Deactivation; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair
    Fleming-R1-32B | 60.6 | 69.0 | 47.5 | 21.5 | 59.2 | 44.1 | 20.2
    Every result from this document
  12. 12GPT-5.2 zero-shot; independent questions; exact option-set match 42.6
    Printed as 42.6Independent run, measured Aug 2026Configuration: zero-shot; independent questions; exact option-set match
    Evaluating Counterfactual Sensitivity to Patient Information in Medication-Safety Reasoning paper, Zhitian Hou, Yuhang Liu, Pengkai Wang, Zeyu Liu, Guanghao Zhu, Zheng Liu, Shuo Cai, Congkai Xie, Zhijie Sang, Kun Zeng, Hongxia Yang (The Hong Kong Polytechnic University; InfiX.ai; Sun Yat-sen University), 4 Aug 2026. arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row GPT-5.2, column Deactivation; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair
    GPT-5.2 | 68.1 | 78.2 | 52.5 | 25.7 | 63.4 | 42.6 | 29.2
    Every result from this document
  13. 13HealthGPT-Pro-4B zero-shot; independent questions; exact option-set match 41.2
    Printed as 41.2Independent run, measured Aug 2026Configuration: zero-shot; independent questions; exact option-set match
    Evaluating Counterfactual Sensitivity to Patient Information in Medication-Safety Reasoning paper, Zhitian Hou, Yuhang Liu, Pengkai Wang, Zeyu Liu, Guanghao Zhu, Zheng Liu, Shuo Cai, Congkai Xie, Zhijie Sang, Kun Zeng, Hongxia Yang (The Hong Kong Polytechnic University; InfiX.ai; Sun Yat-sen University), 4 Aug 2026. arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row HealthGPT-Pro-4B, column Deactivation; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair
    HealthGPT-Pro-4B | 36.0 | 37.7 | 33.3 | 4.3 | 36.6 | 41.2 | 11.2
    Every result from this document
  14. 14Hulu-Med-7B zero-shot; independent questions; exact option-set match 38.2
    Printed as 38.2Independent run, measured Aug 2026Configuration: zero-shot; independent questions; exact option-set match
    Evaluating Counterfactual Sensitivity to Patient Information in Medication-Safety Reasoning paper, Zhitian Hou, Yuhang Liu, Pengkai Wang, Zeyu Liu, Guanghao Zhu, Zheng Liu, Shuo Cai, Congkai Xie, Zhijie Sang, Kun Zeng, Hongxia Yang (The Hong Kong Polytechnic University; InfiX.ai; Sun Yat-sen University), 4 Aug 2026. arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row Hulu-Med-7B, column Deactivation; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair
    Hulu-Med-7B | 42.0 | 48.2 | 32.2 | 16.0 | 36.6 | 38.2 | 14.6
    Every result from this document
  15. 15Claude-Sonnet-4.6 zero-shot; independent questions; exact option-set match 35.3
    Printed as 35.3Independent run, measured Aug 2026Configuration: zero-shot; independent questions; exact option-set match
    Evaluating Counterfactual Sensitivity to Patient Information in Medication-Safety Reasoning paper, Zhitian Hou, Yuhang Liu, Pengkai Wang, Zeyu Liu, Guanghao Zhu, Zheng Liu, Shuo Cai, Congkai Xie, Zhijie Sang, Kun Zeng, Hongxia Yang (The Hong Kong Polytechnic University; InfiX.ai; Sun Yat-sen University), 4 Aug 2026. arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row Claude-Sonnet-4.6, column Deactivation; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair
    Claude-Sonnet-4.6 | 64.2 | 74.6 | 48.1 | 26.6 | 62.0 | 35.3 | 21.3
    Every result from this document
  16. 16Lingshu-7B zero-shot; independent questions; exact option-set match 32.4
    Printed as 32.4Independent run, measured Aug 2026Configuration: zero-shot; independent questions; exact option-set match
    Evaluating Counterfactual Sensitivity to Patient Information in Medication-Safety Reasoning paper, Zhitian Hou, Yuhang Liu, Pengkai Wang, Zeyu Liu, Guanghao Zhu, Zheng Liu, Shuo Cai, Congkai Xie, Zhijie Sang, Kun Zeng, Hongxia Yang (The Hong Kong Polytechnic University; InfiX.ai; Sun Yat-sen University), 4 Aug 2026. arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row Lingshu-7B, column Deactivation; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair
    Lingshu-7B | 43.3 | 50.7 | 31.7 | 19.0 | 39.4 | 32.4 | 6.7
    Every result from this document
  17. 16Llama-3.1-70B zero-shot; independent questions; exact option-set match 32.4
    Printed as 32.4Independent run, measured Aug 2026Configuration: zero-shot; independent questions; exact option-set match
    Evaluating Counterfactual Sensitivity to Patient Information in Medication-Safety Reasoning paper, Zhitian Hou, Yuhang Liu, Pengkai Wang, Zeyu Liu, Guanghao Zhu, Zheng Liu, Shuo Cai, Congkai Xie, Zhijie Sang, Kun Zeng, Hongxia Yang (The Hong Kong Polytechnic University; InfiX.ai; Sun Yat-sen University), 4 Aug 2026. arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row Llama-3.1-70B, column Deactivation; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair
    Llama-3.1-70B | 55.7 | 61.3 | 47.0 | 14.3 | 63.4 | 32.4 | 16.9
    Every result from this document
  18. 16MedGemma-27B-Text zero-shot; independent questions; exact option-set match 32.4
    Printed as 32.4Independent run, measured Aug 2026Configuration: zero-shot; independent questions; exact option-set match
    Evaluating Counterfactual Sensitivity to Patient Information in Medication-Safety Reasoning paper, Zhitian Hou, Yuhang Liu, Pengkai Wang, Zeyu Liu, Guanghao Zhu, Zheng Liu, Shuo Cai, Congkai Xie, Zhijie Sang, Kun Zeng, Hongxia Yang (The Hong Kong Polytechnic University; InfiX.ai; Sun Yat-sen University), 4 Aug 2026. arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row MedGemma-27B-Text, column Deactivation; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair
    MedGemma-27B-Text | 61.2 | 67.6 | 51.4 | 16.2 | 76.1 | 32.4 | 23.6
    Every result from this document
  19. 19Baichuan-M2-32B zero-shot; independent questions; exact option-set match 30.9
    Printed as 30.9Independent run, measured Aug 2026Configuration: zero-shot; independent questions; exact option-set match
    Evaluating Counterfactual Sensitivity to Patient Information in Medication-Safety Reasoning paper, Zhitian Hou, Yuhang Liu, Pengkai Wang, Zeyu Liu, Guanghao Zhu, Zheng Liu, Shuo Cai, Congkai Xie, Zhijie Sang, Kun Zeng, Hongxia Yang (The Hong Kong Polytechnic University; InfiX.ai; Sun Yat-sen University), 4 Aug 2026. arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row Baichuan-M2-32B, column Deactivation; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair
    Baichuan-M2-32B | 47.8 | 54.6 | 37.2 | 17.4 | 49.3 | 30.9 | 11.2
    Every result from this document
  20. 20Fleming-R1-7B zero-shot; independent questions; exact option-set match 29.4
    Printed as 29.4Independent run, measured Aug 2026Configuration: zero-shot; independent questions; exact option-set match
    Evaluating Counterfactual Sensitivity to Patient Information in Medication-Safety Reasoning paper, Zhitian Hou, Yuhang Liu, Pengkai Wang, Zeyu Liu, Guanghao Zhu, Zheng Liu, Shuo Cai, Congkai Xie, Zhijie Sang, Kun Zeng, Hongxia Yang (The Hong Kong Polytechnic University; InfiX.ai; Sun Yat-sen University), 4 Aug 2026. arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row Fleming-R1-7B, column Deactivation; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair
    Fleming-R1-7B | 50.7 | 59.9 | 36.6 | 23.2 | 53.5 | 29.4 | 12.4
    Every result from this document
  21. 21Gemini-2.5-Pro zero-shot; independent questions; exact option-set match 27.9
    Printed as 27.9Independent run, measured Aug 2026Configuration: zero-shot; independent questions; exact option-set match
    Evaluating Counterfactual Sensitivity to Patient Information in Medication-Safety Reasoning paper, Zhitian Hou, Yuhang Liu, Pengkai Wang, Zeyu Liu, Guanghao Zhu, Zheng Liu, Shuo Cai, Congkai Xie, Zhijie Sang, Kun Zeng, Hongxia Yang (The Hong Kong Polytechnic University; InfiX.ai; Sun Yat-sen University), 4 Aug 2026. arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row Gemini-2.5-Pro, column Deactivation; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair
    Gemini-2.5-Pro | 54.4 | 60.6 | 44.8 | 15.8 | 63.4 | 27.9 | 20.2
    Every result from this document
  22. 22HuatuoGPT-o1-70B zero-shot; independent questions; exact option-set match 23.5
    Printed as 23.5Independent run, measured Aug 2026Configuration: zero-shot; independent questions; exact option-set match
    Evaluating Counterfactual Sensitivity to Patient Information in Medication-Safety Reasoning paper, Zhitian Hou, Yuhang Liu, Pengkai Wang, Zeyu Liu, Guanghao Zhu, Zheng Liu, Shuo Cai, Congkai Xie, Zhijie Sang, Kun Zeng, Hongxia Yang (The Hong Kong Polytechnic University; InfiX.ai; Sun Yat-sen University), 4 Aug 2026. arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row HuatuoGPT-o1-70B, column Deactivation; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair
    HuatuoGPT-o1-70B | 55.2 | 63.7 | 42.1 | 21.7 | 63.4 | 23.5 | 14.6
    Every result from this document
  23. 23HealthGPT-Pro-8B zero-shot; independent questions; exact option-set match 20.6
    Printed as 20.6Independent run, measured Aug 2026Configuration: zero-shot; independent questions; exact option-set match
    Evaluating Counterfactual Sensitivity to Patient Information in Medication-Safety Reasoning paper, Zhitian Hou, Yuhang Liu, Pengkai Wang, Zeyu Liu, Guanghao Zhu, Zheng Liu, Shuo Cai, Congkai Xie, Zhijie Sang, Kun Zeng, Hongxia Yang (The Hong Kong Polytechnic University; InfiX.ai; Sun Yat-sen University), 4 Aug 2026. arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row HealthGPT-Pro-8B, column Deactivation; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair
    HealthGPT-Pro-8B | 42.6 | 51.8 | 28.4 | 23.3 | 42.3 | 20.6 | 5.6
    Every result from this document
  24. 23HuatuoGPT-o1-8B zero-shot; independent questions; exact option-set match 20.6
    Printed as 20.6Independent run, measured Aug 2026Configuration: zero-shot; independent questions; exact option-set match
    Evaluating Counterfactual Sensitivity to Patient Information in Medication-Safety Reasoning paper, Zhitian Hou, Yuhang Liu, Pengkai Wang, Zeyu Liu, Guanghao Zhu, Zheng Liu, Shuo Cai, Congkai Xie, Zhijie Sang, Kun Zeng, Hongxia Yang (The Hong Kong Polytechnic University; InfiX.ai; Sun Yat-sen University), 4 Aug 2026. arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row HuatuoGPT-o1-8B, column Deactivation; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair
    HuatuoGPT-o1-8B | 44.3 | 50.0 | 35.5 | 14.5 | 49.3 | 20.6 | 13.5
    Every result from this document
  25. 25Gemma-3-27B zero-shot; independent questions; exact option-set match 10.3
    Printed as 10.3Independent run, measured Aug 2026Configuration: zero-shot; independent questions; exact option-set match
    Evaluating Counterfactual Sensitivity to Patient Information in Medication-Safety Reasoning paper, Zhitian Hou, Yuhang Liu, Pengkai Wang, Zeyu Liu, Guanghao Zhu, Zheng Liu, Shuo Cai, Congkai Xie, Zhijie Sang, Kun Zeng, Hongxia Yang (The Hong Kong Polytechnic University; InfiX.ai; Sun Yat-sen University), 4 Aug 2026. arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row Gemma-3-27B, column Deactivation; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair
    Gemma-3-27B | 40.3 | 44.7 | 33.3 | 11.4 | 53.5 | 10.3 | 5.6
    Every result from this document
  26. 25Llama-3.1-8B zero-shot; independent questions; exact option-set match 10.3
    Printed as 10.3Independent run, measured Aug 2026Configuration: zero-shot; independent questions; exact option-set match
    Evaluating Counterfactual Sensitivity to Patient Information in Medication-Safety Reasoning paper, Zhitian Hou, Yuhang Liu, Pengkai Wang, Zeyu Liu, Guanghao Zhu, Zheng Liu, Shuo Cai, Congkai Xie, Zhijie Sang, Kun Zeng, Hongxia Yang (The Hong Kong Polytechnic University; InfiX.ai; Sun Yat-sen University), 4 Aug 2026. arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row Llama-3.1-8B, column Deactivation; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair
    Llama-3.1-8B | 40.0 | 44.4 | 33.3 | 11.0 | 54.9 | 10.3 | 5.6
    Every result from this document
  27. 25MedGemma-4B zero-shot; independent questions; exact option-set match 10.3
    Printed as 10.3Independent run, measured Aug 2026Configuration: zero-shot; independent questions; exact option-set match
    Evaluating Counterfactual Sensitivity to Patient Information in Medication-Safety Reasoning paper, Zhitian Hou, Yuhang Liu, Pengkai Wang, Zeyu Liu, Guanghao Zhu, Zheng Liu, Shuo Cai, Congkai Xie, Zhijie Sang, Kun Zeng, Hongxia Yang (The Hong Kong Polytechnic University; InfiX.ai; Sun Yat-sen University), 4 Aug 2026. arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row MedGemma-4B, column Deactivation; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair
    MedGemma-4B | 39.6 | 45.4 | 30.6 | 14.8 | 46.5 | 10.3 | 5.6
    Every result from this document
  28. 28Gemma-3-12B zero-shot; independent questions; exact option-set match 7.4
    Printed as 7.4Independent run, measured Aug 2026Configuration: zero-shot; independent questions; exact option-set match
    Evaluating Counterfactual Sensitivity to Patient Information in Medication-Safety Reasoning paper, Zhitian Hou, Yuhang Liu, Pengkai Wang, Zeyu Liu, Guanghao Zhu, Zheng Liu, Shuo Cai, Congkai Xie, Zhijie Sang, Kun Zeng, Hongxia Yang (The Hong Kong Polytechnic University; InfiX.ai; Sun Yat-sen University), 4 Aug 2026. arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row Gemma-3-12B, column Deactivation; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair
    Gemma-3-12B | 39.6 | 44.0 | 32.8 | 11.2 | 57.7 | 7.4 | 3.4
    Every result from this document
Published GF-minus-CF accuracy gap, 28 rows
  1. –Claude-Sonnet-4.6 zero-shot; independent questions; exact option-set match 26.6
    Printed as 26.6Independent run, measured Aug 2026Configuration: zero-shot; independent questions; exact option-set match
    Evaluating Counterfactual Sensitivity to Patient Information in Medication-Safety Reasoning paper, Zhitian Hou, Yuhang Liu, Pengkai Wang, Zeyu Liu, Guanghao Zhu, Zheng Liu, Shuo Cai, Congkai Xie, Zhijie Sang, Kun Zeng, Hongxia Yang (The Hong Kong Polytechnic University; InfiX.ai; Sun Yat-sen University), 4 Aug 2026. arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row Claude-Sonnet-4.6, column Δ_GF−CF; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair
    Claude-Sonnet-4.6 | 64.2 | 74.6 | 48.1 | 26.6 | 62.0 | 35.3 | 21.3
    Every result from this document
  2. –Qwen3.5-9B zero-shot; independent questions; exact option-set match 25.9
    Printed as 25.9Independent run, measured Aug 2026Configuration: zero-shot; independent questions; exact option-set match
    Evaluating Counterfactual Sensitivity to Patient Information in Medication-Safety Reasoning paper, Zhitian Hou, Yuhang Liu, Pengkai Wang, Zeyu Liu, Guanghao Zhu, Zheng Liu, Shuo Cai, Congkai Xie, Zhijie Sang, Kun Zeng, Hongxia Yang (The Hong Kong Polytechnic University; InfiX.ai; Sun Yat-sen University), 4 Aug 2026. arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row Qwen3.5-9B, column Δ_GF−CF; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair
    Qwen3.5-9B | 63.8 | 73.9 | 48.1 | 25.9 | 54.9 | 47.1 | 24.7
    Every result from this document
  3. –GPT-5.2 zero-shot; independent questions; exact option-set match 25.7
    Printed as 25.7Independent run, measured Aug 2026Configuration: zero-shot; independent questions; exact option-set match
    Evaluating Counterfactual Sensitivity to Patient Information in Medication-Safety Reasoning paper, Zhitian Hou, Yuhang Liu, Pengkai Wang, Zeyu Liu, Guanghao Zhu, Zheng Liu, Shuo Cai, Congkai Xie, Zhijie Sang, Kun Zeng, Hongxia Yang (The Hong Kong Polytechnic University; InfiX.ai; Sun Yat-sen University), 4 Aug 2026. arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row GPT-5.2, column Δ_GF−CF; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair
    GPT-5.2 | 68.1 | 78.2 | 52.5 | 25.7 | 63.4 | 42.6 | 29.2
    Every result from this document
  4. –DeepSeek-V4-Pro zero-shot; independent questions; exact option-set match 24.5
    Printed as 24.5Independent run, measured Aug 2026Configuration: zero-shot; independent questions; exact option-set match
    Evaluating Counterfactual Sensitivity to Patient Information in Medication-Safety Reasoning paper, Zhitian Hou, Yuhang Liu, Pengkai Wang, Zeyu Liu, Guanghao Zhu, Zheng Liu, Shuo Cai, Congkai Xie, Zhijie Sang, Kun Zeng, Hongxia Yang (The Hong Kong Polytechnic University; InfiX.ai; Sun Yat-sen University), 4 Aug 2026. arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row DeepSeek-V4-Pro, column Δ_GF−CF; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair
    DeepSeek-V4-Pro | 71.7 | 81.3 | 56.8 | 24.5 | 67.6 | 50.0 | 32.6
    Every result from this document
  5. –Qwen3.5-Plus zero-shot; independent questions; exact option-set match 23.8
    Printed as 23.8Independent run, measured Aug 2026Configuration: zero-shot; independent questions; exact option-set match
    Evaluating Counterfactual Sensitivity to Patient Information in Medication-Safety Reasoning paper, Zhitian Hou, Yuhang Liu, Pengkai Wang, Zeyu Liu, Guanghao Zhu, Zheng Liu, Shuo Cai, Congkai Xie, Zhijie Sang, Kun Zeng, Hongxia Yang (The Hong Kong Polytechnic University; InfiX.ai; Sun Yat-sen University), 4 Aug 2026. arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row Qwen3.5-Plus, column Δ_GF−CF; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair
    Qwen3.5-Plus | 72.4 | 81.7 | 57.9 | 23.8 | 66.2 | 55.9 | 32.6
    Every result from this document
  6. –HealthGPT-Pro-8B zero-shot; independent questions; exact option-set match 23.3
    Printed as 23.3Independent run, measured Aug 2026Configuration: zero-shot; independent questions; exact option-set match
    Evaluating Counterfactual Sensitivity to Patient Information in Medication-Safety Reasoning paper, Zhitian Hou, Yuhang Liu, Pengkai Wang, Zeyu Liu, Guanghao Zhu, Zheng Liu, Shuo Cai, Congkai Xie, Zhijie Sang, Kun Zeng, Hongxia Yang (The Hong Kong Polytechnic University; InfiX.ai; Sun Yat-sen University), 4 Aug 2026. arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row HealthGPT-Pro-8B, column Δ_GF−CF; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair
    HealthGPT-Pro-8B | 42.6 | 51.8 | 28.4 | 23.3 | 42.3 | 20.6 | 5.6
    Every result from this document
  7. –Fleming-R1-7B zero-shot; independent questions; exact option-set match 23.2
    Printed as 23.2Independent run, measured Aug 2026Configuration: zero-shot; independent questions; exact option-set match
    Evaluating Counterfactual Sensitivity to Patient Information in Medication-Safety Reasoning paper, Zhitian Hou, Yuhang Liu, Pengkai Wang, Zeyu Liu, Guanghao Zhu, Zheng Liu, Shuo Cai, Congkai Xie, Zhijie Sang, Kun Zeng, Hongxia Yang (The Hong Kong Polytechnic University; InfiX.ai; Sun Yat-sen University), 4 Aug 2026. arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row Fleming-R1-7B, column Δ_GF−CF; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair
    Fleming-R1-7B | 50.7 | 59.9 | 36.6 | 23.2 | 53.5 | 29.4 | 12.4
    Every result from this document
  8. –Qwen3.5-35B-A3B zero-shot; independent questions; exact option-set match 22.8
    Printed as 22.8Independent run, measured Aug 2026Configuration: zero-shot; independent questions; exact option-set match
    Evaluating Counterfactual Sensitivity to Patient Information in Medication-Safety Reasoning paper, Zhitian Hou, Yuhang Liu, Pengkai Wang, Zeyu Liu, Guanghao Zhu, Zheng Liu, Shuo Cai, Congkai Xie, Zhijie Sang, Kun Zeng, Hongxia Yang (The Hong Kong Polytechnic University; InfiX.ai; Sun Yat-sen University), 4 Aug 2026. arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row Qwen3.5-35B-A3B, column Δ_GF−CF; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair
    Qwen3.5-35B-A3B | 68.5 | 77.5 | 54.6 | 22.8 | 70.4 | 45.6 | 30.3
    Every result from this document
  9. –GPT-OSS-120B zero-shot; independent questions; exact option-set match 22.7
    Printed as 22.7Independent run, measured Aug 2026Configuration: zero-shot; independent questions; exact option-set match
    Evaluating Counterfactual Sensitivity to Patient Information in Medication-Safety Reasoning paper, Zhitian Hou, Yuhang Liu, Pengkai Wang, Zeyu Liu, Guanghao Zhu, Zheng Liu, Shuo Cai, Congkai Xie, Zhijie Sang, Kun Zeng, Hongxia Yang (The Hong Kong Polytechnic University; InfiX.ai; Sun Yat-sen University), 4 Aug 2026. arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row GPT-OSS-120B, column Δ_GF−CF; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair
    GPT-OSS-120B | 70.7 | 79.6 | 56.8 | 22.7 | 62.0 | 55.9 | 30.3
    Every result from this document
  10. –Qwen3.5-27B zero-shot; independent questions; exact option-set match 22.4
    Printed as 22.4Independent run, measured Aug 2026Configuration: zero-shot; independent questions; exact option-set match
    Evaluating Counterfactual Sensitivity to Patient Information in Medication-Safety Reasoning paper, Zhitian Hou, Yuhang Liu, Pengkai Wang, Zeyu Liu, Guanghao Zhu, Zheng Liu, Shuo Cai, Congkai Xie, Zhijie Sang, Kun Zeng, Hongxia Yang (The Hong Kong Polytechnic University; InfiX.ai; Sun Yat-sen University), 4 Aug 2026. arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row Qwen3.5-27B, column Δ_GF−CF; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair
    Qwen3.5-27B | 69.4 | 78.2 | 55.7 | 22.4 | 67.6 | 51.5 | 31.5
    Every result from this document
  11. –HuatuoGPT-o1-70B zero-shot; independent questions; exact option-set match 21.7
    Printed as 21.7Independent run, measured Aug 2026Configuration: zero-shot; independent questions; exact option-set match
    Evaluating Counterfactual Sensitivity to Patient Information in Medication-Safety Reasoning paper, Zhitian Hou, Yuhang Liu, Pengkai Wang, Zeyu Liu, Guanghao Zhu, Zheng Liu, Shuo Cai, Congkai Xie, Zhijie Sang, Kun Zeng, Hongxia Yang (The Hong Kong Polytechnic University; InfiX.ai; Sun Yat-sen University), 4 Aug 2026. arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row HuatuoGPT-o1-70B, column Δ_GF−CF; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair
    HuatuoGPT-o1-70B | 55.2 | 63.7 | 42.1 | 21.7 | 63.4 | 23.5 | 14.6
    Every result from this document
  12. –Fleming-R1-32B zero-shot; independent questions; exact option-set match 21.5
    Printed as 21.5Independent run, measured Aug 2026Configuration: zero-shot; independent questions; exact option-set match
    Evaluating Counterfactual Sensitivity to Patient Information in Medication-Safety Reasoning paper, Zhitian Hou, Yuhang Liu, Pengkai Wang, Zeyu Liu, Guanghao Zhu, Zheng Liu, Shuo Cai, Congkai Xie, Zhijie Sang, Kun Zeng, Hongxia Yang (The Hong Kong Polytechnic University; InfiX.ai; Sun Yat-sen University), 4 Aug 2026. arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row Fleming-R1-32B, column Δ_GF−CF; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair
    Fleming-R1-32B | 60.6 | 69.0 | 47.5 | 21.5 | 59.2 | 44.1 | 20.2
    Every result from this document
  13. –GPT-5 zero-shot; independent questions; exact option-set match 20.1
    Printed as 20.1Independent run, measured Aug 2026Configuration: zero-shot; independent questions; exact option-set match
    Evaluating Counterfactual Sensitivity to Patient Information in Medication-Safety Reasoning paper, Zhitian Hou, Yuhang Liu, Pengkai Wang, Zeyu Liu, Guanghao Zhu, Zheng Liu, Shuo Cai, Congkai Xie, Zhijie Sang, Kun Zeng, Hongxia Yang (The Hong Kong Polytechnic University; InfiX.ai; Sun Yat-sen University), 4 Aug 2026. arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row GPT-5, column Δ_GF−CF; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair
    GPT-5 | 75.6 | 83.5 | 63.4 | 20.1 | 70.4 | 61.8 | 38.2
    Every result from this document
  14. –GPT-OSS-20B zero-shot; independent questions; exact option-set match 19.0
    Printed as 19.0Independent run, measured Aug 2026Configuration: zero-shot; independent questions; exact option-set match
    Evaluating Counterfactual Sensitivity to Patient Information in Medication-Safety Reasoning paper, Zhitian Hou, Yuhang Liu, Pengkai Wang, Zeyu Liu, Guanghao Zhu, Zheng Liu, Shuo Cai, Congkai Xie, Zhijie Sang, Kun Zeng, Hongxia Yang (The Hong Kong Polytechnic University; InfiX.ai; Sun Yat-sen University), 4 Aug 2026. arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row GPT-OSS-20B, column Δ_GF−CF; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair
    GPT-OSS-20B | 59.1 | 66.5 | 47.5 | 19.0 | 52.1 | 48.5 | 24.7
    Every result from this document
  15. –Lingshu-7B zero-shot; independent questions; exact option-set match 19.0
    Printed as 19.0Independent run, measured Aug 2026Configuration: zero-shot; independent questions; exact option-set match
    Evaluating Counterfactual Sensitivity to Patient Information in Medication-Safety Reasoning paper, Zhitian Hou, Yuhang Liu, Pengkai Wang, Zeyu Liu, Guanghao Zhu, Zheng Liu, Shuo Cai, Congkai Xie, Zhijie Sang, Kun Zeng, Hongxia Yang (The Hong Kong Polytechnic University; InfiX.ai; Sun Yat-sen University), 4 Aug 2026. arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row Lingshu-7B, column Δ_GF−CF; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair
    Lingshu-7B | 43.3 | 50.7 | 31.7 | 19.0 | 39.4 | 32.4 | 6.7
    Every result from this document
  16. –Gemini-3.1-Pro zero-shot; independent questions; exact option-set match 17.7
    Printed as 17.7Independent run, measured Aug 2026Configuration: zero-shot; independent questions; exact option-set match
    Evaluating Counterfactual Sensitivity to Patient Information in Medication-Safety Reasoning paper, Zhitian Hou, Yuhang Liu, Pengkai Wang, Zeyu Liu, Guanghao Zhu, Zheng Liu, Shuo Cai, Congkai Xie, Zhijie Sang, Kun Zeng, Hongxia Yang (The Hong Kong Polytechnic University; InfiX.ai; Sun Yat-sen University), 4 Aug 2026. arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row Gemini-3.1-Pro, column Δ_GF−CF; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair
    Gemini-3.1-Pro | 80.7 | 87.7 | 69.9 | 17.7 | 74.6 | 77.9 | 48.3
    Every result from this document
  17. –Baichuan-M2-32B zero-shot; independent questions; exact option-set match 17.4
    Printed as 17.4Independent run, measured Aug 2026Configuration: zero-shot; independent questions; exact option-set match
    Evaluating Counterfactual Sensitivity to Patient Information in Medication-Safety Reasoning paper, Zhitian Hou, Yuhang Liu, Pengkai Wang, Zeyu Liu, Guanghao Zhu, Zheng Liu, Shuo Cai, Congkai Xie, Zhijie Sang, Kun Zeng, Hongxia Yang (The Hong Kong Polytechnic University; InfiX.ai; Sun Yat-sen University), 4 Aug 2026. arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row Baichuan-M2-32B, column Δ_GF−CF; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair
    Baichuan-M2-32B | 47.8 | 54.6 | 37.2 | 17.4 | 49.3 | 30.9 | 11.2
    Every result from this document
  18. –MedGemma-27B-Text zero-shot; independent questions; exact option-set match 16.2
    Printed as 16.2Independent run, measured Aug 2026Configuration: zero-shot; independent questions; exact option-set match
    Evaluating Counterfactual Sensitivity to Patient Information in Medication-Safety Reasoning paper, Zhitian Hou, Yuhang Liu, Pengkai Wang, Zeyu Liu, Guanghao Zhu, Zheng Liu, Shuo Cai, Congkai Xie, Zhijie Sang, Kun Zeng, Hongxia Yang (The Hong Kong Polytechnic University; InfiX.ai; Sun Yat-sen University), 4 Aug 2026. arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row MedGemma-27B-Text, column Δ_GF−CF; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair
    MedGemma-27B-Text | 61.2 | 67.6 | 51.4 | 16.2 | 76.1 | 32.4 | 23.6
    Every result from this document
  19. –Hulu-Med-7B zero-shot; independent questions; exact option-set match 16.0
    Printed as 16.0Independent run, measured Aug 2026Configuration: zero-shot; independent questions; exact option-set match
    Evaluating Counterfactual Sensitivity to Patient Information in Medication-Safety Reasoning paper, Zhitian Hou, Yuhang Liu, Pengkai Wang, Zeyu Liu, Guanghao Zhu, Zheng Liu, Shuo Cai, Congkai Xie, Zhijie Sang, Kun Zeng, Hongxia Yang (The Hong Kong Polytechnic University; InfiX.ai; Sun Yat-sen University), 4 Aug 2026. arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row Hulu-Med-7B, column Δ_GF−CF; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair
    Hulu-Med-7B | 42.0 | 48.2 | 32.2 | 16.0 | 36.6 | 38.2 | 14.6
    Every result from this document
  20. –Gemini-2.5-Pro zero-shot; independent questions; exact option-set match 15.8
    Printed as 15.8Independent run, measured Aug 2026Configuration: zero-shot; independent questions; exact option-set match
    Evaluating Counterfactual Sensitivity to Patient Information in Medication-Safety Reasoning paper, Zhitian Hou, Yuhang Liu, Pengkai Wang, Zeyu Liu, Guanghao Zhu, Zheng Liu, Shuo Cai, Congkai Xie, Zhijie Sang, Kun Zeng, Hongxia Yang (The Hong Kong Polytechnic University; InfiX.ai; Sun Yat-sen University), 4 Aug 2026. arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row Gemini-2.5-Pro, column Δ_GF−CF; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair
    Gemini-2.5-Pro | 54.4 | 60.6 | 44.8 | 15.8 | 63.4 | 27.9 | 20.2
    Every result from this document
  21. –MedGemma-4B zero-shot; independent questions; exact option-set match 14.8
    Printed as 14.8Independent run, measured Aug 2026Configuration: zero-shot; independent questions; exact option-set match
    Evaluating Counterfactual Sensitivity to Patient Information in Medication-Safety Reasoning paper, Zhitian Hou, Yuhang Liu, Pengkai Wang, Zeyu Liu, Guanghao Zhu, Zheng Liu, Shuo Cai, Congkai Xie, Zhijie Sang, Kun Zeng, Hongxia Yang (The Hong Kong Polytechnic University; InfiX.ai; Sun Yat-sen University), 4 Aug 2026. arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row MedGemma-4B, column Δ_GF−CF; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair
    MedGemma-4B | 39.6 | 45.4 | 30.6 | 14.8 | 46.5 | 10.3 | 5.6
    Every result from this document
  22. –HuatuoGPT-o1-8B zero-shot; independent questions; exact option-set match 14.5
    Printed as 14.5Independent run, measured Aug 2026Configuration: zero-shot; independent questions; exact option-set match
    Evaluating Counterfactual Sensitivity to Patient Information in Medication-Safety Reasoning paper, Zhitian Hou, Yuhang Liu, Pengkai Wang, Zeyu Liu, Guanghao Zhu, Zheng Liu, Shuo Cai, Congkai Xie, Zhijie Sang, Kun Zeng, Hongxia Yang (The Hong Kong Polytechnic University; InfiX.ai; Sun Yat-sen University), 4 Aug 2026. arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row HuatuoGPT-o1-8B, column Δ_GF−CF; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair
    HuatuoGPT-o1-8B | 44.3 | 50.0 | 35.5 | 14.5 | 49.3 | 20.6 | 13.5
    Every result from this document
  23. –Llama-3.1-70B zero-shot; independent questions; exact option-set match 14.3
    Printed as 14.3Independent run, measured Aug 2026Configuration: zero-shot; independent questions; exact option-set match
    Evaluating Counterfactual Sensitivity to Patient Information in Medication-Safety Reasoning paper, Zhitian Hou, Yuhang Liu, Pengkai Wang, Zeyu Liu, Guanghao Zhu, Zheng Liu, Shuo Cai, Congkai Xie, Zhijie Sang, Kun Zeng, Hongxia Yang (The Hong Kong Polytechnic University; InfiX.ai; Sun Yat-sen University), 4 Aug 2026. arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row Llama-3.1-70B, column Δ_GF−CF; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair
    Llama-3.1-70B | 55.7 | 61.3 | 47.0 | 14.3 | 63.4 | 32.4 | 16.9
    Every result from this document
  24. –Gemma-3-27B zero-shot; independent questions; exact option-set match 11.4
    Printed as 11.4Independent run, measured Aug 2026Configuration: zero-shot; independent questions; exact option-set match
    Evaluating Counterfactual Sensitivity to Patient Information in Medication-Safety Reasoning paper, Zhitian Hou, Yuhang Liu, Pengkai Wang, Zeyu Liu, Guanghao Zhu, Zheng Liu, Shuo Cai, Congkai Xie, Zhijie Sang, Kun Zeng, Hongxia Yang (The Hong Kong Polytechnic University; InfiX.ai; Sun Yat-sen University), 4 Aug 2026. arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row Gemma-3-27B, column Δ_GF−CF; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair
    Gemma-3-27B | 40.3 | 44.7 | 33.3 | 11.4 | 53.5 | 10.3 | 5.6
    Every result from this document
  25. –Gemma-3-12B zero-shot; independent questions; exact option-set match 11.2
    Printed as 11.2Independent run, measured Aug 2026Configuration: zero-shot; independent questions; exact option-set match
    Evaluating Counterfactual Sensitivity to Patient Information in Medication-Safety Reasoning paper, Zhitian Hou, Yuhang Liu, Pengkai Wang, Zeyu Liu, Guanghao Zhu, Zheng Liu, Shuo Cai, Congkai Xie, Zhijie Sang, Kun Zeng, Hongxia Yang (The Hong Kong Polytechnic University; InfiX.ai; Sun Yat-sen University), 4 Aug 2026. arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row Gemma-3-12B, column Δ_GF−CF; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair
    Gemma-3-12B | 39.6 | 44.0 | 32.8 | 11.2 | 57.7 | 7.4 | 3.4
    Every result from this document
  26. –Llama-3.1-8B zero-shot; independent questions; exact option-set match 11.0
    Printed as 11.0Independent run, measured Aug 2026Configuration: zero-shot; independent questions; exact option-set match
    Evaluating Counterfactual Sensitivity to Patient Information in Medication-Safety Reasoning paper, Zhitian Hou, Yuhang Liu, Pengkai Wang, Zeyu Liu, Guanghao Zhu, Zheng Liu, Shuo Cai, Congkai Xie, Zhijie Sang, Kun Zeng, Hongxia Yang (The Hong Kong Polytechnic University; InfiX.ai; Sun Yat-sen University), 4 Aug 2026. arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row Llama-3.1-8B, column Δ_GF−CF; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair
    Llama-3.1-8B | 40.0 | 44.4 | 33.3 | 11.0 | 54.9 | 10.3 | 5.6
    Every result from this document
  27. –Lingshu-32B zero-shot; independent questions; exact option-set match 10.7
    Printed as 10.7Independent run, measured Aug 2026Configuration: zero-shot; independent questions; exact option-set match
    Evaluating Counterfactual Sensitivity to Patient Information in Medication-Safety Reasoning paper, Zhitian Hou, Yuhang Liu, Pengkai Wang, Zeyu Liu, Guanghao Zhu, Zheng Liu, Shuo Cai, Congkai Xie, Zhijie Sang, Kun Zeng, Hongxia Yang (The Hong Kong Polytechnic University; InfiX.ai; Sun Yat-sen University), 4 Aug 2026. arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row Lingshu-32B, column Δ_GF−CF; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair
    Lingshu-32B | 59.5 | 63.7 | 53.0 | 10.7 | 60.6 | 54.4 | 25.8
    Every result from this document
  28. –HealthGPT-Pro-4B zero-shot; independent questions; exact option-set match 4.3
    Printed as 4.3Independent run, measured Aug 2026Configuration: zero-shot; independent questions; exact option-set match
    Evaluating Counterfactual Sensitivity to Patient Information in Medication-Safety Reasoning paper, Zhitian Hou, Yuhang Liu, Pengkai Wang, Zeyu Liu, Guanghao Zhu, Zheng Liu, Shuo Cai, Congkai Xie, Zhijie Sang, Kun Zeng, Hongxia Yang (The Hong Kong Polytechnic University; InfiX.ai; Sun Yat-sen University), 4 Aug 2026. arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row HealthGPT-Pro-4B, column Δ_GF−CF; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair
    HealthGPT-Pro-4B | 36.0 | 37.7 | 33.3 | 4.3 | 36.6 | 41.2 | 11.2
    Every result from this document

What the items cover

Item counts; an item can carry more than one label.

Benchmark task family

Item counts by benchmark task family
LabelAll itemsCounterfactualGuideline following
Guideline following284–284
Counterfactual183183–

Clinical system

Item counts by clinical system
LabelAll itemsCounterfactualGuideline following
Nervous system1627092
Urinary system1384494
Cardiovascular system623527
Reproductive system492029
Digestive system19613
Lymphatic and immune system1129
Skeletal system10–10
Integumentary system844
Respiratory system826

Drug categories

Item counts by drug categories
LabelAll itemsCounterfactualGuideline following
Nsaids/antipyretic analgesics501634
Opioids412219
Antiarrhythmics331518
Antidiabetics33–33
Antihistamines33–33
Antipsychotics332211
Antipyretic analgesics22220
Factor xa inhibitors/anticoagulants22814
Pde3 inhibitors/peripheral vascular drugs21912
H2-receptor antagonists19127
Alpha-1 receptor blockers18810
Calcium-channel blockers18414
Cholinesterase inhibitors17134
Antibacterials1569
Gabapentinoids1569
Ras inhibitors15–15
Gastrointestinal drugs1367
Antiepileptics1266
Multiple drug categories (composite rules)1239
Statins12210
Anticoagulants1055
Antibiotics927
Thyroid hormones9–9
Antigout agents844
Opioid-like analgesics844
Psychiatric medications844
H2-receptor blockers725
Local anesthetics/topical drugs642
Tricyclic antidepressants624
Benzodiazepines5–5
Lithium523
Theophyllines523
Antidepressants/analgesics422
Antivirals422
Direct thrombin inhibitors/anticoagulants422
Fluoroquinolone antibacterials422
Immunomodulators44–
Low-molecular-weight heparin anticoagulants422
Muscle relaxants422
Nitroimidazole antibacterials4–4
Ophthalmic drugs422
Potassium-sparing diuretics4–4
Thiazolidinedione antidiabetics422
Tricyclic antidepressants/anticholinergics422
Urinary antibacterials422
Z-drugs4–4
Retinoids33–
Sulfonamide antibacterials3–3
Analgesics2–2
Antimetabolites/immunosuppressants22–
Cardiovascular drugs2–2
Central nervous system depressants22–
Hormonal drugs22–
Nonselective alpha-1 receptor blockers2–2
Prostaglandins22–
Ras inhibitors/potassium-sparing diuretics22–
Renin-angiotensin system inhibitors2–2
Aldosterone receptor antagonists1–1
Anesthetics1–1
Antihistamines/antiemetics1–1
Dermatologic drugs1–1
First-generation antihistamines/anticholinergics1–1
Opioid antagonists1–1
Sulfonamides/urinary antibacterials1–1
Topical drugs1–1
Z-class hypnotics1–1

Patient info type

Item counts by patient info type
LabelAll itemsCounterfactualGuideline following
Disease presence1365977
Dose threshold1184474
Composite clinical context662046
Drug interaction451827
Age threshold421824
Gestational-week threshold29–29
Pregnancy status2020–
Pediatric-adult contraindication1147

Population source

Item counts by population source
LabelAll itemsCounterfactualGuideline following
Older adults352137215
Pediatrics662640
Pregnancy492029

Clinical department

Item counts by clinical department
LabelAll itemsCounterfactualGuideline following
Nephrology1184474
Neurology1104664
Pediatrics662640
Cardiology603525
Obstetrics and gynecology492029
Urology20–20
Psychiatry1569
Geriatric medicine10–10
Pain medicine743
Gastroenterology6–6
Pulmonology624

Reasoning operation

Item counts by reasoning operation
LabelAll itemsCounterfactualGuideline following
Static risk identification284–284
Risk activation7171–
Risk deactivation6868–
Risk redistribution2626–
Interaction identification1818–

Results by department

As printed, Table 3.

Mean counterfactual accuracy (%) by model group; departments with at least 20 CF questions

Mean counterfactual accuracy (%) by model group; departments with at least 20 CF questions
DepartmentGeneralProprietaryMedical specific
Neurology31.640.726.4
Cardiology3342.927.9
Pediatrics47.454.942.9
Nephrology5367.241.1
Obstetrics & Gynecology81.196.480

Documents

1