Clinical Benchmarks

CHI-Bench

Long-horizon healthcare operations workflows for agents: prior authorization, utilization management, and care management.

actAVAOfficial page
More about this boardLess

Long-horizon healthcare operations workflows for agents: prior authorization, utilization management, and care management.

Long-horizon US healthcare operations workflows for agents (prior authorization, utilization management, care management), 60-80 step tasks across 4-6 stages, judged by deterministic unit tests plus an LLM judge for evidence grounding, consent, and cross-stage consistency.

Released May 20, 2026; updated August 12, 2026 across 45 harness configurations. The launch report led with reliability, not capability: no agent stayed above 20 percent across three identical runs. Harness choice matters as much as model choice, and the board accepts community submissions, so rows mix author-run and submitted results.

Published by actAVA.ai, released May 2026. 75 workflows (25 per domain), 21 healthcare applications, 200+ MCP tools. Pass@1 with binary 0/1 reward, higher better.

Headline metric
pass@1 accuracy, 75 tasks (25 each: prior authorization, utilization management, care management)
Rows on the board
45
Board last updated
12 Aug 2026
Board version
CHI-Bench v1.0.0
Rows
43
Models
25 of 45on the board
Labs
13
Last measured
Aug 2026
Leader
54.7erius + claude-opus-5

Ranking

Pass@1 with binary 0/1 reward, higher better

  • Humana (harness) / Anthropic (model)
  • OpenAI
  • Alibaba
  • Moonshot AI
  • Other labs
  1. 1erius + claude-opus-5community-submitted harness config vali…54.7
  2. 2claude-code + claude-opus-537.3
  3. 2erius + claude-opus-4-837.3
  4. 4claude-code + claude-opus-4-833.3
  5. 5claude-code + claude-opus-4-628.0
  6. 6claude-code + claude-sonnet-4-626.2
  7. 7codex + gpt-5.6-sol25.3
  8. 7openai-agents + kimi-k325.3
  9. 9claude-code + claude-opus-4-724.4
  10. 10claude-code + claude-fable-524.0
  11. 11hermes + MedGuardAll Domains pass@122.7
  12. 12codex + gpt-5.520.9
  13. 13claude-code + claude-sonnet-520.0
  14. 14hermes + glm-5.1All Domains pass@118.7
  15. 14openai-agents + glm-5.1All Domains pass@118.7
  16. 14openai-agents + glm-5.2All Domains pass@118.7
  17. 17openclaw + claude-opus-4-7All Domains pass@117.3
  18. 18openclaw + glm-5.1All Domains pass@116.9
  19. 19hermes + qwen-3.6-maxAll Domains pass@116.4
  20. 20codex + gpt-5.4All Domains pass@116.0

20 of 43 rows

Rows and sources

Open a row for the quote, the page and the document.

  1. 1erius + claude-opus-5 community-submitted harness config validated by automated workspace judge 54.7
    Printed as 54.7%Independent run, measured Aug 2026Configuration: community-submitted harness config validated by automated workspace judge
    CHI-Bench leaderboard (actAVA) official leaderboard, actAVA, 12 Aug 2026. CHI-Bench v1.0.0, All Domains, rank 01, Agent/Model and Accuracy columns; PA/UM/CM follow; board last updated 2026-08-12; board Date column 2026-07-26; accessed 2026-09-30.
    01 | erius submitted by Michael Johnson (MJ) | claude-opus-5 | Proprietary | 54.7% | 72.0% | 36.0% | 56.0% | 2026-07-26
    Every result from this document
  2. 2claude-code + claude-opus-5 37.3
    Printed as 37.3%Official leaderboard, measured Jul 2026
    CHI-Bench leaderboard (actAVA) official leaderboard, actAVA, 12 Aug 2026. CHI-Bench v1.0.0, All Domains, rank 03, Agent/Model and Accuracy columns; PA/UM/CM follow; board last updated 2026-08-12; board Date column 2026-07-24; accessed 2026-09-30. Linked submission provenance: run 2026-07-24T19:08:02Z to 2026-07-24T23:22:36Z; submitted_at 2026-08-12T20:20:00Z.
    03 | claude-code | claude-opus-5 | Proprietary | 37.3% | 20.0% | 32.0% | 60.0% | 2026-07-24
    Every result from this document
  3. 2erius + claude-opus-4-8 37.3
    Printed as 37.3%Independent run, measured Aug 2026
    CHI-Bench leaderboard (actAVA) official leaderboard, actAVA, 12 Aug 2026. CHI-Bench v1.0.0, All Domains, rank 02, Agent/Model and Accuracy columns; PA/UM/CM follow; board last updated 2026-08-12; board Date column 2026-06-05; accessed 2026-09-30.
    02 | erius submitted by Michael Johnson (MJ) | claude-opus-4-8 | Proprietary | 37.3% | 40.0% | 16.0% | 56.0% | 2026-06-05
    Every result from this document
  4. 4claude-code + claude-opus-4-8 33.3
    Printed as 33.3%Official leaderboard, measured May 2026
    CHI-Bench leaderboard (actAVA) official leaderboard, actAVA, 12 Aug 2026. CHI-Bench v1.0.0, All Domains, rank 04, Agent/Model and Accuracy columns; PA/UM/CM follow; board last updated 2026-08-12; board Date column 2026-05-28; accessed 2026-09-30. Linked submission provenance: run 2026-05-28T17:19:15Z to 2026-05-28T20:48:59Z; submitted_at 2026-05-28T20:53:26Z.
    04 | claude-code | claude-opus-4-8 | Proprietary | 33.3% | 32.0% | 28.0% | 40.0% | 2026-05-28
    Every result from this document
  5. 5claude-code + claude-opus-4-6 28.0
    Printed as 28.0%Official leaderboard, measured May 2026
    CHI-Bench leaderboard (actAVA) official leaderboard, actAVA, 12 Aug 2026. CHI-Bench v1.0.0, All Domains, rank 05, Agent/Model and Accuracy columns; PA/UM/CM follow; board last updated 2026-08-12; board Date column 2026-05-01; accessed 2026-09-30. Linked submission provenance: run 2026-05-12T18:55:45Z to 2026-05-12T18:55:46Z; submitted_at 2026-05-12T22:39:33Z.
    05 | claude-code | claude-opus-4-6 | Proprietary | 28.0% | 20.0% | 36.0% | 28.0% | 2026-05-01
    Every result from this document
  6. 6claude-code + claude-sonnet-4-6 26.2
    Printed as 26.2%Official leaderboard, measured Apr 2026
    CHI-Bench leaderboard (actAVA) official leaderboard, actAVA, 12 Aug 2026. CHI-Bench v1.0.0, All Domains, rank 06, Agent/Model and Accuracy columns; PA/UM/CM follow; board last updated 2026-08-12; board Date column 2026-05-01; accessed 2026-09-30. Linked submission provenance: run 2026-04-29T14:43:27Z to 2026-04-30T15:00:19Z; submitted_at 2026-04-30T15:00:19Z.
    06 | claude-code | claude-sonnet-4-6 | Proprietary | 26.2% | 24.0% | 34.7% | 20.0% | 2026-05-01
    Every result from this document
  7. 7codex + gpt-5.6-sol 25.3
    Printed as 25.3%Official leaderboard, measured Jul 2026
    CHI-Bench leaderboard (actAVA) official leaderboard, actAVA, 12 Aug 2026. CHI-Bench v1.0.0, All Domains, rank 07, Agent/Model and Accuracy columns; PA/UM/CM follow; board last updated 2026-08-12; board Date column 2026-07-24; accessed 2026-09-30. Linked submission provenance: run 2026-07-22T02:41:32.718905Z to 2026-07-22T04:52:39.419746Z; submitted_at 2026-07-24T23:01:25Z.
    07 | codex | gpt-5.6-sol | Proprietary | 25.3% | 36.0% | 28.0% | 12.0% | 2026-07-24
    Every result from this document
  8. 7openai-agents + kimi-k3 25.3
    Printed as 25.3%Official leaderboard, measured Jul 2026
    CHI-Bench leaderboard (actAVA) official leaderboard, actAVA, 12 Aug 2026. CHI-Bench v1.0.0, All Domains, rank 08, Agent/Model and Accuracy columns; PA/UM/CM follow; board last updated 2026-08-12; board Date column 2026-07-24; accessed 2026-09-30. Linked submission provenance: run 2026-07-22T08:03:23.788878Z to 2026-07-22T14:22:38.269923Z; submitted_at 2026-08-12T20:20:00Z.
    08 | openai-agents | kimi-k3 | Open-source | 25.3% | 28.0% | 32.0% | 16.0% | 2026-07-24
    Every result from this document
  9. 9claude-code + claude-opus-4-7 24.4
    Printed as 24.4%Official leaderboard, measured Apr 2026
    CHI-Bench leaderboard (actAVA) official leaderboard, actAVA, 12 Aug 2026. CHI-Bench v1.0.0, All Domains, rank 09, Agent/Model and Accuracy columns; PA/UM/CM follow; board last updated 2026-08-12; board Date column 2026-05-01; accessed 2026-09-30. Linked submission provenance: run 2026-04-29T06:06:32Z to 2026-04-30T10:00:03Z; submitted_at 2026-04-30T10:00:03Z.
    09 | claude-code | claude-opus-4-7 | Proprietary | 24.4% | 24.0% | 17.3% | 32.0% | 2026-05-01
    Every result from this document
  10. 10claude-code + claude-fable-5 24.0
    Printed as 24.0%Official leaderboard, measured Jul 2026
    CHI-Bench leaderboard (actAVA) official leaderboard, actAVA, 12 Aug 2026. CHI-Bench v1.0.0, All Domains, rank 10, Agent/Model and Accuracy columns; PA/UM/CM follow; board last updated 2026-08-12; board Date column 2026-07-22; accessed 2026-09-30. Linked submission provenance: run 2026-07-22T07:16:27.207645Z to 2026-07-22T10:25:29.658582Z; submitted_at 2026-07-22T23:50:25Z.
    10 | claude-code | claude-fable-5 | Proprietary | 24.0% | 24.0% | 24.0% | 24.0% | 2026-07-22
    Every result from this document
  11. 11hermes + MedGuard All Domains pass@1; hermes harness; community submission 22.7
    Printed as 22.7%Independent run, measured Jul 2026Configuration: All Domains pass@1; hermes harness; community submission
    CHI-Bench leaderboard (actAVA) official leaderboard, actAVA, 12 Aug 2026. CHI-Bench v1.0.0, All Domains, rank 11, Agent/Model and Accuracy columns; PA/UM/CM follow; board last updated 2026-08-12; board Date column 2026-07-06; accessed 2026-09-30. Linked submission provenance: run 2026-07-01T06:59:37Z to 2026-07-01T08:17:44Z; submitted_at 2026-07-06T09:10:18Z.
    11 | hermes submitted by cuilinke | MedGuard | Open-source | 22.7% | 4.0% | 4.0% | 60.0% | 2026-07-06
    Every result from this document
  12. 12codex + gpt-5.5 20.9
    Printed as 20.9%Official leaderboard, measured Apr 2026
    CHI-Bench leaderboard (actAVA) official leaderboard, actAVA, 12 Aug 2026. CHI-Bench v1.0.0, All Domains, rank 12, Agent/Model and Accuracy columns; PA/UM/CM follow; board last updated 2026-08-12; board Date column 2026-05-01; accessed 2026-09-30. Linked submission provenance: run 2026-04-29T06:06:32Z to 2026-04-30T09:41:04Z; submitted_at 2026-04-30T09:41:04Z.
    12 | codex | gpt-5.5 | Proprietary | 20.9% | 29.3% | 32.0% | 1.3% | 2026-05-01
    Every result from this document
  13. 13claude-code + claude-sonnet-5 20.0
    Printed as 20.0%Official leaderboard, measured Jul 2026
    CHI-Bench leaderboard (actAVA) official leaderboard, actAVA, 12 Aug 2026. CHI-Bench v1.0.0, All Domains, rank 13, Agent/Model and Accuracy columns; PA/UM/CM follow; board last updated 2026-08-12; board Date column 2026-07-06; accessed 2026-09-30. Linked submission provenance: run 2026-07-02T04:22:04Z to 2026-07-02T04:31:46Z; submitted_at 2026-07-06T05:23:12Z.
    13 | claude-code | claude-sonnet-5 | Proprietary | 20.0% | 24.0% | 24.0% | 12.0% | 2026-07-06
    Every result from this document
  14. 14hermes + glm-5.1 All Domains pass@1; hermes harness 18.7
    Printed as 18.7%Official leaderboard, measured May 2026Configuration: All Domains pass@1; hermes harness
    CHI-Bench leaderboard (actAVA) official leaderboard, actAVA, 12 Aug 2026. CHI-Bench v1.0.0, All Domains, rank 15, Agent/Model and Accuracy columns; PA/UM/CM follow; board last updated 2026-08-12; board Date column 2026-05-01; accessed 2026-09-30. Linked submission provenance: run 2026-05-03T07:01:48Z to 2026-05-04T05:47:30Z; submitted_at 2026-05-04T05:47:30Z.
    15 | hermes | glm-5.1 | Open-source | 18.7% | 10.7% | 34.7% | 10.7% | 2026-05-01
    Every result from this document
  15. 14openai-agents + glm-5.1 All Domains pass@1; openai-agents harness 18.7
    Printed as 18.7%Official leaderboard, measured May 2026Configuration: All Domains pass@1; openai-agents harness
    CHI-Bench leaderboard (actAVA) official leaderboard, actAVA, 12 Aug 2026. CHI-Bench v1.0.0, All Domains, rank 14, Agent/Model and Accuracy columns; PA/UM/CM follow; board last updated 2026-08-12; board Date column 2026-05-01; accessed 2026-09-30. Linked submission provenance: run 2026-05-03T08:41:54Z to 2026-05-04T07:03:18Z; submitted_at 2026-05-04T07:03:18Z.
    14 | openai-agents | glm-5.1 | Open-source | 18.7% | 18.7% | 33.3% | 4.0% | 2026-05-01
    Every result from this document
  16. 14openai-agents + glm-5.2 All Domains pass@1; openai-agents harness 18.7
    Printed as 18.7%Official leaderboard, measured Jul 2026Configuration: All Domains pass@1; openai-agents harness
    CHI-Bench leaderboard (actAVA) official leaderboard, actAVA, 12 Aug 2026. CHI-Bench v1.0.0, All Domains, rank 16, Agent/Model and Accuracy columns; PA/UM/CM follow; board last updated 2026-08-12; board Date column 2026-07-06; accessed 2026-09-30. Linked submission provenance: run 2026-07-02T04:22:04Z to 2026-07-02T04:48:05Z; submitted_at 2026-07-06T05:23:05Z.
    16 | openai-agents | glm-5.2 | Open-source | 18.7% | 20.0% | 32.0% | 4.0% | 2026-07-06
    Every result from this document
  17. 17openclaw + claude-opus-4-7 All Domains pass@1; openclaw harness 17.3
    Printed as 17.3%Official leaderboard, measured May 2026Configuration: All Domains pass@1; openclaw harness
    CHI-Bench leaderboard (actAVA) official leaderboard, actAVA, 12 Aug 2026. CHI-Bench v1.0.0, All Domains, rank 17, Agent/Model and Accuracy columns; PA/UM/CM follow; board last updated 2026-08-12; board Date column 2026-05-01; accessed 2026-09-30. Linked submission provenance: run 2026-05-04T13:50:05Z to 2026-05-04T15:47:22Z; submitted_at 2026-05-04T15:47:22Z.
    17 | openclaw | claude-opus-4-7 | Proprietary | 17.3% | 18.7% | 13.3% | 20.0% | 2026-05-01
    Every result from this document
  18. 18openclaw + glm-5.1 All Domains pass@1; openclaw harness 16.9
    Printed as 16.9%Official leaderboard, measured May 2026Configuration: All Domains pass@1; openclaw harness
    CHI-Bench leaderboard (actAVA) official leaderboard, actAVA, 12 Aug 2026. CHI-Bench v1.0.0, All Domains, rank 18, Agent/Model and Accuracy columns; PA/UM/CM follow; board last updated 2026-08-12; board Date column 2026-05-01; accessed 2026-09-30. Linked submission provenance: run 2026-05-02T09:10:57Z to 2026-05-04T02:49:29Z; submitted_at 2026-05-04T02:49:29Z.
    18 | openclaw | glm-5.1 | Open-source | 16.9% | 13.3% | 26.7% | 10.7% | 2026-05-01
    Every result from this document
  19. 19hermes + qwen-3.6-max All Domains pass@1; hermes harness; Qwen3.6-Max preview endpoint 16.4
    Printed as 16.4%Official leaderboard, measured May 2026Configuration: All Domains pass@1; hermes harness; Qwen3.6-Max preview endpoint
    CHI-Bench leaderboard (actAVA) official leaderboard, actAVA, 12 Aug 2026. CHI-Bench v1.0.0, All Domains, rank 19, Agent/Model and Accuracy columns; PA/UM/CM follow; board last updated 2026-08-12; board Date column 2026-05-01; accessed 2026-09-30. Linked submission provenance: run 2026-05-03T07:32:36Z to 2026-05-04T06:18:19Z; submitted_at 2026-05-04T06:18:19Z.
    19 | hermes | qwen-3.6-max | Open-source | 16.4% | 9.3% | 26.7% | 13.3% | 2026-05-01
    Every result from this document
  20. 20codex + gpt-5.4 All Domains pass@1; codex harness 16.0
    Printed as 16.0%Official leaderboard, measured Apr 2026Configuration: All Domains pass@1; codex harness
    CHI-Bench leaderboard (actAVA) official leaderboard, actAVA, 12 Aug 2026. CHI-Bench v1.0.0, All Domains, rank 20, Agent/Model and Accuracy columns; PA/UM/CM follow; board last updated 2026-08-12; board Date column 2026-05-01; accessed 2026-09-30. Linked submission provenance: run 2026-04-29T19:36:29Z to 2026-04-30T22:58:31Z; submitted_at 2026-04-30T22:58:31Z.
    20 | codex | gpt-5.4 | Proprietary | 16.0% | 24.0% | 17.3% | 6.7% | 2026-05-01
    Every result from this document
  21. 21hermes + kimi-k2.6 All Domains pass@1; hermes harness 15.6
    Printed as 15.6%Official leaderboard, measured May 2026Configuration: All Domains pass@1; hermes harness
    CHI-Bench leaderboard (actAVA) official leaderboard, actAVA, 12 Aug 2026. CHI-Bench v1.0.0, All Domains, rank 22, Agent/Model and Accuracy columns; PA/UM/CM follow; board last updated 2026-08-12; board Date column 2026-05-01; accessed 2026-09-30. Linked submission provenance: run 2026-05-03T07:19:12Z to 2026-05-04T06:35:44Z; submitted_at 2026-05-04T06:35:44Z.
    22 | hermes | kimi-k2.6 | Open-source | 15.6% | 18.7% | 21.3% | 6.7% | 2026-05-01
    Every result from this document
  22. 21openai-agents + qwen-3.6-max All Domains pass@1; openai-agents harness; Qwen3.6-Max preview endpoint 15.6
    Printed as 15.6%Official leaderboard, measured May 2026Configuration: All Domains pass@1; openai-agents harness; Qwen3.6-Max preview endpoint
    CHI-Bench leaderboard (actAVA) official leaderboard, actAVA, 12 Aug 2026. CHI-Bench v1.0.0, All Domains, rank 21, Agent/Model and Accuracy columns; PA/UM/CM follow; board last updated 2026-08-12; board Date column 2026-05-01; accessed 2026-09-30. Linked submission provenance: run 2026-05-03T09:19:32Z to 2026-05-04T07:25:50Z; submitted_at 2026-05-04T07:25:50Z.
    21 | openai-agents | qwen-3.6-max | Open-source | 15.6% | 16.0% | 26.7% | 4.0% | 2026-05-01
    Every result from this document
  23. 23openai-agents + kimi-k2.6 All Domains pass@1; openai-agents harness 15.1
    Printed as 15.1%Official leaderboard, measured May 2026Configuration: All Domains pass@1; openai-agents harness
    CHI-Bench leaderboard (actAVA) official leaderboard, actAVA, 12 Aug 2026. CHI-Bench v1.0.0, All Domains, rank 23, Agent/Model and Accuracy columns; PA/UM/CM follow; board last updated 2026-08-12; board Date column 2026-05-01; accessed 2026-09-30. Linked submission provenance: run 2026-05-03T09:00:39Z to 2026-05-04T07:47:06Z; submitted_at 2026-05-04T07:47:06Z.
    23 | openai-agents | kimi-k2.6 | Open-source | 15.1% | 17.3% | 25.3% | 2.7% | 2026-05-01
    Every result from this document
  24. 24openai-agents + deepseek-v4-pro All Domains pass@1; openai-agents harness 14.2
    Printed as 14.2%Official leaderboard, measured May 2026Configuration: All Domains pass@1; openai-agents harness
    CHI-Bench leaderboard (actAVA) official leaderboard, actAVA, 12 Aug 2026. CHI-Bench v1.0.0, All Domains, rank 24, Agent/Model and Accuracy columns; PA/UM/CM follow; board last updated 2026-08-12; board Date column 2026-05-01; accessed 2026-09-30. Linked submission provenance: run 2026-05-03T08:35:38Z to 2026-05-04T06:59:15Z; submitted_at 2026-05-04T06:59:15Z.
    24 | openai-agents | deepseek-v4-pro | Open-source | 14.2% | 10.7% | 28.0% | 4.0% | 2026-05-01
    Every result from this document
  25. 25hermes + deepseek-v4-pro All Domains pass@1; hermes harness 13.8
    Printed as 13.8%Official leaderboard, measured May 2026Configuration: All Domains pass@1; hermes harness
    CHI-Bench leaderboard (actAVA) official leaderboard, actAVA, 12 Aug 2026. CHI-Bench v1.0.0, All Domains, rank 25, Agent/Model and Accuracy columns; PA/UM/CM follow; board last updated 2026-08-12; board Date column 2026-05-01; accessed 2026-09-30. Linked submission provenance: run 2026-05-03T06:28:45Z to 2026-05-04T05:51:44Z; submitted_at 2026-05-04T05:51:44Z.
    25 | hermes | deepseek-v4-pro | Open-source | 13.8% | 8.0% | 25.3% | 8.0% | 2026-05-01
    Every result from this document
  26. 26codex + gpt-5.6-luna All Domains pass@1; codex harness 13.3
    Printed as 13.3%Official leaderboard, measured Jul 2026Configuration: All Domains pass@1; codex harness
    CHI-Bench leaderboard (actAVA) official leaderboard, actAVA, 12 Aug 2026. CHI-Bench v1.0.0, All Domains, rank 27, Agent/Model and Accuracy columns; PA/UM/CM follow; board last updated 2026-08-12; board Date column 2026-07-24; accessed 2026-09-30. Linked submission provenance: run 2026-07-22T06:23:21.720425Z to 2026-07-22T08:02:34.258993Z; submitted_at 2026-07-24T23:01:25Z.
    27 | codex | gpt-5.6-luna | Proprietary | 13.3% | 20.0% | 16.0% | 4.0% | 2026-07-24
    Every result from this document
  27. 26codex + gpt-5.6-terra All Domains pass@1; codex harness 13.3
    Printed as 13.3%Official leaderboard, measured Jul 2026Configuration: All Domains pass@1; codex harness
    CHI-Bench leaderboard (actAVA) official leaderboard, actAVA, 12 Aug 2026. CHI-Bench v1.0.0, All Domains, rank 26, Agent/Model and Accuracy columns; PA/UM/CM follow; board last updated 2026-08-12; board Date column 2026-07-24; accessed 2026-09-30. Linked submission provenance: run 2026-07-22T04:52:42.763467Z to 2026-07-22T06:23:17.742720Z; submitted_at 2026-07-24T23:01:25Z.
    26 | codex | gpt-5.6-terra | Proprietary | 13.3% | 12.0% | 20.0% | 8.0% | 2026-07-24
    Every result from this document
  28. 28gemini-cli + gemini-3-flash All Domains pass@1; gemini-cli harness 12.5
    Printed as 12.5%Official leaderboard, measured Apr 2026Configuration: All Domains pass@1; gemini-cli harness
    CHI-Bench leaderboard (actAVA) official leaderboard, actAVA, 12 Aug 2026. CHI-Bench v1.0.0, All Domains, rank 28, Agent/Model and Accuracy columns; PA/UM/CM follow; board last updated 2026-08-12; board Date column 2026-05-01; accessed 2026-09-30. Linked submission provenance: run 2026-04-30T09:24:38Z to 2026-04-30T20:11:40Z; submitted_at 2026-04-30T20:11:40Z.
    28 | gemini-cli | gemini-3-flash | Proprietary | 12.5% | 18.7% | 18.7% | 0.0% | 2026-05-01
    Every result from this document
  29. 29deepagents + glm-5.1 All Domains pass@1; deepagents harness 11.1
    Printed as 11.1%Official leaderboard, measured May 2026Configuration: All Domains pass@1; deepagents harness
    CHI-Bench leaderboard (actAVA) official leaderboard, actAVA, 12 Aug 2026. CHI-Bench v1.0.0, All Domains, rank 30, Agent/Model and Accuracy columns; PA/UM/CM follow; board last updated 2026-08-12; board Date column 2026-05-01; accessed 2026-09-30. Linked submission provenance: run 2026-05-03T21:51:53Z to 2026-05-04T13:10:34Z; submitted_at 2026-05-04T13:10:34Z.
    30 | deepagents | glm-5.1 | Open-source | 11.1% | 17.3% | 10.7% | 5.3% | 2026-05-01
    Every result from this document
  30. 29openclaw + deepseek-v4-pro All Domains pass@1; openclaw harness 11.1
    Printed as 11.1%Official leaderboard, measured May 2026Configuration: All Domains pass@1; openclaw harness
    CHI-Bench leaderboard (actAVA) official leaderboard, actAVA, 12 Aug 2026. CHI-Bench v1.0.0, All Domains, rank 29, Agent/Model and Accuracy columns; PA/UM/CM follow; board last updated 2026-08-12; board Date column 2026-05-01; accessed 2026-09-30. Linked submission provenance: run 2026-05-02T09:01:51Z to 2026-05-04T03:00:01Z; submitted_at 2026-05-04T03:00:01Z.
    29 | openclaw | deepseek-v4-pro | Open-source | 11.1% | 14.7% | 12.0% | 6.7% | 2026-05-01
    Every result from this document
  31. 31deepagents + deepseek-v4-pro All Domains pass@1; deepagents harness 10.7
    Printed as 10.7%Official leaderboard, measured May 2026Configuration: All Domains pass@1; deepagents harness
    CHI-Bench leaderboard (actAVA) official leaderboard, actAVA, 12 Aug 2026. CHI-Bench v1.0.0, All Domains, rank 31, Agent/Model and Accuracy columns; PA/UM/CM follow; board last updated 2026-08-12; board Date column 2026-05-01; accessed 2026-09-30. Linked submission provenance: run 2026-05-03T07:35:56Z to 2026-05-05T01:34:20Z; submitted_at 2026-05-05T01:34:20Z.
    31 | deepagents | deepseek-v4-pro | Open-source | 10.7% | 14.7% | 10.7% | 6.7% | 2026-05-01
    Every result from this document
  32. 32openclaw + kimi-k2.6 All Domains pass@1; openclaw harness 10.2
    Printed as 10.2%Official leaderboard, measured May 2026Configuration: All Domains pass@1; openclaw harness
    CHI-Bench leaderboard (actAVA) official leaderboard, actAVA, 12 Aug 2026. CHI-Bench v1.0.0, All Domains, rank 32, Agent/Model and Accuracy columns; PA/UM/CM follow; board last updated 2026-08-12; board Date column 2026-05-01; accessed 2026-09-30. Linked submission provenance: run 2026-05-03T06:28:45Z to 2026-05-04T05:50:33Z; submitted_at 2026-05-04T05:50:33Z.
    32 | openclaw | kimi-k2.6 | Open-source | 10.2% | 12.0% | 18.7% | 0.0% | 2026-05-01
    Every result from this document
  33. 33deepagents + qwen-3.6-max All Domains pass@1; deepagents harness; Qwen3.6-Max preview endpoint 9.3
    Printed as 9.3%Official leaderboard, measured May 2026Configuration: All Domains pass@1; deepagents harness; Qwen3.6-Max preview endpoint
    CHI-Bench leaderboard (actAVA) official leaderboard, actAVA, 12 Aug 2026. CHI-Bench v1.0.0, All Domains, rank 33, Agent/Model and Accuracy columns; PA/UM/CM follow; board last updated 2026-08-12; board Date column 2026-05-01; accessed 2026-09-30. Linked submission provenance: run 2026-05-03T22:18:28Z to 2026-05-04T13:19:53Z; submitted_at 2026-05-04T13:19:53Z.
    33 | deepagents | qwen-3.6-max | Open-source | 9.3% | 12.0% | 10.7% | 5.3% | 2026-05-01
    Every result from this document
  34. 34codex + gpt-5.4-mini All Domains pass@1; codex harness 8.4
    Printed as 8.4%Official leaderboard, measured May 2026Configuration: All Domains pass@1; codex harness
    CHI-Bench leaderboard (actAVA) official leaderboard, actAVA, 12 Aug 2026. CHI-Bench v1.0.0, All Domains, rank 34, Agent/Model and Accuracy columns; PA/UM/CM follow; board last updated 2026-08-12; board Date column 2026-05-01; accessed 2026-09-30. Linked submission provenance: run 2026-04-29T19:36:29Z to 2026-05-01T03:47:25Z; submitted_at 2026-05-01T03:47:25Z.
    34 | codex | gpt-5.4-mini | Proprietary | 8.4% | 10.7% | 13.3% | 1.3% | 2026-05-01
    Every result from this document
  35. 35openai-agents + TML Inkling 256K All Domains pass@1; openai-agents; Tinker 256K, high reasoning, 50-turn limit 8.0
    Printed as 8.0%Official leaderboard, measured Jul 2026Configuration: All Domains pass@1; openai-agents; Tinker 256K, high reasoning, 50-turn limit
    CHI-Bench leaderboard (actAVA) official leaderboard, actAVA, 12 Aug 2026. CHI-Bench v1.0.0, All Domains, rank 35, Agent/Model and Accuracy columns; PA/UM/CM follow; board last updated 2026-08-12; board Date column 2026-07-24; accessed 2026-09-30. Linked submission provenance: run 2026-07-22T14:25:50.976659Z to 2026-07-22T16:53:02.466832Z; submitted_at 2026-07-24T23:02:07Z.
    35 | openai-agents | TML Inkling 256K | Open-source | 8.0% | 4.0% | 16.0% | 4.0% | 2026-07-24
    Every result from this document
  36. 36claude-code + claude-haiku-4-5 All Domains pass@1; claude-code harness 6.2
    Printed as 6.2%Official leaderboard, measured Apr 2026Configuration: All Domains pass@1; claude-code harness
    CHI-Bench leaderboard (actAVA) official leaderboard, actAVA, 12 Aug 2026. CHI-Bench v1.0.0, All Domains, rank 37, Agent/Model and Accuracy columns; PA/UM/CM follow; board last updated 2026-08-12; board Date column 2026-05-01; accessed 2026-09-30. Linked submission provenance: run 2026-04-29T19:36:29Z to 2026-04-30T19:34:41Z; submitted_at 2026-04-30T19:34:41Z.
    37 | claude-code | claude-haiku-4-5 | Proprietary | 6.2% | 0.0% | 14.7% | 4.0% | 2026-05-01
    Every result from this document
  37. 37openai-agents + grok-4.3 All Domains pass@1; openai-agents harness 5.8
    Printed as 5.8%Official leaderboard, measured May 2026Configuration: All Domains pass@1; openai-agents harness
    CHI-Bench leaderboard (actAVA) official leaderboard, actAVA, 12 Aug 2026. CHI-Bench v1.0.0, All Domains, rank 38, Agent/Model and Accuracy columns; PA/UM/CM follow; board last updated 2026-08-12; board Date column 2026-05-01; accessed 2026-09-30. Linked submission provenance: run 2026-05-03T09:26:38Z to 2026-05-04T07:18:48Z; submitted_at 2026-05-04T07:18:48Z.
    38 | openai-agents | grok-4.3 | Open-source | 5.8% | 0.0% | 16.0% | 1.3% | 2026-05-01
    Every result from this document
  38. 38openclaw + qwen-3.6-max All Domains pass@1; openclaw harness; Qwen3.6-Max preview endpoint 4.9
    Printed as 4.9%Official leaderboard, measured May 2026Configuration: All Domains pass@1; openclaw harness; Qwen3.6-Max preview endpoint
    CHI-Bench leaderboard (actAVA) official leaderboard, actAVA, 12 Aug 2026. CHI-Bench v1.0.0, All Domains, rank 39, Agent/Model and Accuracy columns; PA/UM/CM follow; board last updated 2026-08-12; board Date column 2026-05-01; accessed 2026-09-30. Linked submission provenance: run 2026-05-02T09:26:43Z to 2026-05-04T03:15:50Z; submitted_at 2026-05-04T03:15:50Z.
    39 | openclaw | qwen-3.6-max | Open-source | 4.9% | 10.7% | 4.0% | 0.0% | 2026-05-01
    Every result from this document
  39. 39hermes + grok-4.3 All Domains pass@1; hermes harness 4.4
    Printed as 4.4%Official leaderboard, measured May 2026Configuration: All Domains pass@1; hermes harness
    CHI-Bench leaderboard (actAVA) official leaderboard, actAVA, 12 Aug 2026. CHI-Bench v1.0.0, All Domains, rank 40, Agent/Model and Accuracy columns; PA/UM/CM follow; board last updated 2026-08-12; board Date column 2026-05-01; accessed 2026-09-30. Linked submission provenance: run 2026-05-03T07:34:26Z to 2026-05-04T06:20:37Z; submitted_at 2026-05-04T06:20:37Z.
    40 | hermes | grok-4.3 | Open-source | 4.4% | 0.0% | 13.3% | 0.0% | 2026-05-01
    Every result from this document
  40. 40deepagents + kimi-k2.6 All Domains pass@1; deepagents harness 3.1
    Printed as 3.1%Official leaderboard, measured May 2026Configuration: All Domains pass@1; deepagents harness
    CHI-Bench leaderboard (actAVA) official leaderboard, actAVA, 12 Aug 2026. CHI-Bench v1.0.0, All Domains, rank 41, Agent/Model and Accuracy columns; PA/UM/CM follow; board last updated 2026-08-12; board Date column 2026-05-01; accessed 2026-09-30. Linked submission provenance: run 2026-05-03T21:58:07Z to 2026-05-04T13:39:21Z; submitted_at 2026-05-04T13:39:21Z.
    41 | deepagents | kimi-k2.6 | Open-source | 3.1% | 8.0% | 1.3% | 0.0% | 2026-05-01
    Every result from this document
  41. 41deepagents + grok-4.3 All Domains pass@1; deepagents harness 2.2
    Printed as 2.2%Official leaderboard, measured May 2026Configuration: All Domains pass@1; deepagents harness
    CHI-Bench leaderboard (actAVA) official leaderboard, actAVA, 12 Aug 2026. CHI-Bench v1.0.0, All Domains, rank 42, Agent/Model and Accuracy columns; PA/UM/CM follow; board last updated 2026-08-12; board Date column 2026-05-01; accessed 2026-09-30. Linked submission provenance: run 2026-05-03T22:19:46Z to 2026-05-04T13:19:20Z; submitted_at 2026-05-04T13:19:20Z.
    42 | deepagents | grok-4.3 | Open-source | 2.2% | 0.0% | 5.3% | 1.3% | 2026-05-01
    Every result from this document
  42. 42openclaw + grok-4.3 All Domains pass@1; openclaw harness 0.4
    Printed as 0.4%Official leaderboard, measured May 2026Configuration: All Domains pass@1; openclaw harness
    CHI-Bench leaderboard (actAVA) official leaderboard, actAVA, 12 Aug 2026. CHI-Bench v1.0.0, All Domains, rank 43, Agent/Model and Accuracy columns; PA/UM/CM follow; board last updated 2026-08-12; board Date column 2026-05-01; accessed 2026-09-30. Linked submission provenance: run 2026-05-02T09:34:46Z to 2026-05-04T00:03:58Z; submitted_at 2026-05-04T00:03:58Z.
    43 | openclaw | grok-4.3 | Open-source | 0.4% | 1.3% | 0.0% | 0.0% | 2026-05-01
    Every result from this document
  43. 43openai-agents + Nemotron 3 Ultra 256K All Domains pass@1; openai-agents; Tinker 256K 0.0
    Printed as 0.0%Official leaderboard, measured Jul 2026Configuration: All Domains pass@1; openai-agents; Tinker 256K
    CHI-Bench leaderboard (actAVA) official leaderboard, actAVA, 12 Aug 2026. CHI-Bench v1.0.0, All Domains, rank 44, Agent/Model and Accuracy columns; PA/UM/CM follow; board last updated 2026-08-12; board Date column 2026-07-24; accessed 2026-09-30. Linked submission provenance: run 2026-07-22T22:31:29.302007Z to 2026-07-22T23:37:41.111758Z; submitted_at 2026-08-12T20:20:00Z.
    44 | openai-agents | Nemotron 3 Ultra 256K | Open-source | 0.0% | 0.0% | 0.0% | 0.0% | 2026-07-24
    Every result from this document

Documents

44