Clinical Benchmarks

CHI-Bench

Long-horizon healthcare operations workflows for agents: prior authorization, utilization management, and care management.

More about this boardLess

Long-horizon US healthcare operations workflows for agents (prior authorization, utilization management, care management), 60-80 step tasks across 4-6 stages, judged by deterministic unit tests plus an LLM judge for evidence grounding, consent, and cross-stage consistency.

Released May 20, 2026; updated August 12, 2026 across 45 harness configurations. The launch report led with reliability, not capability: no agent stayed above 20 percent across three identical runs. Harness choice matters as much as model choice, and the board accepts community submissions, so rows mix author-run and submitted results.

Published by actAVA.ai, released May 2026. 75 workflows (25 per domain), 21 healthcare applications, 200+ MCP tools. Pass@1 with binary 0/1 reward, higher better.

Headline metric
pass@1 accuracy, 75 tasks (25 each: prior authorization, utilization management, care management)
Rows on the board
45
Board last updated
12 Aug 2026
Board version
CHI-Bench v1.0.0
Publisher
actAVA.ai
Rows
43
Models
25 of 45 on the board
Released
May 2026
Measured to
Aug 2026
Unit
75 operations workflows
Documents cited
1

Every result on CHI-Bench, on the board's own scale

43 results, one column each, highest first. Pass@1 with binary 0/1 reward, higher better.

Column colour is the model's lab. The table below says who ran each result.

Every published result, ranked on this board alone

43 rows. Higher is better. Bars run from 0 to 100; whiskers show a published interval. Select a row for its quote, page and document.

1erius + claude-opus-5Humana (harness) / Anthropic (model) · community-submitted harness…54.7Independent runAug 2026[1]
01 | erius submitted by Michael Johnson (MJ) | claude-opus-5 | Proprietary | 54.7% | 72.0% | 36.0% | 56.0% | 2026-07-26
As printed
54.7%
Where
CHI-Bench v1.0.0, All Domains, rank 01, Agent/Model and Accuracy columns; PA/UM/CM follow; board last updated 2026-08-12; board Date column 2026-07-26; accessed 2026-09-30.
Setting
community-submitted harness config validated by automated workspace judge
Reported by
independent run
Document
[1] CHI-Bench leaderboard (actAVA)
2claude-code + claude-opus-5Anthropic37.3Board publisherJul 2026[1]
03 | claude-code | claude-opus-5 | Proprietary | 37.3% | 20.0% | 32.0% | 60.0% | 2026-07-24
As printed
37.3%
Where
CHI-Bench v1.0.0, All Domains, rank 03, Agent/Model and Accuracy columns; PA/UM/CM follow; board last updated 2026-08-12; board Date column 2026-07-24; accessed 2026-09-30. Linked submission provenance: run 2026-07-24T19:08:02Z to 2026-07-24T23:22:36Z; submitted_at 2026-08-12T20:20:00Z.
Reported by
official leaderboard
Document
[1] CHI-Bench leaderboard (actAVA)
2erius + claude-opus-4-8Humana / Anthropic37.3Independent runAug 2026[1]
02 | erius submitted by Michael Johnson (MJ) | claude-opus-4-8 | Proprietary | 37.3% | 40.0% | 16.0% | 56.0% | 2026-06-05
As printed
37.3%
Where
CHI-Bench v1.0.0, All Domains, rank 02, Agent/Model and Accuracy columns; PA/UM/CM follow; board last updated 2026-08-12; board Date column 2026-06-05; accessed 2026-09-30.
Reported by
independent run
Document
[1] CHI-Bench leaderboard (actAVA)
4claude-code + claude-opus-4-8Anthropic33.3Board publisherMay 2026[1]
04 | claude-code | claude-opus-4-8 | Proprietary | 33.3% | 32.0% | 28.0% | 40.0% | 2026-05-28
As printed
33.3%
Where
CHI-Bench v1.0.0, All Domains, rank 04, Agent/Model and Accuracy columns; PA/UM/CM follow; board last updated 2026-08-12; board Date column 2026-05-28; accessed 2026-09-30. Linked submission provenance: run 2026-05-28T17:19:15Z to 2026-05-28T20:48:59Z; submitted_at 2026-05-28T20:53:26Z.
Reported by
official leaderboard
Document
[1] CHI-Bench leaderboard (actAVA)
5claude-code + claude-opus-4-6Anthropic28.0Board publisherMay 2026[1]
05 | claude-code | claude-opus-4-6 | Proprietary | 28.0% | 20.0% | 36.0% | 28.0% | 2026-05-01
As printed
28.0%
Where
CHI-Bench v1.0.0, All Domains, rank 05, Agent/Model and Accuracy columns; PA/UM/CM follow; board last updated 2026-08-12; board Date column 2026-05-01; accessed 2026-09-30. Linked submission provenance: run 2026-05-12T18:55:45Z to 2026-05-12T18:55:46Z; submitted_at 2026-05-12T22:39:33Z.
Reported by
official leaderboard
Document
[1] CHI-Bench leaderboard (actAVA)
6claude-code + claude-sonnet-4-6Anthropic26.2Board publisherApr 2026[1]
06 | claude-code | claude-sonnet-4-6 | Proprietary | 26.2% | 24.0% | 34.7% | 20.0% | 2026-05-01
As printed
26.2%
Where
CHI-Bench v1.0.0, All Domains, rank 06, Agent/Model and Accuracy columns; PA/UM/CM follow; board last updated 2026-08-12; board Date column 2026-05-01; accessed 2026-09-30. Linked submission provenance: run 2026-04-29T14:43:27Z to 2026-04-30T15:00:19Z; submitted_at 2026-04-30T15:00:19Z.
Reported by
official leaderboard
Document
[1] CHI-Bench leaderboard (actAVA)
7codex + gpt-5.6-solOpenAI25.3Board publisherJul 2026[1]
07 | codex | gpt-5.6-sol | Proprietary | 25.3% | 36.0% | 28.0% | 12.0% | 2026-07-24
As printed
25.3%
Where
CHI-Bench v1.0.0, All Domains, rank 07, Agent/Model and Accuracy columns; PA/UM/CM follow; board last updated 2026-08-12; board Date column 2026-07-24; accessed 2026-09-30. Linked submission provenance: run 2026-07-22T02:41:32.718905Z to 2026-07-22T04:52:39.419746Z; submitted_at 2026-07-24T23:01:25Z.
Reported by
official leaderboard
Document
[1] CHI-Bench leaderboard (actAVA)
7openai-agents + kimi-k3Moonshot AI25.3Board publisherJul 2026[1]
08 | openai-agents | kimi-k3 | Open-source | 25.3% | 28.0% | 32.0% | 16.0% | 2026-07-24
As printed
25.3%
Where
CHI-Bench v1.0.0, All Domains, rank 08, Agent/Model and Accuracy columns; PA/UM/CM follow; board last updated 2026-08-12; board Date column 2026-07-24; accessed 2026-09-30. Linked submission provenance: run 2026-07-22T08:03:23.788878Z to 2026-07-22T14:22:38.269923Z; submitted_at 2026-08-12T20:20:00Z.
Reported by
official leaderboard
Document
[1] CHI-Bench leaderboard (actAVA)
9claude-code + claude-opus-4-7Anthropic24.4Board publisherApr 2026[1]
09 | claude-code | claude-opus-4-7 | Proprietary | 24.4% | 24.0% | 17.3% | 32.0% | 2026-05-01
As printed
24.4%
Where
CHI-Bench v1.0.0, All Domains, rank 09, Agent/Model and Accuracy columns; PA/UM/CM follow; board last updated 2026-08-12; board Date column 2026-05-01; accessed 2026-09-30. Linked submission provenance: run 2026-04-29T06:06:32Z to 2026-04-30T10:00:03Z; submitted_at 2026-04-30T10:00:03Z.
Reported by
official leaderboard
Document
[1] CHI-Bench leaderboard (actAVA)
10claude-code + claude-fable-5Anthropic24.0Board publisherJul 2026[1]
10 | claude-code | claude-fable-5 | Proprietary | 24.0% | 24.0% | 24.0% | 24.0% | 2026-07-22
As printed
24.0%
Where
CHI-Bench v1.0.0, All Domains, rank 10, Agent/Model and Accuracy columns; PA/UM/CM follow; board last updated 2026-08-12; board Date column 2026-07-22; accessed 2026-09-30. Linked submission provenance: run 2026-07-22T07:16:27.207645Z to 2026-07-22T10:25:29.658582Z; submitted_at 2026-07-22T23:50:25Z.
Reported by
official leaderboard
Document
[1] CHI-Bench leaderboard (actAVA)
11hermes + MedGuardMedGuard · All Domains pass@122.7Independent runJul 2026[1]
11 | hermes submitted by cuilinke | MedGuard | Open-source | 22.7% | 4.0% | 4.0% | 60.0% | 2026-07-06
As printed
22.7%
Where
CHI-Bench v1.0.0, All Domains, rank 11, Agent/Model and Accuracy columns; PA/UM/CM follow; board last updated 2026-08-12; board Date column 2026-07-06; accessed 2026-09-30. Linked submission provenance: run 2026-07-01T06:59:37Z to 2026-07-01T08:17:44Z; submitted_at 2026-07-06T09:10:18Z.
Setting
All Domains pass@1; hermes harness; community submission
Reported by
independent run
Document
[1] CHI-Bench leaderboard (actAVA)
12codex + gpt-5.5OpenAI20.9Board publisherApr 2026[1]
12 | codex | gpt-5.5 | Proprietary | 20.9% | 29.3% | 32.0% | 1.3% | 2026-05-01
As printed
20.9%
Where
CHI-Bench v1.0.0, All Domains, rank 12, Agent/Model and Accuracy columns; PA/UM/CM follow; board last updated 2026-08-12; board Date column 2026-05-01; accessed 2026-09-30. Linked submission provenance: run 2026-04-29T06:06:32Z to 2026-04-30T09:41:04Z; submitted_at 2026-04-30T09:41:04Z.
Reported by
official leaderboard
Document
[1] CHI-Bench leaderboard (actAVA)
13claude-code + claude-sonnet-5Anthropic20.0Board publisherJul 2026[1]
13 | claude-code | claude-sonnet-5 | Proprietary | 20.0% | 24.0% | 24.0% | 12.0% | 2026-07-06
As printed
20.0%
Where
CHI-Bench v1.0.0, All Domains, rank 13, Agent/Model and Accuracy columns; PA/UM/CM follow; board last updated 2026-08-12; board Date column 2026-07-06; accessed 2026-09-30. Linked submission provenance: run 2026-07-02T04:22:04Z to 2026-07-02T04:31:46Z; submitted_at 2026-07-06T05:23:12Z.
Reported by
official leaderboard
Document
[1] CHI-Bench leaderboard (actAVA)
14hermes + glm-5.1Zhipu · All Domains pass@118.7Board publisherMay 2026[1]
15 | hermes | glm-5.1 | Open-source | 18.7% | 10.7% | 34.7% | 10.7% | 2026-05-01
As printed
18.7%
Where
CHI-Bench v1.0.0, All Domains, rank 15, Agent/Model and Accuracy columns; PA/UM/CM follow; board last updated 2026-08-12; board Date column 2026-05-01; accessed 2026-09-30. Linked submission provenance: run 2026-05-03T07:01:48Z to 2026-05-04T05:47:30Z; submitted_at 2026-05-04T05:47:30Z.
Setting
All Domains pass@1; hermes harness
Reported by
official leaderboard
Document
[1] CHI-Bench leaderboard (actAVA)
14openai-agents + glm-5.1Zhipu · All Domains pass@118.7Board publisherMay 2026[1]
14 | openai-agents | glm-5.1 | Open-source | 18.7% | 18.7% | 33.3% | 4.0% | 2026-05-01
As printed
18.7%
Where
CHI-Bench v1.0.0, All Domains, rank 14, Agent/Model and Accuracy columns; PA/UM/CM follow; board last updated 2026-08-12; board Date column 2026-05-01; accessed 2026-09-30. Linked submission provenance: run 2026-05-03T08:41:54Z to 2026-05-04T07:03:18Z; submitted_at 2026-05-04T07:03:18Z.
Setting
All Domains pass@1; openai-agents harness
Reported by
official leaderboard
Document
[1] CHI-Bench leaderboard (actAVA)
14openai-agents + glm-5.2Zhipu · All Domains pass@118.7Board publisherJul 2026[1]
16 | openai-agents | glm-5.2 | Open-source | 18.7% | 20.0% | 32.0% | 4.0% | 2026-07-06
As printed
18.7%
Where
CHI-Bench v1.0.0, All Domains, rank 16, Agent/Model and Accuracy columns; PA/UM/CM follow; board last updated 2026-08-12; board Date column 2026-07-06; accessed 2026-09-30. Linked submission provenance: run 2026-07-02T04:22:04Z to 2026-07-02T04:48:05Z; submitted_at 2026-07-06T05:23:05Z.
Setting
All Domains pass@1; openai-agents harness
Reported by
official leaderboard
Document
[1] CHI-Bench leaderboard (actAVA)
17openclaw + claude-opus-4-7Anthropic · All Domains pass@117.3Board publisherMay 2026[1]
17 | openclaw | claude-opus-4-7 | Proprietary | 17.3% | 18.7% | 13.3% | 20.0% | 2026-05-01
As printed
17.3%
Where
CHI-Bench v1.0.0, All Domains, rank 17, Agent/Model and Accuracy columns; PA/UM/CM follow; board last updated 2026-08-12; board Date column 2026-05-01; accessed 2026-09-30. Linked submission provenance: run 2026-05-04T13:50:05Z to 2026-05-04T15:47:22Z; submitted_at 2026-05-04T15:47:22Z.
Setting
All Domains pass@1; openclaw harness
Reported by
official leaderboard
Document
[1] CHI-Bench leaderboard (actAVA)
18openclaw + glm-5.1Zhipu · All Domains pass@116.9Board publisherMay 2026[1]
18 | openclaw | glm-5.1 | Open-source | 16.9% | 13.3% | 26.7% | 10.7% | 2026-05-01
As printed
16.9%
Where
CHI-Bench v1.0.0, All Domains, rank 18, Agent/Model and Accuracy columns; PA/UM/CM follow; board last updated 2026-08-12; board Date column 2026-05-01; accessed 2026-09-30. Linked submission provenance: run 2026-05-02T09:10:57Z to 2026-05-04T02:49:29Z; submitted_at 2026-05-04T02:49:29Z.
Setting
All Domains pass@1; openclaw harness
Reported by
official leaderboard
Document
[1] CHI-Bench leaderboard (actAVA)
19hermes + qwen-3.6-maxAlibaba · All Domains pass@116.4Board publisherMay 2026[1]
19 | hermes | qwen-3.6-max | Open-source | 16.4% | 9.3% | 26.7% | 13.3% | 2026-05-01
As printed
16.4%
Where
CHI-Bench v1.0.0, All Domains, rank 19, Agent/Model and Accuracy columns; PA/UM/CM follow; board last updated 2026-08-12; board Date column 2026-05-01; accessed 2026-09-30. Linked submission provenance: run 2026-05-03T07:32:36Z to 2026-05-04T06:18:19Z; submitted_at 2026-05-04T06:18:19Z.
Setting
All Domains pass@1; hermes harness; Qwen3.6-Max preview endpoint
Reported by
official leaderboard
Document
[1] CHI-Bench leaderboard (actAVA)
20codex + gpt-5.4OpenAI · All Domains pass@116.0Board publisherApr 2026[1]
20 | codex | gpt-5.4 | Proprietary | 16.0% | 24.0% | 17.3% | 6.7% | 2026-05-01
As printed
16.0%
Where
CHI-Bench v1.0.0, All Domains, rank 20, Agent/Model and Accuracy columns; PA/UM/CM follow; board last updated 2026-08-12; board Date column 2026-05-01; accessed 2026-09-30. Linked submission provenance: run 2026-04-29T19:36:29Z to 2026-04-30T22:58:31Z; submitted_at 2026-04-30T22:58:31Z.
Setting
All Domains pass@1; codex harness
Reported by
official leaderboard
Document
[1] CHI-Bench leaderboard (actAVA)
21hermes + kimi-k2.6Moonshot AI · All Domains pass@115.6Board publisherMay 2026[1]
22 | hermes | kimi-k2.6 | Open-source | 15.6% | 18.7% | 21.3% | 6.7% | 2026-05-01
As printed
15.6%
Where
CHI-Bench v1.0.0, All Domains, rank 22, Agent/Model and Accuracy columns; PA/UM/CM follow; board last updated 2026-08-12; board Date column 2026-05-01; accessed 2026-09-30. Linked submission provenance: run 2026-05-03T07:19:12Z to 2026-05-04T06:35:44Z; submitted_at 2026-05-04T06:35:44Z.
Setting
All Domains pass@1; hermes harness
Reported by
official leaderboard
Document
[1] CHI-Bench leaderboard (actAVA)
21openai-agents + qwen-3.6-maxAlibaba · All Domains pass@115.6Board publisherMay 2026[1]
21 | openai-agents | qwen-3.6-max | Open-source | 15.6% | 16.0% | 26.7% | 4.0% | 2026-05-01
As printed
15.6%
Where
CHI-Bench v1.0.0, All Domains, rank 21, Agent/Model and Accuracy columns; PA/UM/CM follow; board last updated 2026-08-12; board Date column 2026-05-01; accessed 2026-09-30. Linked submission provenance: run 2026-05-03T09:19:32Z to 2026-05-04T07:25:50Z; submitted_at 2026-05-04T07:25:50Z.
Setting
All Domains pass@1; openai-agents harness; Qwen3.6-Max preview endpoint
Reported by
official leaderboard
Document
[1] CHI-Bench leaderboard (actAVA)
23openai-agents + kimi-k2.6Moonshot AI · All Domains pass@115.1Board publisherMay 2026[1]
23 | openai-agents | kimi-k2.6 | Open-source | 15.1% | 17.3% | 25.3% | 2.7% | 2026-05-01
As printed
15.1%
Where
CHI-Bench v1.0.0, All Domains, rank 23, Agent/Model and Accuracy columns; PA/UM/CM follow; board last updated 2026-08-12; board Date column 2026-05-01; accessed 2026-09-30. Linked submission provenance: run 2026-05-03T09:00:39Z to 2026-05-04T07:47:06Z; submitted_at 2026-05-04T07:47:06Z.
Setting
All Domains pass@1; openai-agents harness
Reported by
official leaderboard
Document
[1] CHI-Bench leaderboard (actAVA)
24openai-agents + deepseek-v4-proDeepSeek · All Domains pass@114.2Board publisherMay 2026[1]
24 | openai-agents | deepseek-v4-pro | Open-source | 14.2% | 10.7% | 28.0% | 4.0% | 2026-05-01
As printed
14.2%
Where
CHI-Bench v1.0.0, All Domains, rank 24, Agent/Model and Accuracy columns; PA/UM/CM follow; board last updated 2026-08-12; board Date column 2026-05-01; accessed 2026-09-30. Linked submission provenance: run 2026-05-03T08:35:38Z to 2026-05-04T06:59:15Z; submitted_at 2026-05-04T06:59:15Z.
Setting
All Domains pass@1; openai-agents harness
Reported by
official leaderboard
Document
[1] CHI-Bench leaderboard (actAVA)
25hermes + deepseek-v4-proDeepSeek · All Domains pass@113.8Board publisherMay 2026[1]
25 | hermes | deepseek-v4-pro | Open-source | 13.8% | 8.0% | 25.3% | 8.0% | 2026-05-01
As printed
13.8%
Where
CHI-Bench v1.0.0, All Domains, rank 25, Agent/Model and Accuracy columns; PA/UM/CM follow; board last updated 2026-08-12; board Date column 2026-05-01; accessed 2026-09-30. Linked submission provenance: run 2026-05-03T06:28:45Z to 2026-05-04T05:51:44Z; submitted_at 2026-05-04T05:51:44Z.
Setting
All Domains pass@1; hermes harness
Reported by
official leaderboard
Document
[1] CHI-Bench leaderboard (actAVA)
26codex + gpt-5.6-lunaOpenAI · All Domains pass@113.3Board publisherJul 2026[1]
27 | codex | gpt-5.6-luna | Proprietary | 13.3% | 20.0% | 16.0% | 4.0% | 2026-07-24
As printed
13.3%
Where
CHI-Bench v1.0.0, All Domains, rank 27, Agent/Model and Accuracy columns; PA/UM/CM follow; board last updated 2026-08-12; board Date column 2026-07-24; accessed 2026-09-30. Linked submission provenance: run 2026-07-22T06:23:21.720425Z to 2026-07-22T08:02:34.258993Z; submitted_at 2026-07-24T23:01:25Z.
Setting
All Domains pass@1; codex harness
Reported by
official leaderboard
Document
[1] CHI-Bench leaderboard (actAVA)
26codex + gpt-5.6-terraOpenAI · All Domains pass@113.3Board publisherJul 2026[1]
26 | codex | gpt-5.6-terra | Proprietary | 13.3% | 12.0% | 20.0% | 8.0% | 2026-07-24
As printed
13.3%
Where
CHI-Bench v1.0.0, All Domains, rank 26, Agent/Model and Accuracy columns; PA/UM/CM follow; board last updated 2026-08-12; board Date column 2026-07-24; accessed 2026-09-30. Linked submission provenance: run 2026-07-22T04:52:42.763467Z to 2026-07-22T06:23:17.742720Z; submitted_at 2026-07-24T23:01:25Z.
Setting
All Domains pass@1; codex harness
Reported by
official leaderboard
Document
[1] CHI-Bench leaderboard (actAVA)
28gemini-cli + gemini-3-flashGoogle · All Domains pass@112.5Board publisherApr 2026[1]
28 | gemini-cli | gemini-3-flash | Proprietary | 12.5% | 18.7% | 18.7% | 0.0% | 2026-05-01
As printed
12.5%
Where
CHI-Bench v1.0.0, All Domains, rank 28, Agent/Model and Accuracy columns; PA/UM/CM follow; board last updated 2026-08-12; board Date column 2026-05-01; accessed 2026-09-30. Linked submission provenance: run 2026-04-30T09:24:38Z to 2026-04-30T20:11:40Z; submitted_at 2026-04-30T20:11:40Z.
Setting
All Domains pass@1; gemini-cli harness
Reported by
official leaderboard
Document
[1] CHI-Bench leaderboard (actAVA)
29deepagents + glm-5.1Zhipu · All Domains pass@111.1Board publisherMay 2026[1]
30 | deepagents | glm-5.1 | Open-source | 11.1% | 17.3% | 10.7% | 5.3% | 2026-05-01
As printed
11.1%
Where
CHI-Bench v1.0.0, All Domains, rank 30, Agent/Model and Accuracy columns; PA/UM/CM follow; board last updated 2026-08-12; board Date column 2026-05-01; accessed 2026-09-30. Linked submission provenance: run 2026-05-03T21:51:53Z to 2026-05-04T13:10:34Z; submitted_at 2026-05-04T13:10:34Z.
Setting
All Domains pass@1; deepagents harness
Reported by
official leaderboard
Document
[1] CHI-Bench leaderboard (actAVA)
29openclaw + deepseek-v4-proDeepSeek · All Domains pass@111.1Board publisherMay 2026[1]
29 | openclaw | deepseek-v4-pro | Open-source | 11.1% | 14.7% | 12.0% | 6.7% | 2026-05-01
As printed
11.1%
Where
CHI-Bench v1.0.0, All Domains, rank 29, Agent/Model and Accuracy columns; PA/UM/CM follow; board last updated 2026-08-12; board Date column 2026-05-01; accessed 2026-09-30. Linked submission provenance: run 2026-05-02T09:01:51Z to 2026-05-04T03:00:01Z; submitted_at 2026-05-04T03:00:01Z.
Setting
All Domains pass@1; openclaw harness
Reported by
official leaderboard
Document
[1] CHI-Bench leaderboard (actAVA)
31deepagents + deepseek-v4-proDeepSeek · All Domains pass@110.7Board publisherMay 2026[1]
31 | deepagents | deepseek-v4-pro | Open-source | 10.7% | 14.7% | 10.7% | 6.7% | 2026-05-01
As printed
10.7%
Where
CHI-Bench v1.0.0, All Domains, rank 31, Agent/Model and Accuracy columns; PA/UM/CM follow; board last updated 2026-08-12; board Date column 2026-05-01; accessed 2026-09-30. Linked submission provenance: run 2026-05-03T07:35:56Z to 2026-05-05T01:34:20Z; submitted_at 2026-05-05T01:34:20Z.
Setting
All Domains pass@1; deepagents harness
Reported by
official leaderboard
Document
[1] CHI-Bench leaderboard (actAVA)
32openclaw + kimi-k2.6Moonshot AI · All Domains pass@110.2Board publisherMay 2026[1]
32 | openclaw | kimi-k2.6 | Open-source | 10.2% | 12.0% | 18.7% | 0.0% | 2026-05-01
As printed
10.2%
Where
CHI-Bench v1.0.0, All Domains, rank 32, Agent/Model and Accuracy columns; PA/UM/CM follow; board last updated 2026-08-12; board Date column 2026-05-01; accessed 2026-09-30. Linked submission provenance: run 2026-05-03T06:28:45Z to 2026-05-04T05:50:33Z; submitted_at 2026-05-04T05:50:33Z.
Setting
All Domains pass@1; openclaw harness
Reported by
official leaderboard
Document
[1] CHI-Bench leaderboard (actAVA)
33deepagents + qwen-3.6-maxAlibaba · All Domains pass@19.3Board publisherMay 2026[1]
33 | deepagents | qwen-3.6-max | Open-source | 9.3% | 12.0% | 10.7% | 5.3% | 2026-05-01
As printed
9.3%
Where
CHI-Bench v1.0.0, All Domains, rank 33, Agent/Model and Accuracy columns; PA/UM/CM follow; board last updated 2026-08-12; board Date column 2026-05-01; accessed 2026-09-30. Linked submission provenance: run 2026-05-03T22:18:28Z to 2026-05-04T13:19:53Z; submitted_at 2026-05-04T13:19:53Z.
Setting
All Domains pass@1; deepagents harness; Qwen3.6-Max preview endpoint
Reported by
official leaderboard
Document
[1] CHI-Bench leaderboard (actAVA)
34codex + gpt-5.4-miniOpenAI · All Domains pass@18.4Board publisherMay 2026[1]
34 | codex | gpt-5.4-mini | Proprietary | 8.4% | 10.7% | 13.3% | 1.3% | 2026-05-01
As printed
8.4%
Where
CHI-Bench v1.0.0, All Domains, rank 34, Agent/Model and Accuracy columns; PA/UM/CM follow; board last updated 2026-08-12; board Date column 2026-05-01; accessed 2026-09-30. Linked submission provenance: run 2026-04-29T19:36:29Z to 2026-05-01T03:47:25Z; submitted_at 2026-05-01T03:47:25Z.
Setting
All Domains pass@1; codex harness
Reported by
official leaderboard
Document
[1] CHI-Bench leaderboard (actAVA)
35openai-agents + TML Inkling 256KThinking Machines · All Domains pass@18.0Board publisherJul 2026[1]
35 | openai-agents | TML Inkling 256K | Open-source | 8.0% | 4.0% | 16.0% | 4.0% | 2026-07-24
As printed
8.0%
Where
CHI-Bench v1.0.0, All Domains, rank 35, Agent/Model and Accuracy columns; PA/UM/CM follow; board last updated 2026-08-12; board Date column 2026-07-24; accessed 2026-09-30. Linked submission provenance: run 2026-07-22T14:25:50.976659Z to 2026-07-22T16:53:02.466832Z; submitted_at 2026-07-24T23:02:07Z.
Setting
All Domains pass@1; openai-agents; Tinker 256K, high reasoning, 50-turn limit
Reported by
official leaderboard
Document
[1] CHI-Bench leaderboard (actAVA)
36claude-code + claude-haiku-4-5Anthropic · All Domains pass@16.2Board publisherApr 2026[1]
37 | claude-code | claude-haiku-4-5 | Proprietary | 6.2% | 0.0% | 14.7% | 4.0% | 2026-05-01
As printed
6.2%
Where
CHI-Bench v1.0.0, All Domains, rank 37, Agent/Model and Accuracy columns; PA/UM/CM follow; board last updated 2026-08-12; board Date column 2026-05-01; accessed 2026-09-30. Linked submission provenance: run 2026-04-29T19:36:29Z to 2026-04-30T19:34:41Z; submitted_at 2026-04-30T19:34:41Z.
Setting
All Domains pass@1; claude-code harness
Reported by
official leaderboard
Document
[1] CHI-Bench leaderboard (actAVA)
37openai-agents + grok-4.3SpaceX AI · All Domains pass@15.8Board publisherMay 2026[1]
38 | openai-agents | grok-4.3 | Open-source | 5.8% | 0.0% | 16.0% | 1.3% | 2026-05-01
As printed
5.8%
Where
CHI-Bench v1.0.0, All Domains, rank 38, Agent/Model and Accuracy columns; PA/UM/CM follow; board last updated 2026-08-12; board Date column 2026-05-01; accessed 2026-09-30. Linked submission provenance: run 2026-05-03T09:26:38Z to 2026-05-04T07:18:48Z; submitted_at 2026-05-04T07:18:48Z.
Setting
All Domains pass@1; openai-agents harness
Reported by
official leaderboard
Document
[1] CHI-Bench leaderboard (actAVA)
38openclaw + qwen-3.6-maxAlibaba · All Domains pass@14.9Board publisherMay 2026[1]
39 | openclaw | qwen-3.6-max | Open-source | 4.9% | 10.7% | 4.0% | 0.0% | 2026-05-01
As printed
4.9%
Where
CHI-Bench v1.0.0, All Domains, rank 39, Agent/Model and Accuracy columns; PA/UM/CM follow; board last updated 2026-08-12; board Date column 2026-05-01; accessed 2026-09-30. Linked submission provenance: run 2026-05-02T09:26:43Z to 2026-05-04T03:15:50Z; submitted_at 2026-05-04T03:15:50Z.
Setting
All Domains pass@1; openclaw harness; Qwen3.6-Max preview endpoint
Reported by
official leaderboard
Document
[1] CHI-Bench leaderboard (actAVA)
39hermes + grok-4.3SpaceX AI · All Domains pass@14.4Board publisherMay 2026[1]
40 | hermes | grok-4.3 | Open-source | 4.4% | 0.0% | 13.3% | 0.0% | 2026-05-01
As printed
4.4%
Where
CHI-Bench v1.0.0, All Domains, rank 40, Agent/Model and Accuracy columns; PA/UM/CM follow; board last updated 2026-08-12; board Date column 2026-05-01; accessed 2026-09-30. Linked submission provenance: run 2026-05-03T07:34:26Z to 2026-05-04T06:20:37Z; submitted_at 2026-05-04T06:20:37Z.
Setting
All Domains pass@1; hermes harness
Reported by
official leaderboard
Document
[1] CHI-Bench leaderboard (actAVA)
40deepagents + kimi-k2.6Moonshot AI · All Domains pass@13.1Board publisherMay 2026[1]
41 | deepagents | kimi-k2.6 | Open-source | 3.1% | 8.0% | 1.3% | 0.0% | 2026-05-01
As printed
3.1%
Where
CHI-Bench v1.0.0, All Domains, rank 41, Agent/Model and Accuracy columns; PA/UM/CM follow; board last updated 2026-08-12; board Date column 2026-05-01; accessed 2026-09-30. Linked submission provenance: run 2026-05-03T21:58:07Z to 2026-05-04T13:39:21Z; submitted_at 2026-05-04T13:39:21Z.
Setting
All Domains pass@1; deepagents harness
Reported by
official leaderboard
Document
[1] CHI-Bench leaderboard (actAVA)
41deepagents + grok-4.3SpaceX AI · All Domains pass@12.2Board publisherMay 2026[1]
42 | deepagents | grok-4.3 | Open-source | 2.2% | 0.0% | 5.3% | 1.3% | 2026-05-01
As printed
2.2%
Where
CHI-Bench v1.0.0, All Domains, rank 42, Agent/Model and Accuracy columns; PA/UM/CM follow; board last updated 2026-08-12; board Date column 2026-05-01; accessed 2026-09-30. Linked submission provenance: run 2026-05-03T22:19:46Z to 2026-05-04T13:19:20Z; submitted_at 2026-05-04T13:19:20Z.
Setting
All Domains pass@1; deepagents harness
Reported by
official leaderboard
Document
[1] CHI-Bench leaderboard (actAVA)
42openclaw + grok-4.3SpaceX AI · All Domains pass@10.4Board publisherMay 2026[1]
43 | openclaw | grok-4.3 | Open-source | 0.4% | 1.3% | 0.0% | 0.0% | 2026-05-01
As printed
0.4%
Where
CHI-Bench v1.0.0, All Domains, rank 43, Agent/Model and Accuracy columns; PA/UM/CM follow; board last updated 2026-08-12; board Date column 2026-05-01; accessed 2026-09-30. Linked submission provenance: run 2026-05-02T09:34:46Z to 2026-05-04T00:03:58Z; submitted_at 2026-05-04T00:03:58Z.
Setting
All Domains pass@1; openclaw harness
Reported by
official leaderboard
Document
[1] CHI-Bench leaderboard (actAVA)
43openai-agents + Nemotron 3 Ultra 256KNVIDIA · All Domains pass@10.0Board publisherJul 2026[1]
44 | openai-agents | Nemotron 3 Ultra 256K | Open-source | 0.0% | 0.0% | 0.0% | 0.0% | 2026-07-24
As printed
0.0%
Where
CHI-Bench v1.0.0, All Domains, rank 44, Agent/Model and Accuracy columns; PA/UM/CM follow; board last updated 2026-08-12; board Date column 2026-07-24; accessed 2026-09-30. Linked submission provenance: run 2026-07-22T22:31:29.302007Z to 2026-07-22T23:37:41.111758Z; submitted_at 2026-08-12T20:20:00Z.
Setting
All Domains pass@1; openai-agents; Tinker 256K
Reported by
official leaderboard
Document
[1] CHI-Bench leaderboard (actAVA)
References

1 document supports this board

  1. 1.CHI-Bench leaderboard (actAVA)official leaderboard, actAVA, 12 Aug 2026. Every result it supports