AutoMedBench Lite Leaderboard
Dotted line = Pareto frontier · ~ = estimate · API pricing via OpenRouter
| # | Model | Date | Overall | Agentic | Task | Cost | Time | Turns |
|---|---|---|---|---|---|---|---|---|
| 1 |  Claude Opus 4.8 |
2026-05-28 | 81.94 | 88.10 | 75.77 | $2.14 | 14.5 min | 18.7 |
| 2 | Claude Opus 4.6 |
2026-02-05 | 77.23 | 89.10 | 65.36 | $6.57 | 18.2 min | 18.3 |
| 3 |  Gemini 3.1 Pro |
2026-02-19 | 75.21 | 83.49 | 66.93 | $3.85 | 18.0 min | 18.1 |
| 4 |  GPT-5.6 Sol |
2026-07-09 | 75.05 | 80.09 | 70.01 | $0.95 | 10.8 min | 11.7 |
| 5 | Claude Sonnet 5 |
2026-06-30 | 75.04 | 80.53 | 69.53 | $2.29 | 15.1 min | 30.5 |
| 6 | GPT-5.5 |
2026-04-23 | 74.14 | 85.38 | 62.90 | $1.90 | 20.5 min | 15.4 |
| 7 |  GLM-5</> |
2026-02-12 | 72.86 | 89.35 | 56.38 | $1.29 | 17.8 min | 31.3 |
| 8 |  DeepSeek V4 Pro</> |
2026-04-24 | 72.43 | 81.34 | 63.51 | $0.30 | 18.3 min | 24.6 |
| 9 |  Qwen3.5-397B</> |
2026-02-15 | 69.87 | 80.46 | 59.27 | $1.01 | 20.8 min | 25.1 |
| 10 | GPT-5.6 Terra |
2026-07-09 | 68.29 | 74.05 | 62.53 | $0.45 | 10.7 min | 10.2 |
| 11 |  MiniMax M2.5</> |
2026-02-12 | 66.31 | 77.71 | 54.91 | $1.36 | 20.6 min | 28.5 |
| 12 | GPT-5.4 |
2026-03-05 | 66.09 | 76.76 | 55.43 | $2.36 | 16.7 min | 10.4 |
| 13 | Qwen3.6-27B</> |
2026-04-21 | 65.29 | 73.27 | 57.30 | free | 23.0 min | 32.0 |
| 14 | GPT-5.6 Luna |
2026-07-09 | 63.98 | 70.77 | 57.20 | $0.21 | 8.9 min | 13.4 |
| 15 | Qwen3.5-35B-A3B</> |
2026-02-24 | 57.94 | 68.91 | 46.97 | free | 28.4 min | 30.5 |
| 16 | GLM-5.2</> |
2026-06-16 | 56.32 | 64.39 | 48.24 | $0.73 | 15.7 min | 28.6 |
| 17 |  Nemotron 3.5 Lightning 30B-A3B</> |
2026-08-11 | 39.65 | 46.93 | 32.38 | $0.09 | 14.3 min | 42.5 |
| 18 |  Gemma 4 26B-A4B</> |
2026-03-11 | 33.89 | 39.63 | 33.97 | $0.05 | 17.3 min | 14.8 |
| 19 | Gemma 4 31B</> |
2026-03-11 | 30.43 | 35.14 | 31.79 | $0.02 | 27.9 min | 8.1 |
| 20 | GPT-OSS-120B</> |
2025-08-05 | 24.33 | 35.35 | 13.31 | $0 | 4.4 min | 9.4 |
| 21 | Gemma 4 12B</> |
2026-05-23 | 22.43 | 20.19 | 36.78 | $0.02 | 24.0 min | 17.1 |
| 22 | Gemma 4 E4B</> |
2026-03-02 | 21.94 | 26.17 | 19.61 | $0.01 | 7.4 min | 6.8 |
| 23 | Qwen3.5-9B</> |
2026-02-27 | 19.87 | 11.33 | 28.41 | free | 19.3 min | 52.4 |
| 24 | Nemotron Nano 9B v2</> |
2025-08-18 | 18.68 | 24.52 | 12.84 | free | 12.0 min | 22.8 |
| 25 | Nemotron 3 Nano 30B-A3B</> |
2025-12-04 | 18.01 | 25.54 | 13.46 | $0.02 | 2.5 min | 16.3 |
| 26 | GPT-OSS-20B</> |
2025-08-05 | 17.63 | 25.33 | 9.93 | $0 | 2.7 min | 5.1 |
| 27 | Nemotron 3 Nano 4B</> |
2026-03-07 | 9.06 | 18.12 | 0.00 | $0.01 | 5.9 min | 9.7 |
| 28 | Gemma 4 E2B</> |
2026-03-02 | 9.02 | 18.04 | 0.00 | $0.02 | 4.3 min | 18.9 |
| 29 |  Mistral 7B Instruct v0.3</> |
2024-05-22 | 6.80 | 13.50 | 0.00 | $0 | 1.5 min | 2.7 |
| 30 | Qwen3.5-4B</> |
2026-02-27 | 6.05 | 12.10 | 0.00 | free | 13.6 min | 72.6 |