AI Benchmark Digest

AI Benchmark Digest — 2026-06-12

2026-06-12T08:17:57.895837+00:00

Daily

New Benchmarks (2)

MathArena - ARXIV_FALSE May (Accuracy (%)): leader GPT-5.5 (xhigh) (50.0), 8 models
MathArena - ARXIV May (Accuracy (%)): leader Claude-Fable-5 (max) (86.67), 8 models

Top-10 New Scores (9)

Claude Fable 5 on Lynchmark: 100.0 (#1)
Claude Fable 5 on MineBench: 1929.84 (#2)
Claude Opus 4.8 on Chess Puzzles (Epoch AI): 34.0 (#12)
Claude Opus 4.8 on Design Arena (Game Dev): 1250.0 (#37)
Claude Opus 4.8 on GRAB-Lite: 60.6 (#6)
Claude Opus 4.8 on OTIS Mock AIME 2024-25: 98.33 (#3)
Claude Opus 4.8 on SimpleQA Verified: 39.5 (#24)
GPT-5.5 on GRAB-Lite: 71.8 (#2)
Qwen 3.7 Max on Position Bias (Lechmazur): 34.8 (#10)

New #1 Leaders (9)

Chatbot Arena (Text-to-Video): gemini-omni-flash (1527.0) beat dreamina-seedance-2.0-720p by 64.0
Design Arena (UI Components): Claude Fable 5 (1411.0) beat Claude Opus 4.7 (Thinking) by 56.0
Design Arena (Game Dev): Claude Fable 5 (1393.0) beat GPT-5.5 by 39.0
Design Arena (SVG): Claude Fable 5 (1384.0) beat prism by 18.0
SEAL - SWE Atlas - Test Writing: Fable-5 (Claude Code) xHigh (58.52) beat Opus 4.8 (Claude Code) by 12.96
MathArena - ARXIV April: Claude 5 (70.73) beat GPT-5.5 (xHigh) by 3.66
GRAB-Lite: Claude Fable 5 (74.0) beat GPT-5.4 by 3.0
WeirdML: Claude 5 (87.85) beat GPT-5.5 (xHigh) by 2.94
Chatbot Arena (Image-to-Video): gemini-omni-flash (1475.0) beat Grok 1.5 by 2.0

AI Benchmark Digest — 2026-06-11

2026-06-11T08:17:01.068404+00:00

Daily

New Benchmarks (1)

GDPval-AA (Elo): leader Claude Fable 5 (Adaptive Reasoning, Max Effort, Opus 4.8 Fallback) (1932.0), 390 models

Top-10 New Scores (3)

Claude Fable 5 on Chatbot Arena (Document): 1495.0 (#5)
Claude Fable 5 on Chatbot Arena (Vision): 1307.0 (#2)
Claude Fable 5 on React Native Evals: 86.96 (#4)

New #1 Leaders (12)

PACT (Lechmazur): Claude Fable 5 (High) (2171.0) beat GPT-5.5 (High) by 155.0
Chatbot Arena (Code): Claude Fable 5 (1665.0) beat Claude Opus 4.7 (Thinking) by 98.0
Design Arena (Data Viz): Claude Fable 5 (1406.0) beat Claude Opus 4.7 (Thinking) by 68.0
Design Arena (Website): Claude Fable 5 (1364.0) beat Claude Opus 4.6 by 23.0
Design Arena (3D): Claude Fable 5 (1383.0) beat Kimi K2.6 by 17.0
FrontierSWE: Claude Fable 5 (90.0) beat Claude Opus 4.8 by 7.0
Chatbot Arena (Text): Claude Fable 5 (1510.0) beat Claude Opus 4.6 (Thinking) by 6.0
SimpleBench: Claude Fable (81.9) beat Gemini 3.1 Pro (Preview) by 2.3
UGI - Writing: Claude 5 (74.23) beat Gemini 3.5 Flash (Thinking, Medium) by 1.69
EQ-Bench Longform Writing: Claude Fable 5 (83.0) beat Claude Opus 4.7 by 1.2
LLM Stats (Video-MME): MiMo-V2.5 (87.7) beat Kimi K2.5 by 0.3
LLM Stats (CMMLU): MiMo-V2.5-Pro (90.2) beat Qwen 2 72B Instruct by 0.1

AI Benchmark Digest — 2026-06-10

2026-06-10T09:55:36.786616+00:00

Daily

New Benchmarks (1)

SkateBench (Success Rate (%)): leader gemini-3.1-pro-preview (96.92), 28 models
Skateboarding-domain knowledge benchmark ranking models by how well they identify technical skateboard tricks from 390 trick definitions. SkateBench v2 reports success rate, cost, and speed.

New Models (1)

Claude Fable 5 — ELO 1871, #31
- Blueprint-Bench 2: 0.386 (#1/14)
- Opper TaskBench: 96.4 (#1/85)
- LLM Stats (OSWorld-Verified): 85.0 (#1/16)
- YC-Bench: 1977.6 (#1/21)
- Vals AI (Vals Index): 75.14 (#1/25)
- Vals AI Multimodal Index: 74.15 (#1/20)
- Vals AI LegalBench: 88.56 (#1/114)
- Vals AI CorpFin v2: 71.83 (#1/111)
- Vals AI MedScribe: 88.52 (#1/62)
- Vals AI ProofBench: 77.0 (#1/37)

New #1 Leaders (55)

YC-Bench: Claude Fable 5 (1977.6) beat Claude Opus 4.7 by 263.1
Multi-turn Debate (Lechmazur): Claude Fable 5 (High) (1770.9) beat Claude Opus 4.7 (High) by 53.8
AA GDPval: Claude Fable 5 (Adaptive Reasoning, Max Effort, Opus 4.8 Fallback) (1932.47) beat Claude Opus 4.8 (Adaptive Reasoning, Max Effort) by 42.67
Evals for Every Language - Language ay: step-3.7-flash-20260528 (77.14) beat Gemini 3.1 Pro (Preview) by 14.23
LiveBench Python: Claude Fable 5 (xHigh) (95.0) beat Claude Opus 4.5 (Thinking 64K, High) (2025-11-01) by 10.0
CursorBench 3.1: Fable 5 Max (72.9) beat Claude Opus 4.7 by 8.1
AA Omniscience - Software Engineering (SWE) - Dart: Claude Fable 5 (Adaptive Reasoning, Max Effort, Opus 4.8 Fallback) (88.0) beat GPT-5.3 Codex (xHigh) by 8.0
AA Omniscience - Software Engineering (SWE) - R: Claude Fable 5 (Adaptive Reasoning, Max Effort, Opus 4.8 Fallback) (82.0) beat GPT-5.5 (Medium) by 8.0
AA Omniscience - Software Engineering (SWE) - Swift: Claude Fable 5 (Adaptive Reasoning, Max Effort, Opus 4.8 Fallback) (100.0) beat GPT-5.5 (xHigh) by 8.0
Vals AI Vibe Code Bench: Claude Fable 5 (90.35) beat Claude Opus 4.8 by 7.63
AA Humanity's Last Exam: Claude Fable 5 (Adaptive Reasoning, Max Effort, Opus 4.8 Fallback) (53.34) beat Claude Opus 4.8 (Adaptive Reasoning, Max Effort) by 7.6
AA Omniscience: Claude Fable 5 (Adaptive Reasoning, Max Effort, Opus 4.8 Fallback) (40.15) beat Gemini 3.1 Pro (Preview) by 7.22
Vellum - HumanEval: Claude Mythos 5 (95.5) beat Claude Opus 4.8 by 6.9
Vellum - Humanity's Last Exam: Claude Mythos 5 (64.5) beat Claude Opus 4.8 by 6.6
Evals for Every Language - Language crh: step-3.7-flash-20260528 (73.05) beat Gemini 3.1 Pro (Preview) by 6.27
AA Omniscience - Software Engineering (SWE) - Java: Claude Fable 5 (Adaptive Reasoning, Max Effort, Opus 4.8 Fallback) (79.0) beat GPT-5.3 Codex (xHigh) by 6.0
Vals AI ProofBench: Claude Fable 5 (77.0) beat aristotle by 6.0
AA Omniscience - Business: Claude Fable 5 (Adaptive Reasoning, Max Effort, Opus 4.8 Fallback) (55.0) beat GPT-5.5 (xHigh) by 5.9
AA Omniscience - Science, Engineering & Mathematics: Claude Fable 5 (Adaptive Reasoning, Max Effort, Opus 4.8 Fallback) (57.1) beat GPT-5.5 (High) by 4.8
Vals AI (Vals Index): Claude Fable 5 (75.14) beat Claude Opus 4.8 by 4.78
Vals AI IOI: Claude Fable 5 (72.25) beat GPT-5.4 (2026-03-05) by 4.42
AA Omniscience - Humanities & Social Sciences: Claude Fable 5 (Adaptive Reasoning, Max Effort, Opus 4.8 Fallback) (60.9) beat Gemini 3 Pro (Preview) (High) by 4.3
AA Omniscience - Software Engineering (SWE) - Go: Claude Fable 5 (Adaptive Reasoning, Max Effort, Opus 4.8 Fallback) (88.0) beat GPT-5.5 (High) by 4.0
Artificial Analysis Intelligence Index: Claude Fable 5 (Adaptive Reasoning, Max Effort, Opus 4.8 Fallback) (64.88) beat Claude Opus 4.8 (Adaptive Reasoning, Max Effort) by 3.44
Evals for Every Language - Language cv: gemma-4-31B-it-20260402 (69.3) beat Claude Opus 4.5 by 3.39
Vals AI CorpFin v2: Claude Fable 5 (71.83) beat Grok 4.3 by 3.3
Vals AI Multimodal Index: Claude Fable 5 (74.15) beat Claude Opus 4.8 by 3.26
AA Omniscience - Software Engineering (SWE): Claude Fable 5 (Adaptive Reasoning, Max Effort, Opus 4.8 Fallback) (87.6) beat GPT-5.5 (xHigh) by 3.2
Evals for Every Language - MGSM: Claude Opus 4.8 (96.62) beat Claude Opus 4.6 by 2.36
Evals for Every Language - Language ban: step-3.7-flash-20260528 (69.03) beat Claude Opus 4.5 by 2.32
AA Terminal-Bench Hard: Claude Fable 5 (Adaptive Reasoning, Max Effort, Opus 4.8 Fallback) (62.88) beat GPT-5.5 (xHigh) by 2.27
LiveBench Plot Unscrambling: Claude Fable 5 (xHigh) (78.09) beat GPT-5.5 (High) by 1.81
LLM Stats (OSWorld-Verified): Claude Fable 5 (85.0) beat Claude Opus 4.8 by 1.6
AA Omniscience - Software Engineering (SWE) - Python: Claude Fable 5 (Adaptive Reasoning, Max Effort, Opus 4.8 Fallback) (92.0) beat GPT-5.5 (xHigh) by 1.5
Evals for Every Language - Language chm: Claude Opus 4.7 (63.6) beat Gemini 3.1 Pro (Preview) by 1.48
Evals for Every Language - Language doi: Claude Opus 4.7 (71.84) beat Gemini 3 Pro (Preview) by 1.46
AA CritPt: Claude Fable 5 (Adaptive Reasoning, Max Effort, Opus 4.8 Fallback) (28.57) beat GPT-5.5 (xHigh) by 1.43
Evals for Every Language - Language es: Gemini 3.1 Flash Lite (76.16) beat Claude Opus 4.6 by 1.42
AA SciCode: Claude Fable 5 (Adaptive Reasoning, Max Effort, Opus 4.8 Fallback) (60.19) beat Gemini 3.1 Pro (Preview) by 1.28
Evals for Every Language - Language ace: step-3.7-flash-20260528 (72.48) beat Gemini 3.1 Pro (Preview) by 1.28
Evals for Every Language - MMLU: intellect-3-20251126 (100.0) beat Claude Sonnet 4.6 by 1.27
Evals for Every Language - ARC: intellect-3-20251126 (100.0) beat Gemini 3.1 Pro (Preview) by 1.26
Vals AI LegalBench: Claude Fable 5 (88.56) beat Gemini 3.1 Pro (Preview) by 1.16
Evals for Every Language - Language ca: Gemini 3.1 Flash Lite (76.29) beat Gemini 3 Pro (Preview) by 1.03
Opper TaskBench: Claude Fable 5 (96.4) beat Claude Opus 4.7 by 1.0
Evals for Every Language - Language ar: Claude Opus 4.8 (71.58) beat Claude Opus 4.5 by 0.95
Evals for Every Language - Language en: Gemini 3.1 Flash Lite (87.28) beat MiniMax-M2.5 by 0.77
Evals for Every Language - Language cy: Gemini 3.1 Flash Lite (82.03) beat Claude Sonnet 4.5 by 0.65
Evals for Every Language - Language am: Gemini 3.1 Flash Lite (68.6) beat Claude Opus 4.6 by 0.59
Vals AI MedScribe: Claude Fable 5 (88.52) beat GPT-5.1 by 0.43
Evals for Every Language - Language be: Claude Opus 4.8 (69.43) beat Gemini 3.1 Pro (Preview) by 0.32
Evals for Every Language - Language ceb: Gemini 3.1 Flash Lite (78.06) beat Gemini 3.1 Pro (Preview) by 0.29
Evals for Every Language - Language el: Gemini 3.1 Flash Lite (73.81) beat Claude Opus 4.5 by 0.15
Blueprint-Bench 2: Claude Fable 5 (0.386) beat GPT-5.5 by 0.02
LiveBench Olympiad: Claude Fable 5 (High) (92.18) beat Claude Opus 4.6 (Thinking, High) by 0.01

AI Benchmark Digest — 2026-06-10

2026-06-10T08:06:50.673963+00:00

Daily

New Benchmarks (1)

SkateBench (Success Rate (%)): leader gemini-3.1-pro-preview (96.92), 28 models
Skateboarding-domain knowledge benchmark ranking models by how well they identify technical skateboard tricks from 390 trick definitions. SkateBench v2 reports success rate, cost, and speed.

New Models (2)

Claude 5 — ELO 1904, #22
- LiveBench Olympiad: 92.18 (#1/124)
- LiveBench Plot Unscrambling: 78.09 (#1/124)
- LiveBench Python: 95.0 (#1/124)
- Opper TaskBench: 96.4 (#1/85)
- Vals AI (Vals Index): 75.14 (#1/25)
- Vals AI Multimodal Index: 74.15 (#1/20)
- Vals AI LegalBench: 88.56 (#1/114)
- Vals AI CorpFin v2: 71.83 (#1/111)
- Vals AI MedScribe: 88.52 (#1/62)
- Vals AI ProofBench: 77.0 (#1/37)
Claude Fable 5 — ELO 1901, #23
- Blueprint-Bench 2: 0.386 (#1/14)
- LLM Stats (OSWorld-Verified): 85.0 (#1/16)
- YC-Bench: 1977.6 (#1/21)
- SEAL - MCP Atlas: 83.3 (#2/23)
- Vellum - HumanEval: 95.0 (#2/38)
- Vellum - GPQA: 94.1 (#3/57)
- ClockBench: 35.0 (#4/27)
- LLM Stats (GDPval-AA): 64.4 (#11/12)

New #1 Leaders (55)

YC-Bench: Claude Fable 5 (1977.6) beat Claude Opus 4.7 by 263.1
Multi-turn Debate (Lechmazur): Claude Fable 5 (High) (1770.9) beat Claude Opus 4.7 (High) by 53.8
AA GDPval: Claude Fable 5 (Adaptive Reasoning, Max Effort, Opus 4.8 Fallback) (1932.47) beat Claude Opus 4.8 (Adaptive Reasoning, Max Effort) by 42.67
Evals for Every Language - Language ay: step-3.7-flash-20260528 (77.14) beat Gemini 3.1 Pro (Preview) by 14.23
LiveBench Python: Claude 5 (95.0) beat Claude Opus 4.5 (Thinking 64K, High) (2025-11-01) by 10.0
CursorBench 3.1: Fable 5 Max (72.9) beat Claude Opus 4.7 by 8.1
AA Omniscience - Software Engineering (SWE) - Dart: Claude Fable 5 (Adaptive Reasoning, Max Effort, Opus 4.8 Fallback) (88.0) beat GPT-5.3 Codex (xHigh) by 8.0
AA Omniscience - Software Engineering (SWE) - R: Claude Fable 5 (Adaptive Reasoning, Max Effort, Opus 4.8 Fallback) (82.0) beat GPT-5.5 (Medium) by 8.0
AA Omniscience - Software Engineering (SWE) - Swift: Claude Fable 5 (Adaptive Reasoning, Max Effort, Opus 4.8 Fallback) (100.0) beat GPT-5.5 (xHigh) by 8.0
Vals AI Vibe Code Bench: Claude 5 (90.35) beat Claude Opus 4.8 by 7.63
AA Humanity's Last Exam: Claude Fable 5 (Adaptive Reasoning, Max Effort, Opus 4.8 Fallback) (53.34) beat Claude Opus 4.8 (Adaptive Reasoning, Max Effort) by 7.6
AA Omniscience: Claude Fable 5 (Adaptive Reasoning, Max Effort, Opus 4.8 Fallback) (40.15) beat Gemini 3.1 Pro (Preview) by 7.22
Vellum - HumanEval: Claude Mythos 5 (95.5) beat Claude Opus 4.8 by 6.9
Vellum - Humanity's Last Exam: Claude Mythos 5 (64.5) beat Claude Opus 4.8 by 6.6
Evals for Every Language - Language crh: step-3.7-flash-20260528 (73.05) beat Gemini 3.1 Pro (Preview) by 6.27
AA Omniscience - Software Engineering (SWE) - Java: Claude Fable 5 (Adaptive Reasoning, Max Effort, Opus 4.8 Fallback) (79.0) beat GPT-5.3 Codex (xHigh) by 6.0
Vals AI ProofBench: Claude 5 (77.0) beat aristotle by 6.0
AA Omniscience - Business: Claude Fable 5 (Adaptive Reasoning, Max Effort, Opus 4.8 Fallback) (55.0) beat GPT-5.5 (xHigh) by 5.9
AA Omniscience - Science, Engineering & Mathematics: Claude Fable 5 (Adaptive Reasoning, Max Effort, Opus 4.8 Fallback) (57.1) beat GPT-5.5 (High) by 4.8
Vals AI (Vals Index): Claude 5 (75.14) beat Claude Opus 4.8 by 4.78
Vals AI IOI: Claude 5 (72.25) beat GPT-5.4 (2026-03-05) by 4.42
AA Omniscience - Humanities & Social Sciences: Claude Fable 5 (Adaptive Reasoning, Max Effort, Opus 4.8 Fallback) (60.9) beat Gemini 3 Pro (Preview) (High) by 4.3
AA Omniscience - Software Engineering (SWE) - Go: Claude Fable 5 (Adaptive Reasoning, Max Effort, Opus 4.8 Fallback) (88.0) beat GPT-5.5 (High) by 4.0
Artificial Analysis Intelligence Index: Claude Fable 5 (Adaptive Reasoning, Max Effort, Opus 4.8 Fallback) (64.88) beat Claude Opus 4.8 (Adaptive Reasoning, Max Effort) by 3.44
Evals for Every Language - Language cv: gemma-4-31B-it-20260402 (69.3) beat Claude Opus 4.5 by 3.39
Vals AI CorpFin v2: Claude 5 (71.83) beat Grok 4.3 by 3.3
Vals AI Multimodal Index: Claude 5 (74.15) beat Claude Opus 4.8 by 3.26
AA Omniscience - Software Engineering (SWE): Claude Fable 5 (Adaptive Reasoning, Max Effort, Opus 4.8 Fallback) (87.6) beat GPT-5.5 (xHigh) by 3.2
Evals for Every Language - MGSM: Claude Opus 4.8 (96.62) beat Claude Opus 4.6 by 2.36
Evals for Every Language - Language ban: step-3.7-flash-20260528 (69.03) beat Claude Opus 4.5 by 2.32
AA Terminal-Bench Hard: Claude Fable 5 (Adaptive Reasoning, Max Effort, Opus 4.8 Fallback) (62.88) beat GPT-5.5 (xHigh) by 2.27
LiveBench Plot Unscrambling: Claude 5 (78.09) beat GPT-5.5 (High) by 1.81
LLM Stats (OSWorld-Verified): Claude Fable 5 (85.0) beat Claude Opus 4.8 by 1.6
AA Omniscience - Software Engineering (SWE) - Python: Claude Fable 5 (Adaptive Reasoning, Max Effort, Opus 4.8 Fallback) (92.0) beat GPT-5.5 (xHigh) by 1.5
Evals for Every Language - Language chm: Claude Opus 4.7 (63.6) beat Gemini 3.1 Pro (Preview) by 1.48
Evals for Every Language - Language doi: Claude Opus 4.7 (71.84) beat Gemini 3 Pro (Preview) by 1.46
AA CritPt: Claude Fable 5 (Adaptive Reasoning, Max Effort, Opus 4.8 Fallback) (28.57) beat GPT-5.5 (xHigh) by 1.43
Evals for Every Language - Language es: Gemini 3.1 Flash Lite (76.16) beat Claude Opus 4.6 by 1.42
AA SciCode: Claude Fable 5 (Adaptive Reasoning, Max Effort, Opus 4.8 Fallback) (60.19) beat Gemini 3.1 Pro (Preview) by 1.28
Evals for Every Language - Language ace: step-3.7-flash-20260528 (72.48) beat Gemini 3.1 Pro (Preview) by 1.28
Evals for Every Language - MMLU: intellect-3-20251126 (100.0) beat Claude Sonnet 4.6 by 1.27
Evals for Every Language - ARC: intellect-3-20251126 (100.0) beat Gemini 3.1 Pro (Preview) by 1.26
Vals AI LegalBench: Claude 5 (88.56) beat Gemini 3.1 Pro (Preview) by 1.16
Evals for Every Language - Language ca: Gemini 3.1 Flash Lite (76.29) beat Gemini 3 Pro (Preview) by 1.03
Opper TaskBench: Claude 5 (96.4) beat Claude Opus 4.7 by 1.0
Evals for Every Language - Language ar: Claude Opus 4.8 (71.58) beat Claude Opus 4.5 by 0.95
Evals for Every Language - Language en: Gemini 3.1 Flash Lite (87.28) beat MiniMax-M2.5 by 0.77
Evals for Every Language - Language cy: Gemini 3.1 Flash Lite (82.03) beat Claude Sonnet 4.5 by 0.65
Evals for Every Language - Language am: Gemini 3.1 Flash Lite (68.6) beat Claude Opus 4.6 by 0.59
Vals AI MedScribe: Claude 5 (88.52) beat GPT-5.1 by 0.43
Evals for Every Language - Language be: Claude Opus 4.8 (69.43) beat Gemini 3.1 Pro (Preview) by 0.32
Evals for Every Language - Language ceb: Gemini 3.1 Flash Lite (78.06) beat Gemini 3.1 Pro (Preview) by 0.29
Evals for Every Language - Language el: Gemini 3.1 Flash Lite (73.81) beat Claude Opus 4.5 by 0.15
Blueprint-Bench 2: Claude Fable 5 (0.386) beat GPT-5.5 by 0.02
LiveBench Olympiad: Claude 5 (92.18) beat Claude Opus 4.6 (Thinking) (High) by 0.01

AI Benchmark Digest — 2026-06-09

2026-06-09T07:53:25.528997+00:00

Daily

Top-10 New Scores (2)

GPT-5.5 (xHigh) on SEAL - SWE Atlas - Codebase QnA: 45.43 (#2)
GPT-5.5 (xHigh) on SEAL - SWE Atlas - Test Writing: 42.59 (#3)

New #1 Leaders (7)

GSMA Open-Telco - TeleTables: TelecomGPT (88.0) beat OTel-LLM-8.3B-QnA by 26.2
GSMA Open-Telco LLM Leaderboard: TelecomGPT (89.64) beat OTel-LLM-8.3B-QnA by 3.66
SEAL - SWE Atlas - Codebase QnA: Opus 4.8 (Claude Code) (48.79) beat GPT-5.5 by 3.36
GSMA Open-Telco - 3GPP: TelecomGPT (84.22) beat OTel-LLM-8.3B-QnA by 2.82
GSMA Open-Telco - TeleLogs: TelecomGPT (98.96) beat OTel-LLM-8.3B-QnA by 2.66
GSMA Open-Telco - srsRAN-Bench: TelecomGPT (91.33) beat OTel-LLM-8.3B-QnA by 1.65
SEAL - SWE Atlas - Test Writing: Opus 4.8 (Claude Code) (45.56) beat GPT-5.4 (xHigh) by 1.2

AI Benchmark Digest — 2026-06-07

2026-06-07T08:34:58.487719+00:00

Weekly

New Models (2)

MiniMax-M3 — ELO 1762, #83
- LLM Stats (OmniDocBench 1.5): 91.6 (#1/13)
- LLM Stats (Video-MME): 85.4 (#2/13)
- OpenClawProBench: 75.1 (#2/65)
- Vals AI MedScribe: 87.25 (#2/61)
- AA IFBench: 82.86 (#3/429)
- LLM Stats (Claw-Eval): 74.5 (#3/9)
- LLM Stats (NL2Repo): 42.13 (#3/7)
- AA GPQA Diamond: 92.93 (#4/501)
- Vals AI CorpFin v2: 68.1 (#4/110)
- Design Arena (3D): 1348.0 (#5/115)
nemotron-3-ultra-550B-a55B — ELO 1587, #292
- PinchBench: 90.58 (#10/49)
- Vals AI CorpFin v2: 65.46 (#16/110)
- Vals AI (Vals Index): 43.99 (#18/24)
- LiveBench Python: 75.0 (#24/122)
- LiveBench Paraphrase: 61.15 (#33/122)
- Vals AI TaxEval v2: 73.1 (#34/116)
- Bullshit Benchmark: 41.8 (#34/148)
- Vals AI MedCode: 38.62 (#35/62)
- AI Chess Leaderboard (Reasoning): 975.0 (#39/277)
- LiveBench Code Generation: 77.47 (#43/122)

Top-10 New Scores (4)

GPT-5.5 (xHigh) on IMO-Bench: 71.9 (#4)
GPT-5.5 Pro on IUMB: 100.0 (#2)
GPT-5.5 Pro (xHigh) on IMO-Bench: 88.1 (#2)
Gemini 3 Deep Think on IUMB: 87.5 (#6)

New #1 Leaders (10)

EQ-Bench Creative Writing v3: Claude Opus 4.7 (2050.8) beat GPT-5.4 by 144.8
Chatbot Arena (Image-to-Video): Grok 1.5 (1473.0) beat dreamina-seedance-2.0-720p by 11.0
LLM Stats (Multi-Challenge): Nova 2 Pro (77.7) beat GPT-5 by 8.1
MathArena - Kangaroo 2025 Levels 11-12: Claude Opus 4.8 (Thinking) (100.0) beat GPT-5.4 (xHigh) by 1.67
MathArena - APEX 2025: Claude Opus 4.8 (Thinking) (81.25) beat GPT-5.5 (xHigh) by 1.04
MathArena - Kangaroo 2025 Levels 7-8: Claude Opus 4.8 (Thinking) (96.67) beat GPT-5.4 (xHigh) by 0.84
MathArena - AIME 2026: Claude Opus 4.8 (Thinking) (100.0) beat GPT-5.4 (xHigh) by 0.83
LLM Stats (OmniDocBench 1.5): MiniMax-M3 (91.6) beat Qwen 3.6 Plus by 0.4
GAIA: CustomGPT.ai Research Lab v44 (93.36) beat Co-Sight Pro v1.0.1 by 0.34
ForecastBench: Grok 4.20 (Beta, D) (68.1) beat green-tree by 0.2

AI Benchmark Digest — 2026-06-06

2026-06-06T07:45:06.870709+00:00

Daily

New Benchmarks (20)

Pencil Puzzle Bench - Yajilin (Direct-ask Success Rate (%)): leader gpt-5.2 (High) (20.0), 51 models
PPBench direct-ask success rate on Yajilin loop-and-shading puzzles from the golden_300 split, testing exact constraint solving from puzz.link grids.
Pencil Puzzle Bench - Slitherlink (Direct-ask Success Rate (%)): leader gpt-5.2 (High) (33.3), 51 models
PPBench direct-ask success rate on Slitherlink loop puzzles, where numbered cells constrain how a single continuous loop surrounds the grid.
Pencil Puzzle Bench - Heyawake (Direct-ask Success Rate (%)): leader claude-opus-4-5-high (0.0), 51 models
PPBench direct-ask success rate on Heyawake room-shading puzzles, testing region constraints, connectivity, and line-of-sight reasoning.
Pencil Puzzle Bench - Mashu (Direct-ask Success Rate (%)): leader gpt-5.2 (xHigh) (60.0), 51 models
PPBench direct-ask success rate on Mashu loop puzzles, where black and white pearls impose turn and straight-line constraints.
Pencil Puzzle Bench - Shakashaka (Direct-ask Success Rate (%)): leader claude-sonnet-4-5 (0.0), 51 models
PPBench direct-ask success rate on Shakashaka triangle-shading puzzles, testing local clue satisfaction and global rectangle formation.
Pencil Puzzle Bench - Nurikabe (Direct-ask Success Rate (%)): leader claude-opus-4-6 (Thinking) (33.3), 51 models
PPBench direct-ask success rate on Nurikabe island puzzles, where numbered islands must be separated by one connected wall region.
Pencil Puzzle Bench - LITS (Direct-ask Success Rate (%)): leader gpt-5.2 (xHigh) (53.3), 51 models
PPBench direct-ask success rate on LITS tetromino-shading puzzles, testing region-wise shape placement and adjacency constraints.
Pencil Puzzle Bench - Light Up (Direct-ask Success Rate (%)): leader claude-opus-4-6 (Thinking) (66.7), 51 models
PPBench direct-ask success rate on Light Up puzzles, where lamps must illuminate every open cell while satisfying numbered black-cell clues.
Pencil Puzzle Bench - Nurimisaki (Direct-ask Success Rate (%)): leader claude-opus-4-6 (Thinking) (33.3), 51 models
PPBench direct-ask success rate on Nurimisaki puzzles, a Nurikabe-family grid task requiring connected-region reasoning around clue cells.
Pencil Puzzle Bench - Shikaku (Direct-ask Success Rate (%)): leader gpt-5.2 (xHigh) (80.0), 51 models
PPBench direct-ask success rate on Shikaku rectangle-partitioning puzzles, where each numbered clue defines one rectangle of matching area.
Pencil Puzzle Bench - Norinori (Direct-ask Success Rate (%)): leader gpt-5.2 (xHigh) (93.3), 51 models
PPBench direct-ask success rate on Norinori shading puzzles, testing room constraints and two-cell adjacency patterns.
Pencil Puzzle Bench - Double Choco (Direct-ask Success Rate (%)): leader gemini-3.1-pro (6.7), 51 models
PPBench direct-ask success rate on Double Choco region-division puzzles, testing balanced partitioning under color and shape constraints.
Pencil Puzzle Bench - Firefly (Direct-ask Success Rate (%)): leader gpt-5.2 (xHigh) (33.3), 51 models
PPBench direct-ask success rate on Firefly line-drawing puzzles, testing path construction from directional clues and grid constraints.
Pencil Puzzle Bench - Sashigane (Direct-ask Success Rate (%)): leader mistral-large-2512 (0.0), 51 models
PPBench direct-ask success rate on Sashigane shape-partitioning puzzles, testing right-angle region construction from numbered and directional clues.
Pencil Puzzle Bench - Sudoku (Direct-ask Success Rate (%)): leader gpt-5.2 (xHigh) (20.0), 51 models
PPBench direct-ask success rate on Sudoku puzzles, testing classic row, column, and box constraint satisfaction through exact move outputs.
Pencil Puzzle Bench - Nurimaze (Direct-ask Success Rate (%)): leader claude-opus-4-6 (Thinking) (26.7), 51 models
PPBench direct-ask success rate on Nurimaze puzzles, testing maze-style path and shading constraints in a connected grid.
Pencil Puzzle Bench - Tapa (Direct-ask Success Rate (%)): leader gpt-5.2 (xHigh) (60.0), 51 models
PPBench direct-ask success rate on Tapa shading puzzles, where clue numbers describe blocks of shaded neighboring cells.
Pencil Puzzle Bench - Kurodoko (Direct-ask Success Rate (%)): leader gpt-5.2 (xHigh) (6.7), 51 models
PPBench direct-ask success rate on Kurodoko visibility puzzles, testing shading, sight-line counts, and connected unshaded cells.
Pencil Puzzle Bench - Country (Direct-ask Success Rate (%)): leader gemini-3.1-pro (6.7), 51 models
PPBench direct-ask success rate on Country region puzzles, testing loop and region constraints over a partitioned grid.
Pencil Puzzle Bench - Hitori (Direct-ask Success Rate (%)): leader claude-opus-4-6 (Thinking) (66.7), 51 models
PPBench direct-ask success rate on Hitori number-grid puzzles, where repeated numbers are shaded while preserving connectivity and non-adjacency constraints.

New #1 Leaders (24)

LLM Stats (Multi-Challenge): Nova 2 Pro (77.7) beat GPT-5 by 8.1
Ukrainian LLM - Global MMLU Full UK World Religions: MamayLM-Gemma-3-27B-IT-v2.0 (87.13) beat gemma-3-12B-pt by 7.6
Ukrainian LLM - Global MMLU Full UK High School US History: MamayLM-Gemma-3-27B-IT-v2.0 (91.67) beat MamayLM-Gemma-3-12B-IT-v1.0 by 5.4
Ukrainian LLM - Global MMLU Full UK Anatomy: MamayLM-Gemma-3-27B-IT-v2.0 (65.19) beat lapa-12B-pt by 5.19
Ukrainian LLM - Global MMLU Full UK Clinical Knowledge: MamayLM-Gemma-3-27B-IT-v2.0 (77.74) beat gemma-3-12B-pt by 4.53
Ukrainian LLM - Global MMLU Full UK Professional LAW: MamayLM-Gemma-3-27B-IT-v2.0 (51.5) beat gemma-3-12B-pt by 4.43
Ukrainian LLM - Global MMLU Full UK Humanities: MamayLM-Gemma-3-27B-IT-v2.0 (61.68) beat Qwen3-8B-Base by 4.12
Ukrainian LLM - Global MMLU Full UK Computer Security: MamayLM-Gemma-3-12B-IT-v2.0 (82.0) beat MamayLM-Gemma-3-12B-IT-v1.0-FP8-Static-Nadiia by 4.0
Ukrainian LLM - Global MMLU Full UK Global Facts: MamayLM-Gemma-3-27B-IT-v2.0 (52.0) beat Gemma 3 12B (IT) by 4.0
Ukrainian LLM - Global MMLU Full UK Miscellaneous: MamayLM-Gemma-3-27B-IT-v2.0 (83.52) beat MamayLM-Gemma-3-12B-IT-v1.0-FP8-Static-Nadiia by 3.95
Ukrainian LLM - Global MMLU Full UK Prehistory: MamayLM-Gemma-3-27B-IT-v2.0 (77.78) beat gemma-3-12B-pt by 3.71
Ukrainian LLM - Global MMLU Full UK Other: MamayLM-Gemma-3-27B-IT-v2.0 (74.57) beat gemma-3-12B-pt by 3.41
Ukrainian LLM - Global MMLU Full UK Business Ethics: MamayLM-Gemma-3-12B-IT-v2.0 (77.0) beat MamayLM-Gemma-3-12B-IT-v1.0 by 3.0
Ukrainian LLM - Global MMLU Full UK High School World History: MamayLM-Gemma-3-27B-IT-v2.0 (86.08) beat gemma-3-12B-pt by 1.69
Ukrainian LLM - Global MMLU Full UK High School Microeconomics: MamayLM-Gemma-3-27B-IT-v2.0 (84.45) beat Qwen3-8B-Base by 1.68
Ukrainian LLM - Global MMLU Full UK Marketing: MamayLM-Gemma-3-27B-IT-v2.0 (88.89) beat MamayLM-Gemma-3-12B-IT-v1.0 by 1.28
Ukrainian LLM - Global MMLU Full UK Professional Psychology: MamayLM-Gemma-3-27B-IT-v2.0 (70.1) beat gemma-3-12B-pt by 0.98
Ukrainian LLM - Global MMLU Full UK Public Relations: MamayLM-Gemma-3-12B-IT-v2.0 (68.18) beat lapa-12B-pt by 0.91
Ukrainian LLM - Global MMLU Full UK High School European History: MamayLM-Gemma-3-27B-IT-v2.0 (84.24) beat MamayLM-Gemma-3-12B-IT-v1.0-FP8-Static-Nadiia by 0.6
Ukrainian LLM - Global MMLU Full UK High School Macroeconomics: MamayLM-Gemma-3-27B-IT-v2.0 (76.67) beat gemma-3-12B-pt by 0.52
Ukrainian LLM - Global MMLU Full UK Sociology: MamayLM-Gemma-3-27B-IT-v2.0 (83.08) beat lapa-v0.1.2-instruct by 0.49
LLM Stats (OmniDocBench 1.5): MiniMax-M3 (91.6) beat Qwen 3.6 Plus by 0.4
Ukrainian LLM - Global MMLU Full UK Professional Medicine: MamayLM-Gemma-3-27B-IT-v2.0 (80.15) beat gemma-3-12B-pt by 0.37
ForecastBench: Grok 4.20 (Beta, D) (68.1) beat green-tree by 0.3

AI Benchmark Digest — 2026-06-04

2026-06-04T08:22:19.073162+00:00

Daily

New #1 Leaders (1)

GAIA: CustomGPT.ai Research Lab v44 (93.36) beat Co-Sight Pro v1.0.1 by 0.34

AI Benchmark Digest — 2026-06-03

2026-06-03T08:25:40.519214+00:00

Daily

Top-10 New Scores (2)

GPT-5.5 Pro on IUMB: 100.0 (#2)
Gemini 3 Deep Think on IUMB: 87.5 (#6)

New #1 Leaders (4)

MathArena - Kangaroo 2025 Levels 11-12: Claude Opus 4.8 (Thinking) (100.0) beat GPT-5.4 (xHigh) by 1.67
MathArena - APEX 2025: Claude Opus 4.8 (Thinking) (81.25) beat GPT-5.5 (xHigh) by 1.04
MathArena - Kangaroo 2025 Levels 7-8: Claude Opus 4.8 (Thinking) (96.67) beat GPT-5.4 (xHigh) by 0.84
MathArena - AIME 2026: Claude Opus 4.8 (Thinking) (100.0) beat GPT-5.4 (xHigh) by 0.83

AI Benchmark Digest — 2026-06-02

2026-06-02T08:19:29.198019+00:00

Daily

New Benchmarks (1)

GIM (IRT ability (theta)): leader GPT-5.4 Pro (High) (2.16), 46 models
Grounded Integration Measure from Meta FAIR: 820 multimodal and text-grounded problems testing integrated reasoning across quantitative, spatial, language, world-knowledge, and document tasks. Scores are reported as IRT ability on GIM-820.

Top-10 New Scores (2)

GPT-5.5 (xHigh) on IMO-Bench: 71.9 (#4)
GPT-5.5 Pro (xHigh) on IMO-Bench: 88.1 (#2)

AI Benchmark Digest — 2026-06-01

2026-06-01T08:29:45.265204+00:00

Daily

New #1 Leaders (3)

EQ-Bench Creative Writing v3: Claude Opus 4.7 (2050.8) beat GPT-5.4 by 144.8
Design Arena (Data Viz): GLM-5.1 (1367.0) beat Claude Opus 4.7 (Thinking) by 23.0
Chatbot Arena (Image-to-Video): Grok 1.5 (1473.0) beat dreamina-seedance-2.0-720p by 11.0

AI Benchmark Digest — 2026-05-30

2026-05-30T07:49:09.779753+00:00

Daily

Top-10 New Scores (5)

Claude Opus 4.8 (Adaptive Reasoning, Max Effort) on UGI - Natural Intelligence: 65.39 (#30)
Claude Opus 4.8 (Adaptive Reasoning, Max Effort) on UGI - Willingness (W/10): 2.2 (#1094)
Claude Opus 4.8 (Adaptive Reasoning, Max Effort) on UGI - Writing: 65.88 (#34)
Claude Opus 4.8 (Adaptive Reasoning, Max Effort) on UGI Leaderboard: 52.64 (#69)
GPT-5.4 (xHigh) on Creative Writing (Lechmazur): 3.4 (#2)

New #1 Leaders (2)

Bullshit Benchmark: Claude Opus 4.8 (96.4) beat Claude Sonnet 4.6 by 1.9
Creative Writing (Lechmazur): GPT-5.5 (xHigh) (3.5) beat GPT-5.5 (Thinking, xHigh) by 0.3

AI Benchmark Digest — 2026-05-29

2026-05-29T08:06:41.324282+00:00

Daily

New Benchmarks (1)

DeepSWE (Pass@1 (%)): leader GPT-5.5 (xHigh) (70.0), 12 models
DataCurve benchmark measuring frontier coding agents on original, long-horizon software engineering tasks. Reports pass rates for model configurations on realistic repository work.

New Models (1)

Claude Opus 4.8 — ELO 1801, #52
- Clerk LLM Leaderboard: 91.3 (#1/19)
- Vellum - HumanEval: 88.6 (#1/36)
- Vellum - Humanity's Last Exam: 57.9 (#1/20)
- LLM Stats (DeepSearchQA): 93.1 (#1/6)
- LLM Stats (Include): 87.6 (#1/30)
- LLM Stats (OSWorld-Verified): 83.4 (#1/14)
- LLM Stats (ScreenSpot Pro): 87.9 (#1/22)
- LLM Stats (Toolathlon): 59.9 (#1/20)
- FrontierSWE: 83.0 (#1/11)
- Vals AI (Vals Index): 70.17 (#1/20)

Top-10 New Scores (2)

GPT-5.5 (High) on WebDev Arena: 1478.93 (#16)
GPT-5.5 (xHigh) on WebDev Arena: 1504.74 (#12)

New #1 Leaders (16)

AA GDPval: Claude Opus 4.8 (Adaptive Reasoning, Max Effort) (1889.8) beat GPT-5.5 (xHigh) by 120.5
Vellum - Humanity's Last Exam: Claude Opus 4.8 (57.9) beat Gemini 3 Pro by 12.1
Clerk LLM Leaderboard: Claude Opus 4.8 (91.3) beat GPT-5.4 by 11.8
Vals AI Vibe Code Bench: Claude Opus 4.8 (82.72) beat Claude Opus 4.7 by 11.72
Epoch AI - Apex Agents: gemini-3.5-flash_unknown (49.6) beat GPT-5.5 (xHigh) by 11.2
LLM Stats (OSWorld-Verified): Claude Opus 4.8 (83.4) beat Claude Mythos Preview by 3.8
LLM Stats (Toolathlon): Claude Opus 4.8 (59.9) beat Gemini 3.5 Flash by 3.4
Vals AI Multimodal Index: Claude Opus 4.8 (70.71) beat GPT-5.5 by 2.94
Vals AI (Vals Index): Claude Opus 4.8 (70.17) beat GPT-5.5 by 2.55
LLM Stats (DeepSearchQA): Claude Opus 4.8 (93.1) beat Claude Opus 4.6 by 1.8
LLM Stats (ScreenSpot Pro): Claude Opus 4.8 (87.9) beat GPT-5.2 by 1.6
LLM Stats (Include): Claude Opus 4.8 (87.6) beat Qwen 3.7 Max by 1.4
Artificial Analysis Intelligence Index: Claude Opus 4.8 (Adaptive Reasoning, Max Effort) (61.44) beat GPT-5.5 (xHigh) by 1.2
PinchBench: Claude Opus 4.8 Fast (94.49) beat Qwen Max by 1.05
AA Humanity's Last Exam: Claude Opus 4.8 (Adaptive Reasoning, Max Effort) (45.74) beat Gemini 3.1 Pro (Preview) by 1.02
Vellum - HumanEval: Claude Opus 4.8 (88.6) beat Claude Opus 4.7 by 1.0

AI Benchmark Digest — 2026-05-28

2026-05-28T08:13:42.023730+00:00

Daily

Top-10 New Scores (1)

GPT-5.5 (xHigh) on SWE-rebench: 62.73 (#1)

New #1 Leaders (2)

Kaggle FACTS Grounding: Gemma 4 26B A4B (80.87) beat GPT-5.2 by 4.7
PinchBench: Qwen Max (93.44) beat Grok 0.1 by 1.37

AI Benchmark Digest — 2026-05-27

2026-05-27T08:20:58.056719+00:00

Daily

Top-10 New Scores (1)

GPT-5.4 (xHigh) on Creative Writing (Lechmazur): 3.2 (#2)

New #1 Leaders (11)

LLM Chess (Saplin): GPT-5.5 (Medium) (1532.2) beat Gemini 3.1 Pro by 20.8
LLM Stats (PolyMATH): Qwen 3.7 Max (86.5) beat Qwen 3.6 Plus by 9.1
LLM Stats (MCP-Mark): Qwen 3.7 Max (60.8) beat Kimi K2.6 by 4.9
LLM Stats (NL2Repo): Qwen 3.7 Max (47.2) beat GLM-5.1 by 4.5
LLM Stats (MMLU-ProX): Qwen 3.7 Max (87.0) beat Qwen 3.6 Plus by 2.3
LLM Stats (HMMT Feb 26): Qwen 3.7 Max (97.1) beat DeepSeek V4 Pro (Max) by 1.9
LLM Stats (MAXIFE): Qwen 3.7 Max (89.2) beat Qwen 3.6 Plus by 1.0
LLM Stats (Include): Qwen 3.7 Max (86.2) beat Qwen 3.5 397B A17B by 0.6
LLM Stats (IMO-AnswerBench): Qwen 3.7 Max (90.0) beat DeepSeek V4 Pro (Max) by 0.2
Creative Writing (Lechmazur): GPT-5.5 (Thinking, xHigh) (3.2) beat GPT-5.5 by 0.2
LLM Stats (MMLU-Redux): Qwen 3.7 Max (95.0) beat Qwen 3.5 397B A17B by 0.1

AI Benchmark Digest — 2026-05-25

2026-05-25T08:26:35.093083+00:00

Daily

New Benchmarks (6)

LLMEval-Logic Base (Accuracy (%)): leader Seed 2.0 Pro (Thinking) (75.5), 14 models
LLMEval-Logic Hard (Accuracy (%)): leader Gemini 3.1 Pro (Thinking) (37.5), 14 models
LLMEval-Logic Hard Sub-Q (Accuracy (%)): leader Claude Opus 4.6 (Thinking) (76.6), 14 models
LLMEval-Logic Formalization Free (Accuracy (%)): leader Gemini 3.1 Pro (Thinking) (45.1), 14 models
LLMEval-Logic Formalization Fixed (Accuracy (%)): leader GPT-5.4 Pro (No-Think) (60.2), 14 models
ExploitBench v8-bench (Mean Capability (%)): leader Claude Mythos Preview (69.0), 9 models
V8 exploitation ladder benchmark measuring how far AI agents climb from code reachability through crash reproduction, exploit primitives, and arbitrary code execution. Reports mean capability across 41 V8 bug environments.

AI Benchmark Digest — 2026-05-24

2026-05-24T07:56:34.401567+00:00

Daily

New Benchmarks (14)

NanoGPT-Bench (% of Human Progress Recovered): leader Claude Opus 4.6 (9.3), 2 models
Autonomous research benchmark built on the NanoGPT Speedrun, measuring how much of five months of human pretraining-speedup progress coding agents recover under a fixed H100 compute budget.
CursorBench 3.1 (Score (%)): leader Claude Opus 4.7 (64.8), 7 models
Cursor benchmark of ambiguous, multi-file coding tasks from real Cursor sessions, with models scored by task success percentage and average cost per task.
SMDD-Bench (Pass Rate (%)): leader GPT-5.4 (Medium) (40.2), 7 models
Small molecule drug design agent benchmark with sandboxed Python, Boltz structure prediction, and ADMET tooling. Measures pass rate across 502 computationally verifiable chemistry tasks.
SMDD-Bench Diversity (Avg Successful): leader Claude Sonnet 4.6 (8.4), 7 models
SMDD-Bench diversity slice measuring whether agents generate multiple distinct, novel, successful molecule designs across repeated Lead Optimization rollouts.
Blueprint-Bench 2 (Connectivity Similarity Score): leader GPT 5.5 (0.362), 12 models
Andon Labs spatial reasoning benchmark where agents convert apartment photographs into 2D floor plans, scored by normalized connectivity similarity against ground truth layouts.
PACT (Lechmazur) (CMS Points): leader GPT-5.5 (high) (59.0), 25 models
Pairwise Auction Conversation Testbed for multi-round buyer-seller bargaining. LLMs negotiate over 20 rounds with hidden private values, scored by Composite Model Score from head-to-head surplus capture.
FormationEval (Accuracy (%)): leader gemini-3-pro-preview (99.8), 72 models
Chinese Classical Bench (Average Score (%)): leader claude-opus-4-7 (66.21), 10 models
Chinese Classical Bench - Translate Judge (Score (%)): leader claude-opus-4-7-thinking (80.2), 10 models
Chinese Classical Bench - Punctuate Punct F1 (Score (%)): leader claude-opus-4-7 (80.02), 10 models
Chinese Classical Bench - Char-Gloss Judge (Score (%)): leader claude-opus-4-7-thinking (73.6), 10 models
Chinese Classical Bench - Idiom-Source Book EM (Score (%)): leader deepseek-3.2 (74.0), 10 models
Chinese Classical Bench - Fill-In Exact (Score (%)): leader claude-opus-4-7-thinking (88.0), 10 models
Chinese Classical Bench - Compress Efficiency (Score (%)): leader deepseek-3.2 (16.32), 9 models

Top-10 New Scores (1)

Gemini 3.1 Pro (High) on CLBench: 20.8 (#8)

New #1 Leaders (5)

Evals for Every Language: Gemini 3.1 Pro (69.11) beat Gemini 2.5 Flash by 6.52
CLBench: GPT-5.4 (xHigh) (27.9) beat GPT-5.1 (High) by 4.2
LiveBench Logic With Navigation: Qwen Max (84.0) beat Claude Opus 4.6 (Thinking) by 4.0
Spider 2.0-Lite: DivSkill-SQL (73.13) beat SOMA-SQL by 1.11
PinchBench: Grok 0.1 (92.07) beat Claude Opus 4.7 by 0.49

AI Benchmark Digest — 2026-05-23

2026-05-23T07:20:10.541511+00:00

Daily

New #1 Leaders (1)

OSWorld: Opus 4.7 (83.64) beat Holo3-35B-A3B by 1.08

AI Benchmark Digest — 2026-05-22

2026-05-22T07:36:15.662013+00:00

Daily

Top-10 New Scores (1)

GPT-5.5 (High) on Sycophancy (Lechmazur): 3.5 (#11)

New #1 Leaders (2)

UGI - Writing: gemini-3.5-flash (thinking_level=medium) (72.54) beat gemini-3.1-pro-preview (thinking_level=low) by 0.39
Arabic Broad Leaderboard: gemini-3.5-flash (9.253) beat gemini-3-pro-preview by 0.05

AI Benchmark Digest — 2026-05-21

2026-05-21T07:40:34.045646+00:00

Daily

Top-10 New Scores (1)

Gemini 3.5 Flash (High) on WeirdML: 62.64 (#17)

New #1 Leaders (3)

Kaggle Game Arena Poker (Heads Up): GPT-5.5 (73.93) beat GPT-5.2 by 33.93
AA APEX-Agents: Gemini 3.5 Flash (high) (47.05) beat GPT-5.5 (xhigh) by 9.37
LA Leaderboard: Qwen2.5-14B-Instruct-GPTQ-Int8 (63.6) beat gemma-2-9b-it by 0.27

AI Benchmark Digest — 2026-05-20

2026-05-20T07:43:37.557151+00:00

Daily

New Models (1)

Gemini 3.5 Flash (High) — ELO 1942, #9
- AA MMMU-Pro: 84.28 (#1/190)
- SEAL - MCP Atlas: 83.6 (#1/21)
- AA Omniscience: 22.68 (#3/393)
- AA Omniscience - Law: 57.4 (#4/393)
- AA Omniscience - Software Engineering (SWE) - PHP: 84.0 (#4/393)
- AA Humanity's Last Exam: 40.96 (#5/484)
- AA GPQA Diamond: 92.22 (#6/488)
- AA Omniscience - Science, Engineering & Mathematics: 50.1 (#6/393)
- AA GDPval: 1655.7 (#7/365)
- AA Omniscience - Humanities & Social Sciences: 52.3 (#7/393)

Top-10 New Scores (34)

GPT-5.5 (High) on Multi-turn Debate (Lechmazur): 1583.6 (#5)
Gemini 3.5 Flash (High) on AA CritPt: 13.14 (#8)
Gemini 3.5 Flash (High) on AA GDPval: 1655.7 (#7)
Gemini 3.5 Flash (High) on AA GPQA Diamond: 92.22 (#6)
Gemini 3.5 Flash (High) on AA Humanity's Last Exam: 40.96 (#5)
Gemini 3.5 Flash (High) on AA IFBench: 76.33 (#17)
Gemini 3.5 Flash (High) on AA Long Context Reasoning: 69.33 (#27)
Gemini 3.5 Flash (High) on AA Omniscience: 22.68 (#3)
Gemini 3.5 Flash (High) on AA Omniscience - Business: 45.8 (#8)
Gemini 3.5 Flash (High) on AA Omniscience - Health: 40.2 (#14)
Gemini 3.5 Flash (High) on AA Omniscience - Humanities & Social Sciences: 52.3 (#7)
Gemini 3.5 Flash (High) on AA Omniscience - Law: 57.4 (#4)
Gemini 3.5 Flash (High) on AA Omniscience - Science, Engineering & Mathematics: 50.1 (#6)
Gemini 3.5 Flash (High) on AA Omniscience - Software Engineering (SWE): 65.5 (#16)
Gemini 3.5 Flash (High) on AA Omniscience - Software Engineering (SWE) - C: 80.0 (#18)
Gemini 3.5 Flash (High) on AA Omniscience - Software Engineering (SWE) - Dart: 60.0 (#14)
Gemini 3.5 Flash (High) on AA Omniscience - Software Engineering (SWE) - Go: 50.0 (#32)
Gemini 3.5 Flash (High) on AA Omniscience - Software Engineering (SWE) - HTML: 72.0 (#17)
Gemini 3.5 Flash (High) on AA Omniscience - Software Engineering (SWE) - Java: 51.0 (#16)
Gemini 3.5 Flash (High) on AA Omniscience - Software Engineering (SWE) - JavaScript: 71.82 (#14)
Gemini 3.5 Flash (High) on AA Omniscience - Software Engineering (SWE) - Julia: 60.0 (#13)
Gemini 3.5 Flash (High) on AA Omniscience - Software Engineering (SWE) - Kotlin: 56.0 (#22)
Gemini 3.5 Flash (High) on AA Omniscience - Software Engineering (SWE) - PHP: 84.0 (#4)
Gemini 3.5 Flash (High) on AA Omniscience - Software Engineering (SWE) - Python: 61.0 (#24)
Gemini 3.5 Flash (High) on AA Omniscience - Software Engineering (SWE) - R: 56.0 (#18)
Gemini 3.5 Flash (High) on AA Omniscience - Software Engineering (SWE) - Rust: 80.0 (#8)
Gemini 3.5 Flash (High) on AA Omniscience - Software Engineering (SWE) - Swift: 72.0 (#20)
Gemini 3.5 Flash (High) on AA Omniscience - Software Engineering (SWE) - TypeScript: 67.78 (#16)
Gemini 3.5 Flash (High) on AA SciCode: 53.12 (#11)
Gemini 3.5 Flash (High) on AA TAU-2 Bench: 95.32 (#20)
Gemini 3.5 Flash (High) on AA Terminal-Bench Hard: 40.91 (#36)
Gemini 3.5 Flash (High) on ARC-AGI-1: 92.5 (#16)
Gemini 3.5 Flash (High) on ARC-AGI-2: 72.08 (#12)
Gemini 3.5 Flash (High) on Artificial Analysis Intelligence Index: 55.33 (#8)

New #1 Leaders (5)

LLM Stats (GDPval-AA): Gemini 3.5 Flash (165600.0) beat Claude Sonnet 4.6 by 2300.0
LLM Stats (MCP Atlas): Gemini 3.5 Flash (83.6) beat Claude Opus 4.7 by 6.3
AA MMMU-Pro: Gemini 3.5 Flash (high) (84.28) beat Gemini 3.1 Pro Preview by 1.85
SEAL - MCP Atlas: gemini-3.5-flash (high) (83.6) beat Muse Spark by 1.4
LLM Stats (Toolathlon): Gemini 3.5 Flash (56.5) beat GPT-5.5 by 0.9

AI Benchmark Digest — 2026-05-17

2026-05-17T08:02:54.093472+00:00

Daily

New #1 Leaders (1)

OpenClawProBench: intern-s2-preview (76.7) beat Sensenova 6.7 Flash Lite by 3.0

Weekly

Top-10 New Scores (3)

Claude Opus 4.7 (Thinking) on SEAL Showdown: 1115.7 (#12)
Claude Opus 4.7 (Thinking) on WeirdML: 75.45 (#8)
GPT-5.5 (xHigh) on Chatbot Arena (Code): 1501.0 (#9)

New #1 Leaders (16)

MathArena - ARXIVLEAN March: AlephProver (34.15) beat Aristotle by 17.08
OpenCompass Reasoning - Common: Doubao-Seed-2-0-Pro-260215 (high) (82.1) beat Gemini-3-Pro-Preview by 8.5
OpenCompass Math - College: Doubao-Seed-2-0-Pro-260215 (high) (83.8) beat Kimi-K2.5 by 7.3
OpenClawProBench: intern-s2-preview (76.7) beat qwen3.5-397b-a17b by 6.3
Tau3-Bench Banking_Knowledge: GPT-5.5 (37.4) beat Distyl ButtonAgent by 6.2
OpenCompass Knowledge - Social Science: Gemini-3.1-Pro-Preview (97.5) beat Gemini-3-Pro-Preview by 4.3
OpenCompass LLM - Math: Doubao-Seed-2-0-Pro-260215 (high) (77.3) beat Qwen3-Max-2026-01-23 by 4.1
OpenCompass LLM - Reasoning: Doubao-Seed-2-0-Pro-260215 (high) (65.2) beat Gemini-3-Pro-Preview by 3.7
OpenCompass Math - Competition: Kimi-K2.6 (72.1) beat Qwen3-Max-2026-01-23 by 2.1
OpenCompass Reasoning - Academic: GPT-5.4-2026-03-05 (high) (52.0) beat GPT-5.2-2025-12-11 (high) by 1.5
VisuLogic: PEREA-1.0new (52.8) beat Human by 1.4
WeirdML: gpt-5.5 (xhigh) (84.91) beat gpt-5.5 (high) by 1.01
GAIA: Co-Sight Pro v1.0.1 (93.02) beat OPS-Agentic-Search by 0.66
OpenCompass Knowledge - Engineering: GPT-5.4-2026-03-05 (high) (96.2) beat Gemini-3-Pro-Preview by 0.4
AA TAU-2 Bench: JT-35B-Flash (99.12) beat GLM-4.7-Flash (Reasoning) by 0.32
AISI Cyber TLO 10M: GPT-5.5 (10.0) beat Claude Opus 4.6 by 0.2

AI Benchmark Digest — 2026-05-16

2026-05-16T07:15:27.727063+00:00

Daily

Top-10 New Scores (1)

GPT-5.5 (xHigh) on Chatbot Arena (Code): 1501.0 (#9)

New #1 Leaders (2)

MathArena - ARXIVLEAN March: AlephProver (34.15) beat Aristotle by 17.08
GAIA: Co-Sight Pro v1.0.1 (93.02) beat OPS-Agentic-Search by 0.66

AI Benchmark Digest — 2026-05-14

2026-05-14T07:26:43.169192+00:00

Daily

New Models (4)

Doubao-Seed-2-0-Pro-260215 (High) — ELO 1781, #73
- OpenCompass LLM - Reasoning: 65.2 (#1/23)
- OpenCompass LLM - Math: 77.3 (#1/23)
- OpenCompass Knowledge - Humanities: 95.0 (#1/23)
- OpenCompass Reasoning - Common: 82.1 (#1/23)
- OpenCompass Math - College: 83.8 (#1/23)
- OpenCompass LLM - Language: 77.3 (#3/23)
- OpenCompass Language - Creation: 77.1 (#3/23)
- OpenCompass Knowledge - Science: 94.6 (#3/23)
- OpenCompass LLM - Agent: 44.2 (#4/23)
- OpenCompass Language - NLP: 69.6 (#4/23)
Doubao-Seed-2-0-Lite-260215 (High) — ELO 1741, #103
- OpenCompass Reasoning - Common: 78.1 (#2/23)
- OpenCompass Language - Creation: 77.1 (#4/23)
- OpenCompass LLM - Language: 74.4 (#6/23)
- OpenCompass LLM - Agent: 42.4 (#6/23)
- OpenCompass Agent - Tool Use: 42.4 (#6/23)
- OpenCompass Knowledge - Science: 91.7 (#7/23)
- OpenCompass LLM - Reasoning: 59.5 (#8/23)
- OpenCompass Language - NLP: 67.1 (#8/23)
- OpenCompass Language - Instruction Following: 72.5 (#8/23)
- OpenCompass Math - College: 77.1 (#8/23)
Hy3-preview (High) — ELO 1729, #110
- OpenCompass Math - College: 81.3 (#3/23)
- OpenCompass Language - Instruction Following: 76.0 (#4/23)
- OpenCompass LLM - Math: 74.5 (#5/23)
- OpenCompass Language - Creation: 75.4 (#5/23)
- OpenCompass LLM - Language: 74.4 (#7/23)
- OpenCompass Reasoning - Academic: 43.6 (#8/23)
- OpenCompass LLM - Reasoning: 58.5 (#10/23)
- OpenCompass Math - Competition: 67.6 (#10/23)
- OpenCompass LLM - Agent: 28.7 (#12/23)
- OpenCompass Reasoning - Common: 73.5 (#12/23)
Ring-2.5-1T — ELO 1711, #119
- OpenCompass Knowledge - Social Science: 92.9 (#5/23)
- OpenCompass Language - NLP: 65.4 (#11/23)
- OpenCompass Language - Creation: 68.8 (#12/23)
- OpenCompass Knowledge - Humanities: 90.0 (#12/23)
- OpenCompass LLM - Agent: 25.0 (#13/23)
- OpenCompass Math - College: 75.0 (#13/23)
- OpenCompass Agent - Tool Use: 25.0 (#13/23)
- OpenCompass LLM - Knowledge: 89.4 (#14/23)
- OpenCompass Knowledge - Engineering: 90.8 (#14/23)
- OpenCompass LLM - Language: 69.8 (#15/23)

Top-10 New Scores (1)

Claude Opus 4.7 (Thinking) on WeirdML: 75.45 (#8)

New #1 Leaders (9)

OpenCompass Reasoning - Common: Doubao-Seed-2-0-Pro-260215 (high) (82.1) beat Gemini-3-Pro-Preview by 8.5
OpenCompass Math - College: Doubao-Seed-2-0-Pro-260215 (high) (83.8) beat Kimi-K2.5 by 7.3
Tau3-Bench Banking_Knowledge: GPT-5.5 (37.4) beat Distyl ButtonAgent by 6.2
OpenCompass Knowledge - Social Science: Gemini-3.1-Pro-Preview (97.5) beat Gemini-3-Pro-Preview by 4.3
OpenCompass LLM - Math: Doubao-Seed-2-0-Pro-260215 (high) (77.3) beat Qwen3-Max-2026-01-23 by 4.1
OpenCompass LLM - Reasoning: Doubao-Seed-2-0-Pro-260215 (high) (65.2) beat Gemini-3-Pro-Preview by 3.7
OpenCompass Math - Competition: Kimi-K2.6 (72.1) beat Qwen3-Max-2026-01-23 by 2.1
OpenCompass Reasoning - Academic: GPT-5.4-2026-03-05 (high) (52.0) beat GPT-5.2-2025-12-11 (high) by 1.5
OpenCompass Knowledge - Engineering: GPT-5.4-2026-03-05 (high) (96.2) beat Gemini-3-Pro-Preview by 0.4

AI Benchmark Digest — 2026-05-13

2026-05-13T07:29:12.582080+00:00

Daily

New Benchmarks (2)

ProgramBench (Resolved (%)): leader GPT-5.5 (xHigh) (0.5), 13 models
Meta and Stanford benchmark testing whether language-model agents can rebuild complete programs from only a compiled binary and documentation. Agents use mini-SWE-agent across 200 open-source program recreation tasks and are scored by hidden behavioral tests.
ProgramBench Almost (Almost (%)): leader GPT-5.5 (xHigh) (13.5), 13 models
Companion ProgramBench metric that counts near-complete program recreations: tasks where the generated implementation passes most hidden behavioral tests but does not fully resolve the benchmark task.

New Models (1)

JT-35B-Flash — ELO 1693, #141
- AA TAU-2 Bench: 99.1 (#1/405)
- AA Omniscience - Software Engineering (SWE) - Go: 36.0 (#50/391)
- AA Omniscience - Software Engineering (SWE) - Java: 29.0 (#58/391)
- AA Omniscience - Software Engineering (SWE) - HTML: 48.0 (#60/391)
- AA Omniscience - Software Engineering (SWE) - JavaScript: 41.82 (#75/391)
- AA GPQA Diamond: 82.9 (#76/486)
- AA Omniscience - Software Engineering (SWE) - C: 53.0 (#78/391)
- AA Omniscience - Software Engineering (SWE) - PHP: 38.0 (#79/391)
- AA Omniscience - Software Engineering (SWE) - TypeScript: 36.67 (#82/391)
- AA Omniscience - Software Engineering (SWE): 35.0 (#83/391)

Top-10 New Scores (1)

GPT-5.5 (xHigh) on WeirdML: 84.91 (#1)

New #1 Leaders (2)

WeirdML: gpt-5.5 (xhigh) (84.91) beat gpt-5.5 (high) by 1.01
AA TAU-2 Bench: JT-35B-Flash (99.1) beat GLM-4.7-Flash (Reasoning) by 0.3

AI Benchmark Digest — 2026-05-11

2026-05-11T08:08:39.844852+00:00

Daily

New #1 Leaders (2)

OpenClawProBench: Sensenova 6.7 Flash Lite (73.7) beat qwen3.5-397b-a17b by 3.3
VisuLogic: PEREA-1.0new (52.8) beat Human by 1.4

AI Benchmark Digest — 2026-05-10

2026-05-10T07:49:15.895022+00:00

Daily

New Benchmarks (43)

AA Global-MMLU-Lite - Arabic (Accuracy (%)): leader Gemini 3.1 Pro Preview (93.0), 119 models
AA Global-MMLU-Lite - Bengali (Accuracy (%)): leader Gemini 3.1 Pro Preview (92.17), 119 models
AA Global-MMLU-Lite - German (Accuracy (%)): leader Gemini 3.1 Pro Preview (94.75), 119 models
AA Global-MMLU-Lite - English (Accuracy (%)): leader Claude Opus 4.6 (Adaptive Reasoning, Max Effort) (95.17), 120 models
AA Global-MMLU-Lite - Spanish (Accuracy (%)): leader Gemini 3.1 Pro Preview (94.42), 118 models
AA Global-MMLU-Lite - French (Accuracy (%)): leader Gemini 3.1 Pro Preview (93.67), 118 models
AA Global-MMLU-Lite - Hindi (Accuracy (%)): leader Gemini 3.1 Pro Preview (93.67), 117 models
AA Global-MMLU-Lite - Indonesian (Accuracy (%)): leader Gemini 3.1 Pro Preview (93.67), 118 models
AA Global-MMLU-Lite - Italian (Accuracy (%)): leader Gemini 3.1 Pro Preview (94.58), 117 models
AA Global-MMLU-Lite - Japanese (Accuracy (%)): leader Gemini 3.1 Pro Preview (93.67), 116 models
AA Global-MMLU-Lite - Korean (Accuracy (%)): leader Claude Opus 4.6 (Adaptive Reasoning, Max Effort) (93.0), 116 models
AA Global-MMLU-Lite - Burmese (Accuracy (%)): leader Gemini 3.1 Pro Preview (91.17), 111 models
AA Global-MMLU-Lite - Portuguese (Accuracy (%)): leader Gemini 3.1 Pro Preview (94.25), 113 models
AA Global-MMLU-Lite - Swahili (Accuracy (%)): leader Gemini 3.1 Pro Preview (92.33), 112 models
AA Global-MMLU-Lite - Yoruba (Accuracy (%)): leader Gemini 3.1 Pro Preview (88.75), 112 models
AA Global-MMLU-Lite - Chinese (Accuracy (%)): leader Claude Opus 4.6 (Adaptive Reasoning, Max Effort) (93.58), 113 models
AA Omniscience - Business (Accuracy (%)): leader GPT-5.5 (xhigh) (49.1), 388 models
AA Omniscience - Health (Accuracy (%)): leader GPT-5.5 (medium) (48.8), 388 models
AA Omniscience - Humanities & Social Sciences (Accuracy (%)): leader Gemini 3 Pro Preview (high) (56.6), 388 models
AA Omniscience - Law (Accuracy (%)): leader Gemini 3 Pro Preview (high) (64.3), 388 models
AA Omniscience - Science, Engineering & Mathematics (Accuracy (%)): leader GPT-5.5 (high) (52.3), 388 models
AA Omniscience - Software Engineering (SWE) (Accuracy (%)): leader GPT-5.5 (xhigh) (84.4), 388 models
AA Omniscience - Software Engineering (SWE) - C (Accuracy (%)): leader GPT-5.5 (high) (92.0), 388 models
AA Omniscience - Software Engineering (SWE) - Dart (Accuracy (%)): leader GPT-5.3 Codex (xhigh) (80.0), 388 models
AA Omniscience - Software Engineering (SWE) - Go (Accuracy (%)): leader GPT-5.5 (high) (84.0), 388 models
AA Omniscience - Software Engineering (SWE) - HTML (Accuracy (%)): leader GPT-5.5 (medium) (90.0), 388 models
AA Omniscience - Software Engineering (SWE) - Java (Accuracy (%)): leader GPT-5.3 Codex (xhigh) (73.0), 388 models
AA Omniscience - Software Engineering (SWE) - JavaScript (Accuracy (%)): leader GPT-5.3 Codex (xhigh) (90.91), 388 models
AA Omniscience - Software Engineering (SWE) - Julia (Accuracy (%)): leader GPT-5.4 (low) (88.0), 388 models
AA Omniscience - Software Engineering (SWE) - Kotlin (Accuracy (%)): leader GPT-5.3 Codex (xhigh) (90.0), 388 models
AA Omniscience - Software Engineering (SWE) - PHP (Accuracy (%)): leader GPT-5.5 (medium) (92.0), 388 models
AA Omniscience - Software Engineering (SWE) - Python (Accuracy (%)): leader GPT-5.5 (xhigh) (90.5), 388 models
AA Omniscience - Software Engineering (SWE) - R (Accuracy (%)): leader GPT-5.5 (medium) (74.0), 388 models
AA Omniscience - Software Engineering (SWE) - Rust (Accuracy (%)): leader GPT-5.5 (xhigh) (92.0), 388 models
AA Omniscience - Software Engineering (SWE) - Swift (Accuracy (%)): leader GPT-5.5 (xhigh) (92.0), 388 models
AA Omniscience - Software Engineering (SWE) - TypeScript (Accuracy (%)): leader GPT-5.5 (xhigh) (91.11), 388 models
EuroEval Albanian NLU - MMS SQ (Sentiment classification Score (%)): leader gemini-3-flash-preview#no-thinking (32.13), 196 models
EuroEval Albanian NLU task column for the MMS SQ dataset, measuring sentiment classification from the public albanian_nlu.csv leaderboard.
EuroEval Albanian NLU - WikiANN SQ (Named entity recognition Score (%)): leader multilingual-e5-large (86.6), 200 models
EuroEval Albanian NLU task column for the WikiANN SQ dataset, measuring named entity recognition from the public albanian_nlu.csv leaderboard.
EuroEval Albanian NLU - ScaLA SQ (Linguistic acceptability Score (%)): leader gemini-3.1-pro-preview (78.55), 166 models
EuroEval Albanian NLU task column for the ScaLA SQ dataset, measuring linguistic acceptability from the public albanian_nlu.csv leaderboard.
EuroEval Albanian NLU - MultiWikiQA SQ (Reading comprehension Score (%)): leader Qwen3.5-9B-Base (70.8), 200 models
EuroEval Albanian NLU task column for the MultiWikiQA SQ dataset, measuring reading comprehension from the public albanian_nlu.csv leaderboard.
EuroEval Bosnian NLU - MMS BS (Sentiment classification Score (%)): leader gpt-4.1-mini-2025-04-14 (56.43), 208 models
EuroEval Bosnian NLU task column for the MMS BS dataset, measuring sentiment classification from the public bosnian_nlu.csv leaderboard.
EuroEval Bosnian NLU - WikiANN BS (Named entity recognition Score (%)): leader multilingual-e5-large (84.87), 212 models
EuroEval Bosnian NLU task column for the WikiANN BS dataset, measuring named entity recognition from the public bosnian_nlu.csv leaderboard.
EuroEval Bosnian NLU - Multi Wiki QA BS (Reading comprehension Score (%)): leader Olmo-3-1125-32B (78.64), 211 models
EuroEval Bosnian NLU task column for the Multi Wiki QA BS dataset, measuring reading comprehension from the public bosnian_nlu.csv leaderboard.

Weekly

New Benchmarks (43)

AA Global-MMLU-Lite - Arabic (Accuracy (%)): leader Gemini 3.1 Pro Preview (93.0), 119 models
AA Global-MMLU-Lite - Bengali (Accuracy (%)): leader Gemini 3.1 Pro Preview (92.17), 119 models
AA Global-MMLU-Lite - German (Accuracy (%)): leader Gemini 3.1 Pro Preview (94.75), 119 models
AA Global-MMLU-Lite - English (Accuracy (%)): leader Claude Opus 4.6 (Adaptive Reasoning, Max Effort) (95.17), 120 models
AA Global-MMLU-Lite - Spanish (Accuracy (%)): leader Gemini 3.1 Pro Preview (94.42), 118 models
AA Global-MMLU-Lite - French (Accuracy (%)): leader Gemini 3.1 Pro Preview (93.67), 118 models
AA Global-MMLU-Lite - Hindi (Accuracy (%)): leader Gemini 3.1 Pro Preview (93.67), 117 models
AA Global-MMLU-Lite - Indonesian (Accuracy (%)): leader Gemini 3.1 Pro Preview (93.67), 118 models
AA Global-MMLU-Lite - Italian (Accuracy (%)): leader Gemini 3.1 Pro Preview (94.58), 117 models
AA Global-MMLU-Lite - Japanese (Accuracy (%)): leader Gemini 3.1 Pro Preview (93.67), 116 models
AA Global-MMLU-Lite - Korean (Accuracy (%)): leader Claude Opus 4.6 (Adaptive Reasoning, Max Effort) (93.0), 116 models
AA Global-MMLU-Lite - Burmese (Accuracy (%)): leader Gemini 3.1 Pro Preview (91.17), 111 models
AA Global-MMLU-Lite - Portuguese (Accuracy (%)): leader Gemini 3.1 Pro Preview (94.25), 113 models
AA Global-MMLU-Lite - Swahili (Accuracy (%)): leader Gemini 3.1 Pro Preview (92.33), 112 models
AA Global-MMLU-Lite - Yoruba (Accuracy (%)): leader Gemini 3.1 Pro Preview (88.75), 112 models
AA Global-MMLU-Lite - Chinese (Accuracy (%)): leader Claude Opus 4.6 (Adaptive Reasoning, Max Effort) (93.58), 113 models
AA Omniscience - Business (Accuracy (%)): leader GPT-5.5 (xhigh) (49.1), 388 models
AA Omniscience - Health (Accuracy (%)): leader GPT-5.5 (medium) (48.8), 388 models
AA Omniscience - Humanities & Social Sciences (Accuracy (%)): leader Gemini 3 Pro Preview (high) (56.6), 388 models
AA Omniscience - Law (Accuracy (%)): leader Gemini 3 Pro Preview (high) (64.3), 388 models
AA Omniscience - Science, Engineering & Mathematics (Accuracy (%)): leader GPT-5.5 (high) (52.3), 388 models
AA Omniscience - Software Engineering (SWE) (Accuracy (%)): leader GPT-5.5 (xhigh) (84.4), 388 models
AA Omniscience - Software Engineering (SWE) - C (Accuracy (%)): leader GPT-5.5 (high) (92.0), 388 models
AA Omniscience - Software Engineering (SWE) - Dart (Accuracy (%)): leader GPT-5.3 Codex (xhigh) (80.0), 388 models
AA Omniscience - Software Engineering (SWE) - Go (Accuracy (%)): leader GPT-5.5 (high) (84.0), 388 models
AA Omniscience - Software Engineering (SWE) - HTML (Accuracy (%)): leader GPT-5.5 (medium) (90.0), 388 models
AA Omniscience - Software Engineering (SWE) - Java (Accuracy (%)): leader GPT-5.3 Codex (xhigh) (73.0), 388 models
AA Omniscience - Software Engineering (SWE) - JavaScript (Accuracy (%)): leader GPT-5.3 Codex (xhigh) (90.91), 388 models
AA Omniscience - Software Engineering (SWE) - Julia (Accuracy (%)): leader GPT-5.4 (low) (88.0), 388 models
AA Omniscience - Software Engineering (SWE) - Kotlin (Accuracy (%)): leader GPT-5.3 Codex (xhigh) (90.0), 388 models
AA Omniscience - Software Engineering (SWE) - PHP (Accuracy (%)): leader GPT-5.5 (medium) (92.0), 388 models
AA Omniscience - Software Engineering (SWE) - Python (Accuracy (%)): leader GPT-5.5 (xhigh) (90.5), 388 models
AA Omniscience - Software Engineering (SWE) - R (Accuracy (%)): leader GPT-5.5 (medium) (74.0), 388 models
AA Omniscience - Software Engineering (SWE) - Rust (Accuracy (%)): leader GPT-5.5 (xhigh) (92.0), 388 models
AA Omniscience - Software Engineering (SWE) - Swift (Accuracy (%)): leader GPT-5.5 (xhigh) (92.0), 388 models
AA Omniscience - Software Engineering (SWE) - TypeScript (Accuracy (%)): leader GPT-5.5 (xhigh) (91.11), 388 models
EuroEval Albanian NLU - MMS SQ (Sentiment classification Score (%)): leader gemini-3-flash-preview#no-thinking (32.13), 196 models
EuroEval Albanian NLU task column for the MMS SQ dataset, measuring sentiment classification from the public albanian_nlu.csv leaderboard.
EuroEval Albanian NLU - WikiANN SQ (Named entity recognition Score (%)): leader multilingual-e5-large (86.6), 200 models
EuroEval Albanian NLU task column for the WikiANN SQ dataset, measuring named entity recognition from the public albanian_nlu.csv leaderboard.
EuroEval Albanian NLU - ScaLA SQ (Linguistic acceptability Score (%)): leader gemini-3.1-pro-preview (78.55), 166 models
EuroEval Albanian NLU task column for the ScaLA SQ dataset, measuring linguistic acceptability from the public albanian_nlu.csv leaderboard.
EuroEval Albanian NLU - MultiWikiQA SQ (Reading comprehension Score (%)): leader Qwen3.5-9B-Base (70.8), 200 models
EuroEval Albanian NLU task column for the MultiWikiQA SQ dataset, measuring reading comprehension from the public albanian_nlu.csv leaderboard.
EuroEval Bosnian NLU - MMS BS (Sentiment classification Score (%)): leader gpt-4.1-mini-2025-04-14 (56.43), 208 models
EuroEval Bosnian NLU task column for the MMS BS dataset, measuring sentiment classification from the public bosnian_nlu.csv leaderboard.
EuroEval Bosnian NLU - WikiANN BS (Named entity recognition Score (%)): leader multilingual-e5-large (84.87), 212 models
EuroEval Bosnian NLU task column for the WikiANN BS dataset, measuring named entity recognition from the public bosnian_nlu.csv leaderboard.
EuroEval Bosnian NLU - Multi Wiki QA BS (Reading comprehension Score (%)): leader Olmo-3-1125-32B (78.64), 211 models
EuroEval Bosnian NLU task column for the Multi Wiki QA BS dataset, measuring reading comprehension from the public bosnian_nlu.csv leaderboard.

New Models (38)

GLM-5V Turbo (Reasoning) — ELO 1738, #102
- AA TAU-2 Bench: 98.5 (#3/402)
- AA GDPval: 1330.87 (#43/360)
- AA MMMU-Pro: 72.8 (#44/188)
- AA SciCode: 43.5 (#52/477)
- Artificial Analysis Intelligence Index: 42.85 (#56/482)
- AA Terminal-Bench Hard: 32.6 (#79/397)
- AA Omniscience: -18.98 (#80/388)
- AA Long Context Reasoning: 61.0 (#84/411)
- AA Humanity's Last Exam: 15.8 (#91/479)
- AA GPQA Diamond: 80.9 (#96/483)
ERNIE 5.0 Thinking Preview — ELO 1631, #214
- AA LiveCodeBench: 81.2 (#24/343)
- AA Global-MMLU-Lite: 86.5 (#33/120)
- AA AIME 2025: 85.0 (#46/269)
- AA MMLU-Pro: 83.0 (#60/345)
- AA CritPt: 1.4 (#68/388)
- AA MMMU-Pro: 64.6 (#90/188)
- AA TAU-2 Bench: 83.9 (#94/402)
- AA Humanity's Last Exam: 12.7 (#116/479)
- AA Terminal-Bench Hard: 25.0 (#119/397)
- AA GPQA Diamond: 77.7 (#124/483)
K-EXAONE (Reasoning) — ELO 1603, #245
- AA AIME 2025: 90.3 (#25/269)
- AA LiveCodeBench: 76.8 (#41/343)
- AA MMLU-Pro: 83.8 (#44/345)
- AA CritPt: 1.1 (#76/388)
- AA Global-MMLU-Lite: 78.86 (#80/120)
- AA IFBench: 64.7 (#85/411)
- AA Humanity's Last Exam: 13.1 (#111/479)
- AA Long Context Reasoning: 55.7 (#117/411)
- AA GPQA Diamond: 78.3 (#119/483)
- AA TAU-2 Bench: 74.3 (#121/402)
EXAONE 4.5 33B — ELO 1578, #277
- AA MMMU-Pro: 67.3 (#77/188)
- AA GPQA Diamond: 79.4 (#106/483)
- AA IFBench: 58.0 (#107/411)
- AA TAU-2 Bench: 78.1 (#112/402)
- AA CritPt: 0.3 (#128/388)
- AA Humanity's Last Exam: 11.6 (#131/479)
- AA Terminal-Bench Hard: 20.5 (#144/397)
- Artificial Analysis Intelligence Index: 30.23 (#147/482)
- AA Long Context Reasoning: 49.3 (#150/411)
- AA GDPval: 812.72 (#163/360)
K2-V2 (High) — ELO 1562, #294
- AA AIME 2025: 78.3 (#71/269)
- AA LiveCodeBench: 69.4 (#76/343)
- AA Global-MMLU-Lite: 78.6 (#82/120)
- AA IFBench: 60.1 (#102/411)
- AA MMLU-Pro: 78.6 (#135/345)
- AA Humanity's Last Exam: 9.8 (#157/479)
- AA Long Context Reasoning: 33.3 (#211/411)
- AA Terminal-Bench Hard: 9.8 (#212/397)
- AA GPQA Diamond: 68.1 (#222/483)
- Artificial Analysis Intelligence Index: 20.61 (#232/482)
Solar Open 100B (Reasoning) — ELO 1555, #307
- AA Global-MMLU-Lite: 81.58 (#61/120)
- AA IFBench: 57.7 (#110/411)
- AA Humanity's Last Exam: 9.2 (#170/479)
- AA TAU-2 Bench: 48.2 (#180/402)
- AA Long Context Reasoning: 36.0 (#195/411)
- AA CritPt: 0.0 (#204/388)
- AA GDPval: 666.33 (#207/360)
- Artificial Analysis Intelligence Index: 21.67 (#224/482)
- AA GPQA Diamond: 65.7 (#243/483)
- AA Omniscience: -54.1 (#262/388)
JT-MINI — ELO 1546, #324
- AA TAU-2 Bench: 93.0 (#40/402)
- AA Terminal-Bench Hard: 18.2 (#154/397)
- AA GDPval: 831.97 (#157/360)
- Artificial Analysis Intelligence Index: 25.37 (#187/482)
- AA Humanity's Last Exam: 6.6 (#223/479)
- AA GPQA Diamond: 67.6 (#225/483)
- AA CritPt: 0.0 (#263/388)
- AA IFBench: 36.7 (#277/411)
- AA SciCode: 27.2 (#292/477)
- AA Long Context Reasoning: 11.7 (#308/411)
K2 Think V2 — ELO 1545, #328
- AA IFBench: 62.8 (#94/411)
- AA Omniscience: -33.92 (#125/388)
- AA Long Context Reasoning: 52.7 (#135/411)
- AA Humanity's Last Exam: 9.5 (#165/479)
- AA GPQA Diamond: 71.3 (#192/483)
- Artificial Analysis Intelligence Index: 24.12 (#201/482)
- AA GDPval: 607.98 (#222/360)
- AA SciCode: 33.0 (#223/477)
- AA Terminal-Bench Hard: 6.8 (#240/397)
- AA CritPt: 0.0 (#252/388)
HyperCLOVA X SEED Think (32B) — ELO 1537, #342
- AA TAU-2 Bench: 87.4 (#68/402)
- AA Global-MMLU-Lite: 78.6 (#83/120)
- AA LiveCodeBench: 62.9 (#107/343)
- AA AIME 2025: 59.0 (#118/269)
- AA MMLU-Pro: 78.5 (#137/345)
- AA Terminal-Bench Hard: 12.1 (#194/397)
- AA GDPval: 678.83 (#199/360)
- Artificial Analysis Intelligence Index: 23.72 (#204/482)
- AA Omniscience: -52.87 (#255/388)
- AA CritPt: 0.0 (#257/388)
Mi:dm K 2.5 Pro — ELO 1527, #352
- AA TAU-2 Bench: 86.5 (#75/402)
- AA AIME 2025: 76.7 (#77/269)
- AA LiveCodeBench: 65.6 (#92/343)
- AA Global-MMLU-Lite: 74.23 (#94/120)
- AA MMLU-Pro: 80.9 (#97/345)
- AA IFBench: 49.3 (#155/411)
- AA Humanity's Last Exam: 7.7 (#195/479)
- AA GPQA Diamond: 70.1 (#200/483)
- Artificial Analysis Intelligence Index: 23.06 (#213/482)
- AA GDPval: 643.11 (#213/360)
Motif-2-12.7B (Reasoning) — ELO 1520, #366
- AA AIME 2025: 80.3 (#65/269)
- AA LiveCodeBench: 65.1 (#97/343)
- AA IFBench: 57.0 (#113/411)
- AA MMLU-Pro: 79.6 (#122/345)
- AA Humanity's Last Exam: 8.2 (#183/479)
- AA TAU-2 Bench: 46.5 (#185/402)
- AA GPQA Diamond: 69.5 (#210/483)
- Artificial Analysis Intelligence Index: 19.08 (#244/482)
- AA CritPt: 0.0 (#250/388)
- AA GDPval: 485.33 (#255/360)
Mi:dm K 2.5 Pro Preview — ELO 1517, #371
- AA Global-MMLU-Lite: 81.43 (#63/120)
- AA AIME 2025: 78.7 (#70/269)
- AA MMLU-Pro: 81.3 (#92/345)
- AA LiveCodeBench: 57.6 (#125/343)
- AA Humanity's Last Exam: 8.8 (#175/479)
- AA TAU-2 Bench: 49.4 (#177/402)
- AA IFBench: 45.6 (#180/411)
- AA GPQA Diamond: 72.2 (#185/483)
- AA SciCode: 29.7 (#251/477)
- AA CritPt: 0.0 (#255/388)
K2-V2 (Medium) — ELO 1512, #382
- AA Global-MMLU-Lite: 76.7 (#87/120)
- AA AIME 2025: 64.7 (#107/269)
- AA IFBench: 55.1 (#122/411)
- AA LiveCodeBench: 54.1 (#137/343)
- AA MMLU-Pro: 76.1 (#165/345)
- AA Terminal-Bench Hard: 8.3 (#220/397)
- AA Omniscience: -49.97 (#222/388)
- AA GDPval: 578.73 (#227/360)
- AA Long Context Reasoning: 28.0 (#232/411)
- AA CritPt: 0.0 (#251/388)
Granite 4.1 30B — ELO 1491, #425
- AA IFBench: 44.4 (#191/411)
- AA TAU-2 Bench: 42.1 (#198/402)
- AA CritPt: 0.0 (#228/388)
- AA GDPval: 495.5 (#253/360)
- AA Long Context Reasoning: 18.7 (#273/411)
- AA Terminal-Bench Hard: 2.3 (#310/397)
- AA SciCode: 25.8 (#315/477)
- Artificial Analysis Intelligence Index: 14.69 (#324/482)
- AA Omniscience: -67.78 (#342/388)
- AA GPQA Diamond: 48.1 (#354/483)
K-EXAONE (Non-reasoning) — ELO 1487, #432
- AA MMLU-Pro: 81.0 (#94/345)
- AA Global-MMLU-Lite: 71.03 (#104/120)
- AA AIME 2025: 44.0 (#150/269)
- AA Long Context Reasoning: 47.0 (#157/411)
- AA TAU-2 Bench: 59.1 (#162/402)
- AA GDPval: 767.0 (#174/360)
- Artificial Analysis Intelligence Index: 23.41 (#207/482)
- AA GPQA Diamond: 69.5 (#209/483)
- AA Terminal-Bench Hard: 6.8 (#239/397)
- AA CritPt: 0.0 (#242/388)
K2-V2 (Low) — ELO 1483, #444
- AA Global-MMLU-Lite: 71.44 (#103/120)
- AA AIME 2025: 35.3 (#173/269)
- AA LiveCodeBench: 39.3 (#187/343)
- AA MMLU-Pro: 71.3 (#212/345)
- AA Omniscience: -48.07 (#212/388)
- AA IFBench: 41.0 (#233/411)
- AA CritPt: 0.0 (#254/388)
- AA Long Context Reasoning: 19.0 (#271/411)
- AA Terminal-Bench Hard: 4.5 (#277/397)
- AA GDPval: 367.48 (#285/360)
Solar Pro 2 (Reasoning) — ELO 1479, #450
- AA MATH-500: 96.7 (#30/193)
- AA Global-MMLU-Lite: 79.61 (#78/120)
- AA MMLU-Pro: 80.5 (#107/345)
- AA LiveCodeBench: 61.6 (#113/343)
- AA AIME 2025: 61.3 (#115/269)
- AA CritPt: 0.0 (#206/388)
- AA Humanity's Last Exam: 7.0 (#213/479)
- AA GPQA Diamond: 68.7 (#215/483)
- AA SciCode: 30.2 (#246/477)
- AA TAU-2 Bench: 28.1 (#251/402)
Gemma 4 E4B (Reasoning) — ELO 1474, #458
- AA Omniscience: -20.05 (#82/388)
- AA CritPt: 0.6 (#104/388)
- AA MMMU-Pro: 51.4 (#143/188)
- AA IFBench: 44.2 (#193/411)
- AA Terminal-Bench Hard: 8.3 (#218/397)
- AA Long Context Reasoning: 30.7 (#222/411)
- Artificial Analysis Intelligence Index: 18.76 (#250/482)
- AA GPQA Diamond: 57.6 (#297/483)
- AA GDPval: 304.3 (#312/360)
- AA TAU-2 Bench: 20.8 (#314/402)
EXAONE 4.0 32B (Reasoning) — ELO 1473, #461
- AA MATH-500: 97.7 (#21/193)
- AA LiveCodeBench: 74.7 (#48/343)
- AA AIME 2025: 80.0 (#68/269)
- AA MMLU-Pro: 81.8 (#82/345)
- AA Global-MMLU-Lite: 73.46 (#97/120)
- AA Humanity's Last Exam: 10.5 (#145/479)
- AA GPQA Diamond: 73.9 (#167/483)
- AA SciCode: 34.4 (#203/477)
- AA CritPt: 0.0 (#240/388)
- AA GDPval: 499.86 (#249/360)
Tri-21B-Think Preview — ELO 1473, #462
- AA TAU-2 Bench: 93.3 (#38/402)
- AA IFBench: 47.1 (#169/411)
- Artificial Analysis Intelligence Index: 19.99 (#236/482)
- AA Humanity's Last Exam: 5.7 (#257/479)
- AA CritPt: 0.0 (#259/388)
- AA Omniscience: -55.28 (#267/388)
- AA Long Context Reasoning: 14.7 (#294/411)
- AA GDPval: 337.02 (#299/360)
- AA Terminal-Bench Hard: 2.3 (#315/397)
- AA GPQA Diamond: 53.8 (#320/483)
Tri-21B-Think — ELO 1468, #468
- AA TAU-2 Bench: 81.0 (#103/402)
- AA IFBench: 54.6 (#124/411)
- AA CritPt: 0.3 (#132/388)
- AA Humanity's Last Exam: 6.1 (#241/479)
- Artificial Analysis Intelligence Index: 18.62 (#258/482)
- AA GPQA Diamond: 60.1 (#279/483)
- AA GDPval: 374.11 (#282/360)
- AA Long Context Reasoning: 11.0 (#312/411)
- AA Omniscience: -63.3 (#321/388)
- AA Terminal-Bench Hard: 0.8 (#342/397)
GPT-4o (March 2025, chatgpt-4o-latest) — ELO 1449, #500
- AA MATH-500: 89.3 (#73/193)
- AA MMLU-Pro: 80.3 (#110/345)
- AA SciCode: 36.6 (#165/477)
- AA LiveCodeBench: 42.5 (#170/343)
- AA AIME 2025: 25.7 (#196/269)
- AA GPQA Diamond: 65.5 (#247/483)
- Artificial Analysis Intelligence Index: 18.56 (#260/482)
- AA Humanity's Last Exam: 5.0 (#305/479)
Llama 3.3 Nemotron Super 49B v1 (Reasoning) — ELO 1448, #502
- AA MATH-500: 95.9 (#36/193)
- AA AIME 2025: 54.7 (#132/269)
- AA MMLU-Pro: 78.5 (#136/345)
- AA CritPt: 0.0 (#215/388)
- AA Humanity's Last Exam: 6.5 (#227/479)
- AA LiveCodeBench: 27.7 (#238/343)
- AA GPQA Diamond: 64.3 (#251/483)
- Artificial Analysis Intelligence Index: 18.49 (#262/482)
- AA TAU-2 Bench: 26.9 (#262/402)
- AA IFBench: 38.1 (#262/411)
Solar Pro 2 (Non-reasoning) — ELO 1435, #524
- AA MATH-500: 88.9 (#76/193)
- AA Global-MMLU-Lite: 75.34 (#91/120)
- AA LiveCodeBench: 42.4 (#172/343)
- AA MMLU-Pro: 75.0 (#178/345)
- AA AIME 2025: 30.0 (#186/269)
- AA CritPt: 0.0 (#203/388)
- AA TAU-2 Bench: 31.9 (#230/402)
- AA GDPval: 447.04 (#265/360)
- AA Terminal-Bench Hard: 4.5 (#273/397)
- AA IFBench: 33.7 (#306/411)
Llama 3.3 Nemotron Super 49B v1 (Non-reasoning) — ELO 1408, #560
- AA MATH-500: 77.5 (#113/193)
- AA CritPt: 0.0 (#216/388)
- AA Omniscience: -49.68 (#219/388)
- AA MMLU-Pro: 69.8 (#221/345)
- AA LiveCodeBench: 28.0 (#235/343)
- AA AIME 2025: 7.7 (#237/269)
- AA IFBench: 39.5 (#247/411)
- AA Long Context Reasoning: 11.3 (#309/411)
- AA GPQA Diamond: 51.7 (#330/483)
- Artificial Analysis Intelligence Index: 14.35 (#336/482)
NVIDIA Nemotron 3 Nano 4B — ELO 1388, #586
- AA IFBench: 58.2 (#106/411)
- AA CritPt: 0.0 (#211/388)
- AA Terminal-Bench Hard: 6.8 (#238/397)
- AA TAU-2 Bench: 28.1 (#252/402)
- AA GDPval: 476.83 (#258/360)
- AA Long Context Reasoning: 16.7 (#286/411)
- AA Humanity's Last Exam: 4.8 (#323/479)
- Artificial Analysis Intelligence Index: 14.68 (#325/482)
- AA GPQA Diamond: 51.3 (#338/483)
- AA Omniscience: -71.53 (#351/388)
Granite 4.1 3B — ELO 1380, #595
- AA CritPt: 0.0 (#232/388)
- AA GDPval: 366.32 (#286/360)
- AA IFBench: 33.7 (#307/411)
- AA Terminal-Bench Hard: 2.3 (#312/397)
- AA TAU-2 Bench: 19.6 (#323/402)
- AA Long Context Reasoning: 3.0 (#341/411)
- AA Omniscience: -77.38 (#370/388)
- AA SciCode: 11.9 (#412/477)
- Artificial Analysis Intelligence Index: 8.54 (#435/482)
- AA GPQA Diamond: 31.4 (#441/483)
Gemma 4 E2B (Reasoning) — ELO 1376, #604
- AA Omniscience: -23.98 (#94/388)
- AA MMMU-Pro: 44.6 (#160/188)
- AA CritPt: 0.0 (#170/388)
- AA IFBench: 38.0 (#265/411)
- AA Long Context Reasoning: 15.0 (#292/411)
- AA Terminal-Bench Hard: 3.0 (#299/397)
- Artificial Analysis Intelligence Index: 15.21 (#309/482)
- AA TAU-2 Bench: 20.8 (#315/402)
- AA Humanity's Last Exam: 4.8 (#322/479)
- AA GDPval: 272.59 (#338/360)
Llama 3.1 Nemotron Nano 4B v1.1 (Reasoning) — ELO 1351, #630
- AA MATH-500: 94.7 (#41/193)
- AA AIME 2025: 50.0 (#140/269)
- AA LiveCodeBench: 49.3 (#153/343)
- AA MMLU-Pro: 55.6 (#283/345)
- AA Humanity's Last Exam: 5.1 (#289/479)
- Artificial Analysis Intelligence Index: 14.43 (#334/482)
- AA Long Context Reasoning: 0.0 (#358/411)
- AA TAU-2 Bench: 11.7 (#362/402)
- AA IFBench: 25.5 (#375/411)
- AA GPQA Diamond: 40.8 (#393/483)
Ling-mini-2.0 — ELO 1346, #635
- AA AIME 2025: 49.3 (#142/269)
- AA LiveCodeBench: 42.9 (#169/343)
- AA MMLU-Pro: 67.1 (#243/345)
- AA CritPt: 0.0 (#284/388)
- AA Humanity's Last Exam: 5.0 (#304/479)
- AA GPQA Diamond: 56.2 (#306/483)
- AA Long Context Reasoning: 6.7 (#329/411)
- AA GDPval: 264.15 (#341/360)
- AA Terminal-Bench Hard: 0.8 (#345/397)
- AA TAU-2 Bench: 13.2 (#356/402)
Jamba Reasoning 3B — ELO 1320, #657
- AA IFBench: 52.4 (#137/411)
- AA AIME 2025: 10.7 (#231/269)
- AA LiveCodeBench: 21.0 (#267/343)
- AA CritPt: 0.0 (#268/388)
- AA MMLU-Pro: 57.7 (#274/345)
- AA Long Context Reasoning: 7.0 (#323/411)
- AA TAU-2 Bench: 15.8 (#342/402)
- AA Terminal-Bench Hard: 0.8 (#344/397)
- AA GDPval: 257.67 (#345/360)
- AA Humanity's Last Exam: 4.6 (#347/479)
Exaone 4.0 1.2B (Reasoning) — ELO 1266, #696
- AA AIME 2025: 50.3 (#139/269)
- AA LiveCodeBench: 51.6 (#143/343)
- AA CritPt: 0.0 (#241/388)
- AA Humanity's Last Exam: 5.8 (#251/479)
- AA MMLU-Pro: 58.8 (#268/345)
- AA GDPval: 296.88 (#317/360)
- AA GPQA Diamond: 51.5 (#336/483)
- AA TAU-2 Bench: 16.4 (#338/402)
- AA Long Context Reasoning: 0.0 (#370/411)
- AA Terminal-Bench Hard: 0.0 (#377/397)
Exaone 4.0 1.2B (Non-reasoning) — ELO 1262, #697
- AA AIME 2025: 24.0 (#200/269)
- AA LiveCodeBench: 29.3 (#226/343)
- AA CritPt: 0.0 (#239/388)
- AA Humanity's Last Exam: 5.8 (#250/479)
- AA MMLU-Pro: 50.0 (#294/345)
- AA GDPval: 298.76 (#316/360)
- AA TAU-2 Bench: 20.5 (#318/402)
- AA Long Context Reasoning: 0.0 (#369/411)
- AA Terminal-Bench Hard: 0.0 (#376/397)
- AA IFBench: 25.3 (#376/411)
Granite 4.0 1B — ELO 1258, #701
- AA CritPt: 0.0 (#234/388)
- AA AIME 2025: 6.3 (#244/269)
- AA Humanity's Last Exam: 5.1 (#292/479)
- AA TAU-2 Bench: 22.8 (#294/402)
- AA MMLU-Pro: 32.5 (#331/345)
- AA LiveCodeBench: 4.7 (#333/343)
- AA Long Context Reasoning: 4.0 (#340/411)
- AA GDPval: 259.61 (#342/360)
- AA Terminal-Bench Hard: 0.0 (#373/397)
- AA Omniscience: -81.82 (#377/388)
Granite 4.0 H 350M — ELO 1137, #759
- AA CritPt: 0.0 (#227/388)
- AA Humanity's Last Exam: 6.4 (#228/479)
- AA AIME 2025: 1.3 (#262/269)
- AA GDPval: 294.09 (#319/360)
- AA LiveCodeBench: 1.9 (#339/343)
- AA MMLU-Pro: 12.7 (#343/345)
- AA TAU-2 Bench: 14.6 (#349/402)
- AA Long Context Reasoning: 0.0 (#366/411)
- AA Terminal-Bench Hard: 0.0 (#369/397)
- AA Omniscience: -87.25 (#387/388)
OLMo 2 32B — ELO 1037, #780
- AA AIME 2025: 3.3 (#256/269)
- AA IFBench: 38.1 (#264/411)
- AA MMLU-Pro: 51.1 (#292/345)
- AA LiveCodeBench: 6.8 (#328/343)
- AA Terminal-Bench Hard: 0.0 (#391/397)
- AA Long Context Reasoning: 0.0 (#393/411)
- Artificial Analysis Intelligence Index: 10.57 (#397/482)
- AA TAU-2 Bench: 0.0 (#401/402)
- AA GPQA Diamond: 32.8 (#429/483)
- AA SciCode: 8.0 (#437/477)
Phi-3 Mini Instruct 3.8B — ELO 1025, #781
- AA MATH-500: 45.7 (#172/193)
- AA AIME 2025: 0.3 (#265/269)
- AA MMLU-Pro: 43.5 (#308/345)
- AA LiveCodeBench: 11.6 (#308/343)
- AA Long Context Reasoning: 2.0 (#345/411)
- AA Humanity's Last Exam: 4.4 (#372/479)
- AA IFBench: 23.9 (#382/411)
- AA Terminal-Bench Hard: 0.0 (#388/397)
- AA TAU-2 Bench: 0.0 (#398/402)
- Artificial Analysis Intelligence Index: 10.1 (#407/482)
OLMo 2 7B — ELO 958, #787
- AA AIME 2025: 0.7 (#263/269)
- AA Humanity's Last Exam: 5.5 (#265/479)
- AA MMLU-Pro: 28.2 (#334/345)
- AA LiveCodeBench: 4.1 (#335/343)
- AA IFBench: 24.4 (#381/411)
- AA Terminal-Bench Hard: 0.0 (#390/397)
- AA Long Context Reasoning: 0.0 (#391/411)
- AA TAU-2 Bench: 0.0 (#399/402)
- Artificial Analysis Intelligence Index: 9.3 (#423/482)
- AA GPQA Diamond: 28.8 (#455/483)

Top-10 New Scores (7)

Claude Mythos Preview on METR Benchmark: 17.41 (#1)
GPT-5.4 (xHigh) on OpenClawProBench: 68.0 (#8)
GPT-5.5 (xHigh) on OpenClawProBench: 69.3 (#4)
GPT-5.5 (xHigh) on Wolfram LLM Benchmarking Project: 68.8 (#6)
GPT-5.5 Pro on Epoch AI - ECI: 159.5 (#3)
GPT-5.5 Pro on PinchBench: 18.11 (#39)
GPT-5.5 Pro on VoxelBench: 2107.0 (#1)

New #1 Leaders (14)

FoodTruckBench: GPT-5.5 (61408.0) beat Claude Opus 4.6 by 11889.0
LIBRA - ruBABILongQA2: Qwen_Qwen3-30B-A3B-Instruct-2507 (64.72) beat GPT-4o by 28.05
LIBRA - ruQuALITY: 01-ai_Yi-9B-200K (95.9) beat GPT-4o by 12.57
SEAL - AudioMultiChallenge - Audio Output: gpt-realtime-2 (xHigh) (48.45) beat gemini-3.1-flash-live-preview (Thinking) by 12.39
FrontierSWE: GPT-5.5 (83.0) beat Claude Opus 4.7 by 9.0
FrontierMath - Tier 4: AI co-mathematician (47.9) beat GPT-5.5 Pro (xhigh) by 8.3
Story Theory Bench: glm-5 (99.6) beat deepseek-v3.2 by 7.4
Kaggle FACTS Parametric: Gemini 3.1 Pro Preview (78.96) beat Gemini 3 Flash Preview by 6.7
SEAL - SWE Atlas - Codebase QnA: GPT 5.5 (Codex) (45.43) beat Gpt 5.4 xHigh (Codex) by 4.63
LIBRA - ruSciAbstractRetrieval: Qwen_Qwen3-30B-A3B-Instruct-2507 (81.5) beat GLM-4 9B Chat by 3.69
Kaggle FACTS (Google): GPT-5.5 (71.19) beat Gemini 3.1 Pro Preview by 3.48
LIBRA - ruBABILongQA1: Qwen_Qwen3-30B-A3B-Instruct-2507 (80.5) beat GPT-4o by 2.17
Android Bench: GPT 5.5 (74.0) beat GPT-5.4 by 1.6
ForecastBench: green tree (68.2) beat Cassi ensemble_2_crowdadj by 0.4

AI Benchmark Digest — 2026-05-09

2026-05-09T07:40:39.118338+00:00

Daily

New Benchmarks (8)

Factory Code Review Benchmark (Mean F1 (%)): leader GPT-5.2 (60.5), 13 models
Factory benchmark for code review quality, scoring model comments against expected findings with mean F1 across realistic pull request review tasks.
EuroEval Albanian NLU (NLU Average Score (%)): leader gemini-3.1-pro-preview (61.17), 208 models
Albanian-language EuroEval natural-language-understanding suite, separating NLU task performance from the broader all-task EuroEval aggregate.
EuroEval Bosnian NLU (NLU Average Score (%)): leader Ministral-3-14B-Reasoning-2512 (66.0), 214 models
Bosnian-language EuroEval natural-language-understanding suite, separating NLU task performance from the broader all-task EuroEval aggregate.
EuroEval Albanian Knowledge (Knowledge Average Score (%)): leader gemini-3-flash-preview#thinking (96.46), 167 models
EuroEval Albanian knowledge category: language-specific factual or domain-knowledge tasks from EuroEval's public albanian_all.csv leaderboard, scored as the average task score for each model.
EuroEval Albanian Common Sense Reasoning (Common Sense Reasoning Average Score (%)): leader gemini-3.1-pro-preview (85.24), 155 models
EuroEval Albanian common-sense reasoning category: language-specific commonsense tasks from EuroEval's public albanian_all.csv leaderboard, scored as the average task score for each model.
IMO-Bench (Advanced ProofBench Accuracy (%)): leader Aletheia (91.9), 9 models
Advanced IMO-ProofBench leaderboard for rigorous mathematical proof writing on olympiad-level problems.
ChartMuseum (Overall Accuracy (%)): leader Gemini-3.1-Pro (80.7), 22 models
Chart question-answering benchmark over real-world charts, testing visual, textual, and synthesis reasoning.
SvelteBench (Average pass@1 (%)): leader claude-opus-4-6 (100.0), 123 models
Frontend coding benchmark for Svelte component tasks, scored by average pass@1.

New Models (1)

Grok 4.3 (Non-reasoning) — ELO 1647, #259
- AA GDPval: 1306.14 (#52/360)
- AA MMMU-Pro: 64.8 (#88/188)
- AA Omniscience: -32.3 (#121/388)
- Artificial Analysis Intelligence Index: 31.02 (#139/482)
- AA SciCode: 37.4 (#146/477)
- AA TAU-2 Bench: 65.8 (#148/402)
- AA Terminal-Bench Hard: 18.9 (#149/397)
- AA IFBench: 47.6 (#165/411)
- AA CritPt: 0.0 (#182/388)
- AA Humanity's Last Exam: 6.5 (#226/479)

Top-10 New Scores (1)

GPT-5.5 (xHigh) on Wolfram LLM Benchmarking Project: 68.8 (#6)

New #1 Leaders (4)

FrontierMath - Tier 4: AI co-mathematician (47.9) beat GPT-5.5 Pro (xhigh) by 8.3
METR Benchmark: claude mythos preview early (17.41) beat claude opus 4 6 by 5.43
METR Benchmark (80% Horizon): claude mythos preview early (3.1) beat gemini 3 1 pro by 1.6
ForecastBench: green tree (68.2) beat Cassi ensemble_2_crowdadj by 0.4

AI Benchmark Digest — 2026-05-08

2026-05-08T07:40:34.661988+00:00

Daily

New Benchmarks (8)

EuroEval Albanian NLU (NLU Average Score (%)): leader gemini-3.1-pro-preview (61.17), 208 models
EuroEval Bosnian NLU (NLU Average Score (%)): leader Ministral-3-14B-Reasoning-2512 (66.0), 214 models
EuroEval Albanian Knowledge (Knowledge Average Score (%)): leader gemini-3-flash-preview#thinking (96.46), 167 models
EuroEval Albanian Common Sense Reasoning (Common Sense Reasoning Average Score (%)): leader gemini-3.1-pro-preview (85.24), 155 models
MoNaCo (F1): leader o3 (61.18), 15 models
IMO-Bench (Advanced ProofBench Accuracy (%)): leader Aletheia (91.9), 9 models
ChartMuseum (Overall Accuracy (%)): leader Gemini-3.1-Pro (80.7), 22 models
SvelteBench (Average pass@1 (%)): leader claude-opus-4-6 (100.0), 123 models

New #1 Leaders (3)

SEAL - AudioMultiChallenge - Audio Output: gpt-realtime-2 (xHigh) (48.45) beat gemini-3.1-flash-live-preview (Thinking) by 12.39
Story Theory Bench: glm-5 (99.6) beat deepseek-v3.2 by 7.4
SEAL - SWE Atlas - Codebase QnA: GPT 5.5 (Codex) (45.43) beat Gpt 5.4 xHigh (Codex) by 4.63

AI Benchmark Digest — 2026-05-07

2026-05-07T07:40:24.104745+00:00

Daily

New Benchmarks (19)

LIBRA - MatreshkaNames * (Dataset Total Score (%)): leader Qwen_Qwen3-30B-A3B-Instruct-2507 (81.2), 7 models
LIBRA - ruSciPassageCount * (Dataset Total Score (%)): leader Qwen_Qwen3-30B-A3B-Instruct-2507 (25.77), 7 models
LIBRA - ru2WikiMultihopQA * (Dataset Total Score (%)): leader Qwen_Qwen3-30B-A3B-Instruct-2507 (66.63), 7 models
LIBRA - LongContextMultiQ * (Dataset Total Score (%)): leader 01-ai_Yi-34B-200K (53.14), 7 models
LIBRA - LibrusecMHQA * (Dataset Total Score (%)): leader Qwen_Qwen3-30B-A3B-Instruct-2507 (51.0), 7 models
LIBRA - ruBABILongQA3 * (Dataset Total Score (%)): leader Qwen_Qwen3-30B-A3B-Instruct-2507 (38.38), 7 models
Kernel Arena - KernelBench HIP (Mean Correctness+Speedup): leader GPT-5.2 (15.463), 11 models
Kernel Arena - WaferBench NVFP4 (Mean Correctness+Speedup): leader Gemini 3.1 Pro (2.274), 4 models
MathArena - ARXIV_FALSE April (Accuracy (%)): leader GPT-5.5 (xhigh) (72.13), 6 models
MathArena - ARXIV April (Accuracy (%)): leader GPT-5.5 (xhigh) (65.48), 6 models
METR Benchmark (80% Horizon) (80% Time Horizon (hours)): leader gemini 3 1 pro (1.5), 25 models
LLM Stats (HealthBench) (Score (%)): leader Kimi K2-Thinking-0905 (58.0), 5 models
SCORE Robustness (Accuracy) (Average Accuracy (%)): leader Llama-3.1-70B-Instruct (67.02), 6 models
SCORE Robustness (Consistency) (Average Consistency Rate (%)): leader Llama-3.1-70B-Instruct (72.39), 6 models
Multilingual MMLU Leaderboard (Average Accuracy (%)): leader Claude-3.5-Sonnet (77.39), 17 models
Pinocchio Italian Leaderboard (Average Accuracy (%)): leader gemma-2-27b-it (70.97), 45 models
Ukrainian LLM Leaderboard (Average Score (%)): leader gemma-4-26B-A4B-it (reasoning) (63.29), 13 models
Arabic Broad Leaderboard (Average Score (0-10)): leader gemini-3-pro-preview (9.204), 87 models
Darija Chatbot Arena (Elo Rating): leader GPT-4o (1404.8), 13 models

New #1 Leaders (3)

FoodTruckBench: GPT-5.5 (61408.0) beat Claude Opus 4.6 by 11889.0
ASCIIBench: claude-opus-4.5 (1656.0) beat claude-opus-4.1 by 5.0
Kaggle FACTS Parametric: Gemini 3.1 Pro Preview (78.96) beat GPT-5.5 by 0.92