AI Benchmark Digest

AI Benchmark Digest — 2026-06-04

2026-06-04T08:22:19.073162+00:00

Daily

New #1 Leaders (1)

GAIA: CustomGPT.ai Research Lab v44 (93.36) beat Co-Sight Pro v1.0.1 by 0.34

AI Benchmark Digest — 2026-06-03

2026-06-03T08:25:40.519214+00:00

Daily

Top-10 New Scores (2)

GPT-5.5 Pro on IUMB: 100.0 (#2)
Gemini 3 Deep Think on IUMB: 87.5 (#6)

New #1 Leaders (4)

MathArena - Kangaroo 2025 Levels 11-12: Claude Opus 4.8 (Thinking) (100.0) beat GPT-5.4 (xHigh) by 1.67
MathArena - APEX 2025: Claude Opus 4.8 (Thinking) (81.25) beat GPT-5.5 (xHigh) by 1.04
MathArena - Kangaroo 2025 Levels 7-8: Claude Opus 4.8 (Thinking) (96.67) beat GPT-5.4 (xHigh) by 0.84
MathArena - AIME 2026: Claude Opus 4.8 (Thinking) (100.0) beat GPT-5.4 (xHigh) by 0.83

AI Benchmark Digest — 2026-06-02

2026-06-02T08:19:29.198019+00:00

Daily

New Benchmarks (1)

GIM (IRT ability (theta)): leader GPT-5.4 Pro (High) (2.16), 46 models
Grounded Integration Measure from Meta FAIR: 820 multimodal and text-grounded problems testing integrated reasoning across quantitative, spatial, language, world-knowledge, and document tasks. Scores are reported as IRT ability on GIM-820.

Top-10 New Scores (2)

GPT-5.5 (xHigh) on IMO-Bench: 71.9 (#4)
GPT-5.5 Pro (xHigh) on IMO-Bench: 88.1 (#2)

AI Benchmark Digest — 2026-06-01

2026-06-01T08:29:45.265204+00:00

Daily

New #1 Leaders (3)

EQ-Bench Creative Writing v3: Claude Opus 4.7 (2050.8) beat GPT-5.4 by 144.8
Design Arena (Data Viz): GLM-5.1 (1367.0) beat Claude Opus 4.7 (Thinking) by 23.0
Chatbot Arena (Image-to-Video): Grok 1.5 (1473.0) beat dreamina-seedance-2.0-720p by 11.0

AI Benchmark Digest — 2026-05-30

2026-05-30T07:49:09.779753+00:00

Daily

Top-10 New Scores (5)

Claude Opus 4.8 (Adaptive Reasoning, Max Effort) on UGI - Natural Intelligence: 65.39 (#30)
Claude Opus 4.8 (Adaptive Reasoning, Max Effort) on UGI - Willingness (W/10): 2.2 (#1094)
Claude Opus 4.8 (Adaptive Reasoning, Max Effort) on UGI - Writing: 65.88 (#34)
Claude Opus 4.8 (Adaptive Reasoning, Max Effort) on UGI Leaderboard: 52.64 (#69)
GPT-5.4 (xHigh) on Creative Writing (Lechmazur): 3.4 (#2)

New #1 Leaders (2)

Bullshit Benchmark: Claude Opus 4.8 (96.4) beat Claude Sonnet 4.6 by 1.9
Creative Writing (Lechmazur): GPT-5.5 (xHigh) (3.5) beat GPT-5.5 (Thinking, xHigh) by 0.3

AI Benchmark Digest — 2026-05-29

2026-05-29T08:06:41.324282+00:00

Daily

New Benchmarks (1)

DeepSWE (Pass@1 (%)): leader GPT-5.5 (xHigh) (70.0), 12 models
DataCurve benchmark measuring frontier coding agents on original, long-horizon software engineering tasks. Reports pass rates for model configurations on realistic repository work.

New Models (1)

Claude Opus 4.8 — ELO 1801, #52
- Clerk LLM Leaderboard: 91.3 (#1/19)
- Vellum - HumanEval: 88.6 (#1/36)
- Vellum - Humanity's Last Exam: 57.9 (#1/20)
- LLM Stats (DeepSearchQA): 93.1 (#1/6)
- LLM Stats (Include): 87.6 (#1/30)
- LLM Stats (OSWorld-Verified): 83.4 (#1/14)
- LLM Stats (ScreenSpot Pro): 87.9 (#1/22)
- LLM Stats (Toolathlon): 59.9 (#1/20)
- FrontierSWE: 83.0 (#1/11)
- Vals AI (Vals Index): 70.17 (#1/20)

Top-10 New Scores (2)

GPT-5.5 (High) on WebDev Arena: 1478.93 (#16)
GPT-5.5 (xHigh) on WebDev Arena: 1504.74 (#12)

New #1 Leaders (16)

AA GDPval: Claude Opus 4.8 (Adaptive Reasoning, Max Effort) (1889.8) beat GPT-5.5 (xHigh) by 120.5
Vellum - Humanity's Last Exam: Claude Opus 4.8 (57.9) beat Gemini 3 Pro by 12.1
Clerk LLM Leaderboard: Claude Opus 4.8 (91.3) beat GPT-5.4 by 11.8
Vals AI Vibe Code Bench: Claude Opus 4.8 (82.72) beat Claude Opus 4.7 by 11.72
Epoch AI - Apex Agents: gemini-3.5-flash_unknown (49.6) beat GPT-5.5 (xHigh) by 11.2
LLM Stats (OSWorld-Verified): Claude Opus 4.8 (83.4) beat Claude Mythos Preview by 3.8
LLM Stats (Toolathlon): Claude Opus 4.8 (59.9) beat Gemini 3.5 Flash by 3.4
Vals AI Multimodal Index: Claude Opus 4.8 (70.71) beat GPT-5.5 by 2.94
Vals AI (Vals Index): Claude Opus 4.8 (70.17) beat GPT-5.5 by 2.55
LLM Stats (DeepSearchQA): Claude Opus 4.8 (93.1) beat Claude Opus 4.6 by 1.8
LLM Stats (ScreenSpot Pro): Claude Opus 4.8 (87.9) beat GPT-5.2 by 1.6
LLM Stats (Include): Claude Opus 4.8 (87.6) beat Qwen 3.7 Max by 1.4
Artificial Analysis Intelligence Index: Claude Opus 4.8 (Adaptive Reasoning, Max Effort) (61.44) beat GPT-5.5 (xHigh) by 1.2
PinchBench: Claude Opus 4.8 Fast (94.49) beat Qwen Max by 1.05
AA Humanity's Last Exam: Claude Opus 4.8 (Adaptive Reasoning, Max Effort) (45.74) beat Gemini 3.1 Pro (Preview) by 1.02
Vellum - HumanEval: Claude Opus 4.8 (88.6) beat Claude Opus 4.7 by 1.0

AI Benchmark Digest — 2026-05-28

2026-05-28T08:13:42.023730+00:00

Daily

Top-10 New Scores (1)

GPT-5.5 (xHigh) on SWE-rebench: 62.73 (#1)

New #1 Leaders (2)

Kaggle FACTS Grounding: Gemma 4 26B A4B (80.87) beat GPT-5.2 by 4.7
PinchBench: Qwen Max (93.44) beat Grok 0.1 by 1.37

AI Benchmark Digest — 2026-05-27

2026-05-27T08:20:58.056719+00:00

Daily

Top-10 New Scores (1)

GPT-5.4 (xHigh) on Creative Writing (Lechmazur): 3.2 (#2)

New #1 Leaders (11)

LLM Chess (Saplin): GPT-5.5 (Medium) (1532.2) beat Gemini 3.1 Pro by 20.8
LLM Stats (PolyMATH): Qwen 3.7 Max (86.5) beat Qwen 3.6 Plus by 9.1
LLM Stats (MCP-Mark): Qwen 3.7 Max (60.8) beat Kimi K2.6 by 4.9
LLM Stats (NL2Repo): Qwen 3.7 Max (47.2) beat GLM-5.1 by 4.5
LLM Stats (MMLU-ProX): Qwen 3.7 Max (87.0) beat Qwen 3.6 Plus by 2.3
LLM Stats (HMMT Feb 26): Qwen 3.7 Max (97.1) beat DeepSeek V4 Pro (Max) by 1.9
LLM Stats (MAXIFE): Qwen 3.7 Max (89.2) beat Qwen 3.6 Plus by 1.0
LLM Stats (Include): Qwen 3.7 Max (86.2) beat Qwen 3.5 397B A17B by 0.6
LLM Stats (IMO-AnswerBench): Qwen 3.7 Max (90.0) beat DeepSeek V4 Pro (Max) by 0.2
Creative Writing (Lechmazur): GPT-5.5 (Thinking, xHigh) (3.2) beat GPT-5.5 by 0.2
LLM Stats (MMLU-Redux): Qwen 3.7 Max (95.0) beat Qwen 3.5 397B A17B by 0.1

AI Benchmark Digest — 2026-05-25

2026-05-25T08:26:35.093083+00:00

Daily

New Benchmarks (6)

LLMEval-Logic Base (Accuracy (%)): leader Seed 2.0 Pro (Thinking) (75.5), 14 models
LLMEval-Logic Hard (Accuracy (%)): leader Gemini 3.1 Pro (Thinking) (37.5), 14 models
LLMEval-Logic Hard Sub-Q (Accuracy (%)): leader Claude Opus 4.6 (Thinking) (76.6), 14 models
LLMEval-Logic Formalization Free (Accuracy (%)): leader Gemini 3.1 Pro (Thinking) (45.1), 14 models
LLMEval-Logic Formalization Fixed (Accuracy (%)): leader GPT-5.4 Pro (No-Think) (60.2), 14 models
ExploitBench v8-bench (Mean Capability (%)): leader Claude Mythos Preview (69.0), 9 models
V8 exploitation ladder benchmark measuring how far AI agents climb from code reachability through crash reproduction, exploit primitives, and arbitrary code execution. Reports mean capability across 41 V8 bug environments.

AI Benchmark Digest — 2026-05-24

2026-05-24T07:56:34.401567+00:00

Daily

New Benchmarks (14)

NanoGPT-Bench (% of Human Progress Recovered): leader Claude Opus 4.6 (9.3), 2 models
Autonomous research benchmark built on the NanoGPT Speedrun, measuring how much of five months of human pretraining-speedup progress coding agents recover under a fixed H100 compute budget.
CursorBench 3.1 (Score (%)): leader Claude Opus 4.7 (64.8), 7 models
Cursor benchmark of ambiguous, multi-file coding tasks from real Cursor sessions, with models scored by task success percentage and average cost per task.
SMDD-Bench (Pass Rate (%)): leader GPT-5.4 (Medium) (40.2), 7 models
Small molecule drug design agent benchmark with sandboxed Python, Boltz structure prediction, and ADMET tooling. Measures pass rate across 502 computationally verifiable chemistry tasks.
SMDD-Bench Diversity (Avg Successful): leader Claude Sonnet 4.6 (8.4), 7 models
SMDD-Bench diversity slice measuring whether agents generate multiple distinct, novel, successful molecule designs across repeated Lead Optimization rollouts.
Blueprint-Bench 2 (Connectivity Similarity Score): leader GPT 5.5 (0.362), 12 models
Andon Labs spatial reasoning benchmark where agents convert apartment photographs into 2D floor plans, scored by normalized connectivity similarity against ground truth layouts.
PACT (Lechmazur) (CMS Points): leader GPT-5.5 (high) (59.0), 25 models
Pairwise Auction Conversation Testbed for multi-round buyer-seller bargaining. LLMs negotiate over 20 rounds with hidden private values, scored by Composite Model Score from head-to-head surplus capture.
FormationEval (Accuracy (%)): leader gemini-3-pro-preview (99.8), 72 models
Chinese Classical Bench (Average Score (%)): leader claude-opus-4-7 (66.21), 10 models
Chinese Classical Bench - Translate Judge (Score (%)): leader claude-opus-4-7-thinking (80.2), 10 models
Chinese Classical Bench - Punctuate Punct F1 (Score (%)): leader claude-opus-4-7 (80.02), 10 models
Chinese Classical Bench - Char-Gloss Judge (Score (%)): leader claude-opus-4-7-thinking (73.6), 10 models
Chinese Classical Bench - Idiom-Source Book EM (Score (%)): leader deepseek-3.2 (74.0), 10 models
Chinese Classical Bench - Fill-In Exact (Score (%)): leader claude-opus-4-7-thinking (88.0), 10 models
Chinese Classical Bench - Compress Efficiency (Score (%)): leader deepseek-3.2 (16.32), 9 models

Top-10 New Scores (1)

Gemini 3.1 Pro (High) on CLBench: 20.8 (#8)

New #1 Leaders (5)

Evals for Every Language: Gemini 3.1 Pro (69.11) beat Gemini 2.5 Flash by 6.52
CLBench: GPT-5.4 (xHigh) (27.9) beat GPT-5.1 (High) by 4.2
LiveBench Logic With Navigation: Qwen Max (84.0) beat Claude Opus 4.6 (Thinking) by 4.0
Spider 2.0-Lite: DivSkill-SQL (73.13) beat SOMA-SQL by 1.11
PinchBench: Grok 0.1 (92.07) beat Claude Opus 4.7 by 0.49

AI Benchmark Digest — 2026-05-23

2026-05-23T07:20:10.541511+00:00

Daily

New #1 Leaders (1)

OSWorld: Opus 4.7 (83.64) beat Holo3-35B-A3B by 1.08

AI Benchmark Digest — 2026-05-22

2026-05-22T07:36:15.662013+00:00

Daily

Top-10 New Scores (1)

GPT-5.5 (High) on Sycophancy (Lechmazur): 3.5 (#11)

New #1 Leaders (2)

UGI - Writing: gemini-3.5-flash (thinking_level=medium) (72.54) beat gemini-3.1-pro-preview (thinking_level=low) by 0.39
Arabic Broad Leaderboard: gemini-3.5-flash (9.253) beat gemini-3-pro-preview by 0.05

AI Benchmark Digest — 2026-05-21

2026-05-21T07:40:34.045646+00:00

Daily

Top-10 New Scores (1)

Gemini 3.5 Flash (High) on WeirdML: 62.64 (#17)

New #1 Leaders (3)

Kaggle Game Arena Poker (Heads Up): GPT-5.5 (73.93) beat GPT-5.2 by 33.93
AA APEX-Agents: Gemini 3.5 Flash (high) (47.05) beat GPT-5.5 (xhigh) by 9.37
LA Leaderboard: Qwen2.5-14B-Instruct-GPTQ-Int8 (63.6) beat gemma-2-9b-it by 0.27

AI Benchmark Digest — 2026-05-20

2026-05-20T07:43:37.557151+00:00

Daily

New Models (1)

Gemini 3.5 Flash (High) — ELO 1942, #9
- AA MMMU-Pro: 84.28 (#1/190)
- SEAL - MCP Atlas: 83.6 (#1/21)
- AA Omniscience: 22.68 (#3/393)
- AA Omniscience - Law: 57.4 (#4/393)
- AA Omniscience - Software Engineering (SWE) - PHP: 84.0 (#4/393)
- AA Humanity's Last Exam: 40.96 (#5/484)
- AA GPQA Diamond: 92.22 (#6/488)
- AA Omniscience - Science, Engineering & Mathematics: 50.1 (#6/393)
- AA GDPval: 1655.7 (#7/365)
- AA Omniscience - Humanities & Social Sciences: 52.3 (#7/393)

Top-10 New Scores (34)

GPT-5.5 (High) on Multi-turn Debate (Lechmazur): 1583.6 (#5)
Gemini 3.5 Flash (High) on AA CritPt: 13.14 (#8)
Gemini 3.5 Flash (High) on AA GDPval: 1655.7 (#7)
Gemini 3.5 Flash (High) on AA GPQA Diamond: 92.22 (#6)
Gemini 3.5 Flash (High) on AA Humanity's Last Exam: 40.96 (#5)
Gemini 3.5 Flash (High) on AA IFBench: 76.33 (#17)
Gemini 3.5 Flash (High) on AA Long Context Reasoning: 69.33 (#27)
Gemini 3.5 Flash (High) on AA Omniscience: 22.68 (#3)
Gemini 3.5 Flash (High) on AA Omniscience - Business: 45.8 (#8)
Gemini 3.5 Flash (High) on AA Omniscience - Health: 40.2 (#14)
Gemini 3.5 Flash (High) on AA Omniscience - Humanities & Social Sciences: 52.3 (#7)
Gemini 3.5 Flash (High) on AA Omniscience - Law: 57.4 (#4)
Gemini 3.5 Flash (High) on AA Omniscience - Science, Engineering & Mathematics: 50.1 (#6)
Gemini 3.5 Flash (High) on AA Omniscience - Software Engineering (SWE): 65.5 (#16)
Gemini 3.5 Flash (High) on AA Omniscience - Software Engineering (SWE) - C: 80.0 (#18)
Gemini 3.5 Flash (High) on AA Omniscience - Software Engineering (SWE) - Dart: 60.0 (#14)
Gemini 3.5 Flash (High) on AA Omniscience - Software Engineering (SWE) - Go: 50.0 (#32)
Gemini 3.5 Flash (High) on AA Omniscience - Software Engineering (SWE) - HTML: 72.0 (#17)
Gemini 3.5 Flash (High) on AA Omniscience - Software Engineering (SWE) - Java: 51.0 (#16)
Gemini 3.5 Flash (High) on AA Omniscience - Software Engineering (SWE) - JavaScript: 71.82 (#14)
Gemini 3.5 Flash (High) on AA Omniscience - Software Engineering (SWE) - Julia: 60.0 (#13)
Gemini 3.5 Flash (High) on AA Omniscience - Software Engineering (SWE) - Kotlin: 56.0 (#22)
Gemini 3.5 Flash (High) on AA Omniscience - Software Engineering (SWE) - PHP: 84.0 (#4)
Gemini 3.5 Flash (High) on AA Omniscience - Software Engineering (SWE) - Python: 61.0 (#24)
Gemini 3.5 Flash (High) on AA Omniscience - Software Engineering (SWE) - R: 56.0 (#18)
Gemini 3.5 Flash (High) on AA Omniscience - Software Engineering (SWE) - Rust: 80.0 (#8)
Gemini 3.5 Flash (High) on AA Omniscience - Software Engineering (SWE) - Swift: 72.0 (#20)
Gemini 3.5 Flash (High) on AA Omniscience - Software Engineering (SWE) - TypeScript: 67.78 (#16)
Gemini 3.5 Flash (High) on AA SciCode: 53.12 (#11)
Gemini 3.5 Flash (High) on AA TAU-2 Bench: 95.32 (#20)
Gemini 3.5 Flash (High) on AA Terminal-Bench Hard: 40.91 (#36)
Gemini 3.5 Flash (High) on ARC-AGI-1: 92.5 (#16)
Gemini 3.5 Flash (High) on ARC-AGI-2: 72.08 (#12)
Gemini 3.5 Flash (High) on Artificial Analysis Intelligence Index: 55.33 (#8)

New #1 Leaders (5)

LLM Stats (GDPval-AA): Gemini 3.5 Flash (165600.0) beat Claude Sonnet 4.6 by 2300.0
LLM Stats (MCP Atlas): Gemini 3.5 Flash (83.6) beat Claude Opus 4.7 by 6.3
AA MMMU-Pro: Gemini 3.5 Flash (high) (84.28) beat Gemini 3.1 Pro Preview by 1.85
SEAL - MCP Atlas: gemini-3.5-flash (high) (83.6) beat Muse Spark by 1.4
LLM Stats (Toolathlon): Gemini 3.5 Flash (56.5) beat GPT-5.5 by 0.9

AI Benchmark Digest — 2026-05-17

2026-05-17T08:02:54.093472+00:00

Daily

New #1 Leaders (1)

OpenClawProBench: intern-s2-preview (76.7) beat Sensenova 6.7 Flash Lite by 3.0

Weekly

Top-10 New Scores (3)

Claude Opus 4.7 (Thinking) on SEAL Showdown: 1115.7 (#12)
Claude Opus 4.7 (Thinking) on WeirdML: 75.45 (#8)
GPT-5.5 (xHigh) on Chatbot Arena (Code): 1501.0 (#9)

New #1 Leaders (16)

MathArena - ARXIVLEAN March: AlephProver (34.15) beat Aristotle by 17.08
OpenCompass Reasoning - Common: Doubao-Seed-2-0-Pro-260215 (high) (82.1) beat Gemini-3-Pro-Preview by 8.5
OpenCompass Math - College: Doubao-Seed-2-0-Pro-260215 (high) (83.8) beat Kimi-K2.5 by 7.3
OpenClawProBench: intern-s2-preview (76.7) beat qwen3.5-397b-a17b by 6.3
Tau3-Bench Banking_Knowledge: GPT-5.5 (37.4) beat Distyl ButtonAgent by 6.2
OpenCompass Knowledge - Social Science: Gemini-3.1-Pro-Preview (97.5) beat Gemini-3-Pro-Preview by 4.3
OpenCompass LLM - Math: Doubao-Seed-2-0-Pro-260215 (high) (77.3) beat Qwen3-Max-2026-01-23 by 4.1
OpenCompass LLM - Reasoning: Doubao-Seed-2-0-Pro-260215 (high) (65.2) beat Gemini-3-Pro-Preview by 3.7
OpenCompass Math - Competition: Kimi-K2.6 (72.1) beat Qwen3-Max-2026-01-23 by 2.1
OpenCompass Reasoning - Academic: GPT-5.4-2026-03-05 (high) (52.0) beat GPT-5.2-2025-12-11 (high) by 1.5
VisuLogic: PEREA-1.0new (52.8) beat Human by 1.4
WeirdML: gpt-5.5 (xhigh) (84.91) beat gpt-5.5 (high) by 1.01
GAIA: Co-Sight Pro v1.0.1 (93.02) beat OPS-Agentic-Search by 0.66
OpenCompass Knowledge - Engineering: GPT-5.4-2026-03-05 (high) (96.2) beat Gemini-3-Pro-Preview by 0.4
AA TAU-2 Bench: JT-35B-Flash (99.12) beat GLM-4.7-Flash (Reasoning) by 0.32
AISI Cyber TLO 10M: GPT-5.5 (10.0) beat Claude Opus 4.6 by 0.2

AI Benchmark Digest — 2026-05-16

2026-05-16T07:15:27.727063+00:00

Daily

Top-10 New Scores (1)

GPT-5.5 (xHigh) on Chatbot Arena (Code): 1501.0 (#9)

New #1 Leaders (2)

MathArena - ARXIVLEAN March: AlephProver (34.15) beat Aristotle by 17.08
GAIA: Co-Sight Pro v1.0.1 (93.02) beat OPS-Agentic-Search by 0.66

AI Benchmark Digest — 2026-05-14

2026-05-14T07:26:43.169192+00:00

Daily

New Models (4)

Doubao-Seed-2-0-Pro-260215 (High) — ELO 1781, #73
- OpenCompass LLM - Reasoning: 65.2 (#1/23)
- OpenCompass LLM - Math: 77.3 (#1/23)
- OpenCompass Knowledge - Humanities: 95.0 (#1/23)
- OpenCompass Reasoning - Common: 82.1 (#1/23)
- OpenCompass Math - College: 83.8 (#1/23)
- OpenCompass LLM - Language: 77.3 (#3/23)
- OpenCompass Language - Creation: 77.1 (#3/23)
- OpenCompass Knowledge - Science: 94.6 (#3/23)
- OpenCompass LLM - Agent: 44.2 (#4/23)
- OpenCompass Language - NLP: 69.6 (#4/23)
Doubao-Seed-2-0-Lite-260215 (High) — ELO 1741, #103
- OpenCompass Reasoning - Common: 78.1 (#2/23)
- OpenCompass Language - Creation: 77.1 (#4/23)
- OpenCompass LLM - Language: 74.4 (#6/23)
- OpenCompass LLM - Agent: 42.4 (#6/23)
- OpenCompass Agent - Tool Use: 42.4 (#6/23)
- OpenCompass Knowledge - Science: 91.7 (#7/23)
- OpenCompass LLM - Reasoning: 59.5 (#8/23)
- OpenCompass Language - NLP: 67.1 (#8/23)
- OpenCompass Language - Instruction Following: 72.5 (#8/23)
- OpenCompass Math - College: 77.1 (#8/23)
Hy3-preview (High) — ELO 1729, #110
- OpenCompass Math - College: 81.3 (#3/23)
- OpenCompass Language - Instruction Following: 76.0 (#4/23)
- OpenCompass LLM - Math: 74.5 (#5/23)
- OpenCompass Language - Creation: 75.4 (#5/23)
- OpenCompass LLM - Language: 74.4 (#7/23)
- OpenCompass Reasoning - Academic: 43.6 (#8/23)
- OpenCompass LLM - Reasoning: 58.5 (#10/23)
- OpenCompass Math - Competition: 67.6 (#10/23)
- OpenCompass LLM - Agent: 28.7 (#12/23)
- OpenCompass Reasoning - Common: 73.5 (#12/23)
Ring-2.5-1T — ELO 1711, #119
- OpenCompass Knowledge - Social Science: 92.9 (#5/23)
- OpenCompass Language - NLP: 65.4 (#11/23)
- OpenCompass Language - Creation: 68.8 (#12/23)
- OpenCompass Knowledge - Humanities: 90.0 (#12/23)
- OpenCompass LLM - Agent: 25.0 (#13/23)
- OpenCompass Math - College: 75.0 (#13/23)
- OpenCompass Agent - Tool Use: 25.0 (#13/23)
- OpenCompass LLM - Knowledge: 89.4 (#14/23)
- OpenCompass Knowledge - Engineering: 90.8 (#14/23)
- OpenCompass LLM - Language: 69.8 (#15/23)

Top-10 New Scores (1)

Claude Opus 4.7 (Thinking) on WeirdML: 75.45 (#8)

New #1 Leaders (9)

OpenCompass Reasoning - Common: Doubao-Seed-2-0-Pro-260215 (high) (82.1) beat Gemini-3-Pro-Preview by 8.5
OpenCompass Math - College: Doubao-Seed-2-0-Pro-260215 (high) (83.8) beat Kimi-K2.5 by 7.3
Tau3-Bench Banking_Knowledge: GPT-5.5 (37.4) beat Distyl ButtonAgent by 6.2
OpenCompass Knowledge - Social Science: Gemini-3.1-Pro-Preview (97.5) beat Gemini-3-Pro-Preview by 4.3
OpenCompass LLM - Math: Doubao-Seed-2-0-Pro-260215 (high) (77.3) beat Qwen3-Max-2026-01-23 by 4.1
OpenCompass LLM - Reasoning: Doubao-Seed-2-0-Pro-260215 (high) (65.2) beat Gemini-3-Pro-Preview by 3.7
OpenCompass Math - Competition: Kimi-K2.6 (72.1) beat Qwen3-Max-2026-01-23 by 2.1
OpenCompass Reasoning - Academic: GPT-5.4-2026-03-05 (high) (52.0) beat GPT-5.2-2025-12-11 (high) by 1.5
OpenCompass Knowledge - Engineering: GPT-5.4-2026-03-05 (high) (96.2) beat Gemini-3-Pro-Preview by 0.4

AI Benchmark Digest — 2026-05-13

2026-05-13T07:29:12.582080+00:00

Daily

New Benchmarks (2)

ProgramBench (Resolved (%)): leader GPT-5.5 (xHigh) (0.5), 13 models
Meta and Stanford benchmark testing whether language-model agents can rebuild complete programs from only a compiled binary and documentation. Agents use mini-SWE-agent across 200 open-source program recreation tasks and are scored by hidden behavioral tests.
ProgramBench Almost (Almost (%)): leader GPT-5.5 (xHigh) (13.5), 13 models
Companion ProgramBench metric that counts near-complete program recreations: tasks where the generated implementation passes most hidden behavioral tests but does not fully resolve the benchmark task.

New Models (1)

JT-35B-Flash — ELO 1693, #141
- AA TAU-2 Bench: 99.1 (#1/405)
- AA Omniscience - Software Engineering (SWE) - Go: 36.0 (#50/391)
- AA Omniscience - Software Engineering (SWE) - Java: 29.0 (#58/391)
- AA Omniscience - Software Engineering (SWE) - HTML: 48.0 (#60/391)
- AA Omniscience - Software Engineering (SWE) - JavaScript: 41.82 (#75/391)
- AA GPQA Diamond: 82.9 (#76/486)
- AA Omniscience - Software Engineering (SWE) - C: 53.0 (#78/391)
- AA Omniscience - Software Engineering (SWE) - PHP: 38.0 (#79/391)
- AA Omniscience - Software Engineering (SWE) - TypeScript: 36.67 (#82/391)
- AA Omniscience - Software Engineering (SWE): 35.0 (#83/391)

Top-10 New Scores (1)

GPT-5.5 (xHigh) on WeirdML: 84.91 (#1)

New #1 Leaders (2)

WeirdML: gpt-5.5 (xhigh) (84.91) beat gpt-5.5 (high) by 1.01
AA TAU-2 Bench: JT-35B-Flash (99.1) beat GLM-4.7-Flash (Reasoning) by 0.3

AI Benchmark Digest — 2026-05-11

2026-05-11T08:08:39.844852+00:00

Daily

New #1 Leaders (2)

OpenClawProBench: Sensenova 6.7 Flash Lite (73.7) beat qwen3.5-397b-a17b by 3.3
VisuLogic: PEREA-1.0new (52.8) beat Human by 1.4

AI Benchmark Digest — 2026-05-10

2026-05-10T07:49:15.895022+00:00

Daily

New Benchmarks (43)

AA Global-MMLU-Lite - Arabic (Accuracy (%)): leader Gemini 3.1 Pro Preview (93.0), 119 models
AA Global-MMLU-Lite - Bengali (Accuracy (%)): leader Gemini 3.1 Pro Preview (92.17), 119 models
AA Global-MMLU-Lite - German (Accuracy (%)): leader Gemini 3.1 Pro Preview (94.75), 119 models
AA Global-MMLU-Lite - English (Accuracy (%)): leader Claude Opus 4.6 (Adaptive Reasoning, Max Effort) (95.17), 120 models
AA Global-MMLU-Lite - Spanish (Accuracy (%)): leader Gemini 3.1 Pro Preview (94.42), 118 models
AA Global-MMLU-Lite - French (Accuracy (%)): leader Gemini 3.1 Pro Preview (93.67), 118 models
AA Global-MMLU-Lite - Hindi (Accuracy (%)): leader Gemini 3.1 Pro Preview (93.67), 117 models
AA Global-MMLU-Lite - Indonesian (Accuracy (%)): leader Gemini 3.1 Pro Preview (93.67), 118 models
AA Global-MMLU-Lite - Italian (Accuracy (%)): leader Gemini 3.1 Pro Preview (94.58), 117 models
AA Global-MMLU-Lite - Japanese (Accuracy (%)): leader Gemini 3.1 Pro Preview (93.67), 116 models
AA Global-MMLU-Lite - Korean (Accuracy (%)): leader Claude Opus 4.6 (Adaptive Reasoning, Max Effort) (93.0), 116 models
AA Global-MMLU-Lite - Burmese (Accuracy (%)): leader Gemini 3.1 Pro Preview (91.17), 111 models
AA Global-MMLU-Lite - Portuguese (Accuracy (%)): leader Gemini 3.1 Pro Preview (94.25), 113 models
AA Global-MMLU-Lite - Swahili (Accuracy (%)): leader Gemini 3.1 Pro Preview (92.33), 112 models
AA Global-MMLU-Lite - Yoruba (Accuracy (%)): leader Gemini 3.1 Pro Preview (88.75), 112 models
AA Global-MMLU-Lite - Chinese (Accuracy (%)): leader Claude Opus 4.6 (Adaptive Reasoning, Max Effort) (93.58), 113 models
AA Omniscience - Business (Accuracy (%)): leader GPT-5.5 (xhigh) (49.1), 388 models
AA Omniscience - Health (Accuracy (%)): leader GPT-5.5 (medium) (48.8), 388 models
AA Omniscience - Humanities & Social Sciences (Accuracy (%)): leader Gemini 3 Pro Preview (high) (56.6), 388 models
AA Omniscience - Law (Accuracy (%)): leader Gemini 3 Pro Preview (high) (64.3), 388 models
AA Omniscience - Science, Engineering & Mathematics (Accuracy (%)): leader GPT-5.5 (high) (52.3), 388 models
AA Omniscience - Software Engineering (SWE) (Accuracy (%)): leader GPT-5.5 (xhigh) (84.4), 388 models
AA Omniscience - Software Engineering (SWE) - C (Accuracy (%)): leader GPT-5.5 (high) (92.0), 388 models
AA Omniscience - Software Engineering (SWE) - Dart (Accuracy (%)): leader GPT-5.3 Codex (xhigh) (80.0), 388 models
AA Omniscience - Software Engineering (SWE) - Go (Accuracy (%)): leader GPT-5.5 (high) (84.0), 388 models
AA Omniscience - Software Engineering (SWE) - HTML (Accuracy (%)): leader GPT-5.5 (medium) (90.0), 388 models
AA Omniscience - Software Engineering (SWE) - Java (Accuracy (%)): leader GPT-5.3 Codex (xhigh) (73.0), 388 models
AA Omniscience - Software Engineering (SWE) - JavaScript (Accuracy (%)): leader GPT-5.3 Codex (xhigh) (90.91), 388 models
AA Omniscience - Software Engineering (SWE) - Julia (Accuracy (%)): leader GPT-5.4 (low) (88.0), 388 models
AA Omniscience - Software Engineering (SWE) - Kotlin (Accuracy (%)): leader GPT-5.3 Codex (xhigh) (90.0), 388 models
AA Omniscience - Software Engineering (SWE) - PHP (Accuracy (%)): leader GPT-5.5 (medium) (92.0), 388 models
AA Omniscience - Software Engineering (SWE) - Python (Accuracy (%)): leader GPT-5.5 (xhigh) (90.5), 388 models
AA Omniscience - Software Engineering (SWE) - R (Accuracy (%)): leader GPT-5.5 (medium) (74.0), 388 models
AA Omniscience - Software Engineering (SWE) - Rust (Accuracy (%)): leader GPT-5.5 (xhigh) (92.0), 388 models
AA Omniscience - Software Engineering (SWE) - Swift (Accuracy (%)): leader GPT-5.5 (xhigh) (92.0), 388 models
AA Omniscience - Software Engineering (SWE) - TypeScript (Accuracy (%)): leader GPT-5.5 (xhigh) (91.11), 388 models
EuroEval Albanian NLU - MMS SQ (Sentiment classification Score (%)): leader gemini-3-flash-preview#no-thinking (32.13), 196 models
EuroEval Albanian NLU task column for the MMS SQ dataset, measuring sentiment classification from the public albanian_nlu.csv leaderboard.
EuroEval Albanian NLU - WikiANN SQ (Named entity recognition Score (%)): leader multilingual-e5-large (86.6), 200 models
EuroEval Albanian NLU task column for the WikiANN SQ dataset, measuring named entity recognition from the public albanian_nlu.csv leaderboard.
EuroEval Albanian NLU - ScaLA SQ (Linguistic acceptability Score (%)): leader gemini-3.1-pro-preview (78.55), 166 models
EuroEval Albanian NLU task column for the ScaLA SQ dataset, measuring linguistic acceptability from the public albanian_nlu.csv leaderboard.
EuroEval Albanian NLU - MultiWikiQA SQ (Reading comprehension Score (%)): leader Qwen3.5-9B-Base (70.8), 200 models
EuroEval Albanian NLU task column for the MultiWikiQA SQ dataset, measuring reading comprehension from the public albanian_nlu.csv leaderboard.
EuroEval Bosnian NLU - MMS BS (Sentiment classification Score (%)): leader gpt-4.1-mini-2025-04-14 (56.43), 208 models
EuroEval Bosnian NLU task column for the MMS BS dataset, measuring sentiment classification from the public bosnian_nlu.csv leaderboard.
EuroEval Bosnian NLU - WikiANN BS (Named entity recognition Score (%)): leader multilingual-e5-large (84.87), 212 models
EuroEval Bosnian NLU task column for the WikiANN BS dataset, measuring named entity recognition from the public bosnian_nlu.csv leaderboard.
EuroEval Bosnian NLU - Multi Wiki QA BS (Reading comprehension Score (%)): leader Olmo-3-1125-32B (78.64), 211 models
EuroEval Bosnian NLU task column for the Multi Wiki QA BS dataset, measuring reading comprehension from the public bosnian_nlu.csv leaderboard.

Weekly

New Benchmarks (43)

AA Global-MMLU-Lite - Arabic (Accuracy (%)): leader Gemini 3.1 Pro Preview (93.0), 119 models
AA Global-MMLU-Lite - Bengali (Accuracy (%)): leader Gemini 3.1 Pro Preview (92.17), 119 models
AA Global-MMLU-Lite - German (Accuracy (%)): leader Gemini 3.1 Pro Preview (94.75), 119 models
AA Global-MMLU-Lite - English (Accuracy (%)): leader Claude Opus 4.6 (Adaptive Reasoning, Max Effort) (95.17), 120 models
AA Global-MMLU-Lite - Spanish (Accuracy (%)): leader Gemini 3.1 Pro Preview (94.42), 118 models
AA Global-MMLU-Lite - French (Accuracy (%)): leader Gemini 3.1 Pro Preview (93.67), 118 models
AA Global-MMLU-Lite - Hindi (Accuracy (%)): leader Gemini 3.1 Pro Preview (93.67), 117 models
AA Global-MMLU-Lite - Indonesian (Accuracy (%)): leader Gemini 3.1 Pro Preview (93.67), 118 models
AA Global-MMLU-Lite - Italian (Accuracy (%)): leader Gemini 3.1 Pro Preview (94.58), 117 models
AA Global-MMLU-Lite - Japanese (Accuracy (%)): leader Gemini 3.1 Pro Preview (93.67), 116 models
AA Global-MMLU-Lite - Korean (Accuracy (%)): leader Claude Opus 4.6 (Adaptive Reasoning, Max Effort) (93.0), 116 models
AA Global-MMLU-Lite - Burmese (Accuracy (%)): leader Gemini 3.1 Pro Preview (91.17), 111 models
AA Global-MMLU-Lite - Portuguese (Accuracy (%)): leader Gemini 3.1 Pro Preview (94.25), 113 models
AA Global-MMLU-Lite - Swahili (Accuracy (%)): leader Gemini 3.1 Pro Preview (92.33), 112 models
AA Global-MMLU-Lite - Yoruba (Accuracy (%)): leader Gemini 3.1 Pro Preview (88.75), 112 models
AA Global-MMLU-Lite - Chinese (Accuracy (%)): leader Claude Opus 4.6 (Adaptive Reasoning, Max Effort) (93.58), 113 models
AA Omniscience - Business (Accuracy (%)): leader GPT-5.5 (xhigh) (49.1), 388 models
AA Omniscience - Health (Accuracy (%)): leader GPT-5.5 (medium) (48.8), 388 models
AA Omniscience - Humanities & Social Sciences (Accuracy (%)): leader Gemini 3 Pro Preview (high) (56.6), 388 models
AA Omniscience - Law (Accuracy (%)): leader Gemini 3 Pro Preview (high) (64.3), 388 models
AA Omniscience - Science, Engineering & Mathematics (Accuracy (%)): leader GPT-5.5 (high) (52.3), 388 models
AA Omniscience - Software Engineering (SWE) (Accuracy (%)): leader GPT-5.5 (xhigh) (84.4), 388 models
AA Omniscience - Software Engineering (SWE) - C (Accuracy (%)): leader GPT-5.5 (high) (92.0), 388 models
AA Omniscience - Software Engineering (SWE) - Dart (Accuracy (%)): leader GPT-5.3 Codex (xhigh) (80.0), 388 models
AA Omniscience - Software Engineering (SWE) - Go (Accuracy (%)): leader GPT-5.5 (high) (84.0), 388 models
AA Omniscience - Software Engineering (SWE) - HTML (Accuracy (%)): leader GPT-5.5 (medium) (90.0), 388 models
AA Omniscience - Software Engineering (SWE) - Java (Accuracy (%)): leader GPT-5.3 Codex (xhigh) (73.0), 388 models
AA Omniscience - Software Engineering (SWE) - JavaScript (Accuracy (%)): leader GPT-5.3 Codex (xhigh) (90.91), 388 models
AA Omniscience - Software Engineering (SWE) - Julia (Accuracy (%)): leader GPT-5.4 (low) (88.0), 388 models
AA Omniscience - Software Engineering (SWE) - Kotlin (Accuracy (%)): leader GPT-5.3 Codex (xhigh) (90.0), 388 models
AA Omniscience - Software Engineering (SWE) - PHP (Accuracy (%)): leader GPT-5.5 (medium) (92.0), 388 models
AA Omniscience - Software Engineering (SWE) - Python (Accuracy (%)): leader GPT-5.5 (xhigh) (90.5), 388 models
AA Omniscience - Software Engineering (SWE) - R (Accuracy (%)): leader GPT-5.5 (medium) (74.0), 388 models
AA Omniscience - Software Engineering (SWE) - Rust (Accuracy (%)): leader GPT-5.5 (xhigh) (92.0), 388 models
AA Omniscience - Software Engineering (SWE) - Swift (Accuracy (%)): leader GPT-5.5 (xhigh) (92.0), 388 models
AA Omniscience - Software Engineering (SWE) - TypeScript (Accuracy (%)): leader GPT-5.5 (xhigh) (91.11), 388 models
EuroEval Albanian NLU - MMS SQ (Sentiment classification Score (%)): leader gemini-3-flash-preview#no-thinking (32.13), 196 models
EuroEval Albanian NLU task column for the MMS SQ dataset, measuring sentiment classification from the public albanian_nlu.csv leaderboard.
EuroEval Albanian NLU - WikiANN SQ (Named entity recognition Score (%)): leader multilingual-e5-large (86.6), 200 models
EuroEval Albanian NLU task column for the WikiANN SQ dataset, measuring named entity recognition from the public albanian_nlu.csv leaderboard.
EuroEval Albanian NLU - ScaLA SQ (Linguistic acceptability Score (%)): leader gemini-3.1-pro-preview (78.55), 166 models
EuroEval Albanian NLU task column for the ScaLA SQ dataset, measuring linguistic acceptability from the public albanian_nlu.csv leaderboard.
EuroEval Albanian NLU - MultiWikiQA SQ (Reading comprehension Score (%)): leader Qwen3.5-9B-Base (70.8), 200 models
EuroEval Albanian NLU task column for the MultiWikiQA SQ dataset, measuring reading comprehension from the public albanian_nlu.csv leaderboard.
EuroEval Bosnian NLU - MMS BS (Sentiment classification Score (%)): leader gpt-4.1-mini-2025-04-14 (56.43), 208 models
EuroEval Bosnian NLU task column for the MMS BS dataset, measuring sentiment classification from the public bosnian_nlu.csv leaderboard.
EuroEval Bosnian NLU - WikiANN BS (Named entity recognition Score (%)): leader multilingual-e5-large (84.87), 212 models
EuroEval Bosnian NLU task column for the WikiANN BS dataset, measuring named entity recognition from the public bosnian_nlu.csv leaderboard.
EuroEval Bosnian NLU - Multi Wiki QA BS (Reading comprehension Score (%)): leader Olmo-3-1125-32B (78.64), 211 models
EuroEval Bosnian NLU task column for the Multi Wiki QA BS dataset, measuring reading comprehension from the public bosnian_nlu.csv leaderboard.

New Models (38)

GLM-5V Turbo (Reasoning) — ELO 1738, #102
- AA TAU-2 Bench: 98.5 (#3/402)
- AA GDPval: 1330.87 (#43/360)
- AA MMMU-Pro: 72.8 (#44/188)
- AA SciCode: 43.5 (#52/477)
- Artificial Analysis Intelligence Index: 42.85 (#56/482)
- AA Terminal-Bench Hard: 32.6 (#79/397)
- AA Omniscience: -18.98 (#80/388)
- AA Long Context Reasoning: 61.0 (#84/411)
- AA Humanity's Last Exam: 15.8 (#91/479)
- AA GPQA Diamond: 80.9 (#96/483)
ERNIE 5.0 Thinking Preview — ELO 1631, #214
- AA LiveCodeBench: 81.2 (#24/343)
- AA Global-MMLU-Lite: 86.5 (#33/120)
- AA AIME 2025: 85.0 (#46/269)
- AA MMLU-Pro: 83.0 (#60/345)
- AA CritPt: 1.4 (#68/388)
- AA MMMU-Pro: 64.6 (#90/188)
- AA TAU-2 Bench: 83.9 (#94/402)
- AA Humanity's Last Exam: 12.7 (#116/479)
- AA Terminal-Bench Hard: 25.0 (#119/397)
- AA GPQA Diamond: 77.7 (#124/483)
K-EXAONE (Reasoning) — ELO 1603, #245
- AA AIME 2025: 90.3 (#25/269)
- AA LiveCodeBench: 76.8 (#41/343)
- AA MMLU-Pro: 83.8 (#44/345)
- AA CritPt: 1.1 (#76/388)
- AA Global-MMLU-Lite: 78.86 (#80/120)
- AA IFBench: 64.7 (#85/411)
- AA Humanity's Last Exam: 13.1 (#111/479)
- AA Long Context Reasoning: 55.7 (#117/411)
- AA GPQA Diamond: 78.3 (#119/483)
- AA TAU-2 Bench: 74.3 (#121/402)
EXAONE 4.5 33B — ELO 1578, #277
- AA MMMU-Pro: 67.3 (#77/188)
- AA GPQA Diamond: 79.4 (#106/483)
- AA IFBench: 58.0 (#107/411)
- AA TAU-2 Bench: 78.1 (#112/402)
- AA CritPt: 0.3 (#128/388)
- AA Humanity's Last Exam: 11.6 (#131/479)
- AA Terminal-Bench Hard: 20.5 (#144/397)
- Artificial Analysis Intelligence Index: 30.23 (#147/482)
- AA Long Context Reasoning: 49.3 (#150/411)
- AA GDPval: 812.72 (#163/360)
K2-V2 (High) — ELO 1562, #294
- AA AIME 2025: 78.3 (#71/269)
- AA LiveCodeBench: 69.4 (#76/343)
- AA Global-MMLU-Lite: 78.6 (#82/120)
- AA IFBench: 60.1 (#102/411)
- AA MMLU-Pro: 78.6 (#135/345)
- AA Humanity's Last Exam: 9.8 (#157/479)
- AA Long Context Reasoning: 33.3 (#211/411)
- AA Terminal-Bench Hard: 9.8 (#212/397)
- AA GPQA Diamond: 68.1 (#222/483)
- Artificial Analysis Intelligence Index: 20.61 (#232/482)
Solar Open 100B (Reasoning) — ELO 1555, #307
- AA Global-MMLU-Lite: 81.58 (#61/120)
- AA IFBench: 57.7 (#110/411)
- AA Humanity's Last Exam: 9.2 (#170/479)
- AA TAU-2 Bench: 48.2 (#180/402)
- AA Long Context Reasoning: 36.0 (#195/411)
- AA CritPt: 0.0 (#204/388)
- AA GDPval: 666.33 (#207/360)
- Artificial Analysis Intelligence Index: 21.67 (#224/482)
- AA GPQA Diamond: 65.7 (#243/483)
- AA Omniscience: -54.1 (#262/388)
JT-MINI — ELO 1546, #324
- AA TAU-2 Bench: 93.0 (#40/402)
- AA Terminal-Bench Hard: 18.2 (#154/397)
- AA GDPval: 831.97 (#157/360)
- Artificial Analysis Intelligence Index: 25.37 (#187/482)
- AA Humanity's Last Exam: 6.6 (#223/479)
- AA GPQA Diamond: 67.6 (#225/483)
- AA CritPt: 0.0 (#263/388)
- AA IFBench: 36.7 (#277/411)
- AA SciCode: 27.2 (#292/477)
- AA Long Context Reasoning: 11.7 (#308/411)
K2 Think V2 — ELO 1545, #328
- AA IFBench: 62.8 (#94/411)
- AA Omniscience: -33.92 (#125/388)
- AA Long Context Reasoning: 52.7 (#135/411)
- AA Humanity's Last Exam: 9.5 (#165/479)
- AA GPQA Diamond: 71.3 (#192/483)
- Artificial Analysis Intelligence Index: 24.12 (#201/482)
- AA GDPval: 607.98 (#222/360)
- AA SciCode: 33.0 (#223/477)
- AA Terminal-Bench Hard: 6.8 (#240/397)
- AA CritPt: 0.0 (#252/388)
HyperCLOVA X SEED Think (32B) — ELO 1537, #342
- AA TAU-2 Bench: 87.4 (#68/402)
- AA Global-MMLU-Lite: 78.6 (#83/120)
- AA LiveCodeBench: 62.9 (#107/343)
- AA AIME 2025: 59.0 (#118/269)
- AA MMLU-Pro: 78.5 (#137/345)
- AA Terminal-Bench Hard: 12.1 (#194/397)
- AA GDPval: 678.83 (#199/360)
- Artificial Analysis Intelligence Index: 23.72 (#204/482)
- AA Omniscience: -52.87 (#255/388)
- AA CritPt: 0.0 (#257/388)
Mi:dm K 2.5 Pro — ELO 1527, #352
- AA TAU-2 Bench: 86.5 (#75/402)
- AA AIME 2025: 76.7 (#77/269)
- AA LiveCodeBench: 65.6 (#92/343)
- AA Global-MMLU-Lite: 74.23 (#94/120)
- AA MMLU-Pro: 80.9 (#97/345)
- AA IFBench: 49.3 (#155/411)
- AA Humanity's Last Exam: 7.7 (#195/479)
- AA GPQA Diamond: 70.1 (#200/483)
- Artificial Analysis Intelligence Index: 23.06 (#213/482)
- AA GDPval: 643.11 (#213/360)
Motif-2-12.7B (Reasoning) — ELO 1520, #366
- AA AIME 2025: 80.3 (#65/269)
- AA LiveCodeBench: 65.1 (#97/343)
- AA IFBench: 57.0 (#113/411)
- AA MMLU-Pro: 79.6 (#122/345)
- AA Humanity's Last Exam: 8.2 (#183/479)
- AA TAU-2 Bench: 46.5 (#185/402)
- AA GPQA Diamond: 69.5 (#210/483)
- Artificial Analysis Intelligence Index: 19.08 (#244/482)
- AA CritPt: 0.0 (#250/388)
- AA GDPval: 485.33 (#255/360)
Mi:dm K 2.5 Pro Preview — ELO 1517, #371
- AA Global-MMLU-Lite: 81.43 (#63/120)
- AA AIME 2025: 78.7 (#70/269)
- AA MMLU-Pro: 81.3 (#92/345)
- AA LiveCodeBench: 57.6 (#125/343)
- AA Humanity's Last Exam: 8.8 (#175/479)
- AA TAU-2 Bench: 49.4 (#177/402)
- AA IFBench: 45.6 (#180/411)
- AA GPQA Diamond: 72.2 (#185/483)
- AA SciCode: 29.7 (#251/477)
- AA CritPt: 0.0 (#255/388)
K2-V2 (Medium) — ELO 1512, #382
- AA Global-MMLU-Lite: 76.7 (#87/120)
- AA AIME 2025: 64.7 (#107/269)
- AA IFBench: 55.1 (#122/411)
- AA LiveCodeBench: 54.1 (#137/343)
- AA MMLU-Pro: 76.1 (#165/345)
- AA Terminal-Bench Hard: 8.3 (#220/397)
- AA Omniscience: -49.97 (#222/388)
- AA GDPval: 578.73 (#227/360)
- AA Long Context Reasoning: 28.0 (#232/411)
- AA CritPt: 0.0 (#251/388)
Granite 4.1 30B — ELO 1491, #425
- AA IFBench: 44.4 (#191/411)
- AA TAU-2 Bench: 42.1 (#198/402)
- AA CritPt: 0.0 (#228/388)
- AA GDPval: 495.5 (#253/360)
- AA Long Context Reasoning: 18.7 (#273/411)
- AA Terminal-Bench Hard: 2.3 (#310/397)
- AA SciCode: 25.8 (#315/477)
- Artificial Analysis Intelligence Index: 14.69 (#324/482)
- AA Omniscience: -67.78 (#342/388)
- AA GPQA Diamond: 48.1 (#354/483)
K-EXAONE (Non-reasoning) — ELO 1487, #432
- AA MMLU-Pro: 81.0 (#94/345)
- AA Global-MMLU-Lite: 71.03 (#104/120)
- AA AIME 2025: 44.0 (#150/269)
- AA Long Context Reasoning: 47.0 (#157/411)
- AA TAU-2 Bench: 59.1 (#162/402)
- AA GDPval: 767.0 (#174/360)
- Artificial Analysis Intelligence Index: 23.41 (#207/482)
- AA GPQA Diamond: 69.5 (#209/483)
- AA Terminal-Bench Hard: 6.8 (#239/397)
- AA CritPt: 0.0 (#242/388)
K2-V2 (Low) — ELO 1483, #444
- AA Global-MMLU-Lite: 71.44 (#103/120)
- AA AIME 2025: 35.3 (#173/269)
- AA LiveCodeBench: 39.3 (#187/343)
- AA MMLU-Pro: 71.3 (#212/345)
- AA Omniscience: -48.07 (#212/388)
- AA IFBench: 41.0 (#233/411)
- AA CritPt: 0.0 (#254/388)
- AA Long Context Reasoning: 19.0 (#271/411)
- AA Terminal-Bench Hard: 4.5 (#277/397)
- AA GDPval: 367.48 (#285/360)
Solar Pro 2 (Reasoning) — ELO 1479, #450
- AA MATH-500: 96.7 (#30/193)
- AA Global-MMLU-Lite: 79.61 (#78/120)
- AA MMLU-Pro: 80.5 (#107/345)
- AA LiveCodeBench: 61.6 (#113/343)
- AA AIME 2025: 61.3 (#115/269)
- AA CritPt: 0.0 (#206/388)
- AA Humanity's Last Exam: 7.0 (#213/479)
- AA GPQA Diamond: 68.7 (#215/483)
- AA SciCode: 30.2 (#246/477)
- AA TAU-2 Bench: 28.1 (#251/402)
Gemma 4 E4B (Reasoning) — ELO 1474, #458
- AA Omniscience: -20.05 (#82/388)
- AA CritPt: 0.6 (#104/388)
- AA MMMU-Pro: 51.4 (#143/188)
- AA IFBench: 44.2 (#193/411)
- AA Terminal-Bench Hard: 8.3 (#218/397)
- AA Long Context Reasoning: 30.7 (#222/411)
- Artificial Analysis Intelligence Index: 18.76 (#250/482)
- AA GPQA Diamond: 57.6 (#297/483)
- AA GDPval: 304.3 (#312/360)
- AA TAU-2 Bench: 20.8 (#314/402)
EXAONE 4.0 32B (Reasoning) — ELO 1473, #461
- AA MATH-500: 97.7 (#21/193)
- AA LiveCodeBench: 74.7 (#48/343)
- AA AIME 2025: 80.0 (#68/269)
- AA MMLU-Pro: 81.8 (#82/345)
- AA Global-MMLU-Lite: 73.46 (#97/120)
- AA Humanity's Last Exam: 10.5 (#145/479)
- AA GPQA Diamond: 73.9 (#167/483)
- AA SciCode: 34.4 (#203/477)
- AA CritPt: 0.0 (#240/388)
- AA GDPval: 499.86 (#249/360)
Tri-21B-Think Preview — ELO 1473, #462
- AA TAU-2 Bench: 93.3 (#38/402)
- AA IFBench: 47.1 (#169/411)
- Artificial Analysis Intelligence Index: 19.99 (#236/482)
- AA Humanity's Last Exam: 5.7 (#257/479)
- AA CritPt: 0.0 (#259/388)
- AA Omniscience: -55.28 (#267/388)
- AA Long Context Reasoning: 14.7 (#294/411)
- AA GDPval: 337.02 (#299/360)
- AA Terminal-Bench Hard: 2.3 (#315/397)
- AA GPQA Diamond: 53.8 (#320/483)
Tri-21B-Think — ELO 1468, #468
- AA TAU-2 Bench: 81.0 (#103/402)
- AA IFBench: 54.6 (#124/411)
- AA CritPt: 0.3 (#132/388)
- AA Humanity's Last Exam: 6.1 (#241/479)
- Artificial Analysis Intelligence Index: 18.62 (#258/482)
- AA GPQA Diamond: 60.1 (#279/483)
- AA GDPval: 374.11 (#282/360)
- AA Long Context Reasoning: 11.0 (#312/411)
- AA Omniscience: -63.3 (#321/388)
- AA Terminal-Bench Hard: 0.8 (#342/397)
GPT-4o (March 2025, chatgpt-4o-latest) — ELO 1449, #500
- AA MATH-500: 89.3 (#73/193)
- AA MMLU-Pro: 80.3 (#110/345)
- AA SciCode: 36.6 (#165/477)
- AA LiveCodeBench: 42.5 (#170/343)
- AA AIME 2025: 25.7 (#196/269)
- AA GPQA Diamond: 65.5 (#247/483)
- Artificial Analysis Intelligence Index: 18.56 (#260/482)
- AA Humanity's Last Exam: 5.0 (#305/479)
Llama 3.3 Nemotron Super 49B v1 (Reasoning) — ELO 1448, #502
- AA MATH-500: 95.9 (#36/193)
- AA AIME 2025: 54.7 (#132/269)
- AA MMLU-Pro: 78.5 (#136/345)
- AA CritPt: 0.0 (#215/388)
- AA Humanity's Last Exam: 6.5 (#227/479)
- AA LiveCodeBench: 27.7 (#238/343)
- AA GPQA Diamond: 64.3 (#251/483)
- Artificial Analysis Intelligence Index: 18.49 (#262/482)
- AA TAU-2 Bench: 26.9 (#262/402)
- AA IFBench: 38.1 (#262/411)
Solar Pro 2 (Non-reasoning) — ELO 1435, #524
- AA MATH-500: 88.9 (#76/193)
- AA Global-MMLU-Lite: 75.34 (#91/120)
- AA LiveCodeBench: 42.4 (#172/343)
- AA MMLU-Pro: 75.0 (#178/345)
- AA AIME 2025: 30.0 (#186/269)
- AA CritPt: 0.0 (#203/388)
- AA TAU-2 Bench: 31.9 (#230/402)
- AA GDPval: 447.04 (#265/360)
- AA Terminal-Bench Hard: 4.5 (#273/397)
- AA IFBench: 33.7 (#306/411)
Llama 3.3 Nemotron Super 49B v1 (Non-reasoning) — ELO 1408, #560
- AA MATH-500: 77.5 (#113/193)
- AA CritPt: 0.0 (#216/388)
- AA Omniscience: -49.68 (#219/388)
- AA MMLU-Pro: 69.8 (#221/345)
- AA LiveCodeBench: 28.0 (#235/343)
- AA AIME 2025: 7.7 (#237/269)
- AA IFBench: 39.5 (#247/411)
- AA Long Context Reasoning: 11.3 (#309/411)
- AA GPQA Diamond: 51.7 (#330/483)
- Artificial Analysis Intelligence Index: 14.35 (#336/482)
NVIDIA Nemotron 3 Nano 4B — ELO 1388, #586
- AA IFBench: 58.2 (#106/411)
- AA CritPt: 0.0 (#211/388)
- AA Terminal-Bench Hard: 6.8 (#238/397)
- AA TAU-2 Bench: 28.1 (#252/402)
- AA GDPval: 476.83 (#258/360)
- AA Long Context Reasoning: 16.7 (#286/411)
- AA Humanity's Last Exam: 4.8 (#323/479)
- Artificial Analysis Intelligence Index: 14.68 (#325/482)
- AA GPQA Diamond: 51.3 (#338/483)
- AA Omniscience: -71.53 (#351/388)
Granite 4.1 3B — ELO 1380, #595
- AA CritPt: 0.0 (#232/388)
- AA GDPval: 366.32 (#286/360)
- AA IFBench: 33.7 (#307/411)
- AA Terminal-Bench Hard: 2.3 (#312/397)
- AA TAU-2 Bench: 19.6 (#323/402)
- AA Long Context Reasoning: 3.0 (#341/411)
- AA Omniscience: -77.38 (#370/388)
- AA SciCode: 11.9 (#412/477)
- Artificial Analysis Intelligence Index: 8.54 (#435/482)
- AA GPQA Diamond: 31.4 (#441/483)
Gemma 4 E2B (Reasoning) — ELO 1376, #604
- AA Omniscience: -23.98 (#94/388)
- AA MMMU-Pro: 44.6 (#160/188)
- AA CritPt: 0.0 (#170/388)
- AA IFBench: 38.0 (#265/411)
- AA Long Context Reasoning: 15.0 (#292/411)
- AA Terminal-Bench Hard: 3.0 (#299/397)
- Artificial Analysis Intelligence Index: 15.21 (#309/482)
- AA TAU-2 Bench: 20.8 (#315/402)
- AA Humanity's Last Exam: 4.8 (#322/479)
- AA GDPval: 272.59 (#338/360)
Llama 3.1 Nemotron Nano 4B v1.1 (Reasoning) — ELO 1351, #630
- AA MATH-500: 94.7 (#41/193)
- AA AIME 2025: 50.0 (#140/269)
- AA LiveCodeBench: 49.3 (#153/343)
- AA MMLU-Pro: 55.6 (#283/345)
- AA Humanity's Last Exam: 5.1 (#289/479)
- Artificial Analysis Intelligence Index: 14.43 (#334/482)
- AA Long Context Reasoning: 0.0 (#358/411)
- AA TAU-2 Bench: 11.7 (#362/402)
- AA IFBench: 25.5 (#375/411)
- AA GPQA Diamond: 40.8 (#393/483)
Ling-mini-2.0 — ELO 1346, #635
- AA AIME 2025: 49.3 (#142/269)
- AA LiveCodeBench: 42.9 (#169/343)
- AA MMLU-Pro: 67.1 (#243/345)
- AA CritPt: 0.0 (#284/388)
- AA Humanity's Last Exam: 5.0 (#304/479)
- AA GPQA Diamond: 56.2 (#306/483)
- AA Long Context Reasoning: 6.7 (#329/411)
- AA GDPval: 264.15 (#341/360)
- AA Terminal-Bench Hard: 0.8 (#345/397)
- AA TAU-2 Bench: 13.2 (#356/402)
Jamba Reasoning 3B — ELO 1320, #657
- AA IFBench: 52.4 (#137/411)
- AA AIME 2025: 10.7 (#231/269)
- AA LiveCodeBench: 21.0 (#267/343)
- AA CritPt: 0.0 (#268/388)
- AA MMLU-Pro: 57.7 (#274/345)
- AA Long Context Reasoning: 7.0 (#323/411)
- AA TAU-2 Bench: 15.8 (#342/402)
- AA Terminal-Bench Hard: 0.8 (#344/397)
- AA GDPval: 257.67 (#345/360)
- AA Humanity's Last Exam: 4.6 (#347/479)
Exaone 4.0 1.2B (Reasoning) — ELO 1266, #696
- AA AIME 2025: 50.3 (#139/269)
- AA LiveCodeBench: 51.6 (#143/343)
- AA CritPt: 0.0 (#241/388)
- AA Humanity's Last Exam: 5.8 (#251/479)
- AA MMLU-Pro: 58.8 (#268/345)
- AA GDPval: 296.88 (#317/360)
- AA GPQA Diamond: 51.5 (#336/483)
- AA TAU-2 Bench: 16.4 (#338/402)
- AA Long Context Reasoning: 0.0 (#370/411)
- AA Terminal-Bench Hard: 0.0 (#377/397)
Exaone 4.0 1.2B (Non-reasoning) — ELO 1262, #697
- AA AIME 2025: 24.0 (#200/269)
- AA LiveCodeBench: 29.3 (#226/343)
- AA CritPt: 0.0 (#239/388)
- AA Humanity's Last Exam: 5.8 (#250/479)
- AA MMLU-Pro: 50.0 (#294/345)
- AA GDPval: 298.76 (#316/360)
- AA TAU-2 Bench: 20.5 (#318/402)
- AA Long Context Reasoning: 0.0 (#369/411)
- AA Terminal-Bench Hard: 0.0 (#376/397)
- AA IFBench: 25.3 (#376/411)
Granite 4.0 1B — ELO 1258, #701
- AA CritPt: 0.0 (#234/388)
- AA AIME 2025: 6.3 (#244/269)
- AA Humanity's Last Exam: 5.1 (#292/479)
- AA TAU-2 Bench: 22.8 (#294/402)
- AA MMLU-Pro: 32.5 (#331/345)
- AA LiveCodeBench: 4.7 (#333/343)
- AA Long Context Reasoning: 4.0 (#340/411)
- AA GDPval: 259.61 (#342/360)
- AA Terminal-Bench Hard: 0.0 (#373/397)
- AA Omniscience: -81.82 (#377/388)
Granite 4.0 H 350M — ELO 1137, #759
- AA CritPt: 0.0 (#227/388)
- AA Humanity's Last Exam: 6.4 (#228/479)
- AA AIME 2025: 1.3 (#262/269)
- AA GDPval: 294.09 (#319/360)
- AA LiveCodeBench: 1.9 (#339/343)
- AA MMLU-Pro: 12.7 (#343/345)
- AA TAU-2 Bench: 14.6 (#349/402)
- AA Long Context Reasoning: 0.0 (#366/411)
- AA Terminal-Bench Hard: 0.0 (#369/397)
- AA Omniscience: -87.25 (#387/388)
OLMo 2 32B — ELO 1037, #780
- AA AIME 2025: 3.3 (#256/269)
- AA IFBench: 38.1 (#264/411)
- AA MMLU-Pro: 51.1 (#292/345)
- AA LiveCodeBench: 6.8 (#328/343)
- AA Terminal-Bench Hard: 0.0 (#391/397)
- AA Long Context Reasoning: 0.0 (#393/411)
- Artificial Analysis Intelligence Index: 10.57 (#397/482)
- AA TAU-2 Bench: 0.0 (#401/402)
- AA GPQA Diamond: 32.8 (#429/483)
- AA SciCode: 8.0 (#437/477)
Phi-3 Mini Instruct 3.8B — ELO 1025, #781
- AA MATH-500: 45.7 (#172/193)
- AA AIME 2025: 0.3 (#265/269)
- AA MMLU-Pro: 43.5 (#308/345)
- AA LiveCodeBench: 11.6 (#308/343)
- AA Long Context Reasoning: 2.0 (#345/411)
- AA Humanity's Last Exam: 4.4 (#372/479)
- AA IFBench: 23.9 (#382/411)
- AA Terminal-Bench Hard: 0.0 (#388/397)
- AA TAU-2 Bench: 0.0 (#398/402)
- Artificial Analysis Intelligence Index: 10.1 (#407/482)
OLMo 2 7B — ELO 958, #787
- AA AIME 2025: 0.7 (#263/269)
- AA Humanity's Last Exam: 5.5 (#265/479)
- AA MMLU-Pro: 28.2 (#334/345)
- AA LiveCodeBench: 4.1 (#335/343)
- AA IFBench: 24.4 (#381/411)
- AA Terminal-Bench Hard: 0.0 (#390/397)
- AA Long Context Reasoning: 0.0 (#391/411)
- AA TAU-2 Bench: 0.0 (#399/402)
- Artificial Analysis Intelligence Index: 9.3 (#423/482)
- AA GPQA Diamond: 28.8 (#455/483)

Top-10 New Scores (7)

Claude Mythos Preview on METR Benchmark: 17.41 (#1)
GPT-5.4 (xHigh) on OpenClawProBench: 68.0 (#8)
GPT-5.5 (xHigh) on OpenClawProBench: 69.3 (#4)
GPT-5.5 (xHigh) on Wolfram LLM Benchmarking Project: 68.8 (#6)
GPT-5.5 Pro on Epoch AI - ECI: 159.5 (#3)
GPT-5.5 Pro on PinchBench: 18.11 (#39)
GPT-5.5 Pro on VoxelBench: 2107.0 (#1)

New #1 Leaders (14)

FoodTruckBench: GPT-5.5 (61408.0) beat Claude Opus 4.6 by 11889.0
LIBRA - ruBABILongQA2: Qwen_Qwen3-30B-A3B-Instruct-2507 (64.72) beat GPT-4o by 28.05
LIBRA - ruQuALITY: 01-ai_Yi-9B-200K (95.9) beat GPT-4o by 12.57
SEAL - AudioMultiChallenge - Audio Output: gpt-realtime-2 (xHigh) (48.45) beat gemini-3.1-flash-live-preview (Thinking) by 12.39
FrontierSWE: GPT-5.5 (83.0) beat Claude Opus 4.7 by 9.0
FrontierMath - Tier 4: AI co-mathematician (47.9) beat GPT-5.5 Pro (xhigh) by 8.3
Story Theory Bench: glm-5 (99.6) beat deepseek-v3.2 by 7.4
Kaggle FACTS Parametric: Gemini 3.1 Pro Preview (78.96) beat Gemini 3 Flash Preview by 6.7
SEAL - SWE Atlas - Codebase QnA: GPT 5.5 (Codex) (45.43) beat Gpt 5.4 xHigh (Codex) by 4.63
LIBRA - ruSciAbstractRetrieval: Qwen_Qwen3-30B-A3B-Instruct-2507 (81.5) beat GLM-4 9B Chat by 3.69
Kaggle FACTS (Google): GPT-5.5 (71.19) beat Gemini 3.1 Pro Preview by 3.48
LIBRA - ruBABILongQA1: Qwen_Qwen3-30B-A3B-Instruct-2507 (80.5) beat GPT-4o by 2.17
Android Bench: GPT 5.5 (74.0) beat GPT-5.4 by 1.6
ForecastBench: green tree (68.2) beat Cassi ensemble_2_crowdadj by 0.4

AI Benchmark Digest — 2026-05-09

2026-05-09T07:40:39.118338+00:00

Daily

New Benchmarks (8)

Factory Code Review Benchmark (Mean F1 (%)): leader GPT-5.2 (60.5), 13 models
Factory benchmark for code review quality, scoring model comments against expected findings with mean F1 across realistic pull request review tasks.
EuroEval Albanian NLU (NLU Average Score (%)): leader gemini-3.1-pro-preview (61.17), 208 models
Albanian-language EuroEval natural-language-understanding suite, separating NLU task performance from the broader all-task EuroEval aggregate.
EuroEval Bosnian NLU (NLU Average Score (%)): leader Ministral-3-14B-Reasoning-2512 (66.0), 214 models
Bosnian-language EuroEval natural-language-understanding suite, separating NLU task performance from the broader all-task EuroEval aggregate.
EuroEval Albanian Knowledge (Knowledge Average Score (%)): leader gemini-3-flash-preview#thinking (96.46), 167 models
EuroEval Albanian knowledge category: language-specific factual or domain-knowledge tasks from EuroEval's public albanian_all.csv leaderboard, scored as the average task score for each model.
EuroEval Albanian Common Sense Reasoning (Common Sense Reasoning Average Score (%)): leader gemini-3.1-pro-preview (85.24), 155 models
EuroEval Albanian common-sense reasoning category: language-specific commonsense tasks from EuroEval's public albanian_all.csv leaderboard, scored as the average task score for each model.
IMO-Bench (Advanced ProofBench Accuracy (%)): leader Aletheia (91.9), 9 models
Advanced IMO-ProofBench leaderboard for rigorous mathematical proof writing on olympiad-level problems.
ChartMuseum (Overall Accuracy (%)): leader Gemini-3.1-Pro (80.7), 22 models
Chart question-answering benchmark over real-world charts, testing visual, textual, and synthesis reasoning.
SvelteBench (Average pass@1 (%)): leader claude-opus-4-6 (100.0), 123 models
Frontend coding benchmark for Svelte component tasks, scored by average pass@1.

New Models (1)

Grok 4.3 (Non-reasoning) — ELO 1647, #259
- AA GDPval: 1306.14 (#52/360)
- AA MMMU-Pro: 64.8 (#88/188)
- AA Omniscience: -32.3 (#121/388)
- Artificial Analysis Intelligence Index: 31.02 (#139/482)
- AA SciCode: 37.4 (#146/477)
- AA TAU-2 Bench: 65.8 (#148/402)
- AA Terminal-Bench Hard: 18.9 (#149/397)
- AA IFBench: 47.6 (#165/411)
- AA CritPt: 0.0 (#182/388)
- AA Humanity's Last Exam: 6.5 (#226/479)

Top-10 New Scores (1)

GPT-5.5 (xHigh) on Wolfram LLM Benchmarking Project: 68.8 (#6)

New #1 Leaders (4)

FrontierMath - Tier 4: AI co-mathematician (47.9) beat GPT-5.5 Pro (xhigh) by 8.3
METR Benchmark: claude mythos preview early (17.41) beat claude opus 4 6 by 5.43
METR Benchmark (80% Horizon): claude mythos preview early (3.1) beat gemini 3 1 pro by 1.6
ForecastBench: green tree (68.2) beat Cassi ensemble_2_crowdadj by 0.4

AI Benchmark Digest — 2026-05-08

2026-05-08T07:40:34.661988+00:00

Daily

New Benchmarks (8)

EuroEval Albanian NLU (NLU Average Score (%)): leader gemini-3.1-pro-preview (61.17), 208 models
EuroEval Bosnian NLU (NLU Average Score (%)): leader Ministral-3-14B-Reasoning-2512 (66.0), 214 models
EuroEval Albanian Knowledge (Knowledge Average Score (%)): leader gemini-3-flash-preview#thinking (96.46), 167 models
EuroEval Albanian Common Sense Reasoning (Common Sense Reasoning Average Score (%)): leader gemini-3.1-pro-preview (85.24), 155 models
MoNaCo (F1): leader o3 (61.18), 15 models
IMO-Bench (Advanced ProofBench Accuracy (%)): leader Aletheia (91.9), 9 models
ChartMuseum (Overall Accuracy (%)): leader Gemini-3.1-Pro (80.7), 22 models
SvelteBench (Average pass@1 (%)): leader claude-opus-4-6 (100.0), 123 models

New #1 Leaders (3)

SEAL - AudioMultiChallenge - Audio Output: gpt-realtime-2 (xHigh) (48.45) beat gemini-3.1-flash-live-preview (Thinking) by 12.39
Story Theory Bench: glm-5 (99.6) beat deepseek-v3.2 by 7.4
SEAL - SWE Atlas - Codebase QnA: GPT 5.5 (Codex) (45.43) beat Gpt 5.4 xHigh (Codex) by 4.63

AI Benchmark Digest — 2026-05-07

2026-05-07T07:40:24.104745+00:00

Daily

New Benchmarks (19)

LIBRA - MatreshkaNames * (Dataset Total Score (%)): leader Qwen_Qwen3-30B-A3B-Instruct-2507 (81.2), 7 models
LIBRA - ruSciPassageCount * (Dataset Total Score (%)): leader Qwen_Qwen3-30B-A3B-Instruct-2507 (25.77), 7 models
LIBRA - ru2WikiMultihopQA * (Dataset Total Score (%)): leader Qwen_Qwen3-30B-A3B-Instruct-2507 (66.63), 7 models
LIBRA - LongContextMultiQ * (Dataset Total Score (%)): leader 01-ai_Yi-34B-200K (53.14), 7 models
LIBRA - LibrusecMHQA * (Dataset Total Score (%)): leader Qwen_Qwen3-30B-A3B-Instruct-2507 (51.0), 7 models
LIBRA - ruBABILongQA3 * (Dataset Total Score (%)): leader Qwen_Qwen3-30B-A3B-Instruct-2507 (38.38), 7 models
Kernel Arena - KernelBench HIP (Mean Correctness+Speedup): leader GPT-5.2 (15.463), 11 models
Kernel Arena - WaferBench NVFP4 (Mean Correctness+Speedup): leader Gemini 3.1 Pro (2.274), 4 models
MathArena - ARXIV_FALSE April (Accuracy (%)): leader GPT-5.5 (xhigh) (72.13), 6 models
MathArena - ARXIV April (Accuracy (%)): leader GPT-5.5 (xhigh) (65.48), 6 models
METR Benchmark (80% Horizon) (80% Time Horizon (hours)): leader gemini 3 1 pro (1.5), 25 models
LLM Stats (HealthBench) (Score (%)): leader Kimi K2-Thinking-0905 (58.0), 5 models
SCORE Robustness (Accuracy) (Average Accuracy (%)): leader Llama-3.1-70B-Instruct (67.02), 6 models
SCORE Robustness (Consistency) (Average Consistency Rate (%)): leader Llama-3.1-70B-Instruct (72.39), 6 models
Multilingual MMLU Leaderboard (Average Accuracy (%)): leader Claude-3.5-Sonnet (77.39), 17 models
Pinocchio Italian Leaderboard (Average Accuracy (%)): leader gemma-2-27b-it (70.97), 45 models
Ukrainian LLM Leaderboard (Average Score (%)): leader gemma-4-26B-A4B-it (reasoning) (63.29), 13 models
Arabic Broad Leaderboard (Average Score (0-10)): leader gemini-3-pro-preview (9.204), 87 models
Darija Chatbot Arena (Elo Rating): leader GPT-4o (1404.8), 13 models

New #1 Leaders (3)

FoodTruckBench: GPT-5.5 (61408.0) beat Claude Opus 4.6 by 11889.0
ASCIIBench: claude-opus-4.5 (1656.0) beat claude-opus-4.1 by 5.0
Kaggle FACTS Parametric: Gemini 3.1 Pro Preview (78.96) beat GPT-5.5 by 0.92

AI Benchmark Digest — 2026-05-04

2026-05-04T07:41:09.001799+00:00

Daily

New Models (62)

Doubao Seed Code — ELO 1645, #209
K-EXAONE (Reasoning) — ELO 1645, #210
Gemini 2.5 Flash Preview (Sep '25) (Reasoning) — ELO 1638, #221
Gemma 4 31B (Non-reasoning) — ELO 1626, #232
ERNIE 5.0 Thinking Preview — ELO 1622, #240
EXAONE 4.5 33B — ELO 1619, #245
Nemotron Cascade 2 30B A3B — ELO 1591, #288
Gemini 2.5 Flash Preview (Sep '25) (Non-reasoning) — ELO 1579, #309
Gemma 4 26B A4B (Non-reasoning) — ELO 1579, #311
JT-MINI — ELO 1567, #329
MiniMax M1 40k — ELO 1551, #347
K2 Think V2 — ELO 1550, #349
HyperCLOVA X SEED Think (32B) — ELO 1546, #354
Mi:dm K 2.5 Pro — ELO 1536, #369
Gemini 2.0 Flash Thinking Experimental (Jan '25) — ELO 1534, #373
K-EXAONE (Non-reasoning) — ELO 1527, #380
Solar Pro 3 — ELO 1525, #383
Solar Open 100B (Reasoning) — ELO 1521, #388
Mi:dm K 2.5 Pro Preview — ELO 1514, #392
EXAONE 4.0 32B (Reasoning) — ELO 1511, #396
GPT-4o (ChatGPT) — ELO 1493, #416
GPT-4o (March 2025, chatgpt-4o-latest) — ELO 1492, #418
Gemma 4 E4B (Reasoning) — ELO 1475, #438
Solar Pro 2 (Preview) (Reasoning) — ELO 1474, #441
Solar Pro 2 (Reasoning) — ELO 1456, #463
Solar Pro 2 (Preview) (Non-reasoning) — ELO 1456, #464
Llama 3.3 Nemotron Super 49B v1 (Reasoning) — ELO 1448, #476
Step3 VL 10B — ELO 1446, #479
Tri-21B-Think — ELO 1441, #489
NVIDIA Nemotron 3 Nano 4B — ELO 1436, #496
Gemini 2.0 Flash-Lite (Feb '25) — ELO 1433, #498
Granite 4.1 30B — ELO 1431, #501
Llama 3.1 Tulu3 405B — ELO 1425, #509
Gemma 4 E2B (Reasoning) — ELO 1405, #531
Gemma 4 E4B (Non-reasoning) — ELO 1405, #532
Solar Pro 2 (Non-reasoning) — ELO 1403, #538
Gemini 1.5 Flash-8B — ELO 1400, #540
QwQ 32B-Preview — ELO 1385, #556
EXAONE 4.0 32B (Non-reasoning) — ELO 1382, #559
Llama 3.3 Nemotron Super 49B v1 (Non-reasoning) — ELO 1364, #576
Gemma 4 E2B (Non-reasoning) — ELO 1355, #586
Llama 3.1 Nemotron Nano 4B v1.1 (Reasoning) — ELO 1347, #593
DeepHermes 3 - Mistral 24B Preview (Non-reasoning) — ELO 1343, #596
Qwen2.5 Coder Instruct 7B — ELO 1312, #622
Ling-mini-2.0 — ELO 1305, #627
Gemma 3n E4B Instruct Preview (May '25) — ELO 1304, #629
Granite 4.1 3B — ELO 1302, #632
Jamba Reasoning 3B — ELO 1275, #641
LFM 40B — ELO 1265, #650
Exaone 4.0 1.2B (Reasoning) — ELO 1259, #655
Llama 2 Chat 13B — ELO 1242, #667
Exaone 4.0 1.2B (Non-reasoning) — ELO 1234, #671
Granite 4.0 H 1B — ELO 1205, #688
Molmo 7B-D — ELO 1204, #690
DeepHermes 3 - Llama-3.1 8B Preview (Non-reasoning) — ELO 1198, #694
Granite 4.0 1B — ELO 1197, #695
OLMo 2 32B — ELO 1141, #728
Phi-3 Mini Instruct 3.8B — ELO 1126, #733
Granite 4.0 350M — ELO 1103, #739
Gemma 3 270M — ELO 1088, #744
Granite 4.0 H 350M — ELO 1077, #748
OLMo 2 7B — ELO 1071, #752

Top-10 New Scores (1)

GPT-5.5 Pro on VoxelBench: 2125.0 (#1)

New #1 Leaders (1)

VoxelBench: GPT-5.5 Pro (2125.0) beat GPT-5.5 (xHigh) by 103.0

AI Benchmark Digest — 2026-05-04

2026-05-04T00:19:05.942276+00:00

Daily

New Models (62)

Doubao Seed Code — ELO 1645, #209
K-EXAONE (Reasoning) — ELO 1645, #210
Gemini 2.5 Flash Preview (Sep '25) (Reasoning) — ELO 1638, #221
Gemma 4 31B (Non-reasoning) — ELO 1626, #232
ERNIE 5.0 Thinking Preview — ELO 1622, #241
EXAONE 4.5 33B — ELO 1619, #248
Nemotron Cascade 2 30B A3B — ELO 1591, #288
Gemma 4 26B A4B (Non-reasoning) — ELO 1580, #309
Gemini 2.5 Flash Preview (Sep '25) (Non-reasoning) — ELO 1579, #310
JT-MINI — ELO 1567, #329
MiniMax M1 40k — ELO 1551, #347
K2 Think V2 — ELO 1550, #349
HyperCLOVA X SEED Think (32B) — ELO 1547, #352
Mi:dm K 2.5 Pro — ELO 1536, #370
Gemini 2.0 Flash Thinking Experimental (Jan '25) — ELO 1534, #373
K-EXAONE (Non-reasoning) — ELO 1527, #380
Solar Pro 3 — ELO 1525, #384
Solar Open 100B (Reasoning) — ELO 1521, #388
Mi:dm K 2.5 Pro Preview — ELO 1515, #392
EXAONE 4.0 32B (Reasoning) — ELO 1511, #397
GPT-4o (March 2025, chatgpt-4o-latest) — ELO 1493, #416
GPT-4o (ChatGPT) — ELO 1493, #417
Gemma 4 E4B (Reasoning) — ELO 1475, #438
Solar Pro 2 (Preview) (Reasoning) — ELO 1474, #441
Solar Pro 2 (Preview) (Non-reasoning) — ELO 1457, #463
Solar Pro 2 (Reasoning) — ELO 1456, #466
Llama 3.3 Nemotron Super 49B v1 (Reasoning) — ELO 1448, #476
Step3 VL 10B — ELO 1446, #481
Tri-21B-Think — ELO 1441, #490
NVIDIA Nemotron 3 Nano 4B — ELO 1436, #497
Gemini 2.0 Flash-Lite (Feb '25) — ELO 1434, #498
Granite 4.1 30B — ELO 1431, #502
Llama 3.1 Tulu3 405B — ELO 1425, #510
Gemma 4 E2B (Reasoning) — ELO 1405, #531
Gemma 4 E4B (Non-reasoning) — ELO 1405, #532
Solar Pro 2 (Non-reasoning) — ELO 1403, #537
Gemini 1.5 Flash-8B — ELO 1400, #540
QwQ 32B-Preview — ELO 1386, #556
EXAONE 4.0 32B (Non-reasoning) — ELO 1382, #559
Llama 3.3 Nemotron Super 49B v1 (Non-reasoning) — ELO 1365, #575
Gemma 4 E2B (Non-reasoning) — ELO 1355, #585
Llama 3.1 Nemotron Nano 4B v1.1 (Reasoning) — ELO 1348, #593
DeepHermes 3 - Mistral 24B Preview (Non-reasoning) — ELO 1343, #595
Qwen2.5 Coder Instruct 7B — ELO 1312, #622
Ling-mini-2.0 — ELO 1305, #628
Gemma 3n E4B Instruct Preview (May '25) — ELO 1304, #629
Granite 4.1 3B — ELO 1302, #632
Jamba Reasoning 3B — ELO 1275, #642
LFM 40B — ELO 1265, #650
Exaone 4.0 1.2B (Reasoning) — ELO 1260, #655
Llama 2 Chat 13B — ELO 1242, #666
Exaone 4.0 1.2B (Non-reasoning) — ELO 1234, #671
Granite 4.0 H 1B — ELO 1206, #688
Molmo 7B-D — ELO 1204, #689
DeepHermes 3 - Llama-3.1 8B Preview (Non-reasoning) — ELO 1198, #693
Granite 4.0 1B — ELO 1197, #694
OLMo 2 32B — ELO 1142, #726
Phi-3 Mini Instruct 3.8B — ELO 1126, #732
Granite 4.0 350M — ELO 1103, #738
Gemma 3 270M — ELO 1088, #744
Granite 4.0 H 350M — ELO 1077, #748
OLMo 2 7B — ELO 1072, #752

Top-10 New Scores (1)

GPT-5.5 Pro on VoxelBench: 2122.0 (#1)

New #1 Leaders (1)

VoxelBench: GPT-5.5 Pro (2122.0) beat GPT-5.5 (xHigh) by 100.0

AI Benchmark Digest — 2026-05-03

2026-05-03T07:35:30.739888+00:00

Daily

New Benchmarks (9)

Open-R1 Eval Leaderboard (Average Accuracy (%)): leader Qwen3-32B (73.74), 37 models
SciEvalKit (Scientific Capability Score): leader Gemini-3-Pro (48.74), 10 models
LLM Benchmarker Suite (Average Score (%)): leader LLaMA-2 (70B) (62.53), 8 models
FastEval (Total Score): leader GPT-4-0613 (77.78), 33 models
LMArena Preference Proxy (Evaluator Accuracy (%)): leader gemma-2-9b-it (64.63), 4 models
SeaEval (Average Score (%)): leader GPT4o_0513 (72.86), 30 models
LLMZSZL Leaderboard (Score): leader Qwen2.5-72B-Instruct (69.06), 99 models
Swahili LLM Leaderboard (Average Score (%)): leader Swahili Gemma (61.32), 5 models
MMLU-by-task Leaderboard (MMLU Average (%)): leader FashionGPT-70B-V1.1 (70.99), 1257 models

New #1 Leaders (5)

Spider 2.0-DBT: Databao Agent (58.82) beat SignalPilot Agent by 7.26
Design Arena (Video Editing): happy-horse-1.0 (1329.0) beat wan-v2.7-v2v by 7.0
Chess Puzzles (Epoch AI): gpt-5.5-pro-pre-release_xhigh (64.0) beat gpt-5.4-pro-2026-03-05_xhigh by 5.4
WeirdML: gpt-5.5 (high) (83.9) beat gpt-5.3-codex (xhigh) by 4.6
BridgeBench Hallucination: Grok 4.3 (79.8) beat Gemini 3.1 Pro by 0.7

AI Benchmark Digest — 2026-05-01

2026-05-01T07:44:31.560358+00:00

Daily

New Benchmarks (56)

LIBRA - Passkey (Dataset Total Score (%)): leader GLM-4 9B Chat (100.0), 17 models
LIBRA - MatreshkaYesNo (Dataset Total Score (%)): leader GPT-4o (80.0), 17 models
LIBRA - MatreshkaNames (Dataset Total Score (%)): leader GPT-4o (51.67), 17 models
LIBRA - PasskeyWithLibrusec (Dataset Total Score (%)): leader GLM-4 9B Chat (100.0), 17 models
LIBRA - LibrusecHistory (Dataset Total Score (%)): leader GPT-4o (97.5), 17 models
LIBRA - ruGSM100 (Dataset Total Score (%)): leader GPT-4o (100.0), 17 models
LIBRA - ruSciPassageCount (Dataset Total Score (%)): leader GPT-4o (35.0), 17 models
LIBRA - ru2WikiMultihopQA (Dataset Total Score (%)): leader GPT-4o (76.67), 17 models
LIBRA - LongContextMultiQ (Dataset Total Score (%)): leader GPT-4o (36.67), 17 models
LIBRA - ruSciAbstractRetrieval (Dataset Total Score (%)): leader GLM-4 9B Chat (77.81), 17 models
LIBRA - ruTREC (Dataset Total Score (%)): leader GPT-4o (75.0), 17 models
LIBRA - ruSciFi (Dataset Total Score (%)): leader GPT-4o (75.0), 17 models
LIBRA - LibrusecMHQA (Dataset Total Score (%)): leader GPT-4o (50.0), 17 models
LIBRA - ruBABILongQA1 (Dataset Total Score (%)): leader GPT-4o (78.33), 17 models
LIBRA - ruBABILongQA2 (Dataset Total Score (%)): leader GPT-4o (36.67), 17 models
LIBRA - ruBABILongQA3 (Dataset Total Score (%)): leader Llama 3.1 8B (29.65), 17 models
LIBRA - ruBABILongQA4 (Dataset Total Score (%)): leader GPT-4o (78.95), 17 models
LIBRA - ruBABILongQA5 (Dataset Total Score (%)): leader GPT-4o (90.0), 17 models
LIBRA - ruQuALITY (Dataset Total Score (%)): leader GPT-4o (83.33), 17 models
LIBRA - ruTPO (Dataset Total Score (%)): leader GPT-4o (100.0), 17 models
LIBRA - ruQasper (Dataset Total Score (%)): leader GPT-4o (31.72), 17 models
Wolfram LLM Benchmarking Project (Correct Functionality (%)): leader Claude Opus 4.7 thinking on (72.5), 443 models
MathArena - Project Euler 943-970 (Accuracy (%, direct Project Euler problems 943-970)): leader GPT-5.4 (xhigh) (87.5), 17 models
MathArena - Project Euler 971-984 (Accuracy (%, direct Project Euler problems 971-984)): leader Claude-Opus-4.6 (High) (92.86), 10 models
MathArena - Project Euler 985-988 (Accuracy (%, direct Project Euler problems 985-988)): leader Gemini 3.1 Pro Preview (100.0), 5 models
OpenVLM OCRBench (Score (normalized)): leader JT-VL-Chat-V3.0 (95.0), 285 models
Vals AI Vibe Code Bench (Accuracy (%)): leader claude-opus-4-7 (71.0), 41 models
EuroEval Albanian (Average Score (%)): leader gemini-3.1-pro-preview (65.43), 209 models
EuroEval Bosnian (Average Score (%)): leader gpt-4.1-mini-2025-04-14 (63.93), 218 models
EuroEval Bulgarian (Average Score (%)): leader gemini-3-pro-preview (74.47), 219 models
EuroEval Catalan (Average Score (%)): leader gemini-2.5-flash#thinking (68.12), 219 models
EuroEval Croatian (Average Score (%)): leader gemini-3-pro-preview (69.99), 218 models
EuroEval Czech (Average Score (%)): leader gemini-2.5-pro (70.02), 236 models
EuroEval Danish (Average Score (%)): leader gpt-5-2025-08-07#high (78.81), 454 models
EuroEval Dutch (Average Score (%)): leader Llama-3.1-405B (78.43), 350 models
EuroEval Estonian (Average Score (%)): leader gemini-2.5-pro (62.38), 258 models
EuroEval Faroese (Average Score (%)): leader gemini-3-pro-preview (70.72), 391 models
EuroEval Finnish (Average Score (%)): leader gpt-5-2025-08-07#high (72.92), 382 models
EuroEval French (Average Score (%)): leader gemini-3-pro-preview (74.38), 383 models
EuroEval German (Average Score (%)): leader Qwen3-235B-A22B-Thinking-2507-FP8 (68.41), 329 models
EuroEval Greek (Average Score (%)): leader gpt-5-2025-08-07 (72.28), 209 models
EuroEval Hungarian (Average Score (%)): leader gemini-2.5-pro (67.51), 208 models
EuroEval Icelandic (Average Score (%)): leader gpt-5-2025-08-07 (70.59), 399 models
EuroEval Italian (Average Score (%)): leader Qwen3-235B-A22B-Thinking-2507-FP8 (73.12), 435 models
EuroEval Latvian (Average Score (%)): leader gpt-5-2025-08-07 (70.85), 238 models
EuroEval Lithuanian (Average Score (%)): leader Qwen3-235B-A22B-Thinking-2507-FP8 (66.49), 235 models
EuroEval Norwegian (Average Score (%)): leader gpt-5-2025-08-07 (76.81), 466 models
EuroEval Polish (Average Score (%)): leader gpt-5-2025-08-07 (71.84), 241 models
EuroEval Portuguese (Average Score (%)): leader Qwen3-235B-A22B-Thinking-2507-FP8 (73.86), 445 models
EuroEval Romanian (Average Score (%)): leader gpt-5-2025-08-07 (72.03), 212 models
EuroEval Serbian (Average Score (%)): leader gpt-5-2025-08-07 (72.24), 209 models
EuroEval Slovak (Average Score (%)): leader gemini-3-pro-preview (68.36), 208 models
EuroEval Slovene (Average Score (%)): leader claude-sonnet-4-5-20250929#thinking (67.68), 208 models
EuroEval Spanish (Average Score (%)): leader Qwen3-235B-A22B-Thinking-2507-FP8 (68.78), 419 models
EuroEval Swedish (Average Score (%)): leader gpt-5-2025-08-07#high (78.64), 410 models
EuroEval Ukrainian (Average Score (%)): leader gpt-5-2025-08-07 (67.31), 205 models

New Models (3)

Grok 4.3 — ELO 1826, #104
- AA IFBench: 81.3 (#2/409)
Mistral Medium 3.5 — ELO 1749, #189
Hy3-preview (Non-reasoning) — ELO 1711, #233

New #1 Leaders (5)

Design Arena (Data Viz): mimo-v2.5-pro (1375.0) beat claude-sonnet-4-6 by 29.0
Vals AI CaseLaw v2: grok-4.3 (79.31) beat gpt-5.1-2025-11-13 by 5.89
Vals AI Terminal-Bench 2.0: gpt-5.5 (73.2) beat claude-opus-4-7 by 4.66
OpenClawProBench: qwen3.5-397b-a17b (70.4) beat qwen3.5-plus by 0.3
Vals AI CorpFin v2: grok-4.3 (68.53) beat gpt-5.5 by 0.11

AI Benchmark Digest — 2026-04-30

2026-04-30T07:34:47.603527+00:00

Daily

New Models (9)

kimi-k2.6_nitro — ELO 1888, #57
- GACL - WordMatrix: 66.72 (#2/21)
Kimi K2.6 (Non-reasoning) — ELO 1808, #106
deepseek-v4-flash_nitro — ELO 1785, #126
DeepSeek V4 Pro (Non-reasoning) — ELO 1763, #148
DeepSeek V4 Flash (Non-reasoning) — ELO 1733, #187
MiMo-V2.5-Pro (Non-reasoning) — ELO 1729, #190
Granite 4.1 30B — ELO 1502, #485
Granite 4.1 8B — ELO 1445, #600
Granite 4.1 3B — ELO 1372, #742

New #1 Leaders (1)

GACL - Tic-Tac-Toe: claude-sonnet-4.6 (83.6) beat claude-opus-4.6 by 20.46

AI Benchmark Digest — 2026-04-29

2026-04-29T07:19:43.831412+00:00

Daily

New Benchmarks (29)

OpenVLM MME (Overall Score): leader InternVL3-78B (2538.6), 235 models
OpenVLM ScienceQA Test (Accuracy (%)): leader InternVL2.5-78B-MPO (99.5), 218 models
OpenVLM POPE (Overall (%)): leader InternVL2.5-26B-MPO (90.5), 216 models
OpenVLM SEED-Bench 2 Plus (Accuracy (%)): leader Qwen2.5-VL-72B (73.8), 211 models
OpenVLM COCO Captions (CIDEr): leader Emu2_chat (109.2), 211 models
OpenVLM MMT-Bench (Accuracy (%)): leader InternVL3-78B (72.6), 207 models
OpenVLM A-Bench (Accuracy (%)): leader Qwen2.5-VL-72B (81.0), 160 models
OpenVLM MTVQA (Accuracy (%)): leader GPT-4.1-mini-20250414 (36.8), 157 models
OpenVLM OCR-VQA (Accuracy (%)): leader Kimi-VL-A3B-Instruct (82.0), 118 models
OpenVLM SEED-Bench 2 (Accuracy (%)): leader GPT-4.1-20250414 (76.0), 59 models
OpenVLM VCR (Overall Jaccard (%)): leader Qwen2-VL-7B (75.6), 48 models
CLEM Clemscore (Clemscore (%)): leader claude-sonnet-4-5-azure-high (90.1), 31 models
CLEM AdventureGame (Game Clemscore (%)): leader gpt-5.2-azure-high (99.17), 31 models
CLEM Clean Up (Game Clemscore (%)): leader gpt-5.2-azure-high (100.0), 31 models
CLEM Codenames (Game Clemscore (%)): leader gpt-5.2-azure-high (87.69), 31 models
CLEM Deal or No Deal (Game Clemscore (%)): leader gpt-5.2-azure-high (99.12), 31 models
CLEM GuessWhat (Game Clemscore (%)): leader gpt-5.2-azure-high (93.33), 31 models
CLEM Hot Air Balloon (Game Clemscore (%)): leader claude-sonnet-4-5-azure-high (95.53), 31 models
CLEM ImageGame (Game Clemscore (%)): leader gpt-5.2-2025-12-11 (99.92), 31 models
CLEM MatchIt ASCII (Game Clemscore (%)): leader claude-sonnet-4-5-20250929 (100.0), 31 models
CLEM PrivateShared (Game Clemscore (%)): leader claude-sonnet-4-5-20250929 (98.7), 31 models
CLEM ReferenceGame (Game Clemscore (%)): leader claude-sonnet-4-5-20250929 (100.0), 31 models
CLEM Taboo (Game Clemscore (%)): leader claude-sonnet-4-5-azure-low (98.33), 31 models
CLEM TextMapWorld (Game Clemscore (%)): leader gemini-3-flash (91.35), 31 models
CLEM TextMapWorld GraphReasoning (Game Clemscore (%)): leader claude-sonnet-4-5-azure-low (86.34), 31 models
CLEM TextMapWorld SpecificRoom (Game Clemscore (%)): leader Llama-3.1-70B-Instruct (100.0), 31 models
CLEM Wordle (Game Clemscore (%)): leader kimi-k2-thinking (73.0), 31 models
CLEM Wordle with Clue (Game Clemscore (%)): leader claude-sonnet-4-5-azure-high (82.5), 31 models
CLEM Wordle with Critic (Game Clemscore (%)): leader claude-sonnet-4-5-azure-high (86.11), 31 models

New Models (1)

JSL-MedMNX-7B-SFT — ELO 1309, #841

New #1 Leaders (1)

Epoch AI - ECI: GPT-5.5 Pro (xhigh) (158.67) beat GPT-5.4 Pro (xhigh) by 0.38

AI Benchmark Digest — 2026-04-28

2026-04-28T07:42:16.928716+00:00

Daily

New Benchmarks (2)

PredictionArena (Polymarket) (Account Value ($)): leader claude-opus-4-6 (77298.59), 10 models
PredictionArena (Kalshi) (Account Value ($)): leader gemini-3.1-pro-preview (15363.0), 10 models

New Models (4)

Hy3-preview (Reasoning) — ELO 1839, #83
EXAONE 4.5 33B — ELO 1698, #224
llama3-slerp-med — ELO 1338, #770
BioMistralMerged — ELO 1262, #921

New #1 Leaders (4)

MineBench: GPT 5.5 Pro (2080.73) beat GPT 5.4 Pro by 364.29
GSO-Bench: Claude Opus 4.7 (44.12) beat Claude-4.6-Opus by 10.79
Epoch AI - Apex Agents: gpt-5.5_xhigh (38.4) beat gpt-5.4-2026-03-05_xhigh by 2.5
Design Arena (Game Dev): gpt-5.5 (1360.0) beat claude-opus-4-7 by 2.0