AI Benchmark Digest

AI Benchmark Digest — 2026-06-18

2026-06-18T07:17:43.853003+00:00

Daily

New Benchmarks (9)

AISI Cyber Cooling Tower 10M (Avg Steps (/7)): leader Claude Opus 4.6 (0.1), 7 models
AISI cyber range: "Cooling Tower" — a 7-step industrial-control-network attack simulation. Reports average steps completed at a 10M token budget.
AISI Cyber Cooling Tower 100M (Avg Steps (/7)): leader Claude Opus 4.6 (1.4), 5 models
AISI cyber range: "Cooling Tower" — a 7-step industrial-control-network attack simulation. Reports average steps completed at a 100M token budget.
OpenAI CTF (Professional) (pass@12 (%)): leader GPT-5.5 (96.3), 3 models
OpenAI system-card subset of professional capture-the-flag tasks, reporting pass@12 over offensive-security rollouts with a Linux tool harness.
CVE-Bench (pass@1 (%)): leader GPT-5.5 (93.1), 4 models
Cybersecurity benchmark for autonomous web vulnerability exploitation across 40 critical CVEs in zero-day and one-day settings.
OpenAI Cyber Ranges (Combined Pass Rate (%)): leader GPT-5.5 (93.33), 4 models
OpenAI internal cyber-range suite measuring end-to-end cyber operations across realistic emulated networks.
ExploitGym (Successful Intended Exploits (#)): leader Claude Mythos Preview (157.0), 7 models
Real-world cybersecurity agent benchmark measuring whether AI agents can turn known software vulnerabilities into working, intended exploits across userspace, V8, and Linux kernel targets.
CyScenarioBench (Average Success Rate (%)): leader Claude Mythos 5 (36.7), 9 models
Irregular scenario-based offensive security benchmark measuring whether agents can plan and complete full multi-stage attack scenarios in realistic environments.
Lyptus Cyber Time Horizons - InterCode-CTF (pass@1 at 2M tokens (%)): leader Claude Opus 4.6 (100.0), 3 models
Lyptus Research offensive cyber time-horizon run of InterCode-CTF, measuring pass@1 on CTF tasks at a 2M token budget.
Lyptus Cyber Time Horizons - NL2Bash (pass@1 at 2M tokens (%)): leader GPT-5.3 Codex (100.0), 3 models
Lyptus Research offensive cyber time-horizon run of NL2Bash, measuring command-generation success at a 2M token budget.

Top-10 New Scores (2)

GPT-5.4 Pro on FrontierMath - Tier 4 (v2): 58.54 (#5)
GPT-5.4 Pro on FrontierMath - Tiers 1-3 (v2): 82.46 (#4)

New #1 Leaders (2)

Terminal-Bench 2.1 (Claude Code): Claude 5 Fable (83.1) beat Claude Opus 4.8 by 4.2
Terminal-Bench 2.1 (Terminus 2): Claude 5 Fable (80.4) beat GPT-5.5 by 2.2

AI Benchmark Digest — 2026-06-17

2026-06-17T07:26:00.903157+00:00

Daily

New Benchmarks (4)

LLM Stats (Finance Agent v2) (Score (%)): leader Gemini 3.5 Flash (57.86), 25 models
LLM Stats (FrontierSWE) (Score (%)): leader Claude Fable 5 (90.0), 13 models
LLM Stats (Legal Agent Benchmark) (Score (%)): leader Claude Fable 5 (13.3), 11 models
LLM Stats (SkillsBench) (Score (%)): leader Qwen3.7 Max (59.2), 5 models

Top-10 New Scores (12)

Claude Fable 5 on SWE-Marathon: 24.0 (#2)
GLM-5.2 on BenchLM: 94.0 (#3)
GLM-5.2 on LLM Stats (HMMT 2025): 94.4 (#9)
GLM-5.2 on LLM Stats (HMMT Feb 26): 92.5 (#6)
GLM-5.2 on LLM Stats (IMO-AnswerBench): 91.0 (#2)
GLM-5.2 on LLM Stats (MCP Atlas): 76.8 (#4)
GLM-5.2 on LLM Stats (Toolathlon): 48.2 (#8)
GLM-5.2 on PinchBench: 87.79 (#18)
GLM-5.2 on RuneBench: 3230.0 (#4)
GLM-5.2 on SWE-Marathon: 13.0 (#4)
GLM-5.2 on ZeroEval GPQA Diamond: 91.2 (#12)
Qwen 3.7 Max on LLM Stats (GDPval-AA): 1308.0 (#12)

New #1 Leaders (15)

LLM Stats (DeepPlanning): Qwen 3.7 Plus (62.3) beat Qwen 3.6 Plus by 20.8
Coding Agent Leaderboard - swe-bench-pro--ansible: Opus 4.8 + Claude Code (69.8) beat Sonnet 4.6 + Claude Code by 19.8
LLM Stats (MRCR v2): Qwen 3.7 Plus (91.7) beat U2 by 15.09
Coding Agent Leaderboard: Opus 4.8 + Claude Code (78.3) beat Sonnet 4.6 + Claude Code by 13.5
Design Arena (Website): silo (1357.0) beat Claude Fable 5 by 12.0
Coding Agent Leaderboard - swe-bench-verified: Opus 4.8 + Claude Code (86.8) beat Sonnet 4.6 + Claude Code by 7.2
LLM Stats (ERQA): Qwen 3.7 Plus (69.8) beat Qwen 3.6 Plus by 4.1
LLM Stats (SimpleVQA): Qwen 3.7 Plus (81.7) beat GLM-5V Turbo by 3.5
LLM Stats (AIME 2026): GLM-5.2 (99.2) beat Kimi K2.6 by 2.8
LLM Stats (IMO-AnswerBench): Nemotron 3 Ultra (550B A55B) (92.3) beat Qwen 3.7 Max by 2.3
LLM Stats (NL2Repo): GLM-5.2 (48.9) beat Qwen 3.7 Max by 1.7
LLM Stats (RealWorldQA): Qwen 3.7 Plus (86.9) beat Qwen 3.6 Plus by 1.5
LLM Stats (LVBench): Qwen 3.7 Plus (76.2) beat Kimi K2.5 by 0.3
LLM Stats (Video-MME): Qwen 3.7 Plus (88.0) beat MiMo-V2.5 by 0.3
LLM Stats (MLVU): Qwen 3.7 Plus (87.4) beat Qwen 3.5 122B A10B by 0.1

AI Benchmark Digest — 2026-06-16

2026-06-16T08:27:51.523101+00:00

Daily

New Benchmarks (7)

SWE-Marathon (Pass@1 (%)): leader Claude Opus 4.8 (26.0), 9 models
Long-horizon software engineering benchmark where coding agents work on realistic repository tasks under marathon-scale time budgets, reporting pass@1 for end-to-end completed tasks.
InferenceBench (Speedup Score): leader Claude Fable 5 (Low) (8.74), 22 models
Benchmark for coding agents optimizing inference workloads. Agents tune serving configurations and implementation choices across latency, throughput, and all-in-one scenarios.
AgenticVBench (Average Success (%)): leader Claude Fable 5 (32.4), 9 models
Agentic video benchmark where autonomous agents perform multi-step video repurposing, sequencing, repair, and assembly tasks, scored by average task success.
TERMS-Bench (Mean Utility): leader GLM 5.1 (11.7), 15 models
Negotiation benchmark for LLM agents bargaining over terms under changing utility, urgency, and no-deal regimes, reporting mean utility and agreement metrics.
Structured Output Benchmark (Overall (%)): leader GPT-5.4 (87.0), 28 models
Structured-output benchmark measuring schema-constrained generation with value accuracy, faithfulness, JSON validity, path recall, type safety, and perfect-output rates.
BenGER (Aggregate Accuracy (%)): leader Gemini 3.1 Pro (77.0), 12 models
German-law benchmark for subsumption-based legal reasoning, evaluating model answers across Benchathon, ZJS, and doctrinal-principles corpora.
BenchLM (Overall Score): leader Claude Mythos 5 (99.0), 123 models
Composite LLM leaderboard aggregating current model performance across agentic, coding, reasoning, grounded multimodal, knowledge, multilingual, instruction-following, and math categories.

Top-10 New Scores (3)

Claude Fable 5 on Chatbot Arena (Search): 1237.0 (#3)
Claude Fable 5 on Epoch AI - ECI: 160.87 (#3)
Claude Opus 4.8 on Chatbot Arena (Search): 1203.0 (#11)

New #1 Leaders (2)

LLM Stats (MRCR v2): U2 (76.61) beat Gemma 4 31B by 10.21
Epoch AI - ECI: Claude Fable 5 (Max) (160.87) beat GPT-5.5 Pro (xHigh) by 1.97

AI Benchmark Digest — 2026-06-15

2026-06-15T08:24:20.247016+00:00

Daily

New Benchmarks (145)

Open LLM Leaderboard - IFEval (Score): leader Llama-3.3-70B-Instruct (89.98), 4576 models
Open LLM Leaderboard - BBH (Score): leader Benchmaxx-Llama-3.2-1B-Instruct (76.7), 4576 models
Open LLM Leaderboard - MATH Level 5 (Score): leader AceMath-72B-Instruct (71.45), 4576 models
Open LLM Leaderboard - GPQA (Score): leader L3.3-MS-Nevoria-70b (29.42), 4576 models
Open LLM Leaderboard - MuSR (Score): leader T3Q-Qwen2.5-14B-Instruct-1M-e3 (38.69), 4576 models
Open LLM Leaderboard - MMLU-Pro (Score): leader calme-3.2-instruct-78b (70.03), 4576 models
AI for Education Pedagogy (Accuracy (%)): leader GPT-5.5 (92.1), 216 models
AI for Education Pedagogy - Maths (Accuracy (%)): leader Gemini-3.1 Pro (94.44), 216 models
AI for Education Pedagogy - Primary (Accuracy (%)): leader GPT-5.5 (96.71), 216 models
AI for Education Pedagogy - Science (Accuracy (%)): leader Qwen3.5 Plus (95.08), 216 models
AI for Education Pedagogy - Secondary (Accuracy (%)): leader GPT-5.5 (91.04), 216 models
AI for Education Pedagogy - Social studies (Accuracy (%)): leader o3 (91.82), 216 models
AI for Education Pedagogy - Technology (Accuracy (%)): leader Kimi K2.5 (89.62), 216 models
AI for Education SEND (Accuracy (%)): leader GPT-5.5 (88.07), 208 models
AI for Education Visual Maths (Accuracy (%)): leader GPT-5.5 (89.87), 61 models
AI for Education Visual Maths - Algebra (Accuracy (%)): leader Gemini-2.5 Pro (100.0), 61 models
AI for Education Visual Maths - Geometry (Accuracy (%)): leader GPT-5.5 (88.46), 61 models
AI for Education Visual Maths - Measurement (Accuracy (%)): leader GPT-5.5 (97.3), 61 models
AI for Education Visual Maths - Number and Operations (Accuracy (%)): leader GPT-5.5 (83.78), 61 models
AI for Education Visual Maths - Statistics and Probability (Accuracy (%)): leader GPT-5.5 (85.71), 61 models
AI for Education Visual Reasoning (Accuracy (%)): leader Gemini-3.5 Flash (86.0), 63 models
AI for Education Visual Reasoning - match (figure) (Accuracy (%)): leader Gemini-3.5 Flash (85.2), 63 models
AI for Education Visual Reasoning - match (process) (Accuracy (%)): leader Gemini-3 Flash (77.8), 63 models
AI for Education Visual Reasoning - odd one out (Accuracy (%)): leader Gemini-3.5 Flash (80.5), 63 models
AI for Education Visual Reasoning - pattern completion (2d) (Accuracy (%)): leader Gemini-3.1 Pro (86.3), 63 models
AI for Education Visual Reasoning - pattern completion (linear) (Accuracy (%)): leader Gemini-3.5 Flash (91.5), 63 models
AI for Education Visual Reasoning - reasoning by analogy (Accuracy (%)): leader Gemini-3.5 Flash (88.8), 63 models
SWE-bench Verified (Opus 4.6 System Card) (Resolved (%)): leader Claude Opus 4.5 (Thinking) (80.9), 5 models
Terminal-Bench 2.0 (Opus 4.6 System Card) (Pass Rate (%)): leader Claude Opus 4.6 (Thinking) (65.4), 5 models
Tau2 Bench Retail (Opus 4.6 System Card) (Score (%)): leader Claude Opus 4.6 (Thinking) (91.9), 5 models
Tau2 Bench Telecom (Opus 4.6 System Card) (Score (%)): leader Claude Opus 4.6 (Thinking) (99.3), 5 models
MCP-Atlas (Opus 4.6 System Card) (Score (%)): leader Claude Opus 4.5 (Thinking) (62.3), 5 models
ARC-AGI-2 Verified (Opus 4.6 System Card) (Score (%)): leader Claude Opus 4.6 (Thinking) (68.8), 5 models
GPQA Diamond (Opus 4.6 System Card) (Accuracy (%)): leader GPT-5.2 (93.2), 5 models
MMMU-Pro No Tools (Opus 4.6 System Card) (Score (%)): leader Gemini 3 Pro (81.0), 5 models
MMMLU (Opus 4.6 System Card) (Accuracy (%)): leader Gemini 3 Pro (91.8), 5 models
SWE-bench Verified (Fable/Mythos) (Resolved (%)): leader Claude Mythos 5 (95.5), 5 models
Terminal-Bench 2.1 (Fable/Mythos) (Mean Reward (%)): leader Claude Mythos 5 (88.0), 5 models
BrowseComp (Fable/Mythos Single-Agent) (Score (%)): leader Claude Mythos 5 (88.0), 4 models
BrowseComp (Fable/Mythos Multi-Agent) (Score (%)): leader Claude Fable 5 (93.3), 2 models
Humanity's Last Exam (Fable/Mythos No Tools) (Score (%)): leader Claude Mythos 5 (59.0), 5 models
Humanity's Last Exam (Fable/Mythos Tools) (Score (%)): leader Claude Mythos Preview (64.7), 5 models
CharXiv Reasoning (Fable/Mythos No Tools) (Score (%)): leader Claude Mythos 5 (88.9), 3 models
CharXiv Reasoning (Fable/Mythos Tools) (Score (%)): leader Claude Mythos 5 (93.5), 3 models
BioMysteryBench Human Solvable (Fable/Mythos) (Score (%)): leader Claude Mythos 5 (83.9), 4 models
BioMysteryBench Human Difficult (Fable/Mythos) (Score (%)): leader Claude Mythos 5 (46.1), 4 models
OSWorld-Verified (Fable/Mythos) (Score (%)): leader Claude Mythos Preview (85.4), 7 models
CritPt (Fable/Mythos) (Score (%)): leader Claude Mythos 5 (28.6), 4 models
ArxivMath (Fable/Mythos) (Score (%)): leader Claude Mythos 5 (78.5), 5 models
RiemannBench (Fable/Mythos) (Score (%)): leader Claude Mythos 5 (55.0), 3 models
GraphWalks BFS 256K (Fable/Mythos) (Score (%)): leader Claude Mythos 5 (91.1), 4 models
GraphWalks Parents 256K (Fable/Mythos) (Score (%)): leader Claude Mythos 5 (99.96), 4 models
FrontierCode Diamond (Fable/Mythos) (Score (%)): leader Claude Fable 5 (29.3), 3 models
GDPval-AA (Fable/Mythos) (Elo): leader Claude Fable 5 (1932.0), 4 models
GDP.pdf (Fable/Mythos) (Strict Pass Rate (%)): leader Claude Fable 5 (29.8), 4 models
AutomationBench (Fable/Mythos) (Score (%)): leader Claude Fable 5 (17.4), 5 models
Blueprint-Bench 2 (Fable/Mythos) (Score (%)): leader Claude Fable 5 (38.6), 5 models
Legal Agent Benchmark Public Set (Fable/Mythos) (All-Pass Rate (%)): leader Claude Mythos 5 (16.9), 3 models
HealthBench (Fable/Mythos) (Score (%)): leader Claude Mythos 5 (62.7), 4 models
HealthBench Professional (Fable/Mythos) (Score (%)): leader Claude Mythos 5 (66.0), 4 models
OpenAI GPT-5.5 Launch - GDPval (wins or ties) (Score (%)): leader GPT-5.5 (84.9), 6 models
OpenAI GPT-5.5 Launch - FinanceAgent v1.1 (Score (%)): leader Claude Opus 4.7 (64.4), 5 models
OpenAI GPT-5.5 Launch - Investment Banking Modeling Tasks (Score (%)): leader GPT-5.5 Pro (88.6), 4 models
OpenAI GPT-5.5 Launch - BrowseComp (Score (%)): leader GPT-5.5 Pro (90.1), 6 models
OpenAI GPT-5.5 Launch - GeneBench (Score (%)): leader GPT-5.5 Pro (33.2), 4 models
OpenAI GPT-5.5 Launch - FrontierMath Tier 1-3 (Score (%)): leader GPT-5.5 Pro (52.4), 6 models
OpenAI GPT-5.5 Launch - FrontierMath Tier 4 (Score (%)): leader GPT-5.5 Pro (39.6), 6 models
OpenAI GPT-5.5 Launch - GPQA Diamond (Score (%)): leader GPT-5.4 Pro (94.4), 5 models
OpenAI GPT-5.5 Launch - Humanity's Last Exam (no tools) (Score (%)): leader Claude Opus 4.7 (46.9), 6 models
OpenAI GPT-5.5 Launch - Humanity's Last Exam (with tools) (Score (%)): leader GPT-5.4 Pro (58.7), 6 models
OpenAI GPT-5.5 Launch - ARC-AGI-1 (Verified) (Score (%)): leader Gemini 3.1 Pro (98.0), 5 models
OpenAI GPT-5.5 Launch - ARC-AGI-2 (Verified) (Score (%)): leader GPT-5.5 (85.0), 5 models
OpenAI GPT-5.4 Launch - GDPval (Score (%)): leader GPT-5.4 (83.0), 5 models
OpenAI GPT-5.4 Launch - FinanceAgent v1.1 (Score (%)): leader GPT-5.4 Pro (61.5), 4 models
OpenAI GPT-5.4 Launch - Investment Banking Modeling Tasks (Score (%)): leader GPT-5.4 (87.3), 5 models
OpenAI GPT-5.4 Launch - BrowseComp (Score (%)): leader GPT-5.4 Pro (89.3), 5 models
OpenAI GPT-5.4 Launch - Frontier Science Research (Score (%)): leader GPT-5.4 Pro (36.7), 3 models
OpenAI GPT-5.4 Launch - FrontierMath Tier 1-3 (Score (%)): leader GPT-5.4 Pro (50.0), 3 models
OpenAI GPT-5.4 Launch - FrontierMath Tier 4 (Score (%)): leader GPT-5.4 Pro (38.0), 4 models
OpenAI GPT-5.4 Launch - GPQA Diamond (Score (%)): leader GPT-5.4 Pro (94.4), 5 models
OpenAI GPT-5.4 Launch - Humanity's Last Exam (no tools) (Score (%)): leader GPT-5.4 Pro (42.7), 4 models
OpenAI GPT-5.4 Launch - Humanity's Last Exam (with tools) (Score (%)): leader GPT-5.4 Pro (58.7), 4 models
OpenAI GPT-5.4 Launch - ARC-AGI-1 (Verified) (Score (%)): leader GPT-5.4 Pro (94.5), 4 models
OpenAI GPT-5.4 Launch - ARC-AGI-2 (Verified) (Score (%)): leader GPT-5.4 Pro (83.3), 4 models
OpenAI GPT-5.5 System Card - Tacit Knowledge and Troubleshooting (Score (%)): leader GPT-5.5 Pro (81.67), 2 models
OpenAI GPT-5.5 System Card - Biochemistry Knowledge Improvement (reward@4 (%)): leader GPT-5.5 Pro (39.26), 3 models
OpenAI GPT-5.5 System Card - Hard Negative Protein Binding Prediction (pass@4 (%)): leader GPT-5.4 (Thinking) (3.46), 3 models
OpenAI GPT-5.5 System Card - DNA Sequence Design for TF Binding (pass@1 (%)): leader GPT-5.5 Pro (16.5), 3 models
OpenAI GPT-Rosalind-5.5 System Card - ProtocolQA Open-Ended (pass@1 (%)): leader GPT-5.5 (37.3), 3 models
OpenAI GPT-Rosalind-5.5 System Card - TroubleshootingBench (pass@1 (%)): leader GPT-Rosalind-5.5 (53.31), 3 models
OpenAI GPT-Rosalind-5.5 System Card - Biorisk Knowledge (cons@32 (%)): leader GPT-5.5 Pro (81.67), 3 models
OpenAI GPT-Rosalind-5.5 System Card - Multi-select Virology Troubleshooting (pass@1 (%)): leader GPT-5.5 Pro (55.34), 3 models
OpenAI GPT-Rosalind-5.5 System Card - Hard Negative Protein Binding Prediction (pass@4 (%)): leader GPT-Rosalind-5.5 (3.13), 3 models
OpenAI GPT-Rosalind-5.5 System Card - DNA Sequence Design for TF Binding (pass@1 (%)): leader GPT-5.5 Pro (16.5), 3 models
Google Gemini 3 Deep Think - ARC-AGI-2 (Score (%)): leader Gemini 3 Deep Think (84.6), 4 models
Google Gemini 3 Deep Think - Humanity's Last Exam (no tools) (Score (%)): leader Gemini 3 Deep Think (48.4), 4 models
Google Gemini 3 Deep Think - Humanity's Last Exam (search and code) (Score (%)): leader Gemini 3 Deep Think (53.4), 4 models
Google Gemini 3 Deep Think - MMMU-Pro (Score (%)): leader Gemini 3 Deep Think (81.5), 4 models
Google Gemini 3 Deep Think - International Math Olympiad 2025 (Score (%)): leader Gemini 3 Deep Think (81.5), 3 models
Google Gemini 3 Deep Think - Codeforces (Elo): leader Gemini 3 Deep Think (3455.0), 3 models
Google Gemini 3 Deep Think - International Physics Olympiad 2025 (theory) (Score (%)): leader Gemini 3 Deep Think (87.7), 4 models
Google Gemini 3 Deep Think - CMT-Benchmark (Pass@8 (%)): leader Gemini 3 Deep Think (50.5), 4 models
Google Gemini 3 Deep Think - International Chemistry Olympiad 2025 (theory) (Score (%)): leader Gemini 3 Deep Think (82.8), 3 models
Qwen3.7 Launch - Terminal Bench 2.0-Terminus (Score (%)): leader Qwen 3.7 Max (69.7), 6 models
Qwen3.7 Launch - SWE-Verified (Resolved (%)): leader Claude Opus 4.6 (Thinking) (80.8), 5 models
Qwen3.7 Launch - SWE-Pro (Resolved (%)): leader Qwen 3.7 Max (60.6), 6 models
Qwen3.7 Launch - SWE-Multilingual (Resolved (%)): leader Qwen 3.7 Max (78.3), 5 models
Qwen3.7 Launch - NL2repo (Score (%)): leader Claude Opus 4.6 (Thinking) (47.6), 6 models
Qwen3.7 Launch - SciCode (Score (%)): leader Qwen 3.7 Max (53.5), 5 models
Qwen3.7 Launch - QwenWebDev (Elo): leader Claude Opus 4.6 (Thinking) (1617.0), 5 models
Qwen3.7 Launch - QwenSVG (Elo): leader Qwen 3.7 Max (1608.0), 6 models
Qwen3.7 Launch - Qwenclaw (Score (%)): leader Claude Opus 4.6 (Thinking) (65.5), 6 models
Qwen3.7 Launch - CoWorkBench (Score (%)): leader Claude Opus 4.6 (Thinking) (68.2), 6 models
Qwen3.7 Launch - ClawEval (Score (%)): leader Claude Opus 4.6 (Thinking) (70.4), 6 models
Qwen3.7 Launch - Skillsbench (Score (%)): leader Qwen 3.7 Max (59.2), 5 models
Qwen3.7 Launch - BFCL-V4 (Score (%)): leader Claude Opus 4.6 (Thinking) (76.7), 6 models
Qwen3.7 Launch - MCP-Mark (Score (%)): leader Qwen 3.7 Max (60.8), 6 models
Qwen3.7 Launch - MCP-Atlas (Score (%)): leader Qwen 3.7 Max (76.4), 6 models
Qwen3.7 Launch - Vitabench (Score (%)): leader DeepSeek V4 Pro (Reasoning, Max Effort) (51.9), 5 models
Qwen3.7 Launch - SpreadSheetBench-v1 (Score (%)): leader Claude Opus 4.6 (Thinking) (89.3), 6 models
Qwen3.7 Launch - Kernel Bench L3 - Median Speedup (Median speedup (x)): leader Claude Opus 4.6 (Thinking) (2.63), 6 models
Qwen3.7 Launch - Kernel Bench L3 - Win Rate (Problems faster than torch.compile (%)): leader Claude Opus 4.6 (Thinking) (98.0), 6 models
Qwen3.7 Launch - Humanity's Last Exam (with tools) (Score (%)): leader Kimi K2.6 (Thinking) (54.0), 6 models
Qwen3.7 Launch - QwenWorldBench (Score (%)): leader Qwen 3.7 Max (57.3), 6 models
Qwen3.7 Launch - GPQA Diamond (Score (%)): leader Qwen 3.7 Max (92.4), 6 models
Qwen3.7 Launch - Humanity's Last Exam (Score (%)): leader Qwen 3.7 Max (41.4), 6 models
Qwen3.7 Launch - LiveCodeBench (Score (%)): leader DeepSeek V4 Pro (Reasoning, Max Effort) (93.5), 5 models
Qwen3.7 Launch - HMMT 2026 Feb (Score (%)): leader Qwen 3.7 Max (97.1), 6 models
Qwen3.7 Launch - IMOAnswerBench (Score (%)): leader Qwen 3.7 Max (90.0), 6 models
Qwen3.7 Launch - CritPT (Score (%)): leader DeepSeek V4 Pro (Reasoning, Max Effort) (12.9), 6 models
Qwen3.7 Launch - Apex (Score (%)): leader Qwen 3.7 Max (44.5), 6 models
Qwen3.7 Launch - MMLU-Pro (Score (%)): leader Claude Opus 4.6 (Thinking) (89.7), 6 models
Qwen3.7 Launch - MMLU-Redux (Score (%)): leader Kimi K2.6 (Thinking) (95.3), 6 models
Qwen3.7 Launch - SuperGPQA (Score (%)): leader Qwen 3.7 Max (73.6), 6 models
Qwen3.7 Launch - IFEval (Score (%)): leader Kimi K2.6 (Thinking) (94.5), 6 models
Qwen3.7 Launch - IFBench (Score (%)): leader Qwen 3.7 Max (79.1), 6 models
Qwen3.7 Launch - MRCR-v2 128k (Accuracy (%)): leader Qwen 3.7 Max (90.4), 6 models
Qwen3.7 Launch - WMT24++ (Score (%)): leader Qwen 3.7 Max (85.8), 6 models
Qwen3.7 Launch - MAXIFE (Score (%)): leader Qwen 3.7 Max (89.2), 6 models
Qwen3.7 Launch - MMMLU (Score (%)): leader Claude Opus 4.6 (Thinking) (90.6), 6 models
Qwen3.7 Launch - MMLU-ProX (Score (%)): leader Qwen 3.7 Max (87.0), 6 models
Qwen3.7 Launch - NOVA-63 (Score (%)): leader Claude Opus 4.6 (Thinking) (59.1), 6 models
Qwen3.7 Launch - INCLUDE (Score (%)): leader Claude Opus 4.6 (Thinking) (87.4), 6 models
Qwen3.7 Launch - Global PIQA (Score (%)): leader Qwen 3.7 Max (91.4), 6 models
Qwen3.7 Launch - PolyMATH (Score (%)): leader Qwen 3.7 Max (86.5), 6 models

AI Benchmark Digest — 2026-06-14

2026-06-14T09:01:01.779177+00:00

Daily

New Benchmarks (75)

Ramp SWE-Bench (Resolved (%)): leader Claude Fable 5 (87.5), 14 models
Ramp Labs benchmark for background coding agents on realistic financial software engineering work, scored by resolved tasks with the mini-SWE-agent harness.
CADGenBench (Aggregate CAD Score): leader Claude Fable 5 (0.4514), 11 models
CAD generation and editing benchmark scoring generated CAD artifacts on aggregate geometric and validity metrics across validated submissions.
FrontierMath - Tier 4 (v2) (Accuracy (%, 41 private v2 problems)): leader Claude Fable 5 (max) (87.8), 27 models
Current v2 private Tier 4 FrontierMath expansion set from Epoch AI, measuring accuracy on the hardest unpublished research-level mathematics problems.
FrontierMath - Tiers 1-3 (v2) (Accuracy (%, 285 private v2 problems)): leader GPT-5.5 Pro (xhigh) (87.72), 26 models
Current v2 private FrontierMath base set from Epoch AI, covering original problems from undergraduate through early-postdoc difficulty across major areas of modern mathematics.
Benchmarks.bio - SpatialBench (Pass Rate (%)): leader GPT-5.5 (69.57), 11 models
LatchBio agentic benchmark on messy real-world spatial transcriptomics data, with models writing and running analysis workflows across assays, platforms, and task categories.
Benchmarks.bio - scBench (Pass Rate (%)): leader Claude Mythos 5 (59.3), 13 models
LatchBio agentic benchmark for single-cell RNA-seq analysis, requiring models to perform realistic data cleaning, clustering, cell typing, and differential-expression workflows.
Benchmarks.bio - SpatialBench-Long (Pass Rate (%)): leader Gemini 3.5 Flash (11.11), 12 models
Long-form Benchmarks.bio spatial transcriptomics tasks that require multi-step biological data analysis, tool use, and synthesis over larger assay contexts.
Benchmarks.bio - EpiBench (Pass Rate (%)): leader GPT-5.5 (44.97), 11 models
Benchmarks.bio epigenomics benchmark covering real assays such as chromatin accessibility, binding, and methylation analyses with deterministic graders.
Agent Arena (Net Improvement (%)): leader Grok 4.3 xAI · Proprietary (18.3), 25 models
Arena.ai agent leaderboard measuring net improvement on real-world tool orchestration sessions with success, steerability, recovery, and hallucination metrics.
Agent Arena - Confirmed Success (Confirmed Success (%)): leader Claude Fable 5 (High) (17.21), 25 models
Agent Arena submetric tracking confirmed successful completion rate on real-world agent sessions.
Agent Arena - Praise vs Complaint (Praise vs Complaint (%)): leader Claude Fable 5 (High) (27.74), 25 models
Agent Arena submetric comparing user praise against complaints across agent sessions.
Agent Arena - Steerability (Steerability (%)): leader Nemotron 3 Ultra (23.87), 25 models
Agent Arena submetric measuring how well models adapt to user steering during tool-use sessions.
Agent Arena - Bash Recovery (Bash Recovery (%)): leader Grok 4.3 xAI · Proprietary (60.23), 25 models
Agent Arena submetric measuring recovery from shell or command-line failures in agent sessions.
Agent Arena - Tool Hallucination (Tool Hallucination (%)): leader Grok 4.3 xAI · Proprietary (0.26), 25 models
Agent Arena submetric measuring tool hallucination rate; lower values indicate fewer invented or invalid tool uses.
Agents' Last Exam (Pass Rate (%)): leader GPT-5.5 (24.0), 18 models
Snorkel benchmark of long-horizon economically valuable agent tasks across many industries, reporting workflow pass rate and score.
WolfBench (Average Score (%)): leader GPT-5.5 (77.0), 27 models
Agent benchmark based on Terminal-Bench 2.0 that compares harnesses and models across repeated terminal task runs using aggregate score statistics.
Appwrite Arena (With Skills) (Overall Score (%)): leader GPT-5.5 (97.7), 16 models
Appwrite Arena evaluation of model knowledge and reasoning about Appwrite development tasks when models can use Appwrite skills.
Appwrite Arena (Without Skills) (Overall Score (%)): leader Claude Fable 5 (97.7), 16 models
Appwrite Arena evaluation of model knowledge and reasoning about Appwrite development tasks without Appwrite skill assistance.
Terminal-Bench 2.1 (Accuracy (%)): leader GPT-5.5 (83.4), 6 models
Official Terminal-Bench 2.1 leaderboard measuring agent success on realistic command-line tasks, using each model best available harness row.
Terminal-Bench 2.1 (Claude Code) (Accuracy (%)): leader Claude Opus 4.8 (78.9), 3 models
Terminal-Bench 2.1 results for the Claude Code harness, measuring command-line task completion by model.
Terminal-Bench 2.1 (Terminus 2) (Accuracy (%)): leader GPT-5.5 (78.2), 5 models
Terminal-Bench 2.1 results for the Terminus 2 harness, measuring command-line task completion by model.
Vals AI Finance Agent v2 (Accuracy (%)): leader gemini-3.5-flash (57.86), 29 models
Updated Vals AI financial-research agent benchmark over SEC filings and supporting documents, measuring completion accuracy on realistic analyst workflows.
Vals AI Public Benefits Bench (Accuracy (%)): leader claude-fable-5 (71.65), 13 models
SNAP public-benefits guidance benchmark measuring whether models answer benefits questions accurately while following eligibility and documentation rules.
Vals AI Terminal-Bench 2.1 (Accuracy (%)): leader claude-fable-5 (80.52), 30 models
Updated Terminal-Bench 2.1 evaluation from Vals AI, measuring agentic command-line task completion in sandboxed software and systems environments.
Vals AI LiveCodeBench (Accuracy (%)): leader claude-fable-5 (89.78), 121 models
Vals AI run of LiveCodeBench coding problems, measuring pass rates on recent contest-style programming tasks intended to reduce contamination.
Vals AI GPQA (Accuracy (%)): leader gemini-3.1-pro-preview (95.45), 115 models
Vals AI run of GPQA graduate-level science questions, measuring difficult expert-domain reasoning accuracy.
Vals AI MMLU-Pro (Accuracy (%)): leader claude-fable-5 (91.5), 114 models
Vals AI run of MMLU-Pro multitask academic questions, using harder multi-choice problems across STEM, humanities, and professional domains.
Vals AI MMMU (Accuracy (%)): leader claude-fable-5 (89.31), 76 models
Vals AI run of MMMU multimodal college-level subject questions, measuring visual and textual academic reasoning.
Vals AI SWE-bench Verified (Resolved (%)): leader claude-fable-5 (95.0), 57 models
Vals AI SWE-bench Verified leaderboard, measuring the percentage of real GitHub issues resolved by coding agents.
GDP.pdf (Strict Pass Rate (%)): leader Claude Fable 5 (30.0), 12 models
Surge AI document-reasoning benchmark over 100 professional PDF workflows, scored by strict pass rate against expert-written rubrics.
Riemann-bench (Score (%)): leader Claude Fable 5 (55.0), 15 models
Surge AI frontier mathematics benchmark with advanced research-style problems sourced from mathematicians and scored by solution correctness.
SWE-bench Pro (Anthropic Scaffold) (Pass@1 (%)): leader Claude Mythos 5 (80.3), 6 models
Anthropic system-card run of SWE-bench Pro, measuring pass@1 on production software engineering issues using Anthropic scaffold settings.
OfficeQA Pro (Correctness (%)): leader Claude Fable 5 (57.9), 4 models
Hard OfficeQA subset for frontier document agents, requiring grounded search and numerical reasoning over U.S. Treasury Bulletin documents.
Real-World Finance v2 (Elo): leader Claude Fable 5 (1374.0), 4 models
Anthropic long-horizon finance workflow evaluation using pairwise preference grading and Elo ratings over realistic professional deliverables.
Real-World Finance v1 (Score (%)): leader Claude Mythos Preview (70.9), 4 models
Anthropic curated finance benchmark of 53 tasks evaluated against reference answers with a model-based grader.
Legal Agent Benchmark (Harvey Held-Out) (All-Pass Rate (%)): leader Claude Fable 5 (13.3), 5 models
Harvey legal-agent held-out evaluation using closed-universe matter files and expert rubrics, scored by all-pass task success.
Toolathlon (Anthropic Internal Harness) (Pass@1 (%)): leader Claude Fable 5 (61.7), 7 models
Anthropic internal Toolathlon harness over 108 tool-use tasks, reporting pass@1 for agentic workflow completion.
SWE-bench Verified (Anthropic Scaffold) (Resolved (%)): leader Claude Opus 4.8 (88.6), 3 models
Anthropic system-card run of SWE-bench Verified, measuring real GitHub issue resolution with Anthropic scaffold settings.
SWE-bench Multilingual (Anthropic Scaffold) (Resolved (%)): leader Claude Opus 4.8 (84.4), 2 models
Anthropic system-card run of SWE-bench Multilingual, measuring multilingual software issue resolution with Anthropic scaffold settings.
SWE-bench Multimodal (Anthropic Internal Harness) (Resolved (%)): leader Claude Opus 4.8 (38.4), 2 models
Anthropic internal multimodal SWE-bench harness, measuring software issue resolution that requires visual or multimodal context.
Humanity's Last Exam (Anthropic No Tools) (Accuracy (%)): leader Claude Opus 4.8 (49.8), 4 models
Anthropic system-card run of Humanitys Last Exam without tools, covering expert-level academic reasoning across many domains.
Humanity's Last Exam (Anthropic Tools) (Accuracy (%)): leader Claude Opus 4.8 (57.9), 4 models
Anthropic system-card run of Humanitys Last Exam with tools, covering expert-level academic reasoning across many domains.
ChartQAPro (Anthropic No Tools) (Accuracy (%)): leader Claude Opus 4.8 (69.4), 2 models
Anthropic no-tool run of ChartQAPro, testing chart understanding and quantitative visual reasoning.
ChartQAPro (Anthropic Tools) (Accuracy (%)): leader Claude Opus 4.8 (72.3), 2 models
Anthropic tool-enabled run of ChartQAPro, testing chart understanding and quantitative visual reasoning.
ScreenSpot-Pro (Anthropic No Tools) (Accuracy (%)): leader Claude Opus 4.8 (82.3), 2 models
Anthropic no-tool run of ScreenSpot-Pro, evaluating GUI grounding and screen element localization.
ScreenSpot-Pro (Anthropic Tools) (Accuracy (%)): leader Claude Opus 4.8 (87.9), 2 models
Anthropic tool-enabled run of ScreenSpot-Pro, evaluating GUI grounding and screen element localization.
GraphWalks BFS 256K (Anthropic) (F1 Score (%)): leader Claude Opus 4.8 (85.9), 4 models
Anthropic GraphWalks long-context graph traversal evaluation using breadth-first-search tasks at 256K context.
GraphWalks Parents 256K (Anthropic) (F1 Score (%)): leader Claude Opus 4.8 (99.3), 4 models
Anthropic GraphWalks long-context graph traversal evaluation using parent-pointer recovery tasks at 256K context.
USAMO 2026 (Anthropic) (Accuracy (%)): leader Claude Opus 4.8 (96.7), 2 models
Anthropic system-card evaluation on 2026 USAMO-style olympiad math problems, scored by answer correctness.
ArXivMath Mar-Apr 2026 (Anthropic) (Accuracy (%)): leader Claude Opus 4.8 (71.82), 3 models
Anthropic system-card evaluation on recent arXiv mathematics problems from March and April 2026.
OfficeQA (Anthropic Internal Harness) (Exact Match (%)): leader Claude Opus 4.8 (77.6), 2 models
Anthropic internal OfficeQA document-agent benchmark, requiring grounded search and numerical reasoning over office documents.
OfficeQA Pro (Anthropic Internal Harness) (Exact Match (%)): leader Claude Opus 4.8 (66.2), 2 models
Anthropic internal OfficeQA Pro hard subset, requiring grounded search and numerical reasoning over office documents.
ChartMuseum (Anthropic No Tools) (Accuracy (%)): leader Claude Opus 4.8 (75.8), 2 models
Anthropic no-tool run of ChartMuseum, evaluating visual chart interpretation across diverse chart types.
ChartMuseum (Anthropic Tools) (Accuracy (%)): leader Claude Opus 4.8 (89.7), 2 models
Anthropic tool-enabled run of ChartMuseum, evaluating visual chart interpretation across diverse chart types.
LAB-Bench FigQA (Anthropic No Tools) (Accuracy (%)): leader Claude Opus 4.8 (80.4), 2 models
Anthropic no-tool run of LAB-Bench FigQA, testing scientific figure understanding and reasoning.
LAB-Bench FigQA (Anthropic Tools) (Accuracy (%)): leader Claude Opus 4.8 (87.3), 2 models
Anthropic tool-enabled run of LAB-Bench FigQA, testing scientific figure understanding and reasoning.
CharXiv Reasoning (Anthropic No Tools) (Accuracy (%)): leader Claude Opus 4.7 (81.3), 2 models
Anthropic no-tool run of CharXiv Reasoning, evaluating reasoning over scientific charts from arXiv papers.
CharXiv Reasoning (Anthropic Tools) (Accuracy (%)): leader Claude Opus 4.7 (90.1), 2 models
Anthropic tool-enabled run of CharXiv Reasoning, evaluating reasoning over scientific charts from arXiv papers.
HealthBench Professional (Anthropic) (Length-Adjusted Score (%)): leader Claude Opus 4.8 (55.8), 3 models
Anthropic system-card run of HealthBench Professional, measuring clinical and healthcare reasoning with length-adjusted scoring.
GMMLU (Anthropic) (Average Accuracy (%)): leader Gemini 3.1 Pro (92.2), 5 models
Anthropic system-card run of Global MMLU, measuring multilingual academic and professional knowledge.
BioPipelineBench Verified (Anthropic) (Score (%)): leader Claude Mythos Preview (88.1), 4 models
Anthropic system-card run of BioPipelineBench Verified, measuring biological data-analysis workflow completion.
BioMysteryBench Verified - Human Solvable (Anthropic) (Score (%)): leader Claude Mythos Preview (82.6), 4 models
Anthropic system-card run of BioMysteryBench Verified human-solvable tasks, testing biological mystery problem solving.
BioMysteryBench Verified - Human Difficult (Anthropic) (Score (%)): leader Claude Opus 4.8 (40.0), 4 models
Anthropic system-card run of BioMysteryBench Verified human-difficult tasks, testing hard biological mystery problem solving.
LatchBio SpatialBench (Anthropic) (Score (%)): leader Claude Mythos Preview (53.8), 4 models
Anthropic system-card run of LatchBio SpatialBench, measuring spatial transcriptomics analysis workflows.
LatchBio SingleCellBench (Anthropic) (Score (%)): leader Claude Opus 4.8 (58.2), 4 models
Anthropic system-card run of LatchBio SingleCellBench, measuring single-cell RNA-seq analysis workflows.
Structural Biology (Anthropic) (Score (%)): leader Claude Mythos Preview (81.6), 4 models
Anthropic system-card structural biology evaluation, testing biomolecular structure reasoning and analysis.
ProteinGym Hard (Anthropic) (Rank Correlation (%)): leader Claude Mythos Preview (43.1), 4 models
Anthropic system-card run of the hard ProteinGym subset, measuring protein variant effect prediction via rank correlation.
Organic Chemistry (Anthropic) (Score (%)): leader Claude Mythos Preview (86.5), 4 models
Anthropic system-card organic chemistry evaluation, testing reaction and molecule reasoning.
Protocol Troubleshooting (Anthropic) (Score (%)): leader Claude Mythos Preview (69.6), 4 models
Anthropic system-card protocol troubleshooting benchmark, testing diagnosis of laboratory protocol failures.
LABBench2 - Patent Questions (Anthropic) (Score (%)): leader Claude Opus 4.8 (68.8), 3 models
Anthropic system-card LABBench2 patent-question subset, testing life-science document reasoning over patent material.
LABBench2 - Clinical Trial Questions (Anthropic) (Score (%)): leader Claude Mythos Preview (86.3), 3 models
Anthropic system-card LABBench2 clinical-trial subset, testing life-science reasoning over trial documents.
LABBench2 - Table Reading (Anthropic) (Score (%)): leader Claude Opus 4.8 (77.2), 2 models
Anthropic system-card LABBench2 table-reading subset, testing scientific table comprehension.
LABBench2 - Supplementary Materials (Anthropic) (Score (%)): leader Claude Opus 4.8 (58.9), 2 models
Anthropic system-card LABBench2 supplementary-materials subset, testing reasoning over scientific supporting files.
Agent Security League - Functional Correctness (Functional Correctness (%)): leader GPT-5.5 (84.9), 15 models
Endor Labs coding-agent benchmark measuring whether agents functionally complete security-sensitive software tasks.
Agent Security League - Security Correctness (Security Correctness (%)): leader GPT-5.5 (24.0), 15 models
Endor Labs coding-agent benchmark measuring whether completed software tasks avoid introducing or preserving security vulnerabilities.

New #1 Leaders (1)

OpenClawProBench: GLM-5.2 (81.3) beat intern-s2-preview by 4.6

Weekly

New Benchmarks (86)

FrontierCode Diamond (Score (%)): leader Claude Opus 4.8 (13.4), 12 models
Hardest 50 FrontierCode production-code tasks from Cognition, measuring whether maintainers would merge model PRs using blocker criteria and quality rubrics.
FrontierCode Main (Score (%)): leader Claude Opus 4.8 (34.3), 12 models
100 hardest FrontierCode production-code tasks, including Diamond, scored by maintainer-style mergeability criteria across correctness, tests, scope, style, and codebase standards.
FrontierCode Extended (Score (%)): leader Claude Opus 4.8 (51.8), 12 models
Full 150-task FrontierCode benchmark from Cognition, evaluating production-quality coding agents on maintainer-authored open source repository work.
Ramp SWE-Bench (Resolved (%)): leader Claude Fable 5 (87.5), 14 models
Ramp Labs benchmark for background coding agents on realistic financial software engineering work, scored by resolved tasks with the mini-SWE-agent harness.
CADGenBench (Aggregate CAD Score): leader Claude Fable 5 (0.4514), 11 models
CAD generation and editing benchmark scoring generated CAD artifacts on aggregate geometric and validity metrics across validated submissions.
FrontierMath - Tier 4 (v2) (Accuracy (%, 41 private v2 problems)): leader Claude Fable 5 (max) (87.8), 27 models
Current v2 private Tier 4 FrontierMath expansion set from Epoch AI, measuring accuracy on the hardest unpublished research-level mathematics problems.
FrontierMath - Tiers 1-3 (v2) (Accuracy (%, 285 private v2 problems)): leader GPT-5.5 Pro (xhigh) (87.72), 26 models
Current v2 private FrontierMath base set from Epoch AI, covering original problems from undergraduate through early-postdoc difficulty across major areas of modern mathematics.
Benchmarks.bio - SpatialBench (Pass Rate (%)): leader GPT-5.5 (69.57), 11 models
LatchBio agentic benchmark on messy real-world spatial transcriptomics data, with models writing and running analysis workflows across assays, platforms, and task categories.
Benchmarks.bio - scBench (Pass Rate (%)): leader Claude Mythos 5 (59.3), 13 models
LatchBio agentic benchmark for single-cell RNA-seq analysis, requiring models to perform realistic data cleaning, clustering, cell typing, and differential-expression workflows.
Benchmarks.bio - SpatialBench-Long (Pass Rate (%)): leader Gemini 3.5 Flash (11.11), 12 models
Long-form Benchmarks.bio spatial transcriptomics tasks that require multi-step biological data analysis, tool use, and synthesis over larger assay contexts.
Benchmarks.bio - EpiBench (Pass Rate (%)): leader GPT-5.5 (44.97), 11 models
Benchmarks.bio epigenomics benchmark covering real assays such as chromatin accessibility, binding, and methylation analyses with deterministic graders.
Agent Arena (Net Improvement (%)): leader Grok 4.3 xAI · Proprietary (18.3), 25 models
Arena.ai agent leaderboard measuring net improvement on real-world tool orchestration sessions with success, steerability, recovery, and hallucination metrics.
Agent Arena - Confirmed Success (Confirmed Success (%)): leader Claude Fable 5 (High) (17.21), 25 models
Agent Arena submetric tracking confirmed successful completion rate on real-world agent sessions.
Agent Arena - Praise vs Complaint (Praise vs Complaint (%)): leader Claude Fable 5 (High) (27.74), 25 models
Agent Arena submetric comparing user praise against complaints across agent sessions.
Agent Arena - Steerability (Steerability (%)): leader Nemotron 3 Ultra (23.87), 25 models
Agent Arena submetric measuring how well models adapt to user steering during tool-use sessions.
Agent Arena - Bash Recovery (Bash Recovery (%)): leader Grok 4.3 xAI · Proprietary (60.23), 25 models
Agent Arena submetric measuring recovery from shell or command-line failures in agent sessions.
Agent Arena - Tool Hallucination (Tool Hallucination (%)): leader Grok 4.3 xAI · Proprietary (0.26), 25 models
Agent Arena submetric measuring tool hallucination rate; lower values indicate fewer invented or invalid tool uses.
Agents' Last Exam (Pass Rate (%)): leader GPT-5.5 (24.0), 18 models
Snorkel benchmark of long-horizon economically valuable agent tasks across many industries, reporting workflow pass rate and score.
WolfBench (Average Score (%)): leader GPT-5.5 (77.0), 27 models
Agent benchmark based on Terminal-Bench 2.0 that compares harnesses and models across repeated terminal task runs using aggregate score statistics.
Appwrite Arena (With Skills) (Overall Score (%)): leader GPT-5.5 (97.7), 16 models
Appwrite Arena evaluation of model knowledge and reasoning about Appwrite development tasks when models can use Appwrite skills.
Appwrite Arena (Without Skills) (Overall Score (%)): leader Claude Fable 5 (97.7), 16 models
Appwrite Arena evaluation of model knowledge and reasoning about Appwrite development tasks without Appwrite skill assistance.
Terminal-Bench 2.1 (Accuracy (%)): leader GPT-5.5 (83.4), 6 models
Official Terminal-Bench 2.1 leaderboard measuring agent success on realistic command-line tasks, using each model best available harness row.
Terminal-Bench 2.1 (Claude Code) (Accuracy (%)): leader Claude Opus 4.8 (78.9), 3 models
Terminal-Bench 2.1 results for the Claude Code harness, measuring command-line task completion by model.
Terminal-Bench 2.1 (Terminus 2) (Accuracy (%)): leader GPT-5.5 (78.2), 5 models
Terminal-Bench 2.1 results for the Terminus 2 harness, measuring command-line task completion by model.
Vals AI Finance Agent v2 (Accuracy (%)): leader gemini-3.5-flash (57.86), 29 models
Updated Vals AI financial-research agent benchmark over SEC filings and supporting documents, measuring completion accuracy on realistic analyst workflows.
Vals AI Public Benefits Bench (Accuracy (%)): leader claude-fable-5 (71.65), 13 models
SNAP public-benefits guidance benchmark measuring whether models answer benefits questions accurately while following eligibility and documentation rules.
Vals AI Terminal-Bench 2.1 (Accuracy (%)): leader claude-fable-5 (80.52), 30 models
Updated Terminal-Bench 2.1 evaluation from Vals AI, measuring agentic command-line task completion in sandboxed software and systems environments.
Vals AI LiveCodeBench (Accuracy (%)): leader claude-fable-5 (89.78), 121 models
Vals AI run of LiveCodeBench coding problems, measuring pass rates on recent contest-style programming tasks intended to reduce contamination.
Vals AI GPQA (Accuracy (%)): leader gemini-3.1-pro-preview (95.45), 115 models
Vals AI run of GPQA graduate-level science questions, measuring difficult expert-domain reasoning accuracy.
Vals AI MMLU-Pro (Accuracy (%)): leader claude-fable-5 (91.5), 114 models
Vals AI run of MMLU-Pro multitask academic questions, using harder multi-choice problems across STEM, humanities, and professional domains.
Vals AI MMMU (Accuracy (%)): leader claude-fable-5 (89.31), 76 models
Vals AI run of MMMU multimodal college-level subject questions, measuring visual and textual academic reasoning.
Vals AI SWE-bench Verified (Resolved (%)): leader claude-fable-5 (95.0), 57 models
Vals AI SWE-bench Verified leaderboard, measuring the percentage of real GitHub issues resolved by coding agents.
Icelandic LLM Leaderboard - Average (Average Score (%)): leader Gemini 3.1 Pro Preview (88.54), 86 models
Icelandic LLM leaderboard aggregating WinoGrande-IS, GED, Inflection, Belebele-IS, ARC-Challenge-IS, and WikiQA-IS for Icelandic language understanding and reasoning.
Icelandic LLM - WinoGrande-IS (Score (%)): leader Gemini 3.1 Pro Preview (96.14), 86 models
Icelandic WinoGrande common-sense reasoning score.
Icelandic LLM - GED (Score (%)): leader Claude Fable 5 (91.5), 86 models
Icelandic grammatical error detection score.
Icelandic LLM - Inflection (Score (%)): leader GPT-5.5 (97.96), 86 models
Icelandic morphological inflection score.
Icelandic LLM - Belebele-IS (Score (%)): leader Gemini 3.1 Pro Preview (95.0), 86 models
Icelandic Belebele reading-comprehension score.
Icelandic LLM - ARC-Challenge-IS (Score (%)): leader GPT-5.5 (95.22), 86 models
Icelandic ARC-Challenge science and commonsense reasoning score.
Icelandic LLM - WikiQA-IS (Score (%)): leader Claude Fable 5 (75.39), 86 models
Icelandic WikiQA question-answering score.
GDP.pdf (Strict Pass Rate (%)): leader Claude Fable 5 (30.0), 12 models
Surge AI document-reasoning benchmark over 100 professional PDF workflows, scored by strict pass rate against expert-written rubrics.
Riemann-bench (Score (%)): leader Claude Fable 5 (55.0), 15 models
Surge AI frontier mathematics benchmark with advanced research-style problems sourced from mathematicians and scored by solution correctness.
SWE-bench Pro (Anthropic Scaffold) (Pass@1 (%)): leader Claude Mythos 5 (80.3), 6 models
Anthropic system-card run of SWE-bench Pro, measuring pass@1 on production software engineering issues using Anthropic scaffold settings.
OfficeQA Pro (Correctness (%)): leader Claude Fable 5 (57.9), 4 models
Hard OfficeQA subset for frontier document agents, requiring grounded search and numerical reasoning over U.S. Treasury Bulletin documents.
Real-World Finance v2 (Elo): leader Claude Fable 5 (1374.0), 4 models
Anthropic long-horizon finance workflow evaluation using pairwise preference grading and Elo ratings over realistic professional deliverables.
Real-World Finance v1 (Score (%)): leader Claude Mythos Preview (70.9), 4 models
Anthropic curated finance benchmark of 53 tasks evaluated against reference answers with a model-based grader.
Legal Agent Benchmark (Harvey Held-Out) (All-Pass Rate (%)): leader Claude Fable 5 (13.3), 5 models
Harvey legal-agent held-out evaluation using closed-universe matter files and expert rubrics, scored by all-pass task success.
Toolathlon (Anthropic Internal Harness) (Pass@1 (%)): leader Claude Fable 5 (61.7), 7 models
Anthropic internal Toolathlon harness over 108 tool-use tasks, reporting pass@1 for agentic workflow completion.
SWE-bench Verified (Anthropic Scaffold) (Resolved (%)): leader Claude Opus 4.8 (88.6), 3 models
Anthropic system-card run of SWE-bench Verified, measuring real GitHub issue resolution with Anthropic scaffold settings.
SWE-bench Multilingual (Anthropic Scaffold) (Resolved (%)): leader Claude Opus 4.8 (84.4), 2 models
Anthropic system-card run of SWE-bench Multilingual, measuring multilingual software issue resolution with Anthropic scaffold settings.
SWE-bench Multimodal (Anthropic Internal Harness) (Resolved (%)): leader Claude Opus 4.8 (38.4), 2 models
Anthropic internal multimodal SWE-bench harness, measuring software issue resolution that requires visual or multimodal context.
Humanity's Last Exam (Anthropic No Tools) (Accuracy (%)): leader Claude Opus 4.8 (49.8), 4 models
Anthropic system-card run of Humanitys Last Exam without tools, covering expert-level academic reasoning across many domains.
Humanity's Last Exam (Anthropic Tools) (Accuracy (%)): leader Claude Opus 4.8 (57.9), 4 models
Anthropic system-card run of Humanitys Last Exam with tools, covering expert-level academic reasoning across many domains.
ChartQAPro (Anthropic No Tools) (Accuracy (%)): leader Claude Opus 4.8 (69.4), 2 models
Anthropic no-tool run of ChartQAPro, testing chart understanding and quantitative visual reasoning.
ChartQAPro (Anthropic Tools) (Accuracy (%)): leader Claude Opus 4.8 (72.3), 2 models
Anthropic tool-enabled run of ChartQAPro, testing chart understanding and quantitative visual reasoning.
ScreenSpot-Pro (Anthropic No Tools) (Accuracy (%)): leader Claude Opus 4.8 (82.3), 2 models
Anthropic no-tool run of ScreenSpot-Pro, evaluating GUI grounding and screen element localization.
ScreenSpot-Pro (Anthropic Tools) (Accuracy (%)): leader Claude Opus 4.8 (87.9), 2 models
Anthropic tool-enabled run of ScreenSpot-Pro, evaluating GUI grounding and screen element localization.
GraphWalks BFS 256K (Anthropic) (F1 Score (%)): leader Claude Opus 4.8 (85.9), 4 models
Anthropic GraphWalks long-context graph traversal evaluation using breadth-first-search tasks at 256K context.
GraphWalks Parents 256K (Anthropic) (F1 Score (%)): leader Claude Opus 4.8 (99.3), 4 models
Anthropic GraphWalks long-context graph traversal evaluation using parent-pointer recovery tasks at 256K context.
USAMO 2026 (Anthropic) (Accuracy (%)): leader Claude Opus 4.8 (96.7), 2 models
Anthropic system-card evaluation on 2026 USAMO-style olympiad math problems, scored by answer correctness.
ArXivMath Mar-Apr 2026 (Anthropic) (Accuracy (%)): leader Claude Opus 4.8 (71.82), 3 models
Anthropic system-card evaluation on recent arXiv mathematics problems from March and April 2026.
OfficeQA (Anthropic Internal Harness) (Exact Match (%)): leader Claude Opus 4.8 (77.6), 2 models
Anthropic internal OfficeQA document-agent benchmark, requiring grounded search and numerical reasoning over office documents.
OfficeQA Pro (Anthropic Internal Harness) (Exact Match (%)): leader Claude Opus 4.8 (66.2), 2 models
Anthropic internal OfficeQA Pro hard subset, requiring grounded search and numerical reasoning over office documents.
ChartMuseum (Anthropic No Tools) (Accuracy (%)): leader Claude Opus 4.8 (75.8), 2 models
Anthropic no-tool run of ChartMuseum, evaluating visual chart interpretation across diverse chart types.
ChartMuseum (Anthropic Tools) (Accuracy (%)): leader Claude Opus 4.8 (89.7), 2 models
Anthropic tool-enabled run of ChartMuseum, evaluating visual chart interpretation across diverse chart types.
LAB-Bench FigQA (Anthropic No Tools) (Accuracy (%)): leader Claude Opus 4.8 (80.4), 2 models
Anthropic no-tool run of LAB-Bench FigQA, testing scientific figure understanding and reasoning.
LAB-Bench FigQA (Anthropic Tools) (Accuracy (%)): leader Claude Opus 4.8 (87.3), 2 models
Anthropic tool-enabled run of LAB-Bench FigQA, testing scientific figure understanding and reasoning.
CharXiv Reasoning (Anthropic No Tools) (Accuracy (%)): leader Claude Opus 4.7 (81.3), 2 models
Anthropic no-tool run of CharXiv Reasoning, evaluating reasoning over scientific charts from arXiv papers.
CharXiv Reasoning (Anthropic Tools) (Accuracy (%)): leader Claude Opus 4.7 (90.1), 2 models
Anthropic tool-enabled run of CharXiv Reasoning, evaluating reasoning over scientific charts from arXiv papers.
HealthBench Professional (Anthropic) (Length-Adjusted Score (%)): leader Claude Opus 4.8 (55.8), 3 models
Anthropic system-card run of HealthBench Professional, measuring clinical and healthcare reasoning with length-adjusted scoring.
GMMLU (Anthropic) (Average Accuracy (%)): leader Gemini 3.1 Pro (92.2), 5 models
Anthropic system-card run of Global MMLU, measuring multilingual academic and professional knowledge.
BioPipelineBench Verified (Anthropic) (Score (%)): leader Claude Mythos Preview (88.1), 4 models
Anthropic system-card run of BioPipelineBench Verified, measuring biological data-analysis workflow completion.
BioMysteryBench Verified - Human Solvable (Anthropic) (Score (%)): leader Claude Mythos Preview (82.6), 4 models
Anthropic system-card run of BioMysteryBench Verified human-solvable tasks, testing biological mystery problem solving.
BioMysteryBench Verified - Human Difficult (Anthropic) (Score (%)): leader Claude Opus 4.8 (40.0), 4 models
Anthropic system-card run of BioMysteryBench Verified human-difficult tasks, testing hard biological mystery problem solving.
LatchBio SpatialBench (Anthropic) (Score (%)): leader Claude Mythos Preview (53.8), 4 models
Anthropic system-card run of LatchBio SpatialBench, measuring spatial transcriptomics analysis workflows.
LatchBio SingleCellBench (Anthropic) (Score (%)): leader Claude Opus 4.8 (58.2), 4 models
Anthropic system-card run of LatchBio SingleCellBench, measuring single-cell RNA-seq analysis workflows.
Structural Biology (Anthropic) (Score (%)): leader Claude Mythos Preview (81.6), 4 models
Anthropic system-card structural biology evaluation, testing biomolecular structure reasoning and analysis.
ProteinGym Hard (Anthropic) (Rank Correlation (%)): leader Claude Mythos Preview (43.1), 4 models
Anthropic system-card run of the hard ProteinGym subset, measuring protein variant effect prediction via rank correlation.
Organic Chemistry (Anthropic) (Score (%)): leader Claude Mythos Preview (86.5), 4 models
Anthropic system-card organic chemistry evaluation, testing reaction and molecule reasoning.
Protocol Troubleshooting (Anthropic) (Score (%)): leader Claude Mythos Preview (69.6), 4 models
Anthropic system-card protocol troubleshooting benchmark, testing diagnosis of laboratory protocol failures.
LABBench2 - Patent Questions (Anthropic) (Score (%)): leader Claude Opus 4.8 (68.8), 3 models
Anthropic system-card LABBench2 patent-question subset, testing life-science document reasoning over patent material.
LABBench2 - Clinical Trial Questions (Anthropic) (Score (%)): leader Claude Mythos Preview (86.3), 3 models
Anthropic system-card LABBench2 clinical-trial subset, testing life-science reasoning over trial documents.
LABBench2 - Table Reading (Anthropic) (Score (%)): leader Claude Opus 4.8 (77.2), 2 models
Anthropic system-card LABBench2 table-reading subset, testing scientific table comprehension.
LABBench2 - Supplementary Materials (Anthropic) (Score (%)): leader Claude Opus 4.8 (58.9), 2 models
Anthropic system-card LABBench2 supplementary-materials subset, testing reasoning over scientific supporting files.
BoxPwnr CTF Bench (Average Platform Completion (%)): leader z-ai/glm-5.1 (54.37), 15 models
Aggregated BoxPwnr trace leaderboard over public CTF and security-lab platforms including CyBench, Hack The Box, picoCTF, PortSwigger, TryHackMe, Argus, and XBOW.
Agent Security League - Functional Correctness (Functional Correctness (%)): leader GPT-5.5 (84.9), 15 models
Endor Labs coding-agent benchmark measuring whether agents functionally complete security-sensitive software tasks.
Agent Security League - Security Correctness (Security Correctness (%)): leader GPT-5.5 (24.0), 15 models
Endor Labs coding-agent benchmark measuring whether completed software tasks avoid introducing or preserving security vulnerabilities.

New Models (67)

Claude Fable 5 — ELO 2697, #4
- Lynchmark: 100.0 (#1/13)
- Design Arena (Website): 1345.0 (#1/143)
- Design Arena (Game Dev): 1382.0 (#1/129)
- Design Arena (UI Components): 1417.0 (#1/123)
- Design Arena (Data Viz): 1381.0 (#1/125)
- Design Arena (3D): 1370.0 (#1/117)
- Design Arena (SVG): 1370.0 (#1/94)
- Chatbot Arena (Text): 1510.0 (#1/366)
- Chatbot Arena (Code): 1665.0 (#1/86)
- Blueprint-Bench 2: 0.386 (#1/14)
Claude Opus 4.8 — ELO 2449, #6
- Evals for Every Language - MGSM: 96.62 (#1/70)
- Evals for Every Language - Language ar: 71.58 (#1/71)
- Evals for Every Language - Language be: 69.43 (#1/71)
- Evals for Every Language - Language ak: 60.02 (#2/71)
- Evals for Every Language - Language bem: 60.25 (#2/71)
- Evals for Every Language - Language bm: 59.47 (#2/71)
- Evals for Every Language - Language chm: 63.17 (#2/71)
- Evals for Every Language - Language ckb: 71.59 (#2/71)
- Evals for Every Language - Language crh: 69.2 (#2/71)
- Evals for Every Language - Language en: 86.15 (#2/71)
GPT-5.5 — ELO 2384, #7
- Blueprint-Bench 2: 0.362 (#2/14)
- GRAB-Lite: 71.8 (#2/38)
- Evals for Every Language - Language ary: 47.34 (#2/71)
- Evals for Every Language - Language doi: 71.32 (#2/71)
- Evals for Every Language - Language et: 72.25 (#3/71)
- Evals for Every Language - ARC: 97.82 (#4/69)
- Evals for Every Language - Language ay: 59.02 (#4/71)
- Evals for Every Language - Language az: 65.39 (#4/71)
- Evals for Every Language - Language bho: 67.61 (#4/71)
- Evals for Every Language - Language bm: 54.72 (#4/71)
Qwen 3.7 Max — ELO 2370, #8
- Position Bias (Lechmazur): 34.8 (#10/36)
- RuneBench: 2222.0 (#11/23)
- Wolfram LLM Benchmarking Project: 67.5 (#14/483)
Claude Opus 4.7 — ELO 2325, #10
- Evals for Every Language - Language chm: 63.6 (#1/71)
- Evals for Every Language - Language cs: 74.38 (#1/71)
- Evals for Every Language - Language doi: 71.84 (#1/71)
- Evals for Every Language: 66.95 (#2/71)
- Evals for Every Language - MGSM: 95.57 (#2/70)
- Evals for Every Language - Language am: 67.86 (#2/71)
- Evals for Every Language - Language ar: 70.69 (#2/71)
- Evals for Every Language - Language arz: 52.06 (#2/71)
- Evals for Every Language - Language as: 68.11 (#2/71)
- Evals for Every Language - Language awa: 68.23 (#2/71)
Nemotron 3 Ultra — ELO 2288, #13
- YC-Bench: 326.9 (#18/26)
- SimpleBench: 41.7 (#37/74)
Claude Opus 4.6 — ELO 2253, #15
- Android Bench: 66.6 (#5/23)
GPT-5.4 — ELO 2242, #16
- Blueprint-Bench 2: 0.271 (#4/14)
Gemini 3.5 Flash — ELO 2219, #18
- ZeroBench: 19.0 (#4/60)
- GRAB-Lite: 63.0 (#4/38)
- Position Bias (Lechmazur): 29.8 (#5/36)
- Android Bench: 63.7 (#6/23)
- YC-Bench: 987.0 (#12/26)
- SWE-rebench: 49.45 (#30/85)
GPT-5 Pro — ELO 2217, #19
- Epoch AI - ECI: 149.85 (#69/374)
Qwen Max — ELO 2148, #23
- SimpleQA Verified: 58.52 (#10/55)
- OTIS Mock AIME 2024-25: 95.0 (#13/145)
- Chess Puzzles (Epoch AI): 22.0 (#22/46)
DeepSeek V4 Pro — ELO 2097, #30
- RuneBench: 2939.0 (#6/23)
- ProphetArena: 0.9061 (#15/46)
- Position Bias (Lechmazur): 43.6 (#19/36)
Qwen 3.6 Plus — ELO 2092, #31
- ProphetArena: 0.9289 (#3/46)
- Evals for Every Language - Language as: 66.13 (#6/71)
- Evals for Every Language - Language bm: 50.12 (#6/71)
- Evals for Every Language - Language chm: 59.82 (#6/71)
- Evals for Every Language - Language ckb: 68.26 (#6/71)
- Evals for Every Language - Language ace: 65.27 (#7/71)
- Evals for Every Language - Language cv: 61.59 (#7/71)
- Evals for Every Language - Language be: 65.74 (#8/71)
- Evals for Every Language - Language bjn: 45.34 (#8/71)
- Evals for Every Language - Language ban: 62.52 (#9/71)
MiMo-V2.5-Pro — ELO 2059, #36
- LLM Stats (CMMLU): 90.2 (#1/6)
- LLM Stats (DROP): 86.3 (#3/29)
- LLM Stats (TriviaQA): 81.3 (#3/18)
- LLM Stats (C-Eval): 91.5 (#5/18)
- LLM Stats (Claw-Eval): 64.0 (#5/11)
- LLM Stats (GDPval-AA): 1581.0 (#6/13)
- Vals AI ProofBench: 24.0 (#13/42)
- LLM Stats (MMLU-Redux): 92.8 (#14/47)
- Vals AI MedScribe: 83.73 (#14/64)
- Vals AI (Vals Index): 50.74 (#16/29)
MiniMax-M3 — ELO 2054, #37
- OSWorld: 75.19 (#6/61)
- WebDev Arena: 1527.75 (#9/70)
- YC-Bench: 999.5 (#11/26)
- Position Bias (Lechmazur): 34.9 (#11/36)
- Sycophancy (Lechmazur): 3.5 (#12/32)
- Design Arena (SVG): 1255.0 (#18/94)
- Design Arena (Game Dev): 1273.0 (#27/129)
- SWE-rebench: 45.64 (#38/85)
O3 — ELO 2049, #39
- GRAB-Lite: 40.8 (#21/38)
Kimi K2.6 — ELO 2048, #41
- RuneBench: 1256.0 (#16/23)
- Position Bias (Lechmazur): 47.3 (#24/36)
GPT-5.1 — ELO 2045, #42
- GRAB-Lite: 44.4 (#17/38)
kimi-k2.7-code — ELO 2040, #45
- LiveBench Python: 90.0 (#2/125)
- LiveBench TypeScript: 65.0 (#3/124)
- OTIS Mock AIME 2024-25: 96.39 (#6/145)
- Design Arena (Website): 1322.0 (#7/143)
- Design Arena (3D): 1328.0 (#11/117)
- LiveBench Logic With Navigation: 74.0 (#14/125)
- LiveBench Zebra Puzzle: 96.0 (#15/124)
- LiveBench Olympiad: 90.3 (#17/125)
- Vals AI Vibe Code Bench: 47.21 (#18/62)
- LiveBench JavaScript: 55.0 (#19/125)
Claude Sonnet 4.6 — ELO 2023, #50
- ZeroBench: 11.0 (#11/60)
- SWE-rebench: 54.49 (#18/85)
- Terminal-Bench 2.0: 53.4 (#21/58)
GLM-5.1 — ELO 2004, #55
- ProphetArena: 0.9253 (#4/46)
- FrontierSWE: 32.0 (#9/13)
Grok 4.3 — ELO 1973, #64
- ProphetArena: 0.9188 (#6/46)
Step 3.7 Flash — ELO 1962, #71
- Design Arena (Game Dev): 1216.0 (#54/129)
Qwen 3.7 Plus — ELO 1960, #72
- Sycophancy (Lechmazur): 5.0 (#18/32)
Qwen 3.5 Plus — ELO 1951, #77
- Epoch AI - Apex Agents: 13.6 (#29/46)
Grok 4.20 — ELO 1936, #82
- Evals for Every Language - Language fa: 70.2 (#5/71)
- Evals for Every Language - MGSM: 87.39 (#7/70)
- Evals for Every Language - Language ak: 56.11 (#7/71)
- Evals for Every Language - Language cy: 77.85 (#7/71)
- Evals for Every Language - Language en: 84.29 (#7/71)
- Evals for Every Language - Language am: 64.62 (#8/71)
- Evals for Every Language - Language ba: 66.75 (#8/71)
- Evals for Every Language - Language ceb: 74.99 (#8/71)
- Evals for Every Language - Language es: 72.75 (#8/71)
- Evals for Every Language - Language ar: 67.76 (#9/71)
GPT-5.4 Mini — ELO 1912, #91
- ZeroBench: 10.0 (#13/60)
Claude Sonnet 4 (20250514) — ELO 1909, #95
- Epoch AI - Apex Agents: 9.3 (#33/46)
Gemini 3.1 Flash Lite — ELO 1905, #97
- Evals for Every Language - Language am: 68.6 (#1/71)
- Evals for Every Language - Language ca: 76.29 (#1/71)
- Evals for Every Language - Language ceb: 78.06 (#1/71)
- Evals for Every Language - Language cy: 82.03 (#1/71)
- Evals for Every Language - Language el: 73.81 (#1/71)
- Evals for Every Language - Language en: 87.28 (#1/71)
- Evals for Every Language - Language es: 76.16 (#1/71)
- Evals for Every Language - Language aeb: 53.18 (#2/71)
- Evals for Every Language - Language az: 67.76 (#2/71)
- Evals for Every Language - Language eo: 76.43 (#2/71)
Qwen 3.5 122B A10B — ELO 1903, #101
- LIBRA - ruSciPassageCount *: 21.38 (#3/13)
- LIBRA - ruBABILongQA1: 66.8 (#3/13)
- LIBRA - ruBABILongQA2: 53.71 (#3/13)
- LIBRA - ruBABILongQA3 *: 31.85 (#3/13)
- LIBRA - MatreshkaNames *: 67.39 (#4/13)
- LIBRA - LibrusecHistory: 79.77 (#4/13)
- LIBRA - ru2WikiMultihopQA *: 55.3 (#4/13)
- LIBRA - ruSciFi: 50.29 (#4/13)
- LIBRA - LibrusecMHQA *: 42.32 (#4/13)
- LIBRA - ruBABILongQA4: 58.91 (#4/13)
MiMo-V2.5 — ELO 1903, #102
- LLM Stats (Video-MME): 87.7 (#1/14)
- LLM Stats (Claw-Eval): 63.2 (#6/11)
- LLM Stats (CharXiv-R): 81.0 (#12/38)
- Vals AI Multimodal Index: 52.77 (#12/21)
- Vals AI (Vals Index): 51.57 (#15/29)
- Vals AI Vibe Code Bench: 42.17 (#21/62)
- Vals AI ProofBench: 16.0 (#22/42)
- Vals AI SAGE: 43.27 (#26/61)
- Vals AI MortgageTax: 59.26 (#49/80)
- Vals AI MedScribe: 72.15 (#50/64)
qwen3.6-flash — ELO 1872, #116
- Evals for Every Language - Language chm: 55.74 (#12/71)
- Evals for Every Language - Language am: 57.31 (#19/71)
- Evals for Every Language - Language ban: 58.91 (#19/71)
- Evals for Every Language - ARC: 91.99 (#20/69)
- Evals for Every Language - Language ckb: 62.38 (#20/71)
- Evals for Every Language - Language dz: 45.14 (#20/71)
- Evals for Every Language - Language en: 79.89 (#20/71)
- Evals for Every Language - Language ace: 57.48 (#21/71)
- Evals for Every Language - Language cv: 53.08 (#21/71)
- Evals for Every Language - Language ee: 41.46 (#21/71)
MiniMax-M2.7 — ELO 1853, #124
- ProphetArena: 0.9215 (#5/46)
O3 Mini — ELO 1850, #127
- FinBen - FNS: 16.95 (#4/21)
- FinBen - FinNum: 20.98 (#5/21)
nemotron-3-ultra-550B-a55B — ELO 1778, #168
- Vals AI ProofBench: 2.0 (#40/42)
- Vals AI Vibe Code Bench: 7.64 (#49/62)
- WeirdML: 43.45 (#63/131)
- Design Arena (Website): 1144.0 (#97/143)
DeepSeek V3.1 — ELO 1763, #176
- Evals for Every Language - Language da: 76.78 (#2/71)
- Evals for Every Language - Language ban: 65.1 (#4/71)
- Evals for Every Language - ARC: 97.4 (#5/69)
- Evals for Every Language - Language ay: 58.91 (#5/71)
- Evals for Every Language - Language ar: 68.89 (#6/71)
- Evals for Every Language - Language ca: 73.25 (#6/71)
- Evals for Every Language - Language bem: 54.51 (#7/71)
- Evals for Every Language - MMLU: 97.67 (#8/69)
- Evals for Every Language - Language el: 70.9 (#10/71)
- Evals for Every Language - Language as: 64.63 (#11/71)
GPT-4o — ELO 1712, #208
- FinBen (Financial LLM): 46.01 (#1/20)
- FinBen - QA: 78.22 (#1/20)
- FinBen - FNS: 25.5 (#3/21)
- FinBen - MultiFin: 59.26 (#4/20)
- FinBen - FinNum: 9.18 (#6/21)
Mistral Medium 3.5 — ELO 1712, #209
- Position Bias (Lechmazur): 72.5 (#36/36)
Mistral-Small-3.2-24B-Instruct-2506 — ELO 1708, #211
- Evals for Every Language - Classification: 89.59 (#24/70)
- Evals for Every Language - Language en: 76.19 (#29/71)
- Evals for Every Language - Language ars: 46.69 (#31/71)
- Evals for Every Language - Language awa: 61.09 (#31/71)
- Evals for Every Language - Language ca: 69.44 (#31/71)
- Evals for Every Language - Language be: 62.6 (#32/71)
- Evals for Every Language - Language cs: 66.53 (#32/71)
- Evals for Every Language - Language doi: 55.87 (#36/71)
- Evals for Every Language - Language eu: 59.54 (#37/71)
- Evals for Every Language - Language az: 58.06 (#39/71)
Qwen 3.5 35B A3B — ELO 1707, #213
- LIBRA - MatreshkaNames *: 68.97 (#2/13)
- LIBRA - ruSciPassageCount *: 21.89 (#2/13)
- LIBRA - ruSciFi: 51.47 (#2/13)
- LIBRA - ruBABILongQA1: 68.38 (#2/13)
- LIBRA - ruBABILongQA2: 54.97 (#2/13)
- LIBRA - ruBABILongQA3 *: 32.6 (#2/13)
- LIBRA - LibrusecHistory: 81.65 (#3/13)
- LIBRA - ru2WikiMultihopQA *: 56.6 (#3/13)
- LIBRA - LibrusecMHQA *: 43.32 (#3/13)
- LIBRA - ruBABILongQA4: 60.29 (#3/13)
DeepSeek V3 — ELO 1706, #215
- FinBen - FNS: 37.72 (#1/21)
- FinBen - MultiFin: 61.11 (#3/20)
- FinBen - FinNum: 7.43 (#7/21)
- FinBen - QA: 50.0 (#7/20)
- FinBen (Financial LLM): 10.2 (#13/20)
GPT-4.1 Mini — ELO 1705, #216
- GRAB-Lite: 18.6 (#32/38)
GPT-4o (2024-11-20) — ELO 1696, #224
- Epoch AI - Apex Agents: 1.1 (#46/46)
GLM 4.5 Air — ELO 1684, #230
- Evals for Every Language - Language chm: 47.52 (#21/71)
- Evals for Every Language - Language et: 66.43 (#21/71)
- Evals for Every Language - Language ckb: 60.22 (#22/71)
- Evals for Every Language - Language as: 60.21 (#23/71)
- Evals for Every Language - Language az: 62.15 (#24/71)
- Evals for Every Language - Language ak: 41.24 (#26/71)
- Evals for Every Language - Language es: 70.3 (#26/71)
- Evals for Every Language - Language ca: 70.13 (#27/71)
- Evals for Every Language - Language bho: 62.66 (#28/71)
- Evals for Every Language - Language ace: 51.96 (#29/71)
Hermes 4 70B — ELO 1674, #239
- Evals for Every Language - MGSM: 77.91 (#24/70)
- Evals for Every Language - MMLU: 88.52 (#26/69)
- Evals for Every Language - ARC: 83.16 (#38/69)
- Evals for Every Language - Language chm: 34.57 (#40/71)
- Evals for Every Language - Language dz: 28.93 (#40/71)
- Evals for Every Language - Language cv: 31.63 (#44/71)
- Evals for Every Language - Language am: 31.51 (#49/71)
- Evals for Every Language - Language ckb: 41.14 (#49/71)
- Evals for Every Language - Language as: 40.36 (#57/71)
- Evals for Every Language - Language ba: 41.82 (#58/71)
jamba-large-1.7 — ELO 1663, #245
- Evals for Every Language - Classification: 91.29 (#18/70)
- Evals for Every Language - Language af: 71.78 (#24/71)
- Evals for Every Language - Language fa: 65.89 (#24/71)
- Evals for Every Language - Language bg: 70.55 (#25/71)
- Evals for Every Language - Language ee: 30.99 (#26/71)
- Evals for Every Language - Language ar: 63.0 (#27/71)
- Evals for Every Language - Language be: 62.93 (#28/71)
- Evals for Every Language - Language de: 70.42 (#30/71)
- Evals for Every Language - Language aeb: 42.65 (#32/71)
- Evals for Every Language - Language doi: 56.88 (#33/71)
Llama 3.1 70B Instruct — ELO 1658, #251
- FinBen - FinNum: 46.34 (#3/21)
- FinBen - QA: 64.44 (#3/20)
- FinBen - FNS: 13.61 (#7/21)
- FinBen - MultiFin: 50.0 (#7/20)
- FinBen (Financial LLM): 14.07 (#8/20)
Ministral 3 8B (2512) — ELO 1640, #263
- Evals for Every Language - Language bm: 29.35 (#28/71)
- Evals for Every Language - Classification: 84.43 (#39/70)
- Evals for Every Language - Language cs: 65.2 (#41/71)
- Evals for Every Language - Language en: 73.09 (#42/71)
- Evals for Every Language - Language bn: 62.11 (#43/71)
- Evals for Every Language - Language es: 67.86 (#44/71)
- Evals for Every Language - Language el: 63.45 (#45/71)
- Evals for Every Language - Language be: 59.13 (#46/71)
- Evals for Every Language - Language ace: 43.57 (#47/71)
- Evals for Every Language - Language chm: 31.67 (#47/71)
Gemma 3 27B (IT) — ELO 1639, #266
- Evals for Every Language - Language el: 72.48 (#3/71)
- FinBen (Financial LLM): 15.74 (#7/20)
- FinBen - FinNum: 0.0 (#10/21)
- FinBen - MultiFin: 38.89 (#10/20)
- Evals for Every Language - Language eo: 73.18 (#10/71)
- Evals for Every Language - Classification: 95.41 (#11/70)
- Evals for Every Language - Language bg: 73.9 (#11/71)
- Evals for Every Language - Language es: 72.28 (#11/71)
- FinBen - QA: 22.67 (#13/20)
- FinBen - FNS: 0.21 (#14/21)
nova-2-lite-v1 — ELO 1635, #268
- Evals for Every Language - MMLU: 95.33 (#12/69)
- Evals for Every Language - Language en: 81.54 (#12/71)
- Evals for Every Language - Language be: 64.22 (#17/71)
- Evals for Every Language - Language chm: 52.92 (#18/71)
- Evals for Every Language - MGSM: 80.9 (#19/70)
- Evals for Every Language - Language bn: 68.34 (#19/71)
- Evals for Every Language - Language ak: 46.55 (#20/71)
- Evals for Every Language - Language cv: 53.69 (#20/71)
- Evals for Every Language - Language da: 71.55 (#21/71)
- Evals for Every Language - Language bm: 36.11 (#22/71)
Qwen 3.5 9B — ELO 1628, #272
- LIBRA - ruSciPassageCount *: 20.77 (#4/13)
- LIBRA - ruBABILongQA1: 64.88 (#4/13)
- LIBRA - ruBABILongQA2: 52.16 (#4/13)
- LIBRA - ruBABILongQA3 *: 30.94 (#4/13)
- LIBRA - MatreshkaNames *: 65.44 (#5/13)
- LIBRA - LibrusecHistory: 77.47 (#5/13)
- LIBRA - ru2WikiMultihopQA *: 53.7 (#5/13)
- LIBRA - ruSciFi: 48.84 (#5/13)
- LIBRA - LibrusecMHQA *: 41.1 (#5/13)
- LIBRA - ruBABILongQA4: 57.21 (#5/13)
Qwen 3 30B A3B 2507 Instruct — ELO 1615, #280
- Evals for Every Language - Language ars: 50.46 (#10/71)
- Evals for Every Language - Language aeb: 45.53 (#18/71)
- Evals for Every Language - Language en: 78.55 (#23/71)
- Evals for Every Language - Language bs: 68.53 (#30/71)
- Evals for Every Language - Language bg: 69.16 (#32/71)
- Evals for Every Language - Language arz: 42.54 (#33/71)
- Evals for Every Language - Language dz: 32.48 (#33/71)
- Evals for Every Language - Language am: 42.2 (#35/71)
- Evals for Every Language - Language bn: 63.67 (#37/71)
- Evals for Every Language - Language ace: 47.99 (#38/71)
Hunyuan A13B-Instruct — ELO 1579, #307
- Evals for Every Language - Language ars: 42.49 (#55/71)
- Evals for Every Language - Language aeb: 35.99 (#56/71)
- Evals for Every Language - Language apc: 40.5 (#56/71)
- Evals for Every Language - Translation From: 22.42 (#57/71)
- Evals for Every Language - Language ary: 33.19 (#57/71)
- Evals for Every Language - Language ak: 25.6 (#58/71)
- Evals for Every Language - Language cv: 24.99 (#58/71)
- Evals for Every Language - Language arz: 36.21 (#59/71)
- Evals for Every Language - Translation To: 18.34 (#60/71)
- Evals for Every Language - Language bjn: 29.79 (#61/71)
GPT-4o Mini — ELO 1543, #345
- GRAB-Lite: 11.4 (#38/38)
Ministral 3 14B (2512) — ELO 1532, #356
- Evals for Every Language - Classification: 88.17 (#31/70)
- Evals for Every Language - Language be: 62.67 (#31/71)
- Evals for Every Language - Language el: 66.63 (#31/71)
- Evals for Every Language - Language bn: 64.34 (#33/71)
- Evals for Every Language - Language az: 59.29 (#34/71)
- Evals for Every Language - Language af: 69.81 (#35/71)
- Evals for Every Language - Language es: 68.9 (#37/71)
- Evals for Every Language - Language en: 73.16 (#41/71)
- Evals for Every Language - Language arz: 40.92 (#42/71)
- Evals for Every Language - Language bg: 67.27 (#42/71)
GPT-OSS-20B — ELO 1515, #371
- Evals for Every Language - Language en: 77.2 (#26/71)
- Evals for Every Language - Language es: 69.42 (#30/71)
- Evals for Every Language - Language awa: 60.89 (#34/71)
- Evals for Every Language - Language bs: 67.0 (#37/71)
- Evals for Every Language - Language da: 69.16 (#37/71)
- Evals for Every Language - Language dz: 30.01 (#37/71)
- Evals for Every Language - Language as: 54.72 (#38/71)
- Evals for Every Language - Language bem: 33.09 (#39/71)
- Evals for Every Language - Language ak: 34.16 (#40/71)
- Evals for Every Language - Language cs: 65.28 (#40/71)
Llama 4 Scout Instruct — ELO 1498, #384
- FinBen - FinNum: 49.12 (#2/21)
- FinBen - QA: 74.22 (#2/20)
- FinBen (Financial LLM): 20.89 (#3/20)
- FinBen - FNS: 16.9 (#5/21)
- FinBen - MultiFin: 55.56 (#5/20)
Laguna M.1 — ELO 1491, #391
- Vals AI (Vals Index): 35.27 (#27/29)
- Vals AI ProofBench: 0.0 (#42/42)
- Vals AI Terminal-Bench 2.0: 31.46 (#43/68)
- Vals AI Vibe Code Bench: 10.94 (#48/62)
- Vals AI MedCode: 25.24 (#64/67)
- Vals AI CorpFin v2: 58.16 (#68/115)
- Vals AI LegalBench: 75.14 (#86/118)
- Vals AI TaxEval v2: 1.64 (#121/121)
granite-4.0-h-micro — ELO 1486, #399
- Evals for Every Language - Classification: 86.11 (#36/70)
- Evals for Every Language - Language ar: 60.19 (#45/71)
- Evals for Every Language - Language cv: 27.5 (#52/71)
- Evals for Every Language - Language ay: 29.15 (#54/71)
- Evals for Every Language - Language bn: 50.72 (#55/71)
- Evals for Every Language - Language ary: 33.31 (#56/71)
- Evals for Every Language - Language bg: 58.7 (#57/71)
- Evals for Every Language - Language eo: 59.1 (#57/71)
- Evals for Every Language - Language ak: 25.2 (#59/71)
- Evals for Every Language - Language da: 56.25 (#60/71)
Laguna XS.2 — ELO 1486, #401
- Vals AI (Vals Index): 29.15 (#28/29)
- Vals AI ProofBench: 1.0 (#41/42)
- Vals AI Terminal-Bench 2.0: 28.09 (#47/68)
- Vals AI Vibe Code Bench: 3.84 (#53/62)
- Vals AI MedCode: 20.7 (#66/67)
- Vals AI CorpFin v2: 56.33 (#72/115)
- Vals AI LegalBench: 71.03 (#91/118)
- Vals AI TaxEval v2: 59.98 (#107/121)
Gemma 3 4B (IT) — ELO 1463, #424
- FinBen (Financial LLM): 12.74 (#9/20)
- FinBen - FinNum: 0.0 (#9/21)
- FinBen - MultiFin: 38.89 (#9/20)
- FinBen - QA: 22.67 (#12/20)
- FinBen - FNS: 0.24 (#13/21)
Phi-4 Mini Instruct — ELO 1451, #434
- Evals for Every Language - Classification: 79.23 (#54/70)
- Evals for Every Language - Language ckb: 31.9 (#56/71)
- Evals for Every Language - Language aeb: 33.95 (#61/71)
- Evals for Every Language - Language ee: 21.7 (#62/71)
- Evals for Every Language - Language en: 62.96 (#62/71)
- Evals for Every Language - Language es: 51.93 (#65/71)
- Evals for Every Language - MGSM: 16.66 (#66/70)
- Evals for Every Language - MMLU: 43.8 (#66/69)
- Evals for Every Language - ARC: 41.91 (#67/69)
- Evals for Every Language - Language doi: 30.09 (#67/71)
Qwen 3.5 4B — ELO 1430, #455
- LIBRA - ruSciPassageCount *: 19.57 (#5/13)
- LIBRA - ruBABILongQA1: 61.13 (#5/13)
- LIBRA - ruBABILongQA2: 49.14 (#5/13)
- LIBRA - ruBABILongQA3 *: 29.15 (#5/13)
- LIBRA - MatreshkaNames *: 61.66 (#6/13)
- LIBRA - LibrusecHistory: 72.99 (#6/13)
- LIBRA - ru2WikiMultihopQA *: 50.6 (#6/13)
- LIBRA - ruSciFi: 46.02 (#6/13)
- LIBRA - LibrusecMHQA *: 38.73 (#6/13)
- LIBRA - ruBABILongQA4: 53.9 (#6/13)
Qwen3.5 0.8B — ELO 1370, #528
- LIBRA - ruSciPassageCount *: 17.79 (#7/13)
- LIBRA - ruBABILongQA2: 44.67 (#7/13)
- LIBRA - MatreshkaNames *: 56.05 (#8/13)
- LIBRA - ru2WikiMultihopQA *: 46.0 (#8/13)
- LIBRA - LibrusecMHQA *: 35.21 (#8/13)
- LIBRA - ruBABILongQA1: 55.57 (#8/13)
- LIBRA - ruBABILongQA4: 49.0 (#8/13)
- LIBRA - ruSciAbstractRetrieval: 56.26 (#9/13)
- LIBRA - ruSciFi: 41.83 (#9/13)
- LIBRA - ruBABILongQA3 *: 26.5 (#9/13)
Qwen 3.5 2B — ELO 1247, #653
- LIBRA - ruSciPassageCount *: 18.72 (#6/13)
- LIBRA - ruBABILongQA2: 47.01 (#6/13)
- LIBRA - ruBABILongQA3 *: 27.88 (#6/13)
- LIBRA - MatreshkaNames *: 58.98 (#7/13)
- LIBRA - ru2WikiMultihopQA *: 48.4 (#7/13)
- LIBRA - ruSciFi: 44.02 (#7/13)
- LIBRA - LibrusecMHQA *: 37.05 (#7/13)
- LIBRA - ruBABILongQA1: 58.48 (#7/13)
- LIBRA - ruBABILongQA4: 51.56 (#7/13)
- LIBRA - LibrusecHistory: 69.83 (#8/13)
Qwen2.5-Omni-7B — ELO 1227, #667
- FinBen (Financial LLM): 33.53 (#2/20)
- FinBen - FinNum: 0.4 (#8/21)
- FinBen - QA: 48.89 (#8/20)
- FinBen - FNS: 5.6 (#11/21)
- FinBen - MultiFin: 38.89 (#11/20)
Gemma 4 12B — ELO 1100, #731
- LLM Stats (MRCR v2): 43.4 (#3/7)
- LLM Stats (FLEURS): 93.1 (#4/6)
- LLM Stats (MedXpertQA): 48.7 (#8/12)
- LLM Stats (MathVision): 79.7 (#9/28)
- LLM Stats (AIME 2026): 77.5 (#13/16)
- LLM Stats (OmniDocBench 1.5): 16.4 (#13/15)
- LLM Stats (CodeForces): 55.3 (#15/16)
- LLM Stats (MMMLU): 83.4 (#34/48)
- ZeroEval GPQA Diamond: 78.8 (#82/223)

Top-10 New Scores (186)

Claude Fable 5 on AI Chess Leaderboard (Continuation): 1092.0 (#30)
Claude Fable 5 on AI Chess Leaderboard (Reasoning): 1711.0 (#8)
Claude Fable 5 on Chatbot Arena (Document): 1495.0 (#5)
Claude Fable 5 on Chatbot Arena (Vision): 1307.0 (#2)
Claude Fable 5 on ClockBench: 35.0 (#4)
Claude Fable 5 on Epoch AI - Apex Agents: 45.0 (#3)
Claude Fable 5 on LLM Stats (GDPval-AA): 1932.0 (#1)
Claude Fable 5 on Lynchmark: 100.0 (#1)
Claude Fable 5 on MineBench: 1790.51 (#4)
Claude Fable 5 on PM-LLM-Benchmark: 35.6 (#13)
Claude Fable 5 on PinchBench: 59.61 (#44)
Claude Fable 5 on React Native Evals: 86.96 (#4)
Claude Fable 5 on SEAL - MCP Atlas: 83.3 (#2)
Claude Fable 5 on Vals AI MedCode: 56.07 (#2)
Claude Fable 5 on Vals AI MortgageTax: 68.92 (#5)
Claude Fable 5 on Vals AI SAGE: 51.89 (#5)
Claude Fable 5 on Vals AI TaxEval v2: 76.94 (#3)
Claude Fable 5 on Vellum - GPQA: 94.1 (#3)
Claude Fable 5 on Vellum - HumanEval: 95.0 (#2)
Claude Fable 5 on Vending-Bench 2: 4529.94 (#18)
Claude Opus 4.7 on Android Bench: 68.7 (#4)
Claude Opus 4.7 on Evals for Every Language: 66.95 (#2)
Claude Opus 4.7 on Evals for Every Language - ARC: 97.23 (#6)
Claude Opus 4.7 on Evals for Every Language - Classification: 95.98 (#7)
Claude Opus 4.7 on Evals for Every Language - Language ace: 69.04 (#3)
Claude Opus 4.7 on Evals for Every Language - Language aeb: 50.61 (#4)
Claude Opus 4.7 on Evals for Every Language - Language af: 76.97 (#9)
Claude Opus 4.7 on Evals for Every Language - Language ak: 59.75 (#3)
Claude Opus 4.7 on Evals for Every Language - Language am: 67.86 (#2)
Claude Opus 4.7 on Evals for Every Language - Language apc: 55.53 (#5)
Claude Opus 4.7 on Evals for Every Language - Language ar: 70.69 (#2)
Claude Opus 4.7 on Evals for Every Language - Language ars: 49.83 (#13)
Claude Opus 4.7 on Evals for Every Language - Language ary: 44.23 (#12)
Claude Opus 4.7 on Evals for Every Language - Language arz: 52.06 (#2)
Claude Opus 4.7 on Evals for Every Language - Language as: 68.11 (#2)
Claude Opus 4.7 on Evals for Every Language - Language awa: 68.23 (#2)
Claude Opus 4.7 on Evals for Every Language - Language ay: 59.38 (#3)
Claude Opus 4.7 on Evals for Every Language - Language az: 65.04 (#8)
Claude Opus 4.7 on Evals for Every Language - Language ba: 67.46 (#6)
Claude Opus 4.7 on Evals for Every Language - Language ban: 65.75 (#3)
Claude Opus 4.7 on Evals for Every Language - Language be: 66.48 (#4)
Claude Opus 4.7 on Evals for Every Language - Language bem: 59.05 (#4)
Claude Opus 4.7 on Evals for Every Language - Language bg: 74.44 (#4)
Claude Opus 4.7 on Evals for Every Language - Language bho: 67.27 (#8)
Claude Opus 4.7 on Evals for Every Language - Language bjn: 48.88 (#4)
Claude Opus 4.7 on Evals for Every Language - Language bm: 58.41 (#3)
Claude Opus 4.7 on Evals for Every Language - Language bn: 72.35 (#4)
Claude Opus 4.7 on Evals for Every Language - Language bs: 70.22 (#24)
Claude Opus 4.7 on Evals for Every Language - Language ca: 72.35 (#14)
Claude Opus 4.7 on Evals for Every Language - Language ceb: 75.18 (#6)
Claude Opus 4.7 on Evals for Every Language - Language ckb: 70.88 (#3)
Claude Opus 4.7 on Evals for Every Language - Language crh: 66.99 (#3)
Claude Opus 4.7 on Evals for Every Language - Language cs: 74.38 (#1)
Claude Opus 4.7 on Evals for Every Language - Language cv: 62.92 (#4)
Claude Opus 4.7 on Evals for Every Language - Language cy: 79.87 (#5)
Claude Opus 4.7 on Evals for Every Language - Language da: 74.47 (#6)
Claude Opus 4.7 on Evals for Every Language - Language de: 75.66 (#6)
Claude Opus 4.7 on Evals for Every Language - Language dz: 59.16 (#3)
Claude Opus 4.7 on Evals for Every Language - Language ee: 60.86 (#2)
Claude Opus 4.7 on Evals for Every Language - Language el: 71.56 (#8)
Claude Opus 4.7 on Evals for Every Language - Language en: 84.79 (#5)
Claude Opus 4.7 on Evals for Every Language - Language eo: 75.16 (#4)
Claude Opus 4.7 on Evals for Every Language - Language es: 70.89 (#19)
Claude Opus 4.7 on Evals for Every Language - Language et: 71.59 (#5)
Claude Opus 4.7 on Evals for Every Language - Language eu: 68.46 (#7)
Claude Opus 4.7 on Evals for Every Language - Language fa: 69.71 (#8)
Claude Opus 4.7 on Evals for Every Language - MGSM: 95.57 (#2)
Claude Opus 4.7 on Evals for Every Language - MMLU: 95.33 (#13)
Claude Opus 4.7 on Evals for Every Language - Translation From: 40.53 (#7)
Claude Opus 4.7 on Evals for Every Language - Translation To: 39.5 (#4)
Claude Opus 4.7 on GRAB-Lite: 58.2 (#10)
Claude Opus 4.8 on Chess Puzzles (Epoch AI): 34.0 (#13)
Claude Opus 4.8 on Design Arena (Game Dev): 1300.0 (#17)
Claude Opus 4.8 on EQ-Bench Longform Writing: 80.8 (#3)
Claude Opus 4.8 on Epoch AI - Apex Agents: 42.5 (#4)
Claude Opus 4.8 on Epoch AI - ECI: 156.34 (#14)
Claude Opus 4.8 on Evals for Every Language: 66.27 (#3)
Claude Opus 4.8 on Evals for Every Language - ARC: 98.0 (#3)
Claude Opus 4.8 on Evals for Every Language - Classification: 90.31 (#21)
Claude Opus 4.8 on Evals for Every Language - Language ace: 66.63 (#6)
Claude Opus 4.8 on Evals for Every Language - Language aeb: 50.53 (#5)
Claude Opus 4.8 on Evals for Every Language - Language af: 78.38 (#4)
Claude Opus 4.8 on Evals for Every Language - Language ak: 60.02 (#2)
Claude Opus 4.8 on Evals for Every Language - Language am: 65.76 (#5)
Claude Opus 4.8 on Evals for Every Language - Language apc: 49.54 (#21)
Claude Opus 4.8 on Evals for Every Language - Language ars: 47.35 (#26)
Claude Opus 4.8 on Evals for Every Language - Language ary: 40.29 (#25)
Claude Opus 4.8 on Evals for Every Language - Language arz: 49.71 (#6)
Claude Opus 4.8 on Evals for Every Language - Language as: 66.93 (#4)
Claude Opus 4.8 on Evals for Every Language - Language awa: 67.71 (#4)
Claude Opus 4.8 on Evals for Every Language - Language ay: 58.4 (#6)
Claude Opus 4.8 on Evals for Every Language - Language az: 65.38 (#5)
Claude Opus 4.8 on Evals for Every Language - Language ba: 67.66 (#5)
Claude Opus 4.8 on Evals for Every Language - Language ban: 63.8 (#5)
Claude Opus 4.8 on Evals for Every Language - Language bem: 60.25 (#2)
Claude Opus 4.8 on Evals for Every Language - Language bg: 74.44 (#5)
Claude Opus 4.8 on Evals for Every Language - Language bho: 67.32 (#7)
Claude Opus 4.8 on Evals for Every Language - Language bjn: 47.35 (#6)
Claude Opus 4.8 on Evals for Every Language - Language bm: 59.47 (#2)
Claude Opus 4.8 on Evals for Every Language - Language bn: 70.4 (#12)
Claude Opus 4.8 on Evals for Every Language - Language bs: 74.0 (#5)
Claude Opus 4.8 on Evals for Every Language - Language ca: 74.29 (#3)
Claude Opus 4.8 on Evals for Every Language - Language ceb: 75.82 (#5)
Claude Opus 4.8 on Evals for Every Language - Language chm: 63.17 (#2)
Claude Opus 4.8 on Evals for Every Language - Language ckb: 71.59 (#2)
Claude Opus 4.8 on Evals for Every Language - Language crh: 69.2 (#2)
Claude Opus 4.8 on Evals for Every Language - Language cs: 73.8 (#3)
Claude Opus 4.8 on Evals for Every Language - Language cv: 64.32 (#3)
Claude Opus 4.8 on Evals for Every Language - Language cy: 79.83 (#6)
Claude Opus 4.8 on Evals for Every Language - Language da: 74.57 (#5)
Claude Opus 4.8 on Evals for Every Language - Language de: 76.71 (#3)
Claude Opus 4.8 on Evals for Every Language - Language doi: 70.16 (#4)
Claude Opus 4.8 on Evals for Every Language - Language dz: 58.51 (#4)
Claude Opus 4.8 on Evals for Every Language - Language ee: 57.06 (#4)
Claude Opus 4.8 on Evals for Every Language - Language el: 70.34 (#13)
Claude Opus 4.8 on Evals for Every Language - Language en: 86.15 (#2)
Claude Opus 4.8 on Evals for Every Language - Language eo: 74.5 (#6)
Claude Opus 4.8 on Evals for Every Language - Language es: 70.97 (#18)
Claude Opus 4.8 on Evals for Every Language - Language et: 70.93 (#7)
Claude Opus 4.8 on Evals for Every Language - Language eu: 66.0 (#19)
Claude Opus 4.8 on Evals for Every Language - Language fa: 69.54 (#9)
Claude Opus 4.8 on Evals for Every Language - MMLU: 98.31 (#4)
Claude Opus 4.8 on Evals for Every Language - Translation From: 39.86 (#9)
Claude Opus 4.8 on Evals for Every Language - Translation To: 38.22 (#7)
Claude Opus 4.8 on GRAB-Lite: 60.6 (#6)
Claude Opus 4.8 on OTIS Mock AIME 2024-25: 98.33 (#4)
Claude Opus 4.8 on SimpleQA Verified: 39.5 (#26)
Claude Opus 4.8 on WebDev Arena: 1545.05 (#6)
Claude Opus 4.8 on Wolfram LLM Benchmarking Project: 65.9 (#18)
Claude Opus 4.8 on ZeroBench: 17.0 (#7)
GPT-5.5 on Blueprint-Bench 2: 0.362 (#2)
GPT-5.5 on Evals for Every Language: 65.09 (#5)
GPT-5.5 on Evals for Every Language - ARC: 97.82 (#4)
GPT-5.5 on Evals for Every Language - Classification: 82.73 (#42)
GPT-5.5 on Evals for Every Language - Language ace: 67.32 (#5)
GPT-5.5 on Evals for Every Language - Language aeb: 44.61 (#22)
GPT-5.5 on Evals for Every Language - Language af: 77.33 (#8)
GPT-5.5 on Evals for Every Language - Language ak: 57.86 (#5)
GPT-5.5 on Evals for Every Language - Language am: 65.01 (#6)
GPT-5.5 on Evals for Every Language - Language apc: 50.92 (#12)
GPT-5.5 on Evals for Every Language - Language ar: 65.19 (#18)
GPT-5.5 on Evals for Every Language - Language ars: 46.47 (#33)
GPT-5.5 on Evals for Every Language - Language ary: 47.34 (#2)
GPT-5.5 on Evals for Every Language - Language arz: 45.23 (#19)
GPT-5.5 on Evals for Every Language - Language as: 66.04 (#8)
GPT-5.5 on Evals for Every Language - Language awa: 66.14 (#8)
GPT-5.5 on Evals for Every Language - Language ay: 59.02 (#4)
GPT-5.5 on Evals for Every Language - Language az: 65.39 (#4)
GPT-5.5 on Evals for Every Language - Language ba: 64.64 (#14)
GPT-5.5 on Evals for Every Language - Language ban: 62.74 (#8)
GPT-5.5 on Evals for Every Language - Language be: 64.63 (#16)
GPT-5.5 on Evals for Every Language - Language bem: 53.46 (#8)
GPT-5.5 on Evals for Every Language - Language bg: 71.22 (#23)
GPT-5.5 on Evals for Every Language - Language bho: 67.61 (#4)
GPT-5.5 on Evals for Every Language - Language bjn: 44.06 (#12)
GPT-5.5 on Evals for Every Language - Language bm: 54.72 (#4)
GPT-5.5 on Evals for Every Language - Language bn: 69.73 (#14)
GPT-5.5 on Evals for Every Language - Language bs: 71.46 (#13)
GPT-5.5 on Evals for Every Language - Language ca: 73.21 (#7)
GPT-5.5 on Evals for Every Language - Language ceb: 74.54 (#10)
GPT-5.5 on Evals for Every Language - Language chm: 58.46 (#9)
GPT-5.5 on Evals for Every Language - Language ckb: 68.48 (#5)
GPT-5.5 on Evals for Every Language - Language crh: 63.78 (#15)
GPT-5.5 on Evals for Every Language - Language cs: 71.8 (#10)
GPT-5.5 on Evals for Every Language - Language cv: 59.68 (#10)
GPT-5.5 on Evals for Every Language - Language cy: 77.61 (#8)
GPT-5.5 on Evals for Every Language - Language da: 71.48 (#23)
GPT-5.5 on Evals for Every Language - Language de: 73.13 (#20)
GPT-5.5 on Evals for Every Language - Language doi: 71.32 (#2)
GPT-5.5 on Evals for Every Language - Language dz: 58.36 (#6)
GPT-5.5 on Evals for Every Language - Language ee: 56.99 (#5)
GPT-5.5 on Evals for Every Language - Language el: 71.64 (#6)
GPT-5.5 on Evals for Every Language - Language en: 85.03 (#4)
GPT-5.5 on Evals for Every Language - Language eo: 72.05 (#13)
GPT-5.5 on Evals for Every Language - Language es: 70.48 (#23)
GPT-5.5 on Evals for Every Language - Language et: 72.25 (#3)
GPT-5.5 on Evals for Every Language - Language eu: 67.59 (#11)
GPT-5.5 on Evals for Every Language - Language fa: 67.54 (#12)
GPT-5.5 on Evals for Every Language - MGSM: 90.21 (#5)
GPT-5.5 on Evals for Every Language - MMLU: 98.21 (#5)
GPT-5.5 on Evals for Every Language - Translation From: 40.95 (#6)
GPT-5.5 on Evals for Every Language - Translation To: 39.31 (#5)
GPT-5.5 on GRAB-Lite: 71.8 (#2)
Qwen 3.7 Max on Position Bias (Lechmazur): 34.8 (#10)
Qwen 3.7 Max on RuneBench: 2222.0 (#11)
Qwen 3.7 Max on Wolfram LLM Benchmarking Project: 67.5 (#14)

New #1 Leaders (92)

YC-Bench: Claude Fable 5 (1977.6) beat Claude Opus 4.7 by 263.1
PACT (Lechmazur): Claude Fable 5 (High) (2171.0) beat GPT-5.5 (High) by 155.0
Chatbot Arena (Code): Claude Fable 5 (1665.0) beat Claude Opus 4.7 (Thinking) by 98.0
Chatbot Arena (Text-to-Video): gemini-omni-flash (1527.0) beat dreamina-seedance-2.0-720p by 64.0
Design Arena (UI Components): Claude Fable 5 (1417.0) beat Claude Opus 4.7 by 57.0
Multi-turn Debate (Lechmazur): Claude Fable 5 (High) (1770.9) beat Claude Opus 4.7 (High) by 53.8
AA GDPval: Claude Fable 5 (Adaptive Reasoning, Max Effort, Opus 4.8 Fallback) (1932.47) beat Claude Opus 4.8 (Adaptive Reasoning, Max Effort) by 42.67
Design Arena (Data Viz): Claude Fable 5 (1381.0) beat Claude Opus 4.7 (Thinking) by 42.0
Design Arena (Game Dev): Claude Fable 5 (1382.0) beat GPT-5.5 by 27.0
GSMA Open-Telco - TeleTables: TelecomGPT (88.0) beat OTel-LLM-8.3B-QnA by 26.2
LLM Stats (MCP-Mark): Kimi K2.7 Code (81.1) beat Qwen 3.7 Max by 20.3
Design Arena (Image): riverflow-2.5-pro (1419.0) beat gpt-image-2 by 17.0
WDCD: Qwen 3 Max (84.38) beat Claude Opus 4.7 by 14.38
Evals for Every Language - Language ay: step-3.7-flash-20260528 (77.14) beat Gemini 3.1 Pro (Preview) by 14.23
SEAL - SWE Atlas - Test Writing: Fable-5 (Claude Code) xHigh (58.52) beat GPT-5.4 (xHigh) by 14.16
LiveBench Python: Claude Fable 5 (xHigh) (95.0) beat Claude Opus 4.5 (Thinking 64K, High) (2025-11-01) by 10.0
LLM Stats (FLEURS): Qwen2.5-Omni-7B (95.9) beat Gemini 1.5 Flash-8B by 9.5
CursorBench 3.1: Claude Fable 5 (Max) (72.9) beat Claude Opus 4.7 by 8.1
AA Omniscience - Software Engineering (SWE) - Dart: Claude Fable 5 (Adaptive Reasoning, Max Effort, Opus 4.8 Fallback) (88.0) beat GPT-5.3 Codex (xHigh) by 8.0
AA Omniscience - Software Engineering (SWE) - R: Claude Fable 5 (Adaptive Reasoning, Max Effort, Opus 4.8 Fallback) (82.0) beat GPT-5.5 (High) by 8.0
AA Omniscience - Software Engineering (SWE) - Swift: Claude Fable 5 (Adaptive Reasoning, Max Effort, Opus 4.8 Fallback) (100.0) beat GPT-5.5 (xHigh) by 8.0
Vals AI Vibe Code Bench: Claude Fable 5 (90.35) beat Claude Opus 4.8 by 7.63
AA Humanity's Last Exam: Claude Fable 5 (Adaptive Reasoning, Max Effort, Opus 4.8 Fallback) (53.34) beat Claude Opus 4.8 (Adaptive Reasoning, Max Effort) by 7.6
AA Omniscience: Claude Fable 5 (Adaptive Reasoning, Max Effort, Opus 4.8 Fallback) (40.15) beat Gemini 3.1 Pro (Preview) by 7.22
FrontierSWE: Claude Fable 5 (90.0) beat Claude Opus 4.8 by 7.0
Vellum - HumanEval: Claude Mythos 5 (95.5) beat Claude Opus 4.8 by 6.9
Vellum - Humanity's Last Exam: Claude Mythos 5 (64.5) beat Claude Opus 4.8 by 6.6
Evals for Every Language - Language crh: step-3.7-flash-20260528 (73.05) beat Gemini 3.1 Pro (Preview) by 6.27
Chatbot Arena (Text): Claude Fable 5 (1510.0) beat Claude Opus 4.6 (Thinking) by 6.0
AA Omniscience - Software Engineering (SWE) - Java: Claude Fable 5 (Adaptive Reasoning, Max Effort, Opus 4.8 Fallback) (79.0) beat GPT-5.3 Codex (xHigh) by 6.0
Vals AI ProofBench: Claude Fable 5 (77.0) beat aristotle by 6.0
AA Omniscience - Business: Claude Fable 5 (Adaptive Reasoning, Max Effort, Opus 4.8 Fallback) (55.0) beat GPT-5.5 (xHigh) by 5.9
FinBen - MultiFin: plutus-8B-instruct (72.22) beat Qwen 2.5 72B Instruct by 5.55
AA Omniscience - Science, Engineering & Mathematics: Claude Fable 5 (Adaptive Reasoning, Max Effort, Opus 4.8 Fallback) (57.1) beat GPT-5.5 (High) by 4.8
Vals AI (Vals Index): Claude Fable 5 (75.14) beat Claude Opus 4.8 by 4.78
OpenClawProBench: GLM-5.2 (81.3) beat intern-s2-preview by 4.6
Vals AI IOI: Claude Fable 5 (72.25) beat GPT-5.4 (2026-03-05) by 4.42
AA Omniscience - Humanities & Social Sciences: Claude Fable 5 (Adaptive Reasoning, Max Effort, Opus 4.8 Fallback) (60.9) beat Gemini 3 Pro (Preview) (High) by 4.3
Design Arena (Website): Claude Fable 5 (1345.0) beat Claude Opus 4.6 by 4.0
AA Omniscience - Software Engineering (SWE) - Go: Claude Fable 5 (Adaptive Reasoning, Max Effort, Opus 4.8 Fallback) (88.0) beat GPT-5.5 (High) by 4.0
MathArena - ARXIV April: Claude Fable 5 (Max) (70.73) beat GPT-5.5 (xHigh) by 3.66
GSMA Open-Telco LLM Leaderboard: TelecomGPT (89.64) beat OTel-LLM-8.3B-QnA by 3.66
FinBen - QA: GPT-4o (78.22) beat GPT-4.5 (Preview) by 3.55
Artificial Analysis Intelligence Index: Claude Fable 5 (Adaptive Reasoning, Max Effort, Opus 4.8 Fallback) (64.88) beat Claude Opus 4.8 (Adaptive Reasoning, Max Effort) by 3.44
Evals for Every Language - Language cv: gemma-4-31B-it-20260402 (69.3) beat Claude Opus 4.5 by 3.39
SEAL - SWE Atlas - Codebase QnA: Opus 4.8 (Claude Code) (48.79) beat GPT-5.5 by 3.36
Vals AI CorpFin v2: Claude Fable 5 (71.83) beat Grok 4.3 by 3.3
Vals AI Multimodal Index: Claude Fable 5 (74.15) beat Claude Opus 4.8 by 3.26
AA Omniscience - Software Engineering (SWE): Claude Fable 5 (Adaptive Reasoning, Max Effort, Opus 4.8 Fallback) (87.6) beat GPT-5.5 (xHigh) by 3.2
Design Arena (3D): Claude Fable 5 (1370.0) beat Kimi K2.6 by 3.0
GRAB-Lite: Claude Fable 5 (74.0) beat GPT-5.4 by 3.0
WeirdML: Claude Fable 5 (High) (87.85) beat GPT-5.5 (xHigh) by 2.94
BIRD-SQL: Gemini-SQL2 (80.04) beat Gemini-SQL (Multitask SFT + Gemini-2.5-Pro) by 2.9
GSMA Open-Telco - 3GPP: TelecomGPT (84.22) beat OTel-LLM-8.3B-QnA by 2.82
GSMA Open-Telco - TeleLogs: TelecomGPT (98.96) beat OTel-LLM-8.3B-QnA by 2.66
Evals for Every Language - MGSM: Claude Opus 4.8 (96.62) beat Claude Opus 4.6 by 2.36
Evals for Every Language - Language ban: step-3.7-flash-20260528 (69.03) beat Claude Opus 4.5 by 2.32
SimpleBench: Claude Fable 5 (81.9) beat Gemini 3.1 Pro (Preview) by 2.3
AA Terminal-Bench Hard: Claude Fable 5 (Adaptive Reasoning, Max Effort, Opus 4.8 Fallback) (62.88) beat GPT-5.5 (xHigh) by 2.27
Chatbot Arena (Image-to-Video): gemini-omni-flash (1475.0) beat Grok 1.5 by 2.0
LiveBench Plot Unscrambling: Claude Fable 5 (xHigh) (78.09) beat GPT-5.5 (High) by 1.81
UGI - Writing: Claude Fable 5 (Adaptive Reasoning, High Effort) (74.23) beat Gemini 3.5 Flash (Thinking, Medium) by 1.69
GSMA Open-Telco - srsRAN-Bench: TelecomGPT (91.33) beat OTel-LLM-8.3B-QnA by 1.65
LLM Stats (OSWorld-Verified): Claude Fable 5 (85.0) beat Claude Opus 4.8 by 1.6
AA Omniscience - Software Engineering (SWE) - Python: Claude Fable 5 (Adaptive Reasoning, Max Effort, Opus 4.8 Fallback) (92.0) beat GPT-5.5 (xHigh) by 1.5
Evals for Every Language - Language chm: Claude Opus 4.7 (63.6) beat Gemini 3.1 Pro (Preview) by 1.48
Evals for Every Language - Language doi: Claude Opus 4.7 (71.84) beat Gemini 3 Pro (Preview) by 1.46
AA CritPt: Claude Fable 5 (Adaptive Reasoning, Max Effort, Opus 4.8 Fallback) (28.57) beat GPT-5.5 (xHigh) by 1.43
Evals for Every Language - Language es: Gemini 3.1 Flash Lite (76.16) beat Claude Opus 4.6 by 1.42
AA SciCode: Claude Fable 5 (Adaptive Reasoning, Max Effort, Opus 4.8 Fallback) (60.19) beat Gemini 3.1 Pro (Preview) by 1.28
Evals for Every Language - Language ace: step-3.7-flash-20260528 (72.48) beat Gemini 3.1 Pro (Preview) by 1.28
Evals for Every Language - MMLU: intellect-3-20251126 (100.0) beat Claude Sonnet 4.6 by 1.27
Evals for Every Language - ARC: intellect-3-20251126 (100.0) beat Gemini 3.1 Pro (Preview) by 1.26
EQ-Bench Longform Writing: Claude Fable 5 (83.0) beat Claude Opus 4.7 by 1.2
Vals AI LegalBench: Claude Fable 5 (88.56) beat Gemini 3.1 Pro (Preview) by 1.16
Evals for Every Language - Language ca: Gemini 3.1 Flash Lite (76.29) beat Gemini 3 Pro (Preview) by 1.03
Design Arena (SVG): Claude Fable 5 (1370.0) beat prism by 1.0
Opper TaskBench: Claude Fable 5 (96.4) beat Claude Opus 4.7 by 1.0
Evals for Every Language - Language ar: Claude Opus 4.8 (71.58) beat Claude Opus 4.5 by 0.95
Evals for Every Language - Language en: Gemini 3.1 Flash Lite (87.28) beat MiniMax-M2.5 by 0.77
MathArena - HMMT Feb 2026: GPT-5.5 (xHigh) (98.48) beat GPT-5.4 (xHigh) by 0.75
Evals for Every Language - Language cy: Gemini 3.1 Flash Lite (82.03) beat Claude Sonnet 4.5 by 0.65
Evals for Every Language - Language am: Gemini 3.1 Flash Lite (68.6) beat Claude Opus 4.6 by 0.59
Vals AI MedScribe: Claude Fable 5 (88.52) beat GPT-5.1 by 0.43
Evals for Every Language - Language af: Gemini 3.1 Pro (Preview) (79.41) beat Claude Sonnet 4 by 0.43
Evals for Every Language - Language be: Claude Opus 4.8 (69.43) beat Gemini 3.1 Pro (Preview) by 0.32
LLM Stats (Video-MME): MiMo-V2.5 (87.7) beat Kimi K2.5 by 0.3
Evals for Every Language - Language ceb: Gemini 3.1 Flash Lite (78.06) beat Gemini 3.1 Pro (Preview) by 0.29
Evals for Every Language - Language el: Gemini 3.1 Flash Lite (73.81) beat Claude Opus 4.5 by 0.15
LLM Stats (CMMLU): MiMo-V2.5-Pro (90.2) beat Qwen 2 72B Instruct by 0.1
Blueprint-Bench 2: Claude Fable 5 (0.386) beat GPT-5.5 by 0.02
LiveBench Olympiad: Claude Fable 5 (High) (92.18) beat Claude Opus 4.6 (Thinking, High) by 0.01

AI Benchmark Digest — 2026-06-13

2026-06-13T08:02:57.174839+00:00

Daily

Top-10 New Scores (11)

Claude 5 on Chess Puzzles (Epoch AI): 41.0 (#8)
Claude 5 on OTIS Mock AIME 2024-25: 99.72 (#3)
Claude 5 on SimpleQA Verified: 68.3 (#4)
Claude Fable 5 on Epoch AI - Apex Agents: 45.0 (#3)
Claude Fable 5 on Icelandic LLM - ARC-Challenge-IS: 72.95 (#59)
Claude Fable 5 on Icelandic LLM - Belebele-IS: 90.78 (#36)
Claude Fable 5 on Icelandic LLM - Inflection: 97.75 (#2)
Claude Fable 5 on Icelandic LLM - WinoGrande-IS: 96.05 (#2)
Claude Fable 5 on Icelandic LLM Leaderboard - Average: 87.4 (#4)
GPT-5.5 on Blueprint-Bench 2: 0.362 (#2)
Qwen 3.7 Max on Wolfram LLM Benchmarking Project: 67.5 (#14)

New #1 Leaders (6)

Design Arena (Image): riverflow-2.5-pro (1416.0) beat gpt-image-2 by 23.0
LLM Stats (MCP-Mark): Kimi K2.7 Code (81.1) beat Qwen 3.7 Max by 20.3
Icelandic LLM - WikiQA-IS: Claude Fable 5 (75.39) beat Gemini 3.1 Pro (Preview) by 7.65
Icelandic LLM - GED: Claude Fable 5 (91.5) beat Claude Opus 4.7 by 7.0
BIRD-SQL: Gemini-SQL2 (80.04) beat Gemini-SQL (Multitask SFT + Gemini-2.5-Pro) by 2.9
Design Arena (Graphic Design): riverflow-2.5-pro (1474.0) beat gpt-image-2 by 1.0

AI Benchmark Digest — 2026-06-12

2026-06-12T08:17:57.895837+00:00

Daily

New Benchmarks (2)

MathArena - ARXIV_FALSE May (Accuracy (%)): leader GPT-5.5 (xhigh) (50.0), 8 models
MathArena - ARXIV May (Accuracy (%)): leader Claude-Fable-5 (max) (86.67), 8 models

Top-10 New Scores (9)

Claude Fable 5 on Lynchmark: 100.0 (#1)
Claude Fable 5 on MineBench: 1929.84 (#2)
Claude Opus 4.8 on Chess Puzzles (Epoch AI): 34.0 (#12)
Claude Opus 4.8 on Design Arena (Game Dev): 1250.0 (#37)
Claude Opus 4.8 on GRAB-Lite: 60.6 (#6)
Claude Opus 4.8 on OTIS Mock AIME 2024-25: 98.33 (#3)
Claude Opus 4.8 on SimpleQA Verified: 39.5 (#24)
GPT-5.5 on GRAB-Lite: 71.8 (#2)
Qwen 3.7 Max on Position Bias (Lechmazur): 34.8 (#10)

New #1 Leaders (9)

Chatbot Arena (Text-to-Video): gemini-omni-flash (1527.0) beat dreamina-seedance-2.0-720p by 64.0
Design Arena (UI Components): Claude Fable 5 (1411.0) beat Claude Opus 4.7 (Thinking) by 56.0
Design Arena (Game Dev): Claude Fable 5 (1393.0) beat GPT-5.5 by 39.0
Design Arena (SVG): Claude Fable 5 (1384.0) beat prism by 18.0
SEAL - SWE Atlas - Test Writing: Fable-5 (Claude Code) xHigh (58.52) beat Opus 4.8 (Claude Code) by 12.96
MathArena - ARXIV April: Claude 5 (70.73) beat GPT-5.5 (xHigh) by 3.66
GRAB-Lite: Claude Fable 5 (74.0) beat GPT-5.4 by 3.0
WeirdML: Claude 5 (87.85) beat GPT-5.5 (xHigh) by 2.94
Chatbot Arena (Image-to-Video): gemini-omni-flash (1475.0) beat Grok 1.5 by 2.0

AI Benchmark Digest — 2026-06-11

2026-06-11T08:17:01.068404+00:00

Daily

New Benchmarks (1)

GDPval-AA (Elo): leader Claude Fable 5 (Adaptive Reasoning, Max Effort, Opus 4.8 Fallback) (1932.0), 390 models

Top-10 New Scores (3)

Claude Fable 5 on Chatbot Arena (Document): 1495.0 (#5)
Claude Fable 5 on Chatbot Arena (Vision): 1307.0 (#2)
Claude Fable 5 on React Native Evals: 86.96 (#4)

New #1 Leaders (12)

PACT (Lechmazur): Claude Fable 5 (High) (2171.0) beat GPT-5.5 (High) by 155.0
Chatbot Arena (Code): Claude Fable 5 (1665.0) beat Claude Opus 4.7 (Thinking) by 98.0
Design Arena (Data Viz): Claude Fable 5 (1406.0) beat Claude Opus 4.7 (Thinking) by 68.0
Design Arena (Website): Claude Fable 5 (1364.0) beat Claude Opus 4.6 by 23.0
Design Arena (3D): Claude Fable 5 (1383.0) beat Kimi K2.6 by 17.0
FrontierSWE: Claude Fable 5 (90.0) beat Claude Opus 4.8 by 7.0
Chatbot Arena (Text): Claude Fable 5 (1510.0) beat Claude Opus 4.6 (Thinking) by 6.0
SimpleBench: Claude Fable (81.9) beat Gemini 3.1 Pro (Preview) by 2.3
UGI - Writing: Claude 5 (74.23) beat Gemini 3.5 Flash (Thinking, Medium) by 1.69
EQ-Bench Longform Writing: Claude Fable 5 (83.0) beat Claude Opus 4.7 by 1.2
LLM Stats (Video-MME): MiMo-V2.5 (87.7) beat Kimi K2.5 by 0.3
LLM Stats (CMMLU): MiMo-V2.5-Pro (90.2) beat Qwen 2 72B Instruct by 0.1

AI Benchmark Digest — 2026-06-10

2026-06-10T09:55:36.786616+00:00

Daily

New Benchmarks (1)

SkateBench (Success Rate (%)): leader gemini-3.1-pro-preview (96.92), 28 models
Skateboarding-domain knowledge benchmark ranking models by how well they identify technical skateboard tricks from 390 trick definitions. SkateBench v2 reports success rate, cost, and speed.

New Models (1)

Claude Fable 5 — ELO 1871, #31
- Blueprint-Bench 2: 0.386 (#1/14)
- Opper TaskBench: 96.4 (#1/85)
- LLM Stats (OSWorld-Verified): 85.0 (#1/16)
- YC-Bench: 1977.6 (#1/21)
- Vals AI (Vals Index): 75.14 (#1/25)
- Vals AI Multimodal Index: 74.15 (#1/20)
- Vals AI LegalBench: 88.56 (#1/114)
- Vals AI CorpFin v2: 71.83 (#1/111)
- Vals AI MedScribe: 88.52 (#1/62)
- Vals AI ProofBench: 77.0 (#1/37)

New #1 Leaders (55)

YC-Bench: Claude Fable 5 (1977.6) beat Claude Opus 4.7 by 263.1
Multi-turn Debate (Lechmazur): Claude Fable 5 (High) (1770.9) beat Claude Opus 4.7 (High) by 53.8
AA GDPval: Claude Fable 5 (Adaptive Reasoning, Max Effort, Opus 4.8 Fallback) (1932.47) beat Claude Opus 4.8 (Adaptive Reasoning, Max Effort) by 42.67
Evals for Every Language - Language ay: step-3.7-flash-20260528 (77.14) beat Gemini 3.1 Pro (Preview) by 14.23
LiveBench Python: Claude Fable 5 (xHigh) (95.0) beat Claude Opus 4.5 (Thinking 64K, High) (2025-11-01) by 10.0
CursorBench 3.1: Fable 5 Max (72.9) beat Claude Opus 4.7 by 8.1
AA Omniscience - Software Engineering (SWE) - Dart: Claude Fable 5 (Adaptive Reasoning, Max Effort, Opus 4.8 Fallback) (88.0) beat GPT-5.3 Codex (xHigh) by 8.0
AA Omniscience - Software Engineering (SWE) - R: Claude Fable 5 (Adaptive Reasoning, Max Effort, Opus 4.8 Fallback) (82.0) beat GPT-5.5 (Medium) by 8.0
AA Omniscience - Software Engineering (SWE) - Swift: Claude Fable 5 (Adaptive Reasoning, Max Effort, Opus 4.8 Fallback) (100.0) beat GPT-5.5 (xHigh) by 8.0
Vals AI Vibe Code Bench: Claude Fable 5 (90.35) beat Claude Opus 4.8 by 7.63
AA Humanity's Last Exam: Claude Fable 5 (Adaptive Reasoning, Max Effort, Opus 4.8 Fallback) (53.34) beat Claude Opus 4.8 (Adaptive Reasoning, Max Effort) by 7.6
AA Omniscience: Claude Fable 5 (Adaptive Reasoning, Max Effort, Opus 4.8 Fallback) (40.15) beat Gemini 3.1 Pro (Preview) by 7.22
Vellum - HumanEval: Claude Mythos 5 (95.5) beat Claude Opus 4.8 by 6.9
Vellum - Humanity's Last Exam: Claude Mythos 5 (64.5) beat Claude Opus 4.8 by 6.6
Evals for Every Language - Language crh: step-3.7-flash-20260528 (73.05) beat Gemini 3.1 Pro (Preview) by 6.27
AA Omniscience - Software Engineering (SWE) - Java: Claude Fable 5 (Adaptive Reasoning, Max Effort, Opus 4.8 Fallback) (79.0) beat GPT-5.3 Codex (xHigh) by 6.0
Vals AI ProofBench: Claude Fable 5 (77.0) beat aristotle by 6.0
AA Omniscience - Business: Claude Fable 5 (Adaptive Reasoning, Max Effort, Opus 4.8 Fallback) (55.0) beat GPT-5.5 (xHigh) by 5.9
AA Omniscience - Science, Engineering & Mathematics: Claude Fable 5 (Adaptive Reasoning, Max Effort, Opus 4.8 Fallback) (57.1) beat GPT-5.5 (High) by 4.8
Vals AI (Vals Index): Claude Fable 5 (75.14) beat Claude Opus 4.8 by 4.78
Vals AI IOI: Claude Fable 5 (72.25) beat GPT-5.4 (2026-03-05) by 4.42
AA Omniscience - Humanities & Social Sciences: Claude Fable 5 (Adaptive Reasoning, Max Effort, Opus 4.8 Fallback) (60.9) beat Gemini 3 Pro (Preview) (High) by 4.3
AA Omniscience - Software Engineering (SWE) - Go: Claude Fable 5 (Adaptive Reasoning, Max Effort, Opus 4.8 Fallback) (88.0) beat GPT-5.5 (High) by 4.0
Artificial Analysis Intelligence Index: Claude Fable 5 (Adaptive Reasoning, Max Effort, Opus 4.8 Fallback) (64.88) beat Claude Opus 4.8 (Adaptive Reasoning, Max Effort) by 3.44
Evals for Every Language - Language cv: gemma-4-31B-it-20260402 (69.3) beat Claude Opus 4.5 by 3.39
Vals AI CorpFin v2: Claude Fable 5 (71.83) beat Grok 4.3 by 3.3
Vals AI Multimodal Index: Claude Fable 5 (74.15) beat Claude Opus 4.8 by 3.26
AA Omniscience - Software Engineering (SWE): Claude Fable 5 (Adaptive Reasoning, Max Effort, Opus 4.8 Fallback) (87.6) beat GPT-5.5 (xHigh) by 3.2
Evals for Every Language - MGSM: Claude Opus 4.8 (96.62) beat Claude Opus 4.6 by 2.36
Evals for Every Language - Language ban: step-3.7-flash-20260528 (69.03) beat Claude Opus 4.5 by 2.32
AA Terminal-Bench Hard: Claude Fable 5 (Adaptive Reasoning, Max Effort, Opus 4.8 Fallback) (62.88) beat GPT-5.5 (xHigh) by 2.27
LiveBench Plot Unscrambling: Claude Fable 5 (xHigh) (78.09) beat GPT-5.5 (High) by 1.81
LLM Stats (OSWorld-Verified): Claude Fable 5 (85.0) beat Claude Opus 4.8 by 1.6
AA Omniscience - Software Engineering (SWE) - Python: Claude Fable 5 (Adaptive Reasoning, Max Effort, Opus 4.8 Fallback) (92.0) beat GPT-5.5 (xHigh) by 1.5
Evals for Every Language - Language chm: Claude Opus 4.7 (63.6) beat Gemini 3.1 Pro (Preview) by 1.48
Evals for Every Language - Language doi: Claude Opus 4.7 (71.84) beat Gemini 3 Pro (Preview) by 1.46
AA CritPt: Claude Fable 5 (Adaptive Reasoning, Max Effort, Opus 4.8 Fallback) (28.57) beat GPT-5.5 (xHigh) by 1.43
Evals for Every Language - Language es: Gemini 3.1 Flash Lite (76.16) beat Claude Opus 4.6 by 1.42
AA SciCode: Claude Fable 5 (Adaptive Reasoning, Max Effort, Opus 4.8 Fallback) (60.19) beat Gemini 3.1 Pro (Preview) by 1.28
Evals for Every Language - Language ace: step-3.7-flash-20260528 (72.48) beat Gemini 3.1 Pro (Preview) by 1.28
Evals for Every Language - MMLU: intellect-3-20251126 (100.0) beat Claude Sonnet 4.6 by 1.27
Evals for Every Language - ARC: intellect-3-20251126 (100.0) beat Gemini 3.1 Pro (Preview) by 1.26
Vals AI LegalBench: Claude Fable 5 (88.56) beat Gemini 3.1 Pro (Preview) by 1.16
Evals for Every Language - Language ca: Gemini 3.1 Flash Lite (76.29) beat Gemini 3 Pro (Preview) by 1.03
Opper TaskBench: Claude Fable 5 (96.4) beat Claude Opus 4.7 by 1.0
Evals for Every Language - Language ar: Claude Opus 4.8 (71.58) beat Claude Opus 4.5 by 0.95
Evals for Every Language - Language en: Gemini 3.1 Flash Lite (87.28) beat MiniMax-M2.5 by 0.77
Evals for Every Language - Language cy: Gemini 3.1 Flash Lite (82.03) beat Claude Sonnet 4.5 by 0.65
Evals for Every Language - Language am: Gemini 3.1 Flash Lite (68.6) beat Claude Opus 4.6 by 0.59
Vals AI MedScribe: Claude Fable 5 (88.52) beat GPT-5.1 by 0.43
Evals for Every Language - Language be: Claude Opus 4.8 (69.43) beat Gemini 3.1 Pro (Preview) by 0.32
Evals for Every Language - Language ceb: Gemini 3.1 Flash Lite (78.06) beat Gemini 3.1 Pro (Preview) by 0.29
Evals for Every Language - Language el: Gemini 3.1 Flash Lite (73.81) beat Claude Opus 4.5 by 0.15
Blueprint-Bench 2: Claude Fable 5 (0.386) beat GPT-5.5 by 0.02
LiveBench Olympiad: Claude Fable 5 (High) (92.18) beat Claude Opus 4.6 (Thinking, High) by 0.01

AI Benchmark Digest — 2026-06-10

2026-06-10T08:06:50.673963+00:00

Daily

New Benchmarks (1)

SkateBench (Success Rate (%)): leader gemini-3.1-pro-preview (96.92), 28 models
Skateboarding-domain knowledge benchmark ranking models by how well they identify technical skateboard tricks from 390 trick definitions. SkateBench v2 reports success rate, cost, and speed.

New Models (2)

Claude 5 — ELO 1904, #22
- LiveBench Olympiad: 92.18 (#1/124)
- LiveBench Plot Unscrambling: 78.09 (#1/124)
- LiveBench Python: 95.0 (#1/124)
- Opper TaskBench: 96.4 (#1/85)
- Vals AI (Vals Index): 75.14 (#1/25)
- Vals AI Multimodal Index: 74.15 (#1/20)
- Vals AI LegalBench: 88.56 (#1/114)
- Vals AI CorpFin v2: 71.83 (#1/111)
- Vals AI MedScribe: 88.52 (#1/62)
- Vals AI ProofBench: 77.0 (#1/37)
Claude Fable 5 — ELO 1901, #23
- Blueprint-Bench 2: 0.386 (#1/14)
- LLM Stats (OSWorld-Verified): 85.0 (#1/16)
- YC-Bench: 1977.6 (#1/21)
- SEAL - MCP Atlas: 83.3 (#2/23)
- Vellum - HumanEval: 95.0 (#2/38)
- Vellum - GPQA: 94.1 (#3/57)
- ClockBench: 35.0 (#4/27)
- LLM Stats (GDPval-AA): 64.4 (#11/12)

New #1 Leaders (55)

YC-Bench: Claude Fable 5 (1977.6) beat Claude Opus 4.7 by 263.1
Multi-turn Debate (Lechmazur): Claude Fable 5 (High) (1770.9) beat Claude Opus 4.7 (High) by 53.8
AA GDPval: Claude Fable 5 (Adaptive Reasoning, Max Effort, Opus 4.8 Fallback) (1932.47) beat Claude Opus 4.8 (Adaptive Reasoning, Max Effort) by 42.67
Evals for Every Language - Language ay: step-3.7-flash-20260528 (77.14) beat Gemini 3.1 Pro (Preview) by 14.23
LiveBench Python: Claude 5 (95.0) beat Claude Opus 4.5 (Thinking 64K, High) (2025-11-01) by 10.0
CursorBench 3.1: Fable 5 Max (72.9) beat Claude Opus 4.7 by 8.1
AA Omniscience - Software Engineering (SWE) - Dart: Claude Fable 5 (Adaptive Reasoning, Max Effort, Opus 4.8 Fallback) (88.0) beat GPT-5.3 Codex (xHigh) by 8.0
AA Omniscience - Software Engineering (SWE) - R: Claude Fable 5 (Adaptive Reasoning, Max Effort, Opus 4.8 Fallback) (82.0) beat GPT-5.5 (Medium) by 8.0
AA Omniscience - Software Engineering (SWE) - Swift: Claude Fable 5 (Adaptive Reasoning, Max Effort, Opus 4.8 Fallback) (100.0) beat GPT-5.5 (xHigh) by 8.0
Vals AI Vibe Code Bench: Claude 5 (90.35) beat Claude Opus 4.8 by 7.63
AA Humanity's Last Exam: Claude Fable 5 (Adaptive Reasoning, Max Effort, Opus 4.8 Fallback) (53.34) beat Claude Opus 4.8 (Adaptive Reasoning, Max Effort) by 7.6
AA Omniscience: Claude Fable 5 (Adaptive Reasoning, Max Effort, Opus 4.8 Fallback) (40.15) beat Gemini 3.1 Pro (Preview) by 7.22
Vellum - HumanEval: Claude Mythos 5 (95.5) beat Claude Opus 4.8 by 6.9
Vellum - Humanity's Last Exam: Claude Mythos 5 (64.5) beat Claude Opus 4.8 by 6.6
Evals for Every Language - Language crh: step-3.7-flash-20260528 (73.05) beat Gemini 3.1 Pro (Preview) by 6.27
AA Omniscience - Software Engineering (SWE) - Java: Claude Fable 5 (Adaptive Reasoning, Max Effort, Opus 4.8 Fallback) (79.0) beat GPT-5.3 Codex (xHigh) by 6.0
Vals AI ProofBench: Claude 5 (77.0) beat aristotle by 6.0
AA Omniscience - Business: Claude Fable 5 (Adaptive Reasoning, Max Effort, Opus 4.8 Fallback) (55.0) beat GPT-5.5 (xHigh) by 5.9
AA Omniscience - Science, Engineering & Mathematics: Claude Fable 5 (Adaptive Reasoning, Max Effort, Opus 4.8 Fallback) (57.1) beat GPT-5.5 (High) by 4.8
Vals AI (Vals Index): Claude 5 (75.14) beat Claude Opus 4.8 by 4.78
Vals AI IOI: Claude 5 (72.25) beat GPT-5.4 (2026-03-05) by 4.42
AA Omniscience - Humanities & Social Sciences: Claude Fable 5 (Adaptive Reasoning, Max Effort, Opus 4.8 Fallback) (60.9) beat Gemini 3 Pro (Preview) (High) by 4.3
AA Omniscience - Software Engineering (SWE) - Go: Claude Fable 5 (Adaptive Reasoning, Max Effort, Opus 4.8 Fallback) (88.0) beat GPT-5.5 (High) by 4.0
Artificial Analysis Intelligence Index: Claude Fable 5 (Adaptive Reasoning, Max Effort, Opus 4.8 Fallback) (64.88) beat Claude Opus 4.8 (Adaptive Reasoning, Max Effort) by 3.44
Evals for Every Language - Language cv: gemma-4-31B-it-20260402 (69.3) beat Claude Opus 4.5 by 3.39
Vals AI CorpFin v2: Claude 5 (71.83) beat Grok 4.3 by 3.3
Vals AI Multimodal Index: Claude 5 (74.15) beat Claude Opus 4.8 by 3.26
AA Omniscience - Software Engineering (SWE): Claude Fable 5 (Adaptive Reasoning, Max Effort, Opus 4.8 Fallback) (87.6) beat GPT-5.5 (xHigh) by 3.2
Evals for Every Language - MGSM: Claude Opus 4.8 (96.62) beat Claude Opus 4.6 by 2.36
Evals for Every Language - Language ban: step-3.7-flash-20260528 (69.03) beat Claude Opus 4.5 by 2.32
AA Terminal-Bench Hard: Claude Fable 5 (Adaptive Reasoning, Max Effort, Opus 4.8 Fallback) (62.88) beat GPT-5.5 (xHigh) by 2.27
LiveBench Plot Unscrambling: Claude 5 (78.09) beat GPT-5.5 (High) by 1.81
LLM Stats (OSWorld-Verified): Claude Fable 5 (85.0) beat Claude Opus 4.8 by 1.6
AA Omniscience - Software Engineering (SWE) - Python: Claude Fable 5 (Adaptive Reasoning, Max Effort, Opus 4.8 Fallback) (92.0) beat GPT-5.5 (xHigh) by 1.5
Evals for Every Language - Language chm: Claude Opus 4.7 (63.6) beat Gemini 3.1 Pro (Preview) by 1.48
Evals for Every Language - Language doi: Claude Opus 4.7 (71.84) beat Gemini 3 Pro (Preview) by 1.46
AA CritPt: Claude Fable 5 (Adaptive Reasoning, Max Effort, Opus 4.8 Fallback) (28.57) beat GPT-5.5 (xHigh) by 1.43
Evals for Every Language - Language es: Gemini 3.1 Flash Lite (76.16) beat Claude Opus 4.6 by 1.42
AA SciCode: Claude Fable 5 (Adaptive Reasoning, Max Effort, Opus 4.8 Fallback) (60.19) beat Gemini 3.1 Pro (Preview) by 1.28
Evals for Every Language - Language ace: step-3.7-flash-20260528 (72.48) beat Gemini 3.1 Pro (Preview) by 1.28
Evals for Every Language - MMLU: intellect-3-20251126 (100.0) beat Claude Sonnet 4.6 by 1.27
Evals for Every Language - ARC: intellect-3-20251126 (100.0) beat Gemini 3.1 Pro (Preview) by 1.26
Vals AI LegalBench: Claude 5 (88.56) beat Gemini 3.1 Pro (Preview) by 1.16
Evals for Every Language - Language ca: Gemini 3.1 Flash Lite (76.29) beat Gemini 3 Pro (Preview) by 1.03
Opper TaskBench: Claude 5 (96.4) beat Claude Opus 4.7 by 1.0
Evals for Every Language - Language ar: Claude Opus 4.8 (71.58) beat Claude Opus 4.5 by 0.95
Evals for Every Language - Language en: Gemini 3.1 Flash Lite (87.28) beat MiniMax-M2.5 by 0.77
Evals for Every Language - Language cy: Gemini 3.1 Flash Lite (82.03) beat Claude Sonnet 4.5 by 0.65
Evals for Every Language - Language am: Gemini 3.1 Flash Lite (68.6) beat Claude Opus 4.6 by 0.59
Vals AI MedScribe: Claude 5 (88.52) beat GPT-5.1 by 0.43
Evals for Every Language - Language be: Claude Opus 4.8 (69.43) beat Gemini 3.1 Pro (Preview) by 0.32
Evals for Every Language - Language ceb: Gemini 3.1 Flash Lite (78.06) beat Gemini 3.1 Pro (Preview) by 0.29
Evals for Every Language - Language el: Gemini 3.1 Flash Lite (73.81) beat Claude Opus 4.5 by 0.15
Blueprint-Bench 2: Claude Fable 5 (0.386) beat GPT-5.5 by 0.02
LiveBench Olympiad: Claude 5 (92.18) beat Claude Opus 4.6 (Thinking) (High) by 0.01

AI Benchmark Digest — 2026-06-09

2026-06-09T07:53:25.528997+00:00

Daily

Top-10 New Scores (2)

GPT-5.5 (xHigh) on SEAL - SWE Atlas - Codebase QnA: 45.43 (#2)
GPT-5.5 (xHigh) on SEAL - SWE Atlas - Test Writing: 42.59 (#3)

New #1 Leaders (7)

GSMA Open-Telco - TeleTables: TelecomGPT (88.0) beat OTel-LLM-8.3B-QnA by 26.2
GSMA Open-Telco LLM Leaderboard: TelecomGPT (89.64) beat OTel-LLM-8.3B-QnA by 3.66
SEAL - SWE Atlas - Codebase QnA: Opus 4.8 (Claude Code) (48.79) beat GPT-5.5 by 3.36
GSMA Open-Telco - 3GPP: TelecomGPT (84.22) beat OTel-LLM-8.3B-QnA by 2.82
GSMA Open-Telco - TeleLogs: TelecomGPT (98.96) beat OTel-LLM-8.3B-QnA by 2.66
GSMA Open-Telco - srsRAN-Bench: TelecomGPT (91.33) beat OTel-LLM-8.3B-QnA by 1.65
SEAL - SWE Atlas - Test Writing: Opus 4.8 (Claude Code) (45.56) beat GPT-5.4 (xHigh) by 1.2

AI Benchmark Digest — 2026-06-07

2026-06-07T08:34:58.487719+00:00

Weekly

New Models (2)

MiniMax-M3 — ELO 1762, #83
- LLM Stats (OmniDocBench 1.5): 91.6 (#1/13)
- LLM Stats (Video-MME): 85.4 (#2/13)
- OpenClawProBench: 75.1 (#2/65)
- Vals AI MedScribe: 87.25 (#2/61)
- AA IFBench: 82.86 (#3/429)
- LLM Stats (Claw-Eval): 74.5 (#3/9)
- LLM Stats (NL2Repo): 42.13 (#3/7)
- AA GPQA Diamond: 92.93 (#4/501)
- Vals AI CorpFin v2: 68.1 (#4/110)
- Design Arena (3D): 1348.0 (#5/115)
nemotron-3-ultra-550B-a55B — ELO 1587, #292
- PinchBench: 90.58 (#10/49)
- Vals AI CorpFin v2: 65.46 (#16/110)
- Vals AI (Vals Index): 43.99 (#18/24)
- LiveBench Python: 75.0 (#24/122)
- LiveBench Paraphrase: 61.15 (#33/122)
- Vals AI TaxEval v2: 73.1 (#34/116)
- Bullshit Benchmark: 41.8 (#34/148)
- Vals AI MedCode: 38.62 (#35/62)
- AI Chess Leaderboard (Reasoning): 975.0 (#39/277)
- LiveBench Code Generation: 77.47 (#43/122)

Top-10 New Scores (4)

GPT-5.5 (xHigh) on IMO-Bench: 71.9 (#4)
GPT-5.5 Pro on IUMB: 100.0 (#2)
GPT-5.5 Pro (xHigh) on IMO-Bench: 88.1 (#2)
Gemini 3 Deep Think on IUMB: 87.5 (#6)

New #1 Leaders (10)

EQ-Bench Creative Writing v3: Claude Opus 4.7 (2050.8) beat GPT-5.4 by 144.8
Chatbot Arena (Image-to-Video): Grok 1.5 (1473.0) beat dreamina-seedance-2.0-720p by 11.0
LLM Stats (Multi-Challenge): Nova 2 Pro (77.7) beat GPT-5 by 8.1
MathArena - Kangaroo 2025 Levels 11-12: Claude Opus 4.8 (Thinking) (100.0) beat GPT-5.4 (xHigh) by 1.67
MathArena - APEX 2025: Claude Opus 4.8 (Thinking) (81.25) beat GPT-5.5 (xHigh) by 1.04
MathArena - Kangaroo 2025 Levels 7-8: Claude Opus 4.8 (Thinking) (96.67) beat GPT-5.4 (xHigh) by 0.84
MathArena - AIME 2026: Claude Opus 4.8 (Thinking) (100.0) beat GPT-5.4 (xHigh) by 0.83
LLM Stats (OmniDocBench 1.5): MiniMax-M3 (91.6) beat Qwen 3.6 Plus by 0.4
GAIA: CustomGPT.ai Research Lab v44 (93.36) beat Co-Sight Pro v1.0.1 by 0.34
ForecastBench: Grok 4.20 (Beta, D) (68.1) beat green-tree by 0.2

AI Benchmark Digest — 2026-06-06

2026-06-06T07:45:06.870709+00:00

Daily

New Benchmarks (20)

Pencil Puzzle Bench - Yajilin (Direct-ask Success Rate (%)): leader gpt-5.2 (High) (20.0), 51 models
PPBench direct-ask success rate on Yajilin loop-and-shading puzzles from the golden_300 split, testing exact constraint solving from puzz.link grids.
Pencil Puzzle Bench - Slitherlink (Direct-ask Success Rate (%)): leader gpt-5.2 (High) (33.3), 51 models
PPBench direct-ask success rate on Slitherlink loop puzzles, where numbered cells constrain how a single continuous loop surrounds the grid.
Pencil Puzzle Bench - Heyawake (Direct-ask Success Rate (%)): leader claude-opus-4-5-high (0.0), 51 models
PPBench direct-ask success rate on Heyawake room-shading puzzles, testing region constraints, connectivity, and line-of-sight reasoning.
Pencil Puzzle Bench - Mashu (Direct-ask Success Rate (%)): leader gpt-5.2 (xHigh) (60.0), 51 models
PPBench direct-ask success rate on Mashu loop puzzles, where black and white pearls impose turn and straight-line constraints.
Pencil Puzzle Bench - Shakashaka (Direct-ask Success Rate (%)): leader claude-sonnet-4-5 (0.0), 51 models
PPBench direct-ask success rate on Shakashaka triangle-shading puzzles, testing local clue satisfaction and global rectangle formation.
Pencil Puzzle Bench - Nurikabe (Direct-ask Success Rate (%)): leader claude-opus-4-6 (Thinking) (33.3), 51 models
PPBench direct-ask success rate on Nurikabe island puzzles, where numbered islands must be separated by one connected wall region.
Pencil Puzzle Bench - LITS (Direct-ask Success Rate (%)): leader gpt-5.2 (xHigh) (53.3), 51 models
PPBench direct-ask success rate on LITS tetromino-shading puzzles, testing region-wise shape placement and adjacency constraints.
Pencil Puzzle Bench - Light Up (Direct-ask Success Rate (%)): leader claude-opus-4-6 (Thinking) (66.7), 51 models
PPBench direct-ask success rate on Light Up puzzles, where lamps must illuminate every open cell while satisfying numbered black-cell clues.
Pencil Puzzle Bench - Nurimisaki (Direct-ask Success Rate (%)): leader claude-opus-4-6 (Thinking) (33.3), 51 models
PPBench direct-ask success rate on Nurimisaki puzzles, a Nurikabe-family grid task requiring connected-region reasoning around clue cells.
Pencil Puzzle Bench - Shikaku (Direct-ask Success Rate (%)): leader gpt-5.2 (xHigh) (80.0), 51 models
PPBench direct-ask success rate on Shikaku rectangle-partitioning puzzles, where each numbered clue defines one rectangle of matching area.
Pencil Puzzle Bench - Norinori (Direct-ask Success Rate (%)): leader gpt-5.2 (xHigh) (93.3), 51 models
PPBench direct-ask success rate on Norinori shading puzzles, testing room constraints and two-cell adjacency patterns.
Pencil Puzzle Bench - Double Choco (Direct-ask Success Rate (%)): leader gemini-3.1-pro (6.7), 51 models
PPBench direct-ask success rate on Double Choco region-division puzzles, testing balanced partitioning under color and shape constraints.
Pencil Puzzle Bench - Firefly (Direct-ask Success Rate (%)): leader gpt-5.2 (xHigh) (33.3), 51 models
PPBench direct-ask success rate on Firefly line-drawing puzzles, testing path construction from directional clues and grid constraints.
Pencil Puzzle Bench - Sashigane (Direct-ask Success Rate (%)): leader mistral-large-2512 (0.0), 51 models
PPBench direct-ask success rate on Sashigane shape-partitioning puzzles, testing right-angle region construction from numbered and directional clues.
Pencil Puzzle Bench - Sudoku (Direct-ask Success Rate (%)): leader gpt-5.2 (xHigh) (20.0), 51 models
PPBench direct-ask success rate on Sudoku puzzles, testing classic row, column, and box constraint satisfaction through exact move outputs.
Pencil Puzzle Bench - Nurimaze (Direct-ask Success Rate (%)): leader claude-opus-4-6 (Thinking) (26.7), 51 models
PPBench direct-ask success rate on Nurimaze puzzles, testing maze-style path and shading constraints in a connected grid.
Pencil Puzzle Bench - Tapa (Direct-ask Success Rate (%)): leader gpt-5.2 (xHigh) (60.0), 51 models
PPBench direct-ask success rate on Tapa shading puzzles, where clue numbers describe blocks of shaded neighboring cells.
Pencil Puzzle Bench - Kurodoko (Direct-ask Success Rate (%)): leader gpt-5.2 (xHigh) (6.7), 51 models
PPBench direct-ask success rate on Kurodoko visibility puzzles, testing shading, sight-line counts, and connected unshaded cells.
Pencil Puzzle Bench - Country (Direct-ask Success Rate (%)): leader gemini-3.1-pro (6.7), 51 models
PPBench direct-ask success rate on Country region puzzles, testing loop and region constraints over a partitioned grid.
Pencil Puzzle Bench - Hitori (Direct-ask Success Rate (%)): leader claude-opus-4-6 (Thinking) (66.7), 51 models
PPBench direct-ask success rate on Hitori number-grid puzzles, where repeated numbers are shaded while preserving connectivity and non-adjacency constraints.

New #1 Leaders (24)

LLM Stats (Multi-Challenge): Nova 2 Pro (77.7) beat GPT-5 by 8.1
Ukrainian LLM - Global MMLU Full UK World Religions: MamayLM-Gemma-3-27B-IT-v2.0 (87.13) beat gemma-3-12B-pt by 7.6
Ukrainian LLM - Global MMLU Full UK High School US History: MamayLM-Gemma-3-27B-IT-v2.0 (91.67) beat MamayLM-Gemma-3-12B-IT-v1.0 by 5.4
Ukrainian LLM - Global MMLU Full UK Anatomy: MamayLM-Gemma-3-27B-IT-v2.0 (65.19) beat lapa-12B-pt by 5.19
Ukrainian LLM - Global MMLU Full UK Clinical Knowledge: MamayLM-Gemma-3-27B-IT-v2.0 (77.74) beat gemma-3-12B-pt by 4.53
Ukrainian LLM - Global MMLU Full UK Professional LAW: MamayLM-Gemma-3-27B-IT-v2.0 (51.5) beat gemma-3-12B-pt by 4.43
Ukrainian LLM - Global MMLU Full UK Humanities: MamayLM-Gemma-3-27B-IT-v2.0 (61.68) beat Qwen3-8B-Base by 4.12
Ukrainian LLM - Global MMLU Full UK Computer Security: MamayLM-Gemma-3-12B-IT-v2.0 (82.0) beat MamayLM-Gemma-3-12B-IT-v1.0-FP8-Static-Nadiia by 4.0
Ukrainian LLM - Global MMLU Full UK Global Facts: MamayLM-Gemma-3-27B-IT-v2.0 (52.0) beat Gemma 3 12B (IT) by 4.0
Ukrainian LLM - Global MMLU Full UK Miscellaneous: MamayLM-Gemma-3-27B-IT-v2.0 (83.52) beat MamayLM-Gemma-3-12B-IT-v1.0-FP8-Static-Nadiia by 3.95
Ukrainian LLM - Global MMLU Full UK Prehistory: MamayLM-Gemma-3-27B-IT-v2.0 (77.78) beat gemma-3-12B-pt by 3.71
Ukrainian LLM - Global MMLU Full UK Other: MamayLM-Gemma-3-27B-IT-v2.0 (74.57) beat gemma-3-12B-pt by 3.41
Ukrainian LLM - Global MMLU Full UK Business Ethics: MamayLM-Gemma-3-12B-IT-v2.0 (77.0) beat MamayLM-Gemma-3-12B-IT-v1.0 by 3.0
Ukrainian LLM - Global MMLU Full UK High School World History: MamayLM-Gemma-3-27B-IT-v2.0 (86.08) beat gemma-3-12B-pt by 1.69
Ukrainian LLM - Global MMLU Full UK High School Microeconomics: MamayLM-Gemma-3-27B-IT-v2.0 (84.45) beat Qwen3-8B-Base by 1.68
Ukrainian LLM - Global MMLU Full UK Marketing: MamayLM-Gemma-3-27B-IT-v2.0 (88.89) beat MamayLM-Gemma-3-12B-IT-v1.0 by 1.28
Ukrainian LLM - Global MMLU Full UK Professional Psychology: MamayLM-Gemma-3-27B-IT-v2.0 (70.1) beat gemma-3-12B-pt by 0.98
Ukrainian LLM - Global MMLU Full UK Public Relations: MamayLM-Gemma-3-12B-IT-v2.0 (68.18) beat lapa-12B-pt by 0.91
Ukrainian LLM - Global MMLU Full UK High School European History: MamayLM-Gemma-3-27B-IT-v2.0 (84.24) beat MamayLM-Gemma-3-12B-IT-v1.0-FP8-Static-Nadiia by 0.6
Ukrainian LLM - Global MMLU Full UK High School Macroeconomics: MamayLM-Gemma-3-27B-IT-v2.0 (76.67) beat gemma-3-12B-pt by 0.52
Ukrainian LLM - Global MMLU Full UK Sociology: MamayLM-Gemma-3-27B-IT-v2.0 (83.08) beat lapa-v0.1.2-instruct by 0.49
LLM Stats (OmniDocBench 1.5): MiniMax-M3 (91.6) beat Qwen 3.6 Plus by 0.4
Ukrainian LLM - Global MMLU Full UK Professional Medicine: MamayLM-Gemma-3-27B-IT-v2.0 (80.15) beat gemma-3-12B-pt by 0.37
ForecastBench: Grok 4.20 (Beta, D) (68.1) beat green-tree by 0.3

AI Benchmark Digest — 2026-06-04

2026-06-04T08:22:19.073162+00:00

Daily

New #1 Leaders (1)

GAIA: CustomGPT.ai Research Lab v44 (93.36) beat Co-Sight Pro v1.0.1 by 0.34

AI Benchmark Digest — 2026-06-03

2026-06-03T08:25:40.519214+00:00

Daily

Top-10 New Scores (2)

GPT-5.5 Pro on IUMB: 100.0 (#2)
Gemini 3 Deep Think on IUMB: 87.5 (#6)

New #1 Leaders (4)

MathArena - Kangaroo 2025 Levels 11-12: Claude Opus 4.8 (Thinking) (100.0) beat GPT-5.4 (xHigh) by 1.67
MathArena - APEX 2025: Claude Opus 4.8 (Thinking) (81.25) beat GPT-5.5 (xHigh) by 1.04
MathArena - Kangaroo 2025 Levels 7-8: Claude Opus 4.8 (Thinking) (96.67) beat GPT-5.4 (xHigh) by 0.84
MathArena - AIME 2026: Claude Opus 4.8 (Thinking) (100.0) beat GPT-5.4 (xHigh) by 0.83

AI Benchmark Digest — 2026-06-02

2026-06-02T08:19:29.198019+00:00

Daily

New Benchmarks (1)

GIM (IRT ability (theta)): leader GPT-5.4 Pro (High) (2.16), 46 models
Grounded Integration Measure from Meta FAIR: 820 multimodal and text-grounded problems testing integrated reasoning across quantitative, spatial, language, world-knowledge, and document tasks. Scores are reported as IRT ability on GIM-820.

Top-10 New Scores (2)

GPT-5.5 (xHigh) on IMO-Bench: 71.9 (#4)
GPT-5.5 Pro (xHigh) on IMO-Bench: 88.1 (#2)

AI Benchmark Digest — 2026-06-01

2026-06-01T08:29:45.265204+00:00

Daily

New #1 Leaders (3)

EQ-Bench Creative Writing v3: Claude Opus 4.7 (2050.8) beat GPT-5.4 by 144.8
Design Arena (Data Viz): GLM-5.1 (1367.0) beat Claude Opus 4.7 (Thinking) by 23.0
Chatbot Arena (Image-to-Video): Grok 1.5 (1473.0) beat dreamina-seedance-2.0-720p by 11.0

AI Benchmark Digest — 2026-05-30

2026-05-30T07:49:09.779753+00:00

Daily

Top-10 New Scores (5)

Claude Opus 4.8 (Adaptive Reasoning, Max Effort) on UGI - Natural Intelligence: 65.39 (#30)
Claude Opus 4.8 (Adaptive Reasoning, Max Effort) on UGI - Willingness (W/10): 2.2 (#1094)
Claude Opus 4.8 (Adaptive Reasoning, Max Effort) on UGI - Writing: 65.88 (#34)
Claude Opus 4.8 (Adaptive Reasoning, Max Effort) on UGI Leaderboard: 52.64 (#69)
GPT-5.4 (xHigh) on Creative Writing (Lechmazur): 3.4 (#2)

New #1 Leaders (2)

Bullshit Benchmark: Claude Opus 4.8 (96.4) beat Claude Sonnet 4.6 by 1.9
Creative Writing (Lechmazur): GPT-5.5 (xHigh) (3.5) beat GPT-5.5 (Thinking, xHigh) by 0.3

AI Benchmark Digest — 2026-05-29

2026-05-29T08:06:41.324282+00:00

Daily

New Benchmarks (1)

DeepSWE (Pass@1 (%)): leader GPT-5.5 (xHigh) (70.0), 12 models
DataCurve benchmark measuring frontier coding agents on original, long-horizon software engineering tasks. Reports pass rates for model configurations on realistic repository work.

New Models (1)

Claude Opus 4.8 — ELO 1801, #52
- Clerk LLM Leaderboard: 91.3 (#1/19)
- Vellum - HumanEval: 88.6 (#1/36)
- Vellum - Humanity's Last Exam: 57.9 (#1/20)
- LLM Stats (DeepSearchQA): 93.1 (#1/6)
- LLM Stats (Include): 87.6 (#1/30)
- LLM Stats (OSWorld-Verified): 83.4 (#1/14)
- LLM Stats (ScreenSpot Pro): 87.9 (#1/22)
- LLM Stats (Toolathlon): 59.9 (#1/20)
- FrontierSWE: 83.0 (#1/11)
- Vals AI (Vals Index): 70.17 (#1/20)

Top-10 New Scores (2)

GPT-5.5 (High) on WebDev Arena: 1478.93 (#16)
GPT-5.5 (xHigh) on WebDev Arena: 1504.74 (#12)

New #1 Leaders (16)

AA GDPval: Claude Opus 4.8 (Adaptive Reasoning, Max Effort) (1889.8) beat GPT-5.5 (xHigh) by 120.5
Vellum - Humanity's Last Exam: Claude Opus 4.8 (57.9) beat Gemini 3 Pro by 12.1
Clerk LLM Leaderboard: Claude Opus 4.8 (91.3) beat GPT-5.4 by 11.8
Vals AI Vibe Code Bench: Claude Opus 4.8 (82.72) beat Claude Opus 4.7 by 11.72
Epoch AI - Apex Agents: gemini-3.5-flash_unknown (49.6) beat GPT-5.5 (xHigh) by 11.2
LLM Stats (OSWorld-Verified): Claude Opus 4.8 (83.4) beat Claude Mythos Preview by 3.8
LLM Stats (Toolathlon): Claude Opus 4.8 (59.9) beat Gemini 3.5 Flash by 3.4
Vals AI Multimodal Index: Claude Opus 4.8 (70.71) beat GPT-5.5 by 2.94
Vals AI (Vals Index): Claude Opus 4.8 (70.17) beat GPT-5.5 by 2.55
LLM Stats (DeepSearchQA): Claude Opus 4.8 (93.1) beat Claude Opus 4.6 by 1.8
LLM Stats (ScreenSpot Pro): Claude Opus 4.8 (87.9) beat GPT-5.2 by 1.6
LLM Stats (Include): Claude Opus 4.8 (87.6) beat Qwen 3.7 Max by 1.4
Artificial Analysis Intelligence Index: Claude Opus 4.8 (Adaptive Reasoning, Max Effort) (61.44) beat GPT-5.5 (xHigh) by 1.2
PinchBench: Claude Opus 4.8 Fast (94.49) beat Qwen Max by 1.05
AA Humanity's Last Exam: Claude Opus 4.8 (Adaptive Reasoning, Max Effort) (45.74) beat Gemini 3.1 Pro (Preview) by 1.02
Vellum - HumanEval: Claude Opus 4.8 (88.6) beat Claude Opus 4.7 by 1.0

AI Benchmark Digest — 2026-05-28

2026-05-28T08:13:42.023730+00:00

Daily

Top-10 New Scores (1)

GPT-5.5 (xHigh) on SWE-rebench: 62.73 (#1)

New #1 Leaders (2)

Kaggle FACTS Grounding: Gemma 4 26B A4B (80.87) beat GPT-5.2 by 4.7
PinchBench: Qwen Max (93.44) beat Grok 0.1 by 1.37

AI Benchmark Digest — 2026-05-27

2026-05-27T08:20:58.056719+00:00

Daily

Top-10 New Scores (1)

GPT-5.4 (xHigh) on Creative Writing (Lechmazur): 3.2 (#2)

New #1 Leaders (11)

LLM Chess (Saplin): GPT-5.5 (Medium) (1532.2) beat Gemini 3.1 Pro by 20.8
LLM Stats (PolyMATH): Qwen 3.7 Max (86.5) beat Qwen 3.6 Plus by 9.1
LLM Stats (MCP-Mark): Qwen 3.7 Max (60.8) beat Kimi K2.6 by 4.9
LLM Stats (NL2Repo): Qwen 3.7 Max (47.2) beat GLM-5.1 by 4.5
LLM Stats (MMLU-ProX): Qwen 3.7 Max (87.0) beat Qwen 3.6 Plus by 2.3
LLM Stats (HMMT Feb 26): Qwen 3.7 Max (97.1) beat DeepSeek V4 Pro (Max) by 1.9
LLM Stats (MAXIFE): Qwen 3.7 Max (89.2) beat Qwen 3.6 Plus by 1.0
LLM Stats (Include): Qwen 3.7 Max (86.2) beat Qwen 3.5 397B A17B by 0.6
LLM Stats (IMO-AnswerBench): Qwen 3.7 Max (90.0) beat DeepSeek V4 Pro (Max) by 0.2
Creative Writing (Lechmazur): GPT-5.5 (Thinking, xHigh) (3.2) beat GPT-5.5 by 0.2
LLM Stats (MMLU-Redux): Qwen 3.7 Max (95.0) beat Qwen 3.5 397B A17B by 0.1

AI Benchmark Digest — 2026-05-25

2026-05-25T08:26:35.093083+00:00

Daily

New Benchmarks (6)

LLMEval-Logic Base (Accuracy (%)): leader Seed 2.0 Pro (Thinking) (75.5), 14 models
LLMEval-Logic Hard (Accuracy (%)): leader Gemini 3.1 Pro (Thinking) (37.5), 14 models
LLMEval-Logic Hard Sub-Q (Accuracy (%)): leader Claude Opus 4.6 (Thinking) (76.6), 14 models
LLMEval-Logic Formalization Free (Accuracy (%)): leader Gemini 3.1 Pro (Thinking) (45.1), 14 models
LLMEval-Logic Formalization Fixed (Accuracy (%)): leader GPT-5.4 Pro (No-Think) (60.2), 14 models
ExploitBench v8-bench (Mean Capability (%)): leader Claude Mythos Preview (69.0), 9 models
V8 exploitation ladder benchmark measuring how far AI agents climb from code reachability through crash reproduction, exploit primitives, and arbitrary code execution. Reports mean capability across 41 V8 bug environments.

AI Benchmark Digest — 2026-05-24

2026-05-24T07:56:34.401567+00:00

Daily

New Benchmarks (14)

NanoGPT-Bench (% of Human Progress Recovered): leader Claude Opus 4.6 (9.3), 2 models
Autonomous research benchmark built on the NanoGPT Speedrun, measuring how much of five months of human pretraining-speedup progress coding agents recover under a fixed H100 compute budget.
CursorBench 3.1 (Score (%)): leader Claude Opus 4.7 (64.8), 7 models
Cursor benchmark of ambiguous, multi-file coding tasks from real Cursor sessions, with models scored by task success percentage and average cost per task.
SMDD-Bench (Pass Rate (%)): leader GPT-5.4 (Medium) (40.2), 7 models
Small molecule drug design agent benchmark with sandboxed Python, Boltz structure prediction, and ADMET tooling. Measures pass rate across 502 computationally verifiable chemistry tasks.
SMDD-Bench Diversity (Avg Successful): leader Claude Sonnet 4.6 (8.4), 7 models
SMDD-Bench diversity slice measuring whether agents generate multiple distinct, novel, successful molecule designs across repeated Lead Optimization rollouts.
Blueprint-Bench 2 (Connectivity Similarity Score): leader GPT 5.5 (0.362), 12 models
Andon Labs spatial reasoning benchmark where agents convert apartment photographs into 2D floor plans, scored by normalized connectivity similarity against ground truth layouts.
PACT (Lechmazur) (CMS Points): leader GPT-5.5 (high) (59.0), 25 models
Pairwise Auction Conversation Testbed for multi-round buyer-seller bargaining. LLMs negotiate over 20 rounds with hidden private values, scored by Composite Model Score from head-to-head surplus capture.
FormationEval (Accuracy (%)): leader gemini-3-pro-preview (99.8), 72 models
Chinese Classical Bench (Average Score (%)): leader claude-opus-4-7 (66.21), 10 models
Chinese Classical Bench - Translate Judge (Score (%)): leader claude-opus-4-7-thinking (80.2), 10 models
Chinese Classical Bench - Punctuate Punct F1 (Score (%)): leader claude-opus-4-7 (80.02), 10 models
Chinese Classical Bench - Char-Gloss Judge (Score (%)): leader claude-opus-4-7-thinking (73.6), 10 models
Chinese Classical Bench - Idiom-Source Book EM (Score (%)): leader deepseek-3.2 (74.0), 10 models
Chinese Classical Bench - Fill-In Exact (Score (%)): leader claude-opus-4-7-thinking (88.0), 10 models
Chinese Classical Bench - Compress Efficiency (Score (%)): leader deepseek-3.2 (16.32), 9 models

Top-10 New Scores (1)

Gemini 3.1 Pro (High) on CLBench: 20.8 (#8)

New #1 Leaders (5)

Evals for Every Language: Gemini 3.1 Pro (69.11) beat Gemini 2.5 Flash by 6.52
CLBench: GPT-5.4 (xHigh) (27.9) beat GPT-5.1 (High) by 4.2
LiveBench Logic With Navigation: Qwen Max (84.0) beat Claude Opus 4.6 (Thinking) by 4.0
Spider 2.0-Lite: DivSkill-SQL (73.13) beat SOMA-SQL by 1.11
PinchBench: Grok 0.1 (92.07) beat Claude Opus 4.7 by 0.49

AI Benchmark Digest — 2026-05-23

2026-05-23T07:20:10.541511+00:00

Daily

New #1 Leaders (1)

OSWorld: Opus 4.7 (83.64) beat Holo3-35B-A3B by 1.08

AI Benchmark Digest — 2026-05-22

2026-05-22T07:36:15.662013+00:00

Daily

Top-10 New Scores (1)

GPT-5.5 (High) on Sycophancy (Lechmazur): 3.5 (#11)

New #1 Leaders (2)

UGI - Writing: gemini-3.5-flash (thinking_level=medium) (72.54) beat gemini-3.1-pro-preview (thinking_level=low) by 0.39
Arabic Broad Leaderboard: gemini-3.5-flash (9.253) beat gemini-3-pro-preview by 0.05

AI Benchmark Digest — 2026-05-21

2026-05-21T07:40:34.045646+00:00

Daily

Top-10 New Scores (1)

Gemini 3.5 Flash (High) on WeirdML: 62.64 (#17)

New #1 Leaders (3)

Kaggle Game Arena Poker (Heads Up): GPT-5.5 (73.93) beat GPT-5.2 by 33.93
AA APEX-Agents: Gemini 3.5 Flash (high) (47.05) beat GPT-5.5 (xhigh) by 9.37
LA Leaderboard: Qwen2.5-14B-Instruct-GPTQ-Int8 (63.6) beat gemma-2-9b-it by 0.27

AI Benchmark Digest — 2026-05-20

2026-05-20T07:43:37.557151+00:00

Daily

New Models (1)

Gemini 3.5 Flash (High) — ELO 1942, #9
- AA MMMU-Pro: 84.28 (#1/190)
- SEAL - MCP Atlas: 83.6 (#1/21)
- AA Omniscience: 22.68 (#3/393)
- AA Omniscience - Law: 57.4 (#4/393)
- AA Omniscience - Software Engineering (SWE) - PHP: 84.0 (#4/393)
- AA Humanity's Last Exam: 40.96 (#5/484)
- AA GPQA Diamond: 92.22 (#6/488)
- AA Omniscience - Science, Engineering & Mathematics: 50.1 (#6/393)
- AA GDPval: 1655.7 (#7/365)
- AA Omniscience - Humanities & Social Sciences: 52.3 (#7/393)

Top-10 New Scores (34)

GPT-5.5 (High) on Multi-turn Debate (Lechmazur): 1583.6 (#5)
Gemini 3.5 Flash (High) on AA CritPt: 13.14 (#8)
Gemini 3.5 Flash (High) on AA GDPval: 1655.7 (#7)
Gemini 3.5 Flash (High) on AA GPQA Diamond: 92.22 (#6)
Gemini 3.5 Flash (High) on AA Humanity's Last Exam: 40.96 (#5)
Gemini 3.5 Flash (High) on AA IFBench: 76.33 (#17)
Gemini 3.5 Flash (High) on AA Long Context Reasoning: 69.33 (#27)
Gemini 3.5 Flash (High) on AA Omniscience: 22.68 (#3)
Gemini 3.5 Flash (High) on AA Omniscience - Business: 45.8 (#8)
Gemini 3.5 Flash (High) on AA Omniscience - Health: 40.2 (#14)
Gemini 3.5 Flash (High) on AA Omniscience - Humanities & Social Sciences: 52.3 (#7)
Gemini 3.5 Flash (High) on AA Omniscience - Law: 57.4 (#4)
Gemini 3.5 Flash (High) on AA Omniscience - Science, Engineering & Mathematics: 50.1 (#6)
Gemini 3.5 Flash (High) on AA Omniscience - Software Engineering (SWE): 65.5 (#16)
Gemini 3.5 Flash (High) on AA Omniscience - Software Engineering (SWE) - C: 80.0 (#18)
Gemini 3.5 Flash (High) on AA Omniscience - Software Engineering (SWE) - Dart: 60.0 (#14)
Gemini 3.5 Flash (High) on AA Omniscience - Software Engineering (SWE) - Go: 50.0 (#32)
Gemini 3.5 Flash (High) on AA Omniscience - Software Engineering (SWE) - HTML: 72.0 (#17)
Gemini 3.5 Flash (High) on AA Omniscience - Software Engineering (SWE) - Java: 51.0 (#16)
Gemini 3.5 Flash (High) on AA Omniscience - Software Engineering (SWE) - JavaScript: 71.82 (#14)
Gemini 3.5 Flash (High) on AA Omniscience - Software Engineering (SWE) - Julia: 60.0 (#13)
Gemini 3.5 Flash (High) on AA Omniscience - Software Engineering (SWE) - Kotlin: 56.0 (#22)
Gemini 3.5 Flash (High) on AA Omniscience - Software Engineering (SWE) - PHP: 84.0 (#4)
Gemini 3.5 Flash (High) on AA Omniscience - Software Engineering (SWE) - Python: 61.0 (#24)
Gemini 3.5 Flash (High) on AA Omniscience - Software Engineering (SWE) - R: 56.0 (#18)
Gemini 3.5 Flash (High) on AA Omniscience - Software Engineering (SWE) - Rust: 80.0 (#8)
Gemini 3.5 Flash (High) on AA Omniscience - Software Engineering (SWE) - Swift: 72.0 (#20)
Gemini 3.5 Flash (High) on AA Omniscience - Software Engineering (SWE) - TypeScript: 67.78 (#16)
Gemini 3.5 Flash (High) on AA SciCode: 53.12 (#11)
Gemini 3.5 Flash (High) on AA TAU-2 Bench: 95.32 (#20)
Gemini 3.5 Flash (High) on AA Terminal-Bench Hard: 40.91 (#36)
Gemini 3.5 Flash (High) on ARC-AGI-1: 92.5 (#16)
Gemini 3.5 Flash (High) on ARC-AGI-2: 72.08 (#12)
Gemini 3.5 Flash (High) on Artificial Analysis Intelligence Index: 55.33 (#8)

New #1 Leaders (5)

LLM Stats (GDPval-AA): Gemini 3.5 Flash (165600.0) beat Claude Sonnet 4.6 by 2300.0
LLM Stats (MCP Atlas): Gemini 3.5 Flash (83.6) beat Claude Opus 4.7 by 6.3
AA MMMU-Pro: Gemini 3.5 Flash (high) (84.28) beat Gemini 3.1 Pro Preview by 1.85
SEAL - MCP Atlas: gemini-3.5-flash (high) (83.6) beat Muse Spark by 1.4
LLM Stats (Toolathlon): Gemini 3.5 Flash (56.5) beat GPT-5.5 by 0.9

AI Benchmark Digest — 2026-05-17

2026-05-17T08:02:54.093472+00:00

Daily

New #1 Leaders (1)

OpenClawProBench: intern-s2-preview (76.7) beat Sensenova 6.7 Flash Lite by 3.0

Weekly

Top-10 New Scores (3)

Claude Opus 4.7 (Thinking) on SEAL Showdown: 1115.7 (#12)
Claude Opus 4.7 (Thinking) on WeirdML: 75.45 (#8)
GPT-5.5 (xHigh) on Chatbot Arena (Code): 1501.0 (#9)

New #1 Leaders (16)

MathArena - ARXIVLEAN March: AlephProver (34.15) beat Aristotle by 17.08
OpenCompass Reasoning - Common: Doubao-Seed-2-0-Pro-260215 (high) (82.1) beat Gemini-3-Pro-Preview by 8.5
OpenCompass Math - College: Doubao-Seed-2-0-Pro-260215 (high) (83.8) beat Kimi-K2.5 by 7.3
OpenClawProBench: intern-s2-preview (76.7) beat qwen3.5-397b-a17b by 6.3
Tau3-Bench Banking_Knowledge: GPT-5.5 (37.4) beat Distyl ButtonAgent by 6.2
OpenCompass Knowledge - Social Science: Gemini-3.1-Pro-Preview (97.5) beat Gemini-3-Pro-Preview by 4.3
OpenCompass LLM - Math: Doubao-Seed-2-0-Pro-260215 (high) (77.3) beat Qwen3-Max-2026-01-23 by 4.1
OpenCompass LLM - Reasoning: Doubao-Seed-2-0-Pro-260215 (high) (65.2) beat Gemini-3-Pro-Preview by 3.7
OpenCompass Math - Competition: Kimi-K2.6 (72.1) beat Qwen3-Max-2026-01-23 by 2.1
OpenCompass Reasoning - Academic: GPT-5.4-2026-03-05 (high) (52.0) beat GPT-5.2-2025-12-11 (high) by 1.5
VisuLogic: PEREA-1.0new (52.8) beat Human by 1.4
WeirdML: gpt-5.5 (xhigh) (84.91) beat gpt-5.5 (high) by 1.01
GAIA: Co-Sight Pro v1.0.1 (93.02) beat OPS-Agentic-Search by 0.66
OpenCompass Knowledge - Engineering: GPT-5.4-2026-03-05 (high) (96.2) beat Gemini-3-Pro-Preview by 0.4
AA TAU-2 Bench: JT-35B-Flash (99.12) beat GLM-4.7-Flash (Reasoning) by 0.32
AISI Cyber TLO 10M: GPT-5.5 (10.0) beat Claude Opus 4.6 by 0.2

AI Benchmark Digest — 2026-05-16

2026-05-16T07:15:27.727063+00:00

Daily

Top-10 New Scores (1)

GPT-5.5 (xHigh) on Chatbot Arena (Code): 1501.0 (#9)

New #1 Leaders (2)

MathArena - ARXIVLEAN March: AlephProver (34.15) beat Aristotle by 17.08
GAIA: Co-Sight Pro v1.0.1 (93.02) beat OPS-Agentic-Search by 0.66

AI Benchmark Digest — 2026-05-14

2026-05-14T07:26:43.169192+00:00

Daily

New Models (4)

Doubao-Seed-2-0-Pro-260215 (High) — ELO 1781, #73
- OpenCompass LLM - Reasoning: 65.2 (#1/23)
- OpenCompass LLM - Math: 77.3 (#1/23)
- OpenCompass Knowledge - Humanities: 95.0 (#1/23)
- OpenCompass Reasoning - Common: 82.1 (#1/23)
- OpenCompass Math - College: 83.8 (#1/23)
- OpenCompass LLM - Language: 77.3 (#3/23)
- OpenCompass Language - Creation: 77.1 (#3/23)
- OpenCompass Knowledge - Science: 94.6 (#3/23)
- OpenCompass LLM - Agent: 44.2 (#4/23)
- OpenCompass Language - NLP: 69.6 (#4/23)
Doubao-Seed-2-0-Lite-260215 (High) — ELO 1741, #103
- OpenCompass Reasoning - Common: 78.1 (#2/23)
- OpenCompass Language - Creation: 77.1 (#4/23)
- OpenCompass LLM - Language: 74.4 (#6/23)
- OpenCompass LLM - Agent: 42.4 (#6/23)
- OpenCompass Agent - Tool Use: 42.4 (#6/23)
- OpenCompass Knowledge - Science: 91.7 (#7/23)
- OpenCompass LLM - Reasoning: 59.5 (#8/23)
- OpenCompass Language - NLP: 67.1 (#8/23)
- OpenCompass Language - Instruction Following: 72.5 (#8/23)
- OpenCompass Math - College: 77.1 (#8/23)
Hy3-preview (High) — ELO 1729, #110
- OpenCompass Math - College: 81.3 (#3/23)
- OpenCompass Language - Instruction Following: 76.0 (#4/23)
- OpenCompass LLM - Math: 74.5 (#5/23)
- OpenCompass Language - Creation: 75.4 (#5/23)
- OpenCompass LLM - Language: 74.4 (#7/23)
- OpenCompass Reasoning - Academic: 43.6 (#8/23)
- OpenCompass LLM - Reasoning: 58.5 (#10/23)
- OpenCompass Math - Competition: 67.6 (#10/23)
- OpenCompass LLM - Agent: 28.7 (#12/23)
- OpenCompass Reasoning - Common: 73.5 (#12/23)
Ring-2.5-1T — ELO 1711, #119
- OpenCompass Knowledge - Social Science: 92.9 (#5/23)
- OpenCompass Language - NLP: 65.4 (#11/23)
- OpenCompass Language - Creation: 68.8 (#12/23)
- OpenCompass Knowledge - Humanities: 90.0 (#12/23)
- OpenCompass LLM - Agent: 25.0 (#13/23)
- OpenCompass Math - College: 75.0 (#13/23)
- OpenCompass Agent - Tool Use: 25.0 (#13/23)
- OpenCompass LLM - Knowledge: 89.4 (#14/23)
- OpenCompass Knowledge - Engineering: 90.8 (#14/23)
- OpenCompass LLM - Language: 69.8 (#15/23)

Top-10 New Scores (1)

Claude Opus 4.7 (Thinking) on WeirdML: 75.45 (#8)

New #1 Leaders (9)

OpenCompass Reasoning - Common: Doubao-Seed-2-0-Pro-260215 (high) (82.1) beat Gemini-3-Pro-Preview by 8.5
OpenCompass Math - College: Doubao-Seed-2-0-Pro-260215 (high) (83.8) beat Kimi-K2.5 by 7.3
Tau3-Bench Banking_Knowledge: GPT-5.5 (37.4) beat Distyl ButtonAgent by 6.2
OpenCompass Knowledge - Social Science: Gemini-3.1-Pro-Preview (97.5) beat Gemini-3-Pro-Preview by 4.3
OpenCompass LLM - Math: Doubao-Seed-2-0-Pro-260215 (high) (77.3) beat Qwen3-Max-2026-01-23 by 4.1
OpenCompass LLM - Reasoning: Doubao-Seed-2-0-Pro-260215 (high) (65.2) beat Gemini-3-Pro-Preview by 3.7
OpenCompass Math - Competition: Kimi-K2.6 (72.1) beat Qwen3-Max-2026-01-23 by 2.1
OpenCompass Reasoning - Academic: GPT-5.4-2026-03-05 (high) (52.0) beat GPT-5.2-2025-12-11 (high) by 1.5
OpenCompass Knowledge - Engineering: GPT-5.4-2026-03-05 (high) (96.2) beat Gemini-3-Pro-Preview by 0.4