LMC Benchmark Saturation Tracker
Which LLM benchmarks still separate frontier models and which have quietly become noise. We compute each benchmark's discrimination score from the top-5 score spread, the saturation flag, and the ceiling behavior across 146 models. 16 standardized benchmarks analyzed.
Scores last refreshed 2026-07-14 (auto-updated every 6 hours from HuggingFace Open LLM Leaderboard, LMArena, and vendor benchmark reports).
Most discriminating benchmarks (wider top-5 spread = better)
Top-5 spread is the gap between the best and fifth-best model on each benchmark. A wide gap means the benchmark can still tell frontier models apart.
Most saturated benchmarks (narrower top-5 spread = dead)
These benchmarks cluster the top 5 models within a single-digit range. They are no longer useful for ranking frontier LLMs.
Full benchmark ranking
Sorted by discrimination score, high to low. Saturated benchmarks are marked.
| Benchmark | N | Max | Top-5 spread | Status |
|---|---|---|---|---|
HLE Humanity's Last Exam | 32 | 59.0% | 20.0 | Active |
BigCodeBench BigCodeBench (Hard) | 45 | 72 | 22.1 | Active |
LiveBench LiveBench (Dynamic) | 35 | 87 | 8.3 | Active |
AIME 2024 American Invitational Mathematics Examination 2024 | 22 | 96.7% | 7.5 | Active |
SWE-bench Verified Software Engineering Benchmark (Verified) | 53 | 95 | 7.4 | Active |
MMLU-Pro MMLU Professional | 40 | 88.0% | 2.0 | Weak |
Arena Elo LMSYS Chatbot Arena Elo Rating | 126 | 1505 | 14.0 | Weak |
BBH BIG-Bench Hard | 42 | 93.1% | 1.6 | Weak |
IFEval Instruction Following Evaluation | 39 | 94 | 1.0 | Weak |
MATH-500 MATH Benchmark (500-problem subset) | 49 | 99.0% | 1.7 | Weak |
GPQA Diamond Graduate-Level Google-Proof Q&A (Diamond) | 57 | 94.3% | 0.7 | Weak |
HellaSwag HellaSwag Commonsense NLI | 7 | 96.0% | 3.0 | Saturated |
MMLU Massive Multitask Language Understanding | 54 | 94.0% | 1.4 | Saturated |
ARC-Challenge AI2 Reasoning Challenge (Challenge Set) | 8 | 96.9% | 1.8 | Saturated |
HumanEval HumanEval Code Generation | 50 | 98 | 1.0 | Saturated |
GSM8K Grade School Math 8K | 15 | 96.8% | 1.0 | Saturated |
What each benchmark actually measures
Reference notes for the top-ranked benchmarks so you know what you are looking at when a vendor quotes a score.
HLE
reasoning2,500 expert-level questions spanning mathematics, sciences, and humanities. Designed to be 'the final closed-ended academic evaluation' that even top models fail most of.
Why it still matters: The hardest academic benchmark — top models still fail 60-65% of questions. Shows how far we are from genuine expert-level reasoning.
BigCodeBench
codingPractical code generation requiring use of libraries, APIs, and complex program structures. The 'Hard' subset tests non-trivial engineering tasks.
Why it still matters: More realistic than HumanEval — tests practical programming skills including library usage, API calls, and multi-file reasoning.
LiveBench
arenaComprehensive benchmark across 6 categories (math, coding, reasoning, data analysis, instruction following, language) using contamination-resistant, regularly updated questions.
Why it still matters: Contamination-free by design — uses new questions regularly. Top models still score below 70%, making it highly discriminating.
AIME 2024
mathOlympiad-level mathematical problem solving from the real 2024 AIME competition. 30 problems testing advanced algebra, geometry, combinatorics, and number theory.
Why it still matters: Tests mathematical reasoning at competition level. Reasoning models achieve 70-90% while standard models struggle below 30%. Best differentiator for math ability.
SWE-bench Verified
codingCan a model resolve real GitHub issues from popular Python repositories? Human-validated subset ensures accurate evaluation. Tests end-to-end software engineering ability.
Why it still matters: The gold standard for real-world coding ability. Unlike HumanEval, tests understanding of large codebases, debugging, and complex changes. Scores range 20-80%.
MMLU-Pro
knowledgeHarder version of MMLU with reasoning-focused questions and 10 answer choices instead of 4. Contains 12,000+ questions across 14 domains.
Why it still matters: Better at differentiating top models since scores are 16-33% lower than standard MMLU. Tests reasoning in addition to knowledge.
How the discrimination score is computed
The discrimination score is a 0-to-1 rating of how well a benchmark still separates frontier models in 2026. Three inputs feed into it:
- Top-5 score spread. The gap between the best-scoring model and the fifth-best on a benchmark. If the top 5 are within a single percentage point of each other, the benchmark cannot tell them apart. Wider gaps mean more discrimination.
- Saturation flag. Benchmarks we have flagged as saturated in the catalog receive a 0.4x multiplier. Flagged benchmarks include MMLU, HumanEval, GSM8K, ARC-Challenge, and HellaSwag - all of which have multiple frontier models scoring at or above the human baseline.
- Ceiling hit. If the best model is within 5% of the benchmark's theoretical max, we apply a 0.5x penalty because further progress is compressed into a very narrow score range.
Benchmarks with fewer than 5 scored models are excluded from ranking to avoid spurious top-5 spreads on small samples.
MMLU is flagged as saturated in the current LMC data. Across the 54 models we have scored, the maximum is 94.0%, the median is 88.7%, and the top-5 are clustered within 1.4 percentage points of each other. That spread is narrower than MMLU's own annotation noise floor, so it cannot reliably rank frontier LLMs. Use MMLU-Pro, GPQA Diamond, or LiveBench for ranking instead.
By top-5 spread, the most discriminating benchmarks in the current LMC data are HLE (top-5 spread 20.0), BigCodeBench (top-5 spread 22.1), LiveBench (top-5 spread 8.3). A wide spread means a frontier model that improves by a few points still meaningfully moves the ranking, which is the property you want for buyer decisions.
HumanEval in the current LMC data has a max of 97.5%, a median of 90.5%, and a top-5 spread of 1.0 points across 50 scored models. That kind of ceiling behavior means the remaining errors are largely ambiguous test cases rather than real coding failures. For coding evaluation today use SWE-bench Verified (top-5 spread 7.4), LiveCodeBench, or BigCodeBench instead.
We refresh benchmark scores every 6 hours from the HuggingFace Open LLM Leaderboard, LMArena text, vision, and image leaderboards, and vendor-published benchmark reports. The merged dataset is overlaid on the curated LMC benchmark definitions at render time, so every page load reflects the latest available scores. Benchmarks with fewer than 5 scored models are excluded from ranking to avoid spurious top-5 spreads. The current snapshot was fetched at 2026-07-14T00:30:03.726Z.