AI benchmarks reach 96–97%, then need replacing

An overall leader can lose a field

Claude Opus 5.5 (max) leads Artificial Analysis’s Intelligence Index v4.3.2 at 57.62. That index weights Agents at 30%, Coding at 20%, General at 30% and Scientific Reasoning at 20%. Half the total therefore comes from agents and coding, the categories Claude models lead in our reconstruction.

The field leader changes with the task: Anthropic leads the official Terminal-Bench 4.0 board narrowly, Google and OpenAI tie on Epoch’s hard-fact recall test, and science has different leaders on different boards. AA publishes no per-model category scores, and reconstructing its current categories leaves historical gaps because important older models were never run on the current components.

Science reached 96.4%, with five models tied

Epoch’s GPQA Diamond series puts GPT-4 at 35.7%, o1 at 76.8% in December 2024, Gemini 3 Pro at 92.6% in November 2025 and Gemini 4 Argon at 96.4% in September 2026. Diamond contains 198 multiple-choice questions in biology, physics and chemistry. The same prompt and strict answer parsing make these runs more comparable than a collection of launch claims.

The last stretch is less decisive than the first: every later 95% interval overlaps Gemini 3 Pro’s 89.4–95.9 interval. The top five models sit within about one point, against standard errors of 1.2–1.4 points. Argon has the highest displayed mean, but these results do not clearly separate the leading models. Opus 5.5, the overall AA leader, scores 90.6% here, below Opus 5’s 93.9%.

Coding reached 97% on a benchmark evaluators dropped

The official SWE-bench Verified board lists SWE-agent with GPT-4 Turbo at 22.4%, under an April 2024 entry date; Vals AI’s highest result was Claude Opus 5 at 97.0% in July 2026. These are model-and-agent results, and lab scaffolds typically added 2–10 points over independent minimal-harness runs before saturation.

OpenAI stopped evaluating Verified on February 23, 2026 after finding material problems in 59.4% of 138 audited hard tasks and evidence of memorised fixes. That percentage describes the audited subset, not all 500 tasks. Vals stopped running the benchmark on September 1.

Terminal-Bench 4.0 supplies a new endpoint with 66 tasks and an eight-hour limit per task. Opus 4.8 makes the reset concrete: 88.6% on Verified, 23.64% on Terminal-Bench 4.0. On the official new board, Opus 5.5 scores 64.85% ±3.11 and Sonnet 5.5 scores 61.82% ±2.94, with overlapping intervals.

The evaluator still matters: Vals’ 65.15% for Opus 5.5 includes 22 of 198 attempts served by older Claude models; counting those as failures gives 58.08%, behind Sonnet 5.5 and Astra. AA’s run ranks Sonnet 5.5 first at 63.64%. The defensible conclusion is a narrow Anthropic lead, conditional on the run.

Hard-fact recall nearly tripled after MMLU saturated

GPT-4’s March 2023 MMLU score was 86.4%, using five examples in the prompt. Google reported Gemini Ultra at 90.0% that December with a different prompting method. HELM’s fixed five-shot runs instead topped out at 87.3% before its final MMLU release in January 2025.

The handover changes what knowledge means: GPT-4o’s August 2024 snapshot scored 84.3% on HELM’s multiple-choice MMLU and 26.0% on SimpleQA Verified’s exact factual questions, without answer options. Epoch’s SimpleQA Verified accuracy rose to Gemini 4 Argon’s 76.0%, effectively tied with GPT-6 Astra’s 75.6%. Opus 5.5 places fifth at 72.2%.

These are retrospective comparisons: Epoch ran all 90 rows between August 10 and October 13, 2026, using one prompt and grader. Since August 27, that prompt tells models to avoid abstaining, and earlier scores were retired. The metric is accuracy, not Google’s F1 score, and original GPT-4 has no result.

Argon became generally available on October 13, following its September 30 announcement.

Human baselines apply to particular tests

GPQA has measured expert comparisons, with limits: the paper’s 65% expert result comes from a broader question set, while OpenAI’s recruited PhDs scored 69.7% on Diamond. Diamond’s 81.2% validator result is affected by how questions were selected and should not become a human reference line. Epoch’s o1 result passed the reported expert baselines in December 2024; that does not establish superiority to PhDs in every respect.

MMLU’s 89.8% expert level is an estimate extrapolated from exam results, not a measured panel. SimpleQA Verified has no closed-book human baseline, and agentic coding benchmarks publish no measured human solve rate. A useful coding milestone is Google’s advanced Gemini 2.5 Deep Think solving 10 of 12 ICPC 2025 problems under the five-hour limit, with its solutions confirmed by ICPC. It is a contest result, not a longitudinal human benchmark.

A harder exam changes the leader again

Answer-key quality becomes consequential near saturation: MMLU-Redux estimated errors in 6.49% of MMLU questions, while Epoch extrapolated an approximately 8% invalid-question rate for GPQA from a small review. That GPQA estimate is uncertain; it is not a demonstrated ceiling that every model must obey.

Harder exams also change the ranking: AA’s text-only Humanity’s Last Exam run puts Opus 5.5 first at 61.4%, while the official full-set board puts Astra at 54.80% but lacks Opus 5.5 and Argon. Those are different comparison sets, not contradictory scores on one setup.

The authors’ cleaned, 1,000-question HLE-Diamond gives Astra 59.9% and Opus 5.5 54.6% without tools; with web and code, they reach 82.9% and 73.9%. Its launch report supplies neither a human baseline nor an error rate. A useful endpoint therefore names the test, evaluator, tools and uncertainty alongside the model.

How we compiled this

We separate developer-published figures from third-party runs by benchmark maintainers, Epoch, Vals, HELM and AA. Third-party evaluation describes who ran the test; Terminal-Bench’s leaderboard receives funding from AI labs. Release labels identify model vintages, not evaluation dates: GPT-4’s Epoch GPQA run was in October 2025, and early models were retested on the later harness. The early SWE-agent row retains its board-entry date. The chart shows selected release months; the tables preserve the sources’ date precision and distinguish date types.

We retain the researched score precision and confidence intervals, keeping different benchmarks and evaluator setups separate. Claude 3.7 Sonnet’s 78.5% mean is retained without the interval based on its different 79.7% best-scorer value. Missing scores stay missing, and figures we could not verify at their primary source are excluded.