AI model speed: tokens per second rose 18-fold, then levelled off

GPT-4 began around reading pace

An adult silently reading non-fiction takes in about 238 words per minute, or 5.3 tokens per second using about 0.75 English words per token. GPT-4’s May 2023 output was close to that pace. Reading is an input rate; it gives the model’s output a human yardstick rather than an equal-task comparison.

The 5.1 figure came from one independent tester’s regression, about two months after GPT-4 launched. GPT-4 itself reached 10.6 tokens per second by August, and Artificial Analysis measured about 22–24 from December. Dating the reading matters: the first figure describes a period in GPT-4’s life.

The research found no usable, citable speed measurements for the 2020–2022 OpenAI models. The numerical history therefore begins in 2023, with no interpolated baseline for GPT-3 or the original ChatGPT.

Standard flagship writing levelled off after 2024

On October 8, 2026, Artificial Analysis measured the November 2024 GPT-4o at 112.3 tokens per second, Claude Opus 5.5 at maximum effort at 96.4, and GPT-6.1 Sol at maximum effort at 53.9. All three readings use the same 10,000-token input workload.

These two newer standard flagships wrote within the broad speed range established in 2024. Faster small models continued to appear. Gemini 3.8 Flash reached 348.3 tokens per second near launch, and Claude Haiku 5.5 reached 233.6. The split is between tiers as well as generations.

The records served models below the frontier

Groq’s Mixtral endpoint measured 473.1 tokens per second in March 2024. Mixtral was a GPT-3.5-class model when GPT-4 Turbo represented the frontier. In August 2024, Artificial Analysis measured Cerebras serving Llama 3.1 8B above 1,800 tokens per second.

In August 2025, Cerebras reported an Artificial Analysis measurement of about 2,700 tokens per second for gpt-oss-120b. That is an independently measured result relayed by the vendor; Cerebras compared the model with o4-mini. Its Llama 4 Maverick release similarly quoted Artificial Analysis at 2,522.

Nvidia’s August 2026 Groq 3 LPX demonstration reached 3,431 tokens per second on Gemma 4 31B. Artificial Analysis measured it in Nvidia’s own data centre, with no public price or release date. It belongs in the table as a private demonstration.

Celeris-1 reached 1,549 tokens per second on Artificial Analysis’s public-API leaderboard, with an Intelligence Index of 6 against its class median of 9. On the October 8 snapshot’s 10,000-token workload, the figure charted here, it measured 1,418. Its diffusion speed is an average over parallel refinement passes. The change from 2025’s short prompts to 2026’s longer prompts prevents treating the lower headline rate as a decline in public records.

Reasoning moved the wait before the answer

At GPT-4’s May 2023 rate, streaming 500 tokens takes about 98 seconds, calculated without an unmeasured first-token wait. For GPT-4o in June 2024, the 500-token scenario is about 6.5 seconds, calculated from its measured first-token wait and output speed.

By October 8, GPT-6.1 Sol at maximum effort waited 326.9 seconds before its answer started; Artificial Analysis’s whole-answer metric was 336.2 seconds for 500 tokens. Opus 5.5 at maximum effort waited 723.7 seconds before starting, with a whole-answer metric of 728.8 seconds. Almost all of that time comes before the answer text.

The first visible token can also be misleading. DeepSeek R1 began showing reasoning after 3.1 seconds in an April 2025 test, while its first answer token came after 107.2 seconds. Artificial Analysis introduced time to first answer token in 2025 to capture that distinction.

The same model can start in seconds or minutes

On the same October 8 test, Opus 5.5’s first answer arrived after 8.5 seconds at low effort, 21.7 at medium, 38.0 at high, 131.2 at xhigh and 723.7 at maximum. Medium was its default. The full effort comparison below keeps the date and workload fixed.

The maximum setting is opt-in. These are API measurements with 10,000-token prompts, and the research found no distribution of waiting times in Claude.ai. All the long Opus readings were taken within its first three weeks, so launch load remains a possible contributor.

Faster serving can reduce thinking time too. In the October 14 reading, GPT-6.1 Sol reached 301 tokens per second in Ultrafast against 65 on Standard, about 4.6 times faster at six times the standard price. Its maximum-effort wait fell from 330.6 seconds to 74.6. The research found no independent measurement of Anthropic’s advertised fast-mode multiplier.

A speed reading needs a host and a date

DeepSeek R1’s own API fell from 62.4 tokens per second on January 27, 2025 to 26.9 on February 15, during the demand surge. The same model’s speed depends on capacity and host. Artificial Analysis’s provider comparison also showed R1 near 26 on DeepSeek’s API and 312 on Together.

Serving changes can move speed without changing weights: OpenAI reported a roughly 40% improvement for GPT-5.2 after a stack update. A measured rate needs its host, setting, workload and observation date. A launch reading is a dated observation rather than a permanent model specification.

How we compiled this

We combined dated Artificial Analysis readings with Taivo Pungas’s 2023 regressions, retaining the host, workload, effort and availability qualifications. Historical model-level readings that pool hosts are labelled. When a vendor release quotes Artificial Analysis, the table identifies that route to the measurement.

Artificial Analysis’s current performance method re-counts text with OpenAI’s o200k_base tokenizer, so its Claude figures need no extra tokenizer correction. The sources do not establish whether the same normalization applied before March 2026.

On March 2, 2026, Artificial Analysis changed its default from about 1,000 to 10,000 input tokens and upgraded its prompts. Earlier aggregation also changed from averages to medians, then from 14-day to 72-hour windows. Comparisons use a shared snapshot and workload where possible; historical figures keep those qualifications. Aggregate chip throughput, input-processing rates and multi-call agent runs are excluded from the output-speed comparison.