AI context windows grew 4,096-fold, then stalled

Most of the growth came in 15 months

GPT-1 trained on 512-token sequences; GPT-2 doubled that to 1,024. GPT-3 used 2,048 in its May 2020 paper, with private-beta API access starting on June 11. The same 2,048-token length persisted across Gopher, PaLM, OPT and BLOOM into mid-2022, although their access conditions differed.

From GPT-4’s 32,768 in March 2023 to Gemini’s 2,097,152 in June 2024, the published milestone grew 64-fold in about 15 months—roughly one doubling every 2.6 months. The starting point was limited access: GPT-4’s July general release covered 8K, and OpenAI’s help center later said it was not granting 32K access.

Open weights also moved ahead of closed APIs: Yi-34B-200K shipped on November 5, 2023, one day before GPT-4 Turbo’s 128,000-token preview. Turbo’s brief lead applied only among closed APIs; Claude 2.1 reached 200,000 later that month.

A released model’s headline could exceed its hosted window

The report distinguishes public service or trained open weights, limited access, on-paper sizes, and research without public access. An open public preview counts as served, while Gemini 1.5 Pro’s initial private preview and Claude Sonnet 4’s high-tier 1M beta remain limited.

Meta’s Llama 4 Scout carried a 10M headline and a 10,485,760-token configuration, but was trained at 256K. Launch-day hosts offered 128,000–328,000, according to Simon Willison’s account. The later OpenRouter listing reached 1,310,720—still far below Meta’s headline.

Alibaba’s Qwen-Long listed 10M, but ordinary HTTP requests stopped at 1M and longer inputs required file submission; the documentation did not establish that all uploaded text entered one simultaneous window. MiniMax-Text-01 likewise advertised 4M after training at 1M, while its hosted OpenRouter listing was 1,000,192. These figures retain the report’s on-paper label.

New flagships settled around 1M

Google’s Gemini 2.5 Pro arrived at 1,048,576 input tokens in March 2025; its promised 2M extension never shipped. Gemini 1.5 Pro still offered 2,097,152 until its September 29 shutdown, and Grok 4 Fast had launched with 2,000,000 ten days earlier. A smaller successor therefore did not immediately remove the older, larger window.

Grok 4.20’s cut from 2,000,000 to 1,000,000 falls between May 6 and May 20, 2026 document captures; the multi-agent variant followed by July 17. New Grok 4.5, 4.6 and 4.7 models had 500,000. The sources give an interval for the first reduction, rather than a verified effective day.

OpenAI’s selected flagship went from GPT-4.1’s 1,047,576 to GPT-5’s 400,000, then back to 1,050,000 with GPT-5.4. Claude’s 1M reached general availability at standard pricing in March 2026, and Haiku 5.5 extended 1M across the current lineup in October. Long prompts could still cost more: OpenAI charged a premium above 272K input, and Haiku 5.5 above 100K.

A million tokens does not hold the same text

OpenAI and Anthropic count input and output together; Google publishes an input limit with output counted separately. GPT-6.1 Sol’s 1,050,000 total includes a 128,000-token output reservation, leaving 922,000 input tokens. Gemini 3.8 Flash lists 1,048,576 input and a separate 65,536 output.

Anthropic’s newer tokenizer produces about 30% more tokens for the same text than its earlier one. Its own estimate puts 1M at roughly 555,000 English words, compared with about 750,000 previously. OpenAI’s rule of thumb is about 75 words per 100 tokens; Google gives 60–80. These are provider estimates, not measurements of the same text on all three systems.

The research measured 23 public-domain English books with tiktoken: 1.25–1.49 tokens per word, with a median of 1.37 on o200k_base. Named editions are more specific than page counts, which the labs’ examples imply can range from about 400 to 820 tokens per page. The measured King James Bible and complete Shakespeare exceeded 1M o200k tokens, although both fit within 2M.

The window measures capacity, not reliable recall

NVIDIA’s RULER defines effective length as the longest tested input that still beats Llama-2-7B’s 4K baseline of 85.6%. GPT-4 Turbo’s advertised 128K became 64K by that rule, while LWM 7B’s 1M became under 4K. Gemini 1.5 Pro’s result was reported as over 128K, with testing stopping at 128K; it does not establish an effective 1M window.

NoLiMa made retrieval harder by hiding facts without keywords shared with the question. At 32K, 11 of its 13 models fell below half their strong short-input baselines; GPT-4o fell from 99.3% to 69.7%. This was a task-specific result, rather than a universal usable-length limit.

MRCR v2 asks for the right one of several near-identical requests in a long conversation. Anthropic reported 76% for Opus 4.6 and 18.5% for Sonnet 4.5 on its 8-needle, 1M variant; Google reported 26.3% pointwise for Gemini 3.1 Pro at 1M. Different scoring prevents a direct ranking across those reports. The research found no independent 1M test of a 2026 frontier model, and did not collect MRCR scores for later Opus releases or GPT-6.

How we compiled this

We reconciled provider papers, announcements, documentation, model cards, configuration files and archived pages into selected historical milestones. Dates retain their basis: GPT-3’s paper preceded its private-beta API, MPT’s release is month-only, and xAI’s reduction is bounded by archive captures. An observation date for a hosted listing is not a launch date.

Windows retain the provider’s own unit and published shorthand, with different configuration integers kept in notes. Early Kimi claims in Chinese characters are excluded from token comparisons. Word equivalents are English-specific estimates or the research’s identified tokenizer measurements; RULER lengths retain their benchmark’s threshold and test ceiling. The tables separate advertised capacity, access and measured performance.