Tokens and context windows, explained
Why an assistant 'forgets', and how much it can really hold at once.
Jamie Owen Updated 19 Aug 2026
Two words explain a lot of an assistant’s behaviour: tokens and the context window. Once they click, things like “why did it forget what I said earlier?” and “why did it cut off?” stop being mysterious.
What a token is
A model doesn’t read whole words; it reads tokens, meaning short chunks of text. A token is often a word, but long or unusual words split into several, and spaces and punctuation count too. A rough rule for English: about three to four characters per token, or loosely three-quarters of a word. “Unbelievable” might be three tokens; “cat” is one.
You don’t need to count them. You need to know that everything, from your prompt to the files you paste to the model’s reply, is measured in them.
What the context window is
The context window is the maximum number of tokens the model can consider at once: your whole conversation so far, plus anything you’ve pasted, plus the answer it’s writing. It’s a working memory, not a hard drive.
The model can only “see” what fits in the context window. Anything older has effectively fallen out of view.
Why it “forgets”
When a long conversation exceeds the window, the earliest parts drop out of the model’s view. It isn’t ignoring you; that text is no longer in front of it. This is why a model can lose a detail you mentioned an hour ago, and why pasting a very long document can push out your actual instructions.
Practical consequences
On long threads, put the key instructions near your question rather than buried far above. Summarise instead of pasting everything, giving the model the relevant section rather than the entire 80-page report, unless its window genuinely holds it. Start a fresh chat for a new task instead of dragging a bloated context along. And remember that bigger windows cost more without making the model read more carefully. Relevance still beats volume, which is also good prompting.
A window measured in tokens does not fix how much text it holds, either. When a provider changes its tokeniser, the same words come out as a different number of tokens, so a window can shrink in practice while the headline figure stays put. Claude’s Sonnet and Opus tiers show what that costs in words.
Local models often have smaller windows than hosted ones, which is worth remembering if you run one yourself.