What is a token in AI?
The chunk of text a language model actually reads and writes — usually a word-piece, not a whole word — and the unit models are priced in.
A token is the unit of text a language model actually processes. It is usually a word-piece: a common short word often counts as one token, while a longer or rarer word gets split into several. A model never sees whole words the way a person reads them, and it never sees individual letters either — it sees this in-between unit, chosen by a separate process that breaks text into chunks it can compute over efficiently.
Why it matters
Because tokens are the model’s native unit, everything about how it works is measured in them: a model reads your question in tokens, writes its answer in tokens, and the amount of text it can hold in mind at once — its context window — is counted in tokens rather than words or characters. Services built on these models also typically meter usage by token count, though pricing details vary by provider and are outside the scope of this explanation.
The surprising consequence
Because a model works from tokens rather than letters, a task that seems trivial to a human — counting how many times a particular letter appears in a word — can trip it up. The model is not looking at the letters directly; it is reasoning over the tokens the word broke into, which may bundle several letters together or split the word in an unintuitive place. This is the underlying cause of a well-known failure case where large language models miscount letters in everyday words, a pattern researchers at institutions such as Stanford HAI have pointed to as a clear illustration of the gap between how these systems represent language and how humans do. It applies to any chat assistant working this way, from ChatGPT to Claude, Gemini or Copilot, not to one product in particular.