What is inference in AI?
The act of running a trained model to get an answer — what happens every time you send a prompt, and what you pay for per use.
Inference is what a trained AI model does the moment you give it something to respond to: it takes your input and produces an output, using patterns it already learned rather than learning anything new in that moment. Every prompt you send to a chatbot, every photo an app labels, every sentence a translation tool converts — each is one act of inference.
In practice
When you send a prompt, the model does not compose the whole reply at once. It generates the response one token at a time — a token being roughly a word or word-fragment — with each new token chosen based on everything that came before it. That generation process is the phase you’re waiting for when a reply appears gradually rather than all at once. On commercial AI services that charge by usage, inference is also the phase that gets billed, since it’s the work done specifically to answer your request. Stanford HAI’s annual AI Index treats it as distinct from the cost of building a model in the first place.
Researchers are careful about how far to describe this process in human terms. What a model does when producing output resembles reasoning in its results, but NIST and allied research bodies frame that resemblance as a description of behaviour, not a claim about genuine understanding or awareness inside the system.
Not to be confused with
Inference is often paired with, and confused with, training — the earlier, separate phase where a model adjusted itself by processing large amounts of data. Training happens rarely and in advance; inference happens every time the finished model is used. See training vs inference for how the two relate.