How do large language models generate text?

A large language model, or LLM, is a machine-learning model trained to work with language. When you send a prompt, the model does not usually retrieve a finished paragraph from a shelf. It uses the text in its context to estimate a likely continuation and generates a response piece by piece.

Text is broken into tokens

Before processing, text is divided into tokens. A token might be a whole short word, part of a longer word, punctuation, or another text unit. The model represents these tokens numerically so it can process patterns across large amounts of text. Token boundaries do not always match the words a person sees.

Next-token prediction

During training, the model sees text and learns to predict tokens that follow earlier tokens. It adjusts many internal numerical values so its predictions become more useful across its training examples. At use time, it takes the prompt and previous response tokens as context, calculates possible next tokens, and selects one according to its generation settings.

The selected token is added to the context. Then the model predicts another. Repeating this process produces a sentence and eventually a longer answer. Because each new token depends on the context so far, a small change in the prompt can lead to a different continuation.

Why the result can feel knowledgeable

Language contains patterns: words appear in related contexts, explanations follow recognizable structures, and ideas often occur together. By learning these relationships, a model can produce fluent text, summarize, transform writing, and respond to many kinds of prompts. Fluency, however, is not proof that every statement is correct.

Context and limits

The model's available context is limited, and its answer is shaped by that context, its training, and the generation process. Some systems add retrieval or tools that fetch information or perform actions; those are additional system capabilities, not the same thing as next-token prediction itself.

This distinction is useful: the language model generates text, while the surrounding application may provide instructions, search results, safety checks, or other tools. Understanding the pieces makes it easier to judge what an AI system can—and cannot—reliably do.

What to remember