Lesson 14 of 38 · Written learning
Logits, probabilities, and perplexity
Interpret model outputs without confusing a score with a truth judgment.
A language model commonly returns logits for each possible next token. Softmax turns them into a probability distribution. A decoder then chooses or samples a token. The same model can produce different text when the prompt, context, or sampling settings change.
What perplexity can and cannot tell you
Perplexity summarizes how surprised a language model is by a sequence, based on its predicted probabilities. Lower perplexity on comparable data can indicate better next-token prediction. It is not a complete measure of helpfulness, factuality, safety, or writing quality, and values are not directly comparable across different tokenizers or datasets.
- Classification chooses among defined labels; generation produces a sequence of tokens.
- A high-confidence prediction can still be wrong about the world.
- Use task-specific evaluation when deciding whether a model is useful for an application.
- Always record the dataset and tokenizer when reporting perplexity.
Return to the NLP and language-model fundamentals stage · Suggested timing is flexible; move at your own pace.