Lesson 22 of 38 · Written learning
Choose and run a pretrained model
Compare hosted services and open-weight models while accounting for licenses and operating limits.
A hosted model is operated by a provider and reached through an API. An open-weight model can be downloaded under a particular license and run on compatible hardware. “Open weights” does not automatically mean the training data, code, or all usage rights are open. A model card can describe intended use, training, evaluation, and limitations; read it critically.
Prompts have a format
Many chat models expect structured messages with roles such as system, user, and assistant. A chat template converts those messages into the token sequence the model was trained to receive. Incorrect templates can produce confusing behavior. Inference parameters and context limits vary by model, so check the current documentation and test the intended workload.
- Consider privacy, data-retention terms, licensing, capability, and deployment needs.
- Never assume one provider's prompt format works unchanged with another model.
- Measure latency and approximate token cost using representative requests.
- Keep model and tokenizer versions with your experiment notes.
Return to the Using, adapting, and fine-tuning pretrained LLMs stage · Suggested timing is flexible; move at your own pace.