Lesson 37 of 38 · Written learning
Plan an end-to-end LLM capstone
Turn a meaningful problem into a bounded system you can implement and evaluate.
Start by describing one user, one task, and the consequence of a wrong answer. Write a short product specification: what the application will do, what it will not do, what information it needs, and how a user will recognize a useful result. Check whether a simpler non-LLM method could solve the task more reliably.
Design, build, and measure
Choose a model and architecture that fit the privacy, latency, and cost constraints. Build a small vertical slice first. If the application uses retrieval, test retrieval separately; if it uses tools, start with least privilege. Create representative examples before tuning. Measure the output quality and system behavior, document failures, and revise the design based on evidence.
- Keep source code, setup instructions, and data provenance in a repository.
- Include normal, edge, adversarial, and insufficient-evidence cases.
- Report latency and approximate cost alongside quality observations.
- Document what the system cannot safely or reliably do.
Return to the Capstone and advanced directions stage · Suggested timing is flexible; move at your own pace.