Lesson 27 of 38 · Written learning
Evaluate retrieval, evidence, and citations
Find out whether a RAG system retrieves the right passages and stays within what they support.
A response can fail because retrieval missed the relevant passage or because generation misread a passage it received. Evaluate these separately. For retrieval, check whether expected sources appear near the top. For answers, inspect factual support, completeness, citation accuracy, and whether the system abstains when the evidence is missing.
Retrieved text is untrusted input
A document may contain malicious or misleading instructions. Treat retrieved passages as data, not as new system instructions. Limit what tools the model can call, keep secrets out of the prompt, and test documents that include hostile text. Citations make evidence easier to inspect; they do not guarantee that the cited passage supports every sentence.
- Create test questions with known source passages and insufficient-evidence cases.
- Measure retrieval separately from answer quality.
- Check citations manually on a sample and test misleading or conflicting documents.
- Store only necessary data and explain retention in a clear privacy note.
Return to the Embeddings, vector search, and RAG stage · Suggested timing is flexible; move at your own pace.