Lesson 26 of 38 · Written learning
Build a retrieval-augmented generation pipeline
Trace a question from document ingestion through retrieval and answer generation.
A retrieval-augmented generation system first prepares a document collection: parse files, split them into passages, attach source metadata, and compute embeddings or searchable terms. At query time, it may rewrite the question, retrieve candidates, filter them, rerank them, and place the strongest passages into a prompt.
Construct the answer context carefully
The prompt should clearly separate instructions, the user's question, and retrieved content. The model can then produce a response grounded in those passages. Include source identifiers so the application can show citations. Chunk size and overlap affect both retrieval precision and whether a passage contains enough context; test them rather than choosing them by habit.
- Parse and retain document titles, URLs, dates, and page identifiers where possible.
- Use only documents you are permitted to process.
- Keep source text available so a citation can point to evidence a reader can inspect.
- Treat local and hosted storage as different privacy and operational choices.
Return to the Embeddings, vector search, and RAG stage · Suggested timing is flexible; move at your own pace.