How retrieval-augmented generation grounds LLM answers in real documents — the end-to-end pipeline, retrieval strategies, evaluation, and when a system should retrieve at all.
Takeaway 1. A prompt only contains what you put in it — instructions, context, data, and constraints. An LLM "doesn't know" what you want unless it's actually written into the prompt; RAG is fundamentally about automating what goes into that context.
Takeaway 2. RAG doesn't fix a "dumb" model — it fixes a knowledge-access problem. Grounding generation in retrieved documents the model wasn't trained on (or has forgotten) is why retrieval quality, not the LLM itself, is usually the bottleneck.
Takeaway 3. A RAG pipeline has far more moving parts than "retrieve then generate" — data cleaning, chunking strategy, and the choice of retrieval/ranking method each independently affect answer quality, and each is a separate place to debug.
Takeaway 4. No single retrieval method wins outright — sparse methods (TF-IDF, BM25) match exact terms, dense nearest-neighbor search captures semantic similarity, and hybrid retrieval, query rewriting, and cross-encoder reranking each patch a different weakness. Evaluation metrics like precision, recall, and reciprocal rank can disagree, so the right one depends on how the system is actually used.
Takeaway 5. When to retrieve is a design choice, not a fixed rule: once at the start (standard RAG), a few times during generation when the model is uncertain (FLARE), or at every token (kNN-LM, Unlimiformer) — each trades compute cost against how current and precise the retrieved knowledge needs to be.