A language model only knows what was in its training data. Ask it about your company handbook, last week's incident report or a document it has never seen, and it will either say it doesn't know or — worse — invent something plausible. Retrieval-augmented generation (RAG) is the standard fix.
The idea in one paragraph
Before the model answers, you run a search over your own documents, pull back the handful of passages most relevant to the question, and paste them into the prompt as context. The model then answers from that context instead of from memory. That's it. The cleverness is all in the retrieval step and in how you feed the results to the model.
What it looks like in pieces
- Chunking. Split documents into passages small enough to be specific but large enough to make sense on their own.
- Embedding & indexing. Turn each chunk into a vector and store it so you can find “nearby” chunks fast.
- Retrieval. Embed the question, find the closest chunks, optionally re-rank them with a second model.
- Generation. Put the question and the retrieved chunks in the prompt; ask the model to answer and cite which chunk it used.
Where it goes wrong
Most bad RAG systems fail at retrieval, not generation. The model is fine; it just got handed the wrong passages. Common causes: chunks that are too big or too small, an embedding model that doesn't understand your domain's vocabulary, no re-ranking, or no evaluation set to catch regressions when you change something. Teams also forget that “the answer isn't in any document” is a valid outcome the system should handle gracefully.
How we teach it
In the GenAI & LLM Engineering track you build a RAG system end to end on a real corpus, then spend as much time measuring it as building it: a labelled question set, retrieval metrics, answer-quality checks, and a loop for improving each stage. By the end you can tell why a given answer was wrong and which knob to turn.
This is the kind of thing we go deep on in the courses — with a GPU workspace open and a mentor reviewing your code. See the catalogue or talk to an advisor.