💡 Think of it like this:

Imagine you're a student writing an exam, but you're allowed to bring one textbook. Without that textbook, you'd have to answer purely from memory — and your memory has gaps and might be outdated. With it, you can flip to the exact page you need, read the relevant passage, and then write your answer. The answer is still yours, but it's grounded in accurate, current information.

That's what RAG does for AI. An LLM on its own can only answer from what it was trained on (its "memory"). RAG lets it retrieve real, up-to-date information from external sources before generating a response — so the answer is grounded in actual facts, not just training data.

What Is RAG?

NVIDIA Glossary: What is Retrieval-Augmented Generation (RAG)?

Retrieval-Augmented Generation (RAG) is an AI technique where an external data source is connected to a large language model (LLM) so it can generate domain-specific or up-to-date responses in real time.

The core problem RAG solves: LLMs are powerful, but their knowledge is frozen at their training cutoff. They don't know about your company's internal documents, yesterday's news, or your specific database. RAG bridges this gap by letting the model look things up before answering.

<aside> ⚠️

Why this matters: Without RAG, LLMs can "hallucinate" — they confidently generate answers that sound right but are factually wrong. RAG anchors responses in real data, dramatically reducing hallucinations.

</aside>

⚙️ How Does RAG Work?

The RAG pipeline has three main phases:

Phase 1: Extraction (Preparing the Knowledge)

Before RAG can retrieve anything, the external data needs to be prepared:

  1. Collect the data — Documents, PDFs, product catalogs, images, audio transcripts — whatever the organisation has
  2. Chunk it — Break large documents into smaller, manageable pieces (paragraphs or sections)
  3. Create embeddings — An embedding model converts each chunk into a vector (a list of numbers that represents its meaning). This is the same concept as the word vectors and latent space you learned about in your NLP and Generative AI notes
  4. Store in a vector database — The chunks and their vectors get indexed and stored for fast retrieval later

Phase 2: Retrieval (Finding the Right Information)

When a user asks a question:

  1. The query gets converted into a vector using the same embedding model
  2. Similarity search — The system searches the vector database for the chunks whose vectors are most similar to the query vector (this is semantic search — searching by meaning, not exact keywords)