Does RAG eliminate hallucinations?

It reduces them by grounding answers in retrieved text, but it does not remove them. The retriever can miss the right passage and the model can still misread or ignore sources, which is why evaluation matters.

Do I need a vector database to start?

No. For thousands of chunks, a NumPy array is enough. Move to a vector database or a database extension when your data, update rate, or access-control needs grow.

RAG Explained: What Retrieval-Augmented Generation Is and When to Use It

Introduction

An LLM only knows what was in its training data, and that data is frozen at a point in time and does not include your private documents. Retrieval-augmented generation (RAG) fixes this by finding relevant passages from your own content at question time and giving them to the model. This is the first lesson of a six-part RAG course.

Prerequisites

Calling an LLM API lesson
Basic Python

Step-by-Step Guide

Step 1: See the problem RAG solves

Without help, a model cannot answer questions about your company handbook, product docs, or yesterday's news, and it may guess instead. Pasting every document into every prompt is expensive and often impossible. RAG retrieves only the few passages that matter for each question.

Step 2: Learn the two phases

Indexing happens ahead of time: load documents, split them into chunks, convert each chunk into an embedding vector, and store them in an index. Querying happens per question: embed the question, retrieve the most similar chunks, put them in the prompt, and let the model answer using them.

INDEXING (offline)                 QUERYING (per question)
documents                          question
  -> split into chunks               -> embed the question
  -> embed each chunk                -> search the index for top-k chunks
  -> store vectors + metadata        -> build prompt with those chunks
                                     -> LLM answers, citing sources

Step 3: Know what each part does

Chunking decides what a retrievable unit is. Embeddings turn text into vectors so that similar meanings are close together. The vector index finds nearest neighbors quickly. The prompt tells the model to answer only from the retrieved sources. Each part can be tuned and tested independently.

Step 4: Decide between RAG, long context, and fine-tuning

Use RAG for large or changing document collections, when you need citations, or when different users may see different documents. Put the material directly in the prompt when it is small. Fine-tuning changes a model's behavior and style but is a poor way to teach it facts, which are better supplied through retrieval.

Step 5: Plan the project

Pick a focused document set and 20 to 30 realistic questions with known answers. You will use them in the evaluation lesson to measure retrieval and answer quality. The next lessons build the pipeline in Python with minimal dependencies so every step is visible.

FAQs

Does RAG eliminate hallucinations?: It reduces them by grounding answers in retrieved text, but it does not remove them. The retriever can miss the right passage and the model can still misread or ignore sources, which is why evaluation matters.
Do I need a vector database to start?: No. For thousands of chunks, a NumPy array is enough. Move to a vector database or a database extension when your data, update rate, or access-control needs grow.

Summary

RAG retrieves relevant passages from your documents at question time and gives them to the model, keeping answers current, grounded, and citable. Next you will load and chunk documents.