What chunk size is best?

There is no universal answer. Start around 500 to 1000 characters with some overlap, then test different sizes against your own questions and measure retrieval quality.

Why is overlap useful?

It prevents an important sentence or idea from being split across two chunks so that neither half is retrievable on its own.

RAG Step 1: Loading and Chunking Documents

Introduction

Retrieval quality starts with how you split your documents. Chunks that are too big bury the answer in noise, and chunks that are too small lose context. This lesson loads text from files, splits it into overlapping chunks, and keeps metadata so you can cite sources later.

Prerequisites

The RAG introduction lesson
pip install pypdf

Step-by-Step Guide

Step 1: Load text from files

Plain text and markdown can be read directly. For PDFs, extract text page by page and keep the page number, since it is valuable for citations. Check the extracted text, because scanned or oddly formatted PDFs can produce poor results.

from pathlib import Path
from pypdf import PdfReader

def load_text_file(path: str) -> list[dict]:
    text = Path(path).read_text(encoding="utf-8")
    return [{"text": text, "source": Path(path).name, "page": None}]

def load_pdf(path: str) -> list[dict]:
    reader = PdfReader(path)
    pages = []
    for number, page in enumerate(reader.pages, start=1):
        pages.append({
            "text": page.extract_text() or "",
            "source": Path(path).name,
            "page": number,
        })
    return pages

Step 2: Split into overlapping chunks

A simple, effective start is fixed-size chunks with overlap, so a sentence cut at a boundary still appears whole in the neighboring chunk. A few hundred to about a thousand characters is a common range, but the best size depends on your content and questions.

def chunk_text(text: str, size: int = 800, overlap: int = 150) -> list[str]:
    chunks = []
    start = 0
    while start < len(text):
        end = min(start + size, len(text))
        piece = text[start:end].strip()
        if piece:
            chunks.append(piece)
        if end == len(text):
            break
        start = end - overlap
    return chunks

Step 3: Respect natural boundaries

Better chunks follow the document's own structure: split on headings, then paragraphs, and only cut inside a paragraph when it is too long. Keeping a heading with its content gives each chunk the context it needs to be understood on its own.

Step 4: Attach metadata to every chunk

Record the source file, page or section, and any attributes you may want to filter on, such as department, date, or access level. Metadata powers citations and lets you restrict results later.

def build_chunks(pages: list[dict]) -> list[dict]:
    records = []
    for page in pages:
        for piece in chunk_text(page["text"]):
            records.append({
                "text": piece,
                "source": page["source"],
                "page": page["page"],
            })
    return records

Step 5: Inspect your chunks

Read a sample. Do they make sense alone? Are tables or code mangled? Fixing extraction and chunking problems here is far cheaper than debugging bad answers later.

records = build_chunks(load_pdf("handbook.pdf"))
print(len(records), "chunks")
print(records[0]["text"][:300])

FAQs

What chunk size is best?: There is no universal answer. Start around 500 to 1000 characters with some overlap, then test different sizes against your own questions and measure retrieval quality.
Why is overlap useful?: It prevents an important sentence or idea from being split across two chunks so that neither half is retrievable on its own.

Summary

Extract clean text, split along natural boundaries with some overlap, and attach metadata to each chunk. Always inspect the result, because chunking quality limits everything that follows.