Why do I get no useful answers after changing the embedding model?

Vectors from different models are not compatible. Re-ingest your documents using the new embedding model so the stored vectors and the query vectors match.

How do I know retrieval is the problem?

Run the vector store or Parser component on its own and use Inspect output to see which chunks were retrieved. If the right passage is missing, fix chunking or retrieval settings before touching the prompt.

Langflow Module 5: RAG with Your Own Documents

Introduction

Retrieval-augmented generation lets a model answer from your own documents. In Langflow it is built from two flows that share a vector store: one loads data, and one retrieves and answers. This module walks through the Vector Store RAG template and the key settings.

Prerequisites

Modules 2 and 3 completed
A document to test with, such as a PDF or text file
An embedding model available through your provider settings

Step-by-Step Guide

Step 1: Open the Vector Store RAG template

Create a new flow and choose Vector Store RAG. It contains two subflows: a Load Data Flow and a Retriever Flow. If you prefer a local store, you can replace the template's vector database component with Chroma DB. Chroma and several other vector stores are not part of a minimal install, so run uv pip install "langflow[bundles]" first if you see a missing-package error.

uv pip install "langflow[bundles]"

Step 2: Build the Load Data Flow

The loading side reads a file, splits it, embeds the chunks, and stores them. The components are Read File, Split Text, an embedding model, and the vector store. Read File accepts types such as PDF, text, markdown, Word, CSV, and JSON. Split Text defaults to a Chunk Size of 1000 and a Chunk Overlap of 200, which is a reasonable start.

Step 3: Ingest the data

In Read File, choose your file. Then select the vector store component and click Run component to write the chunks into the store. The write happens when you run that component, not when you chat. For Chroma, leaving Persist Directory empty creates an ephemeral store that disappears when the server restarts, so set a directory if you want to keep it.

Step 4: Review the Retriever Flow

The question side takes the user's message from Chat Input, embeds it with the same embedding model, searches the vector store, and turns the results into text with a Parser. A Prompt Template combines the retrieved text and the question, a Language Model answers, and Chat Output displays it. Using the same embedding model for loading and searching is essential.

Step 5: Write a grounded prompt and tune

In the Prompt Template, tell the model to answer only from the retrieved context and to say when it does not know. Then test in the Playground with questions whose answers you know. If answers are weak, adjust Chunk Size, the Number of Results the store returns, and the prompt, and keep chunks within your embedding model's token limit.

Answer the question using only the context below.
If the context does not contain the answer, say you don't know.

Context:
{context}

Question: {question}

FAQs

Why do I get no useful answers after changing the embedding model?: Vectors from different models are not compatible. Re-ingest your documents using the new embedding model so the stored vectors and the query vectors match.
How do I know retrieval is the problem?: Run the vector store or Parser component on its own and use Inspect output to see which chunks were retrieved. If the right passage is missing, fix chunking or retrieval settings before touching the prompt.

Summary

Langflow RAG is two flows around one vector store: load data with Read File, Split Text, embeddings, and a store, then retrieve with the same embedding model and answer with a grounded prompt. Inspect retrieved chunks first when answers are poor.