LangChain Module 7: RAG Part 2, Retrieval Agents and Two-Step Chains

Step 1: Build a two-step RAG function

Retrieve the best chunks, put them in the prompt with clear instructions, and call the model once. This is predictable, fast, and cheap, because there is exactly one search and one model call per question.

def answer_question(question: str) -> str:
    hits = vector_store.similarity_search(question, k=4)
    context = "\n\n".join(
        "[" + str(i + 1) + "] (" + str(d.metadata.get("source")) + ", page "
        + str(d.metadata.get("page")) + ")\n" + d.page_content
        for i, d in enumerate(hits)
    )
    prompt = (
        "Answer using only the numbered sources below and cite them like [1]. "
        "If the sources do not contain the answer, say you don't know.\n\n"
        "<sources>\n" + context + "\n</sources>\n\nQuestion: " + question
    )
    return model.invoke(prompt).text

print(answer_question("How many vacation days do new employees get?"))

Step 2: Turn retrieval into a tool

For an agent, wrap the search as a tool that returns text the model can read. Include the source and page in the returned text so the agent can cite them.

from langchain.tools import tool

@tool
def search_handbook(query: str) -> str:
    """Search the company handbook for passages relevant to a question."""
    hits = vector_store.similarity_search(query, k=4)
    return "\n\n".join(
        "Source: " + str(d.metadata.get("source")) + ", page "
        + str(d.metadata.get("page")) + "\n" + d.page_content
        for d in hits
    )

Step 3: Create the retrieval agent

Give the agent the search tool and tell it when to use it. The agent can search more than once, rephrase queries, or skip searching for simple greetings. That flexibility costs extra model calls and makes behavior less predictable.

from langchain.agents import create_agent

agent = create_agent(
    model=model,
    tools=[search_handbook],
    system_prompt=(
        "You answer questions about the company handbook. "
        "Use search_handbook for factual questions, quote the relevant passages, "
        "and say when the handbook does not contain the answer."
    ),
)

result = agent.invoke(
    {"messages": [{"role": "user", "content": "Can contractors take paid leave?"}]}
)
print(result["messages"][-1].text)

Step 4: Choose between the two approaches

Use the two-step flow when every question needs retrieval and you want low latency, low cost, and predictable behavior. Use the agent when questions vary, some need several searches or other tools, or you want the system to handle follow-ups. Many production systems start with two-step and add an agent only where it measurably helps.

Step 5: Test and secure your RAG

Keep a list of real questions with known answers and check both retrieval (is the right chunk returned?) and answers (are they grounded?). Treat retrieved documents as untrusted input, since they can contain text that tries to instruct the model, and apply access control before retrieval so users only search documents they may read.