Is LangSmith required to use LangChain?

No. It is an optional observability and evaluation platform. LangChain works without it, but tracing makes agents far easier to debug.

How do I test an agent that gives different answers each time?

Check outcomes with assertions such as required facts or tool calls, run each case several times if needed, and track pass rates rather than expecting identical text.

LangChain Module 9: Observability, Testing, and Going to Production

Introduction

A demo that works once is not an application. In this final module of the LangChain course you add tracing so you can see what agents do, a simple test harness, reliability features, and a checklist for production.

Prerequisites

Modules 1 to 8 completed

Step-by-Step Guide

Step 1: Turn on tracing with LangSmith

LangSmith records each model call, tool call, and step, so you can inspect exactly what happened in a run. Enable it with environment variables and a LangSmith API key. No code changes are required.

# .env
LANGSMITH_TRACING=true
LANGSMITH_API_KEY=your-langsmith-key

Step 2: Build a golden test set

Collect realistic questions with expected outcomes and run the agent over them whenever you change prompts, tools, or models. Check outcomes rather than exact wording, because model phrasing varies.

cases = [
    {"question": "What is the weather in Paris?", "must_contain": "sunny"},
    {"question": "How many words are in 'to be or not to be'?", "must_contain": "6"},
]

for case in cases:
    result = agent.invoke(
        {"messages": [{"role": "user", "content": case["question"]}]}
    )
    answer = result["messages"][-1].text
    print("PASS" if case["must_contain"] in answer else "FAIL", "-", case["question"])

Step 3: Handle failures and cost

Models time out and rate-limit. The max_retries setting retries transient errors, and middleware such as model retry and model fallback can switch to another model when one fails. Log token usage from usage_metadata, and use call-limit middleware so a confused agent cannot loop and run up costs.

Step 4: Secure the application

Give tools least privilege and validate their inputs, since the model chooses the arguments. Require approval for irreversible actions, treat retrieved documents and web content as untrusted, and keep API keys in environment variables or a secrets manager, never in code.

Step 5: Pin versions and plan deployment

LangChain evolves quickly, so pin your dependency versions and upgrade deliberately, running your test set each time. Expose your agent behind an API endpoint, use a database-backed checkpointer for memory, and put the whole service behind authentication. If you maintain older code, migrate gradually and use langchain-classic as a bridge for removed legacy imports.

pip freeze > requirements.txt

FAQs

Is LangSmith required to use LangChain?: No. It is an optional observability and evaluation platform. LangChain works without it, but tracing makes agents far easier to debug.
How do I test an agent that gives different answers each time?: Check outcomes with assertions such as required facts or tool calls, run each case several times if needed, and track pass rates rather than expecting identical text.

Summary

Trace runs with LangSmith, test against a golden set, plan for retries and cost, secure tools and secrets, and pin versions. That completes the path from your first model call to a production-ready LangChain application.