Why is my model's accuracy lower in production than in testing?

Common causes are data drift, differences between training and live data pipelines, leakage during evaluation, and a changed real-world process. Monitoring inputs and outcomes helps you find which.

Should I serve the model in real time or in batches?

Use real-time APIs when decisions are needed immediately, and batch scoring when predictions can be computed periodically, which is simpler and cheaper to run.

Saving and Deploying a Machine Learning Model as an API

Introduction

A model only creates value once other systems can use it. This final lesson of the ML course saves a trained Pipeline to disk, wraps it in a small FastAPI service that returns predictions, and covers what to monitor once it is running.

Prerequisites

A trained scikit-learn Pipeline from the previous lesson
pip install fastapi uvicorn joblib

Step-by-Step Guide

Step 1: Save the trained pipeline

Saving the whole Pipeline keeps preprocessing and the model together, so the API applies exactly the same transformations as training. Record the library versions used, because a saved model can break if scikit-learn changes.

import joblib

joblib.dump(clf, "model.joblib")

# pip freeze > requirements.txt   # record versions with the model

Step 2: Load only files you trust

joblib and pickle files can execute code when loaded. Never load a model file from an untrusted source, and keep your own model files in a protected location.

Step 3: Create the prediction service

FastAPI uses Pydantic to validate input. Define the input shape, load the model once at startup, and return the predicted probability from an endpoint.

# main.py
from fastapi import FastAPI
from pydantic import BaseModel
import joblib
import pandas as pd

app = FastAPI()
model = joblib.load("model.joblib")


class Applicant(BaseModel):
    age: int
    income: float
    city: str


@app.post("/predict")
def predict(item: Applicant):
    frame = pd.DataFrame([item.model_dump()])
    probability = float(model.predict_proba(frame)[0][1])
    return {"probability": probability}

Step 4: Run and call it

Start the server with uvicorn, then send a request. FastAPI also generates interactive documentation at /docs.

uvicorn main:app --reload

# in another terminal:
curl -X POST http://127.0.0.1:8000/predict \
  -H "Content-Type: application/json" \
  -d '{"age": 34, "income": 52000, "city": "London"}'

Step 5: Monitor after launch

Deployment is the start, not the end. Track input data drift (inputs that no longer look like training data), prediction distributions, latency and errors, and, once true outcomes arrive, the real accuracy. Plan to retrain on fresh data and keep versioned models so you can roll back.

FAQs

Why is my model's accuracy lower in production than in testing?: Common causes are data drift, differences between training and live data pipelines, leakage during evaluation, and a changed real-world process. Monitoring inputs and outcomes helps you find which.
Should I serve the model in real time or in batches?: Use real-time APIs when decisions are needed immediately, and batch scoring when predictions can be computed periodically, which is simpler and cheaper to run.

Summary

Save the whole Pipeline, load only trusted files, wrap the model in a validated API, and monitor drift and outcomes after launch. With that you have completed an ML project from data to deployment.