Why use a Pipeline instead of preprocessing the data first?

Preprocessing the whole dataset up front leaks information from the test data into training. A Pipeline fits preprocessing on training data only and guarantees the same steps run at prediction time.

Do all models need feature scaling?

No. Distance-based and gradient-based models such as k-nearest neighbors, SVMs, and logistic regression benefit from scaling. Tree-based models generally do not need it.

scikit-learn Pipelines and Feature Engineering

Introduction

Preprocessing steps like imputing, scaling, and encoding must be learned from the training data only, then applied identically to new data. Doing that by hand invites mistakes. A scikit-learn Pipeline bundles preprocessing and the model into a single object that is easy to cross-validate, save, and deploy.

Prerequisites

Data preparation with pandas
Cross-validation basics

Step-by-Step Guide

Step 1: Describe preprocessing by column type

A ColumnTransformer applies different steps to different columns: impute and scale numeric columns, and one-hot encode categorical ones. handle_unknown='ignore' stops the model failing on categories it did not see during training.

from sklearn.compose import ColumnTransformer
from sklearn.pipeline import Pipeline
from sklearn.impute import SimpleImputer
from sklearn.preprocessing import StandardScaler, OneHotEncoder

numeric = ["age", "income"]
categorical = ["city"]

preprocess = ColumnTransformer([
    ("num", Pipeline([
        ("impute", SimpleImputer(strategy="median")),
        ("scale", StandardScaler()),
    ]), numeric),
    ("cat", OneHotEncoder(handle_unknown="ignore"), categorical),
])

Step 2: Attach a model to make a full Pipeline

Put the preprocessing and the estimator in one Pipeline. Calling fit learns the preprocessing statistics from the training data only and then trains the model.

from sklearn.linear_model import LogisticRegression

clf = Pipeline([
    ("prep", preprocess),
    ("model", LogisticRegression(max_iter=1000)),
])

clf.fit(X_train, y_train)
print("Test accuracy:", clf.score(X_test, y_test))

Step 3: Cross-validate without leakage

Because the preprocessing lives inside the Pipeline, each cross-validation fold learns its own imputation and scaling from its own training portion, which prevents leakage from the validation fold.

from sklearn.model_selection import cross_val_score

scores = cross_val_score(clf, X_train, y_train, cv=5)
print(scores.mean())

Step 4: Tune pipeline settings

Reach into a step with the step name, two underscores, and the parameter name. This lets you tune preprocessing and the model together.

from sklearn.model_selection import GridSearchCV

grid = GridSearchCV(
    clf,
    {"model__C": [0.1, 1, 10]},
    cv=5,
)
grid.fit(X_train, y_train)
print(grid.best_params_)

Step 5: Add simple engineered features

Feature engineering means creating inputs that make patterns easier to learn, such as ratios, counts, or date parts. Add them in pandas before the pipeline, and validate each one with cross-validation to confirm it helps.

df["income_per_room"] = df["income"] / df["rooms"]
df["signup_month"] = pd.to_datetime(df["signup_date"]).dt.month

FAQs

Why use a Pipeline instead of preprocessing the data first?: Preprocessing the whole dataset up front leaks information from the test data into training. A Pipeline fits preprocessing on training data only and guarantees the same steps run at prediction time.
Do all models need feature scaling?: No. Distance-based and gradient-based models such as k-nearest neighbors, SVMs, and logistic regression benefit from scaling. Tree-based models generally do not need it.

Summary

Bundle preprocessing and modeling in a Pipeline so every step is learned from training data only and applied the same way later. It makes cross-validation honest and deployment simple.