Aggregation or application code?

Do heavy filtering, grouping, and joining inside the database with a pipeline so you move less data over the network. Keep presentation logic and complex business rules in your application.

MongoDB Module 4: The Aggregation Framework

Introduction

The aggregation framework is MongoDB's analytics tool. You pass documents through a pipeline of stages, and each stage transforms them: filter, group, reshape, join, sort. It is the equivalent of GROUP BY, JOIN, and subqueries in SQL. This module uses the orders from the previous lesson.

Prerequisites

Module 3 completed (the orders collection)

Step-by-Step Guide

Step 1: Understand the pipeline idea

A pipeline is an array of stages. Documents flow through them in order, and each stage receives the output of the previous one. Put $match and $limit early to reduce the amount of data later stages must process, which also lets MongoDB use indexes.

db.collection.aggregate([
  { $match: { ... } },     // filter
  { $group: { ... } },     // aggregate
  { $sort: { ... } },      // order
  { $limit: 10 }           // cap
])

Step 2: Filter and group

$match filters like find. $group groups documents by an _id expression and applies accumulators such as $sum, $avg, $min, $max, and $count. Group by null to compute one total over everything.

db.orders.aggregate([
  { $match: { status: "shipped" } },
  { $group: { _id: "$customer.city", orders: { $sum: 1 } } },
  { $sort: { orders: -1 } }
])

Step 3: Unwind arrays and compute fields

$unwind turns each element of an array into its own document, so you can aggregate line items. $addFields and $project create or reshape fields using expressions such as $multiply.

db.orders.aggregate([
  { $unwind: "$items" },
  { $addFields: { lineTotal: { $multiply: ["$items.qty", "$items.price"] } } },
  { $group: { _id: "$items.sku", unitsSold: { $sum: "$items.qty" }, revenue: { $sum: "$lineTotal" } } },
  { $sort: { revenue: -1 } }
])

Step 4: Join collections with $lookup

$lookup performs a left outer join with another collection, adding the matches as an array. Use it when data is genuinely in separate collections, and consider embedding for data that is always read together. An index on the foreign field makes the lookup much faster.

db.orders.aggregate([
  { $unwind: "$items" },
  { $lookup: { from: "products", localField: "items.sku", foreignField: "sku", as: "product" } },
  { $unwind: "$product" },
  { $project: { _id: 0, orderId: 1, sku: "$items.sku", name: "$product.name", qty: "$items.qty" } }
])

Step 5: Group by time and write results out

Use date operators such as $dateToString or $dateTrunc to group by day or month. $merge or $out write a pipeline's result to a collection, which is how you build summary collections for dashboards.

db.orders.aggregate([
  { $group: {
      _id: { $dateToString: { format: "%Y-%m-%d", date: "$placedAt" } },
      orders: { $sum: 1 } } },
  { $sort: { _id: 1 } },
  { $merge: { into: "daily_orders", whenMatched: "replace", whenNotMatched: "insert" } }
])

FAQs

Aggregation or application code?: Do heavy filtering, grouping, and joining inside the database with a pipeline so you move less data over the network. Keep presentation logic and complex business rules in your application.

Summary

An aggregation pipeline passes documents through stages: $match early, $group with accumulators, $unwind for arrays, $addFields and $project to reshape, $lookup to join, and $merge to save results.