CTCaliberTek

Data Engineering & Analytics

Courses on data science, data lakes, ETL, data warehousing, NoSQL databases, and the lakehouse.

Data ScienceData LakesAWS GlueAmazon RedshiftMongoDBDatabricks

Tutorials

Data Engineering & Analytics

Databricks Module 8: SQL Warehouses, Performance, Cost, and a First Look at MLflow

Serve analysts with Databricks SQL, tune Spark and Delta performance, control cost, and track a model with MLflow.

October 21, 202610 min read
Data Engineering & Analytics

Databricks Module 7: Jobs, Orchestration, and Deploying with Git and Bundles

Schedule multi-task workflows with Lakeflow Jobs, pass parameters, handle failures, and deploy code with Git and Asset Bundles.

October 20, 202610 min read
Data Engineering & Analytics

Databricks Module 6: Declarative Pipelines and the Medallion Architecture

Build bronze, silver, and gold tables declaratively with Lakeflow Spark Declarative Pipelines, including data quality expectations.

October 19, 202610 min read
Data Engineering & Analytics

Databricks Module 5: Ingesting Files with Auto Loader and COPY INTO

Load new files incrementally with Auto Loader, handle schema evolution, and use COPY INTO for simple batch loads.

October 18, 202610 min read
Data Engineering & Analytics

Databricks Module 4: Unity Catalog, Volumes, and Governance

Organize data in catalogs and schemas, store files in volumes, grant permissions, and use lineage.

October 17, 20269 min read
Data Engineering & Analytics

Databricks Module 3: Delta Lake Tables, MERGE, and Time Travel

Create Delta tables, update data with MERGE, query earlier versions, and keep tables healthy with OPTIMIZE and VACUUM.

October 16, 202611 min read
Data Engineering & Analytics

Databricks Module 2: Notebooks and Spark DataFrames with PySpark

Create DataFrames, select, filter, aggregate, join, and use Spark SQL in Databricks notebooks.

October 15, 202610 min read
Data Engineering & Analytics

Databricks Module 1: The Lakehouse Platform and Your First Notebook

Understand what Databricks is, how its workspace and compute fit together, and run your first notebook on Free Edition or a trial workspace.

October 14, 20269 min read
Data Engineering & Analytics

MongoDB Module 8: Security, Atlas Features, and Using MongoDB from Code

Secure a deployment, learn Atlas Search and Vector Search, and connect from Python and Node.js.

October 13, 202610 min read
Data Engineering & Analytics

MongoDB Module 7: Transactions, Replication, and Sharding

Understand replica sets, read and write concerns, multi-document transactions, change streams, and how sharding scales a cluster.

October 12, 202610 min read
Data Engineering & Analytics

MongoDB Module 6: Indexes and Query Performance

Create single, compound, and TTL indexes, read explain plans, and follow the equality, sort, range rule.

October 11, 202610 min read
Data Engineering & Analytics

MongoDB Module 5: Schema Design and Validation

Decide when to embed and when to reference, use common design patterns, and enforce structure with JSON Schema validation.

October 10, 202610 min read
Data Engineering & Analytics

MongoDB Module 4: The Aggregation Framework

Build pipelines with $match, $group, $project, $unwind, $lookup, and $sort to analyze data inside MongoDB.

October 9, 202611 min read
Data Engineering & Analytics

MongoDB Module 3: Query Operators, Arrays, Nested Fields, and Pagination

Write richer queries with comparison and logical operators, query nested fields and arrays, and paginate results.

October 8, 202610 min read
Data Engineering & Analytics

MongoDB Module 2: CRUD Operations

Insert, read, update, and delete documents with mongosh, including update operators, projections, and upserts.

October 7, 202610 min read
Data Engineering & Analytics

MongoDB Module 1: Introduction, the Document Model, and Setup

Learn what MongoDB is, how documents differ from tables, and set up MongoDB with Docker or Atlas and connect with mongosh.

October 6, 20269 min read
Data Engineering & Analytics

Amazon Redshift Module 8: Operations, Cost, and a Capstone Project

Back up and restore, monitor and control cost, and finish by building a small analytics warehouse end to end.

October 5, 202610 min read
Data Engineering & Analytics

Amazon Redshift Module 7: Security, Data Sharing, and Integrations

Control access with roles and fine-grained permissions, share live data between warehouses, and understand zero-ETL integrations.

October 4, 202610 min read
Data Engineering & Analytics

Amazon Redshift Module 6: Redshift Spectrum and Querying the Data Lake

Query data that stays in Amazon S3 using external schemas and the Glue Data Catalog, and combine it with warehouse tables.

October 3, 20269 min read
Data Engineering & Analytics

Amazon Redshift Module 5: Performance Tuning and Maintenance

Read query plans, find slow queries, understand caching and concurrency scaling, and keep tables healthy.

October 2, 202610 min read
Data Engineering & Analytics

Amazon Redshift Module 4: Querying, Window Functions, and Semi-Structured Data

Write analytical SQL with CTEs and window functions, and query JSON with the SUPER data type.

October 1, 202610 min read
Data Engineering & Analytics

Amazon Redshift Module 3: Loading Data with COPY and Exporting with UNLOAD

Load files from S3 into Redshift with COPY, troubleshoot load errors, and export query results with UNLOAD.

September 30, 202610 min read
Data Engineering & Analytics

Amazon Redshift Module 2: Designing Tables with Distribution Keys and Sort Keys

Create tables with the right data types, distribution styles, sort keys, and compression for fast queries.

September 29, 202610 min read
Data Engineering & Analytics

Amazon Redshift Module 1: Architecture and Setting Up Serverless

Understand how Redshift stores and queries data, choose between Serverless and provisioned, and create a workgroup with cost limits.

September 28, 20269 min read
Data Engineering & Analytics

AWS Glue Module 8: Monitoring, Performance, Data Quality, and Cost

Monitor jobs, tune workers and auto scaling, add Glue Data Quality rules, and keep Glue costs under control.

September 27, 202610 min read
Data Engineering & Analytics

AWS Glue Module 7: Triggers, Workflows, and Job Parameters

Schedule and chain crawlers and jobs, pass parameters, and handle failures and retries.

September 26, 20269 min read
Data Engineering & Analytics

AWS Glue Module 6: Partitions, Pushdown Predicates, and Job Bookmarks

Process only new or needed data with partition filters and job bookmarks, and write well-partitioned output.

September 25, 20269 min read
Data Engineering & Analytics

AWS Glue Module 5: Transforms and Cleaning Data

Use ApplyMapping, ResolveChoice, DropNullFields, Filter, and Join, and clean data with Spark functions.

September 24, 202610 min read
Data Engineering & Analytics

AWS Glue Module 4: Writing PySpark Scripts with GlueContext and DynamicFrames

Write a Glue ETL script by hand: read from the catalog, transform with DynamicFrames, and write partitioned Parquet.

September 23, 202610 min read
Data Engineering & Analytics

AWS Glue Module 3: Your First ETL Job with Glue Studio

Build a visual ETL job that reads the cataloged table, filters and renames columns, and writes Parquet to S3.

September 22, 20269 min read
Data Engineering & Analytics

AWS Glue Module 2: The Data Catalog and Crawlers

Create a catalog database, run a crawler over S3 data, and understand how tables, schemas, and partitions are recorded.

September 21, 20269 min read
Data Engineering & Analytics

AWS Glue Module 1: What Glue Is and Setting It Up

Learn the parts of AWS Glue, how it is priced, and create the IAM role and S3 sample data you need for the course.

September 20, 20269 min read
Data Engineering & Analytics

Building a Data Lake Module 7: Governance, Security, Data Quality, and Cost

Protect data with IAM and Lake Formation, add quality checks and lineage, and keep the lake affordable.

September 19, 202610 min read
Data Engineering & Analytics

Building a Data Lake Module 6: Open Table Formats with Iceberg, Delta Lake, and Hudi

Add transactions, updates, deletes, and time travel to lake data with Apache Iceberg, Delta Lake, or Apache Hudi.

September 18, 202610 min read
Data Engineering & Analytics

Building a Data Lake Module 5: Cataloging and Querying with Glue and Athena

Register data in the AWS Glue Data Catalog and query it in place with SQL in Amazon Athena.

September 17, 202610 min read
Data Engineering & Analytics

Building a Data Lake Module 4: Ingesting Data into the Lake

Bring in data from files, databases, applications, and streams, and make ingestion repeatable and safe to rerun.

September 16, 20269 min read
Data Engineering & Analytics

Building a Data Lake Module 3: File Formats, Compression, and Partitioning

Choose between CSV, JSON, Parquet, and Avro, and design partitions that make queries fast and cheap.

September 15, 20269 min read
Data Engineering & Analytics

Building a Data Lake Module 2: Storage Foundations on Amazon S3

Create buckets and a clean folder structure, and turn on encryption, versioning, and lifecycle rules.

September 14, 20269 min read
Data Engineering & Analytics

Building a Data Lake Module 1: What a Data Lake Is and How It Is Organized

Understand data lakes, warehouses, and lakehouses, and learn the raw, clean, and curated layer design.

September 13, 20268 min read
Data Engineering & Analytics

Data Science with Python Module 8: From Notebook to a Finished Project

Structure a project, save and reload a model, write a prediction function, and communicate results clearly.

September 12, 20269 min read
Data Engineering & Analytics

Data Science with Python Module 7: Evaluating and Improving Models

Choose the right metrics, use cross-validation, tune hyperparameters, and avoid data leakage and overfitting.

September 11, 202610 min read
Data Engineering & Analytics

Data Science with Python Module 6: Machine Learning with scikit-learn

Train and test a regression model and a classifier with scikit-learn, and combine preprocessing and a model in a pipeline.

September 10, 202611 min read
Data Engineering & Analytics

Data Science with Python Module 5: Statistics for Data Science

Learn distributions, sampling, confidence intervals, and hypothesis tests, and run them with SciPy.

September 9, 202610 min read
Data Engineering & Analytics

Data Science with Python Module 4: Exploratory Data Analysis and Visualization

Summarize a dataset, look at distributions and relationships, and draw clear charts with matplotlib.

September 8, 202610 min read
Data Engineering & Analytics

Data Science with Python Module 3: Loading and Cleaning Data

Read files into pandas, then fix missing values, wrong types, duplicates, inconsistent text, and outliers.

September 7, 202610 min read
Data Engineering & Analytics

Data Science with Python Module 2: NumPy and pandas Basics

Work with arrays in NumPy and tables in pandas: select, filter, sort, and summarize data with groupby.

September 6, 20269 min read
Data Engineering & Analytics

Data Science with Python Module 1: What Data Scientists Do and Setting Up Python

Understand the data science workflow, then set up Python, a virtual environment, and Jupyter with the core libraries.

September 5, 20268 min read