Amazon Redshift Module 1: Architecture and Setting Up Serverless
Introduction
Amazon Redshift is AWS's cloud data warehouse, built to run analytical SQL over very large tables quickly. This first module of the eight-module Redshift course explains how it works, helps you choose between Serverless and provisioned clusters, and walks through creating a small Serverless environment with spending limits.
Prerequisites
- An AWS account (use a sandbox)
- Basic SQL
Step-by-Step Guide
Step 1: Learn the architecture
Redshift is a massively parallel processing (MPP) system. Queries are split into pieces that run in parallel on many compute nodes, and a leader node plans the work and combines results. Data is stored by column, which compresses well and lets a query read only the columns it needs. With RA3 nodes and Serverless, storage is managed separately from compute (Redshift managed storage on S3), so you scale them independently.
Client (BI tool / SQL editor)
|
Leader node -- plans the query, coordinates
|
Compute nodes (slices) -- each processes part of the data in parallel
|
Columnar storage (Redshift managed storage)Step 2: Choose Serverless or provisioned
Redshift Serverless needs no cluster to manage: you set a base capacity measured in Redshift Processing Units (RPUs), it scales with the workload, and you pay for the compute used, billed per second, plus storage. It suits variable or unpredictable workloads and getting started. Provisioned clusters, built from nodes you choose, can be cheaper for steady, heavy, always-on workloads, especially with reserved pricing. Minimum base capacities and prices change, so check the current Redshift pricing page.
Serverless no clusters, pay per RPU-second used, scales automatically
Provisioned you pick node type and count, pay per node-hour, reserved pricing availableStep 3: Create a namespace and workgroup
In Serverless, a namespace holds your data objects (databases, users, encryption), and a workgroup holds compute settings (base capacity, network). In the console, use the Redshift Serverless creation wizard and choose the smallest base capacity offered. Keep the default VPC for learning and do not make the workgroup publicly accessible unless you need it.
# Equivalent AWS CLI sketch (the console wizard is easier for a first time)
aws redshift-serverless create-namespace --namespace-name lab-namespace \
--admin-username awsuser --admin-user-password 'Use-A-Strong-Password-1' --db-name dev
aws redshift-serverless create-workgroup --workgroup-name lab-workgroup \
--namespace-name lab-namespace --base-capacity 8Step 4: Set a usage limit before you run anything
Add a usage limit that caps the RPU-hours the workgroup can use per day, week, or month, with an action such as logging or turning off user queries when it is reached. This is the single most important guardrail for a learning environment. Check the supported base capacity values for your region.
aws redshift-serverless create-usage-limit \
--resource-arn <your-workgroup-arn> \
--usage-type serverless-compute \
--amount 20 --period daily \
--breach-action emit-metricStep 5: Connect and run your first query
Open the Redshift query editor v2 from the console, connect to the workgroup, and run a simple query. Redshift speaks a PostgreSQL-compatible dialect with its own extensions, and you can also connect from tools such as DBeaver or from Python using the Redshift Data API or the redshift_connector library.
SELECT current_database(), current_user, version();
CREATE TABLE hello (id int, msg varchar(50));
INSERT INTO hello VALUES (1, 'Redshift works');
SELECT * FROM hello;
DROP TABLE hello;FAQs
- Is Redshift the same as PostgreSQL?
- No. Redshift began from a PostgreSQL fork and uses a similar SQL dialect, but it is a different, columnar, distributed engine. Some PostgreSQL features are missing and some Redshift features, such as distribution keys, do not exist in PostgreSQL.
- Will Redshift Serverless cost money while idle?
- Compute is billed only while queries run, but storage is billed continuously. Always delete your lab namespace and workgroup when you finish.
Summary
Redshift is a columnar MPP data warehouse. Serverless suits variable workloads with RPU-second billing, provisioned suits steady heavy use. Create a namespace and workgroup, set a usage limit first, and connect with query editor v2.
Related tutorials
- Databricks Module 8: SQL Warehouses, Performance, Cost, and a First Look at MLflow
- Databricks Module 7: Jobs, Orchestration, and Deploying with Git and Bundles
- Databricks Module 6: Declarative Pipelines and the Medallion Architecture
- Databricks Module 5: Ingesting Files with Auto Loader and COPY INTO
- Databricks Module 4: Unity Catalog, Volumes, and Governance
Comments
Log in or create a free account to join the discussion.