Database Engineering Course

PostgreSQL
Mastery Guide

From your first SELECT to production-grade schemas, indexing strategy, transactions, security, and application integration. Built for engineers who want to understand databases deeply.

PostgreSQL 16 15 Chapters 50+ Queries 3 Projects Fintech Focus
CH 01
Introduction to PostgreSQL
What it is, why it matters, and how to get started
What is PostgreSQL?

PostgreSQL (often called "Postgres") is a free, open-source, object-relational database management system with over 35 years of active development. It's renowned for reliability, standards compliance, and an extensible architecture that supports custom types, functions, and languages.

ACID Compliance

Atomicity, Consistency, Isolation, Durability — all transactions are safe and reliable, even during hardware failure.

Extensibility

Custom data types, operators, functions, full-text search, JSON support, and PostGIS for geospatial data.

Use Cases

Financial systems · SaaS backends · Analytics · Geospatial apps · IoT time-series · Any application requiring data integrity.

FeaturePostgreSQLMySQLSQLite
ACID Compliance✅ Full✅ InnoDB only✅ Partial
JSON Support✅ JSONB (indexed)⚠️ Basic❌ Limited
Full-Text Search✅ Built-in✅ Basic
Window Functions✅ Advanced✅ Basic⚠️ Limited
Best ForComplex apps, fintechWeb apps, CMSEmbedded, mobile
Key Features
Multi-Version Concurrency Control (MVCC)

Readers never block writers. Writers never block readers. Enables high concurrency without locking overhead.

Advanced Indexing

B-tree, Hash, GIN, GiST, BRIN, SP-GiST. Partial, expression, and covering indexes for any query pattern.

Foreign Data Wrappers

Query data in other PostgreSQL instances, MySQL, CSV files, or any external source as if it were a local table.

Logical Replication

Stream changes to read replicas, data warehouses, or downstream systems for real-time reporting and HA setups.

Installation & Setup
terminal — installation
bash
# Ubuntu / Debian
sudo apt update
sudo apt install postgresql postgresql-contrib

# macOS with Homebrew
brew install postgresql@16
brew services start postgresql@16

# Start the psql interactive shell
sudo -u postgres psql

# Check version
SELECT version();

# Cloud: Supabase, Railway, Neon — free PostgreSQL in the cloud
# Connection string format:
# postgresql://user:password@host:5432/dbname
CH 02
Database Fundamentals
Relational concepts, schemas, and PostgreSQL data types
Relational Database Concepts

A relational database organises data into tables (relations). Each table has columns (attributes) and rows (records). Tables relate to each other through keys.

customers
idPK · SERIAL
nameVARCHAR(100)
emailTEXT UNIQUE
created_atTIMESTAMPTZ
orders
idPK · SERIAL
customer_idFK → customers
total_amountNUMERIC(12,2)
statusTEXT
order_items
idPK · SERIAL
order_idFK → orders
product_idFK → products
quantityINTEGER
PostgreSQL Data Types
CategoryTypeExampleNotes
IntegerSMALLINT, INTEGER, BIGINT42, 9876543210Use BIGINT for IDs in large systems
DecimalNUMERIC(p,s), REAL, FLOAT819.99, 3.14NUMERIC for money — no floating point errors
Auto-incrementSERIAL, BIGSERIALauto: 1, 2, 3Or use DEFAULT gen_random_uuid() for UUID
TextVARCHAR(n), TEXT, CHAR(n)'Alice', 'Kigali'TEXT preferred — no performance difference
BooleanBOOLEANTRUE, FALSE, NULLUse for flags and switches
Date/TimeDATE, TIME, TIMESTAMP, TIMESTAMPTZ2024-03-15 10:30:00+02Always use TIMESTAMPTZ in production
JSONJSON, JSONB{"key": "value"}JSONB is indexed and faster for queries
UUIDUUIDa0eebc99-9c0b-4ef8-bb6d-6bb9bd380a11gen_random_uuid() built-in
ArrayINTEGER[], TEXT[]'{1,2,3}', '{"a","b"}'One-dimensional or multi-dimensional
EnumCREATE TYPE … AS ENUM'pending', 'active'Type-safe status fields
CH 03
Basic SQL Operations
CREATE DATABASE, CREATE TABLE, psql commands
Creating & Connecting to Databases
database operations
sql
-- Create a new database
CREATE DATABASE vendorflow
  WITH
  OWNER     = postgres
  ENCODING  = 'UTF8'
  LC_COLLATE = 'en_US.UTF-8';

-- Connect in psql
\c vendorflow

-- Create a schema (namespace for tables)
CREATE SCHEMA IF NOT EXISTS payments;

-- List all databases
\l

-- Show current database
SELECT current_database();

-- Drop a database (must be disconnected first)
DROP DATABASE IF EXISTS test_db;
Creating Tables with Best Practices
schema/customers.sql
sql
-- Custom enum type
CREATE TYPE account_status AS ENUM ('active', 'suspended', 'closed');

-- Customers table — production-grade design
CREATE TABLE customers (
  id           UUID           PRIMARY KEY DEFAULT gen_random_uuid(),
  name         VARCHAR(100)   NOT NULL,
  email        TEXT           NOT NULL UNIQUE,
  phone        VARCHAR(20),
  status       account_status  NOT NULL DEFAULT 'active',
  metadata     JSONB          DEFAULT '{}',
  created_at   TIMESTAMPTZ    NOT NULL DEFAULT NOW(),
  updated_at   TIMESTAMPTZ    NOT NULL DEFAULT NOW()
);

-- Orders table with foreign key
CREATE TABLE orders (
  id           UUID           PRIMARY KEY DEFAULT gen_random_uuid(),
  customer_id  UUID           NOT NULL REFERENCES customers(id) ON DELETE RESTRICT,
  total_amount NUMERIC(12, 2) NOT NULL CHECK (total_amount > 0),
  currency     CHAR(3)        NOT NULL DEFAULT 'RWF',
  status       TEXT           NOT NULL DEFAULT 'pending',
  notes        TEXT,
  created_at   TIMESTAMPTZ    NOT NULL DEFAULT NOW()
);

-- Modify existing table
ALTER TABLE customers ADD COLUMN date_of_birth DATE;
ALTER TABLE customers DROP COLUMN date_of_birth;
ALTER TABLE customers RENAME COLUMN phone TO phone_number;
Essential psql Commands
psql — interactive shell
sql
\l              -- list databases
\c dbname      -- connect to database
\dt            -- list tables in current schema
\dt schema.*  -- list tables in specific schema
\d tablename   -- describe table (columns, types, constraints)
\di            -- list indexes
\dv            -- list views
\du            -- list users and roles
\dn            -- list schemas
\timing        -- toggle query execution time display
\x             -- toggle expanded display (great for wide rows)
\i file.sql   -- execute SQL file
\q             -- quit psql

-- Search pg_catalog for objects
SELECT tablename, schemaname
FROM   pg_catalog.pg_tables
WHERE  schemaname NOT IN ('pg_catalog', 'information_schema');
CH 04
CRUD Operations
INSERT, SELECT, UPDATE, DELETE with real-world patterns
INSERT — Adding Records
crud/insert.sql
sql
-- Single row insert
INSERT INTO customers (name, email, phone)
VALUES ('Alice Mugisha', 'alice@vendorflow.rw', '+250788123456')
RETURNING id, created_at;  -- return generated values

-- Multiple rows in one statement (more efficient)
INSERT INTO customers (name, email) VALUES
  ('Bob Nkurunziza', 'bob@example.rw'),
  ('Claire Uwimana', 'claire@example.rw'),
  ('David Habimana', 'david@example.rw');

-- UPSERT: insert or update on conflict
INSERT INTO customers (email, name)
VALUES ('alice@vendorflow.rw', 'Alice M.')
ON CONFLICT (email) DO UPDATE SET
  name       = EXCLUDED.name,
  updated_at = NOW();

-- Insert from SELECT (copy data between tables)
INSERT INTO customers_archive
SELECT * FROM customers
WHERE  status = 'closed'
  AND  created_at < NOW() - INTERVAL '2 years';
SELECT — Reading & Filtering Data
crud/select.sql
sql
-- Basic SELECT
SELECT id, name, email, status
FROM   customers
WHERE  status = 'active'
ORDER BY created_at DESC
LIMIT  10 OFFSET 0;  -- page 1, 10 per page

-- Aliasing and expressions
SELECT
  id,
  name,
  email,
  UPPER(name)                                     AS name_upper,
  DATE_TRUNC('month', created_at)                AS joined_month,
  NOW() - created_at                               AS account_age,
  CASE status
    WHEN 'active'    THEN '✓ Active'
    WHEN 'suspended' THEN '⚠ Suspended'
    ELSE '✗ Closed'
  END                                             AS status_label
FROM customers
WHERE status != 'closed';

-- Distinct values
SELECT DISTINCT status FROM customers;

-- Count and pagination info
SELECT
  COUNT(*) OVER()   AS total_count,  -- window: total without extra query
  id, name, email
FROM  customers
ORDER BY name
LIMIT  10 OFFSET 20;  -- page 3
Example Output
idnameemailstatus_label
a0ee…bc99Alice Mugishaalice@vendorflow.rw✓ Active
b1ff…cd00Bob Nkurunzizabob@example.rw✓ Active
c2gg…de11Claire Uwimanaclaire@example.rw⚠ Suspended
(3 rows)
UPDATE — Modifying Records
crud/update.sql
sql
-- Update a single record
UPDATE customers
SET
  name       = 'Alice Mugisha-Nkusi',
  updated_at = NOW()
WHERE id = 'a0eebc99-9c0b-4ef8-bb6d-6bb9bd380a11'
RETURNING id, name, updated_at;

-- Update with expression
UPDATE orders
SET    total_amount = total_amount * 1.10  -- apply 10% increase
WHERE  currency = 'USD'
  AND  status   = 'pending'
RETURNING id, total_amount;

-- Update with JOIN using FROM clause
UPDATE orders o
SET    status = 'cancelled'
FROM   customers c
WHERE  o.customer_id = c.id
  AND  c.status     = 'suspended'
  AND  o.status     = 'pending';

-- ⚠️ Always include WHERE — without it, updates ALL rows!
DELETE — Removing Records Safely
crud/delete.sql
sql
-- Preview before deleting (always do this first!)
SELECT * FROM customers
WHERE  status = 'closed'
  AND  created_at < NOW() - INTERVAL '1 year';

-- Then delete
DELETE FROM customers
WHERE  status = 'closed'
  AND  created_at < NOW() - INTERVAL '1 year'
RETURNING id, name, email;  -- see what was deleted

-- Delete with JOIN using USING clause
DELETE FROM orders o
USING  customers c
WHERE  o.customer_id = c.id
  AND  c.status     = 'closed'
  AND  o.status     = 'pending';

-- TRUNCATE: fast delete of ALL rows (no RETURNING, no row-level triggers)
TRUNCATE TABLE temp_import_staging RESTART IDENTITY CASCADE;

-- Soft delete pattern (preferred in production)
ALTER TABLE customers ADD COLUMN deleted_at TIMESTAMPTZ;

UPDATE customers
SET    deleted_at = NOW()
WHERE  id = '...';

-- Query excludes soft-deleted rows
SELECT * FROM customers WHERE deleted_at IS NULL;
⚠️ Golden Rule
Always run a SELECT with the same WHERE clause before executing DELETE or UPDATE. In production systems, prefer soft deletes (deleted_at column) so records can be recovered.
CH 05
Filtering & Querying Data
WHERE operators, logical conditions, and pattern matching
WHERE Clause Operators
filtering/where_operators.sql
sql
-- Comparison operators
SELECT * FROM orders WHERE total_amount =  1000.00;
SELECT * FROM orders WHERE total_amount != 0;
SELECT * FROM orders WHERE total_amount >  5000;
SELECT * FROM orders WHERE total_amount <= 100;

-- BETWEEN (inclusive)
SELECT * FROM orders
WHERE  created_at BETWEEN '2024-01-01' AND '2024-03-31';

-- IN — match a list
SELECT * FROM orders
WHERE  status IN ('pending', 'processing', 'shipped');

-- IS NULL / IS NOT NULL
SELECT * FROM customers WHERE phone_number IS NULL;
SELECT * FROM customers WHERE phone_number IS NOT NULL;

-- Logical operators
SELECT * FROM orders
WHERE  total_amount > 1000
  AND  currency    = 'USD'
  AND  status      != 'cancelled';

SELECT * FROM customers
WHERE  status = 'suspended'
    OR (status = 'active' AND created_at > NOW() - INTERVAL '7 days');
Pattern Matching
filtering/patterns.sql
sql
-- LIKE: % = any chars, _ = single char
SELECT * FROM customers WHERE email LIKE '%@vendorflow.rw';
SELECT * FROM customers WHERE name  LIKE 'Ali_e%';

-- ILIKE: case-insensitive LIKE
SELECT * FROM customers WHERE name ILIKE 'alice%';  -- matches Alice, ALICE, alice

-- SIMILAR TO: SQL regex
SELECT * FROM customers
WHERE  phone_number SIMILAR TO '\+250[0-9]{9}';

-- ~ operator: POSIX regex (most powerful)
SELECT * FROM customers
WHERE  email ~ '^[a-z]+\.[a-z]+@';          -- case-sensitive

SELECT * FROM customers
WHERE  email ~* '^[a-z]+@vendorflow';        -- case-insensitive

-- Full-text search
SELECT id, name, email,
       ts_rank(to_tsvector('english', name), plainto_tsquery('alice')) AS rank
FROM   customers
WHERE  to_tsvector('english', name) @@ plainto_tsquery('alice')
ORDER BY rank DESC;
CH 06
Joins & Relationships
INNER, LEFT, RIGHT, FULL OUTER JOIN and relational schema design
Primary & Foreign Keys
schema/e-commerce.sql
sql
CREATE TABLE products (
  id       UUID            PRIMARY KEY DEFAULT gen_random_uuid(),
  name     VARCHAR(200)    NOT NULL,
  price    NUMERIC(10, 2)  NOT NULL,
  stock    INTEGER         NOT NULL DEFAULT 0
);

CREATE TABLE order_items (
  id         UUID           PRIMARY KEY DEFAULT gen_random_uuid(),
  order_id   UUID           NOT NULL REFERENCES orders(id)   ON DELETE CASCADE,
  product_id UUID           NOT NULL REFERENCES products(id) ON DELETE RESTRICT,
  quantity   INTEGER        NOT NULL CHECK (quantity > 0),
  unit_price NUMERIC(10, 2) NOT NULL,
  UNIQUE (order_id, product_id)  -- composite unique constraint
);
ON DELETE / ON UPDATEBehaviourUse Case
RESTRICTPrevent deletion if referencedProducts with orders
CASCADEDelete/update child rows automaticallyOrder items when order deleted
SET NULLSet FK column to NULLOptional relationships
NO ACTIONLike RESTRICT but deferredDeferred constraint checking
Join Types — Complete Reference
joins/all_join_types.sql
sql
-- ── INNER JOIN: only matching rows ────────────────────────
SELECT
  c.name            AS customer,
  o.id              AS order_id,
  o.total_amount,
  o.status
FROM       customers   c
INNER JOIN orders     o ON o.customer_id = c.id
ORDER BY   o.created_at DESC;

-- ── LEFT JOIN: all customers, even those without orders ───
SELECT
  c.name,
  c.email,
  COUNT(o.id)       AS order_count,
  COALESCE(SUM(o.total_amount), 0) AS total_spent
FROM      customers  c
LEFT JOIN orders    o ON o.customer_id = c.id
GROUP BY  c.id, c.name, c.email
ORDER BY  total_spent DESC;

-- ── FULL OUTER JOIN: all rows from both sides ────────────
SELECT
  c.name      AS customer_name,
  p.name      AS product_name,
  oi.quantity
FROM            customers   c
FULL OUTER JOIN orders      o  ON o.customer_id  = c.id
FULL OUTER JOIN order_items oi ON oi.order_id    = o.id
FULL OUTER JOIN products    p  ON p.id          = oi.product_id;

-- ── MULTI-TABLE JOIN: real order report ──────────────────
SELECT
  c.name                              AS customer,
  o.id                                AS order_ref,
  o.created_at::date                  AS order_date,
  p.name                              AS product,
  oi.quantity,
  oi.unit_price,
  oi.quantity * oi.unit_price         AS line_total,
  o.total_amount
FROM       customers   c
INNER JOIN orders      o  ON o.customer_id  = c.id
INNER JOIN order_items oi ON oi.order_id    = o.id
INNER JOIN products    p  ON p.id          = oi.product_id
ORDER BY   o.created_at DESC, c.name;
CH 07
Aggregate Functions & Reporting
COUNT, SUM, AVG, GROUP BY, HAVING — real-world analytics
Aggregate Functions
analytics/aggregates.sql
sql
-- Basic aggregates
SELECT
  COUNT(*)                          AS total_orders,
  COUNT(DISTINCT customer_id)       AS unique_customers,
  SUM(total_amount)                 AS revenue,
  AVG(total_amount)                 AS avg_order_value,
  MIN(total_amount)                 AS smallest_order,
  MAX(total_amount)                 AS largest_order,
  PERCENTILE_CONT(0.5) WITHIN GROUP
    (ORDER BY total_amount)          AS median_order
FROM   orders
WHERE  status != 'cancelled'
  AND  created_at >= DATE_TRUNC('month', NOW());
Output
total_ordersunique_customersrevenueavg_order_valuemedian_order
3421984,827,500.0014,112.579,800.00
GROUP BY, HAVING & Reporting Queries
analytics/reports.sql
sql
-- Revenue by month
SELECT
  DATE_TRUNC('month', created_at)    AS month,
  COUNT(*)                              AS order_count,
  SUM(total_amount)                    AS monthly_revenue,
  ROUND(AVG(total_amount), 2)         AS avg_order
FROM     orders
WHERE    status != 'cancelled'
GROUP BY DATE_TRUNC('month', created_at)
ORDER BY month DESC;

-- Top 10 customers by spending — HAVING filters groups
SELECT
  c.name,
  c.email,
  COUNT(o.id)                 AS orders,
  SUM(o.total_amount)         AS total_spent
FROM       customers c
INNER JOIN orders    o ON o.customer_id = c.id
WHERE      o.status != 'cancelled'
GROUP BY   c.id, c.name, c.email
HAVING     SUM(o.total_amount) > 100000  -- minimum spend threshold
ORDER BY   total_spent DESC
LIMIT      10;

-- Product performance report
SELECT
  p.name,
  SUM(oi.quantity)               AS units_sold,
  SUM(oi.quantity * oi.unit_price) AS revenue,
  COUNT(DISTINCT o.customer_id)  AS unique_buyers
FROM       products    p
INNER JOIN order_items oi ON oi.product_id = p.id
INNER JOIN orders      o  ON o.id = oi.order_id
WHERE      o.status = 'completed'
GROUP BY   p.id, p.name
ORDER BY   revenue DESC;
CH 08
Indexes & Performance
Index types, creation strategies, EXPLAIN ANALYZE
Creating & Managing Indexes
performance/indexes.sql
sql
-- Basic B-tree index (default) — for equality and range queries
CREATE INDEX idx_orders_customer_id
  ON orders (customer_id);

-- Composite index — for queries filtering on multiple columns
CREATE INDEX idx_orders_status_created
  ON orders (status, created_at DESC);

-- Partial index — only index rows matching a condition
CREATE INDEX idx_orders_pending
  ON orders (created_at)
  WHERE status = 'pending';         -- only index pending orders!

-- Expression index — index on computed value
CREATE INDEX idx_customers_email_lower
  ON customers (LOWER(email));    -- for case-insensitive lookups

-- GIN index — for JSONB, arrays, and full-text search
CREATE INDEX idx_customers_metadata
  ON customers USING GIN (metadata);

-- Covering index — avoid table lookup entirely
CREATE INDEX idx_orders_covering
  ON orders (customer_id, status) INCLUDE (total_amount, created_at);

-- CONCURRENTLY — create without locking the table (for production)
CREATE INDEX CONCURRENTLY idx_orders_amount
  ON orders (total_amount);

-- List all indexes on a table
SELECT indexname, indexdef
FROM   pg_indexes
WHERE  tablename = 'orders';

-- Drop index
DROP INDEX CONCURRENTLY idx_orders_amount;
EXPLAIN ANALYZE — Reading Query Plans
performance/explain.sql
sql
-- EXPLAIN: show query plan without executing
EXPLAIN
SELECT * FROM orders WHERE customer_id = '...';

-- EXPLAIN ANALYZE: execute AND show actual timings
EXPLAIN (ANALYZE, BUFFERS, FORMAT JSON)
SELECT
  c.name, SUM(o.total_amount)
FROM       customers c
INNER JOIN orders    o ON o.customer_id = c.id
GROUP BY   c.id;

-- What to look for in the plan:
-- ✅ Index Scan / Bitmap Index Scan = index being used
-- ⚠️  Seq Scan on large table = missing index
-- ⚠️  Large "rows" vs "actual rows" = stale statistics
-- ✅ Nested Loop / Hash Join = efficient join strategy

-- Update statistics (run after bulk loads)
ANALYZE orders;
ANALYZE customers;
💡 Index Design Rules
1. Index columns used in WHERE, JOIN ON, and ORDER BY clauses. 2. Composite indexes: put high-selectivity columns first. 3. Partial indexes for filtered queries. 4. Never index columns with < 3 distinct values (booleans). 5. Use CONCURRENTLY in production to avoid locks.
CH 09
Constraints & Data Integrity
Enforce data quality at the database level
All Constraint Types
schema/constraints.sql
sql
CREATE TABLE payments (
  -- NOT NULL: field must always have a value
  id              UUID           PRIMARY KEY DEFAULT gen_random_uuid(),
  order_id        UUID           NOT NULL,
  amount          NUMERIC(12,2)  NOT NULL,
  currency        CHAR(3)        NOT NULL DEFAULT 'RWF',

  -- UNIQUE: no duplicate values in column
  reference       TEXT           NOT NULL UNIQUE,

  -- CHECK: custom validation rule
  status          TEXT           NOT NULL CHECK (status IN ('pending', 'success', 'failed')),
  fee_amount      NUMERIC(10,2)  NOT NULL DEFAULT 0 CHECK (fee_amount >= 0),

  -- FOREIGN KEY with explicit action
  provider        TEXT           NOT NULL CHECK (provider IN ('stripe', 'mtn_momo', 'airtel')),
  created_at      TIMESTAMPTZ    NOT NULL DEFAULT NOW(),

  -- Table-level CHECK constraint (multi-column)
  CONSTRAINT chk_amount_positive CHECK (amount > 0 AND amount >= fee_amount),

  -- Named foreign key constraint
  CONSTRAINT fk_payment_order
    FOREIGN KEY (order_id)
    REFERENCES orders(id)
    ON DELETE RESTRICT ON UPDATE CASCADE
);

-- Add constraint to existing table
ALTER TABLE customers
  ADD CONSTRAINT chk_email_format
  CHECK (email ~ '^[^@]+@[^@]+\.[^@]+$');

-- Add NOT NULL to existing column
ALTER TABLE customers
  ALTER COLUMN name SET NOT NULL;
CH 10
Advanced PostgreSQL Features
JSONB, CTEs, Window Functions, and Views
JSON & JSONB Queries
advanced/jsonb.sql
sql
-- Insert with JSONB metadata
INSERT INTO customers (name, email, metadata)
VALUES ('Alice', 'alice@test.rw', '{"tier": "gold", "country": "RW", "preferences": {"notifications": true}}');

-- Access JSONB fields
SELECT
  name,
  metadata ->> 'tier'                    AS tier,       -- text value
  metadata -> 'preferences'             AS prefs,      -- JSON value
  metadata -> 'preferences' ->> 'notifications' AS notifs
FROM customers
WHERE metadata @> '{"tier": "gold"}';         -- @> contains operator

-- Filter by JSONB value
SELECT * FROM customers
WHERE  metadata ->> 'country' = 'RW'
  AND  (metadata -> 'preferences' ->> 'notifications')::boolean = true;

-- JSONB key exists
SELECT * FROM customers WHERE metadata ? 'tier';

-- Update JSONB field
UPDATE customers
SET    metadata = metadata || '{"tier": "platinum"}'::jsonb
WHERE  id = '...';

-- Aggregate JSONB to array
SELECT jsonb_agg(jsonb_build_object('name', name, 'email', email))
FROM   customers
WHERE  status = 'active';
Common Table Expressions (CTEs)
advanced/cte.sql
sql
-- ── BASIC CTE: customer revenue summary ─────────────────
WITH customer_revenue AS (
  SELECT
    c.id,
    c.name,
    c.email,
    SUM(o.total_amount) AS total_spent,
    COUNT(o.id)          AS order_count
  FROM       customers c
  INNER JOIN orders    o ON o.customer_id = c.id
  WHERE      o.status = 'completed'
  GROUP BY   c.id, c.name, c.email
)
SELECT
  name,
  email,
  order_count,
  total_spent,
  CASE
    WHEN total_spent >= 500000 THEN 'Platinum'
    WHEN total_spent >= 100000 THEN 'Gold'
    ELSE                            'Standard'
  END AS tier
FROM  customer_revenue
ORDER BY total_spent DESC;

-- ── CHAINED CTEs ────────────────────────────────────────
WITH
  monthly_revenue AS (
    SELECT
      DATE_TRUNC('month', created_at) AS month,
      SUM(total_amount)               AS revenue
    FROM   orders
    WHERE  status != 'cancelled'
    GROUP BY 1
  ),
  growth AS (
    SELECT
      month,
      revenue,
      LAG(revenue) OVER (ORDER BY month) AS prev_month,
      revenue - LAG(revenue) OVER (ORDER BY month) AS change
    FROM monthly_revenue
  )
SELECT
  month,
  revenue,
  change,
  ROUND(change::numeric / prev_month * 100, 2) AS growth_pct
FROM  growth
WHERE prev_month IS NOT NULL
ORDER BY month;
Window Functions — Ranking & Running Totals
advanced/window_functions.sql
sql
-- Rank customers by spend (no gaps: DENSE_RANK)
SELECT
  c.name,
  SUM(o.total_amount)  AS total_spent,
  RANK()       OVER (ORDER BY SUM(o.total_amount) DESC) AS rank,
  DENSE_RANK() OVER (ORDER BY SUM(o.total_amount) DESC) AS dense_rank,
  NTILE(4)     OVER (ORDER BY SUM(o.total_amount) DESC) AS quartile
FROM       customers c
INNER JOIN orders    o ON o.customer_id = c.id
GROUP BY   c.id, c.name;

-- Running total (cumulative sum)
SELECT
  created_at::date       AS date,
  total_amount,
  SUM(total_amount) OVER (
    ORDER BY created_at
    ROWS BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW
  )                      AS running_total
FROM   orders
WHERE  status = 'completed'
ORDER BY created_at;

-- Month-over-month comparison with LAG
SELECT
  DATE_TRUNC('month', created_at)         AS month,
  SUM(total_amount)                        AS revenue,
  LAG(SUM(total_amount)) OVER (ORDER BY DATE_TRUNC('month', created_at))
                                           AS prev_month_revenue
FROM     orders
WHERE    status = 'completed'
GROUP BY DATE_TRUNC('month', created_at)
ORDER BY month;

-- Row number within partition (top 3 orders per customer)
WITH ranked AS (
  SELECT
    customer_id,
    total_amount,
    created_at,
    ROW_NUMBER() OVER (
      PARTITION BY customer_id
      ORDER BY total_amount DESC
    ) AS rn
  FROM orders
)
SELECT * FROM ranked WHERE rn <= 3;
Views & Materialized Views
advanced/views.sql
sql
-- Regular view: stored query, always fresh
CREATE OR REPLACE VIEW v_active_customers AS
SELECT
  id, name, email, created_at,
  metadata ->> 'tier' AS tier
FROM  customers
WHERE status = 'active'
  AND deleted_at IS NULL;

-- Query the view like a table
SELECT * FROM v_active_customers WHERE tier = 'gold';

-- Materialized view: cached result, manually refreshed
CREATE MATERIALIZED VIEW mv_customer_stats AS
SELECT
  c.id,
  c.name,
  COUNT(o.id)          AS order_count,
  SUM(o.total_amount)  AS total_spent,
  MAX(o.created_at)    AS last_order
FROM      customers c
LEFT JOIN orders    o ON o.customer_id = c.id
GROUP BY  c.id, c.name
WITH DATA;  -- populate immediately

-- Create index on materialized view
CREATE INDEX ON mv_customer_stats (total_spent DESC);

-- Refresh materialized view (schedule with pg_cron)
REFRESH MATERIALIZED VIEW CONCURRENTLY mv_customer_stats;
CH 11
Transactions & Concurrency
BEGIN/COMMIT/ROLLBACK, isolation levels, and safe concurrent operations
Transactions — All or Nothing
transactions/payment_tx.sql
sql
-- Transfer funds between wallets — must be atomic
BEGIN;

  -- Debit sender
  UPDATE wallets
  SET    balance = balance - 50000
  WHERE  user_id = 'sender-id'
    AND  balance >= 50000     -- check sufficient funds atomically
    AND  currency = 'RWF';

  -- Verify debit succeeded
  DO $$
  BEGIN
    IF NOT FOUND THEN
      RAISE EXCEPTION 'Insufficient funds';
    END IF;
  END $$;

  -- Credit recipient
  UPDATE wallets
  SET    balance = balance + 50000
  WHERE  user_id = 'recipient-id'
    AND  currency = 'RWF';

  -- Record the transaction
  INSERT INTO payment_ledger (from_user, to_user, amount, reference)
  VALUES ('sender-id', 'recipient-id', 50000, 'TXN-2024-001');

COMMIT;  -- everything succeeds or nothing happens

-- If anything fails, use ROLLBACK
-- ROLLBACK; -- undoes all changes since BEGIN

-- SAVEPOINT: partial rollback
BEGIN;
  INSERT INTO orders (...) VALUES (...);
  SAVEPOINT after_order;
  INSERT INTO order_items (...) VALUES (...);
  -- If item insert fails:
  ROLLBACK TO after_order;  -- keeps the order, rolls back items
COMMIT;
Isolation Levels
LevelDirty ReadNon-repeatable ReadPhantom ReadUse Case
READ UNCOMMITTEDPossiblePossiblePossibleNot used in PostgreSQL
READ COMMITTED (default)✅ SafePossiblePossibleMost OLTP workloads
REPEATABLE READ✅ Safe✅ Safe✅ Safe*Reports, analytics
SERIALIZABLE✅ Safe✅ Safe✅ SafeFinancial transactions
transactions/isolation.sql
sql
-- Set isolation level for a transaction
BEGIN TRANSACTION ISOLATION LEVEL SERIALIZABLE;
  -- Safe for financial operations
  UPDATE wallets SET balance = balance - 100 WHERE id = '...';
COMMIT;

-- FOR UPDATE: lock rows to prevent concurrent modifications
BEGIN;
  SELECT * FROM wallets
  WHERE  user_id = '...'
  FOR UPDATE;         -- row-level lock

  UPDATE wallets SET balance = balance - 100 WHERE user_id = '...';
COMMIT;
CH 12
User Management & Security
Roles, permissions, row-level security
Creating Users & Roles
security/users.sql
sql
-- Create role (group of permissions)
CREATE ROLE app_reader;
CREATE ROLE app_writer;
CREATE ROLE app_admin;

-- Create user with password
CREATE USER vendorflow_app
  WITH PASSWORD 'strong-password-here'
  LOGIN
  CONNECTION LIMIT 50;

-- Assign role to user
GRANT app_reader TO vendorflow_app;

-- Create read-only analytics user
CREATE USER analytics_user
  WITH PASSWORD 'analytics-pass'
  LOGIN
  VALID UNTIL '2025-12-31';  -- expiry date

GRANT app_reader TO analytics_user;
Granting Permissions & Row-Level Security
security/permissions.sql
sql
-- Grant schema access
GRANT USAGE ON SCHEMA public TO app_reader;
GRANT USAGE ON SCHEMA public TO app_writer;

-- Reader: SELECT only
GRANT SELECT ON ALL TABLES IN SCHEMA public TO app_reader;

-- Writer: CRUD on specific tables
GRANT SELECT, INSERT, UPDATE, DELETE ON
  customers, orders, order_items
TO app_writer;

-- Grant sequence (for SERIAL/BIGSERIAL inserts)
GRANT USAGE ON ALL SEQUENCES IN SCHEMA public TO app_writer;

-- Revoke permission
REVOKE DELETE ON orders FROM app_writer;

-- ROW LEVEL SECURITY: users only see their own data
ALTER TABLE orders ENABLE ROW LEVEL SECURITY;

CREATE POLICY orders_user_isolation
  ON orders
  USING (customer_id = current_setting('app.current_user_id')::uuid);

-- Set the app context variable
SET app.current_user_id = 'user-uuid-here';

-- Admin bypasses RLS
ALTER TABLE orders FORCE ROW LEVEL SECURITY;
CREATE POLICY admin_all ON orders TO app_admin USING (true);
CH 13
Backup & Restore
pg_dump, pg_restore, and disaster recovery strategies
pg_dump & Restore
terminal — backup & restore
bash
# Dump database to SQL file
pg_dump -U postgres -d vendorflow -f backup.sql

# Dump in custom format (recommended — smaller, faster restore)
pg_dump -U postgres -d vendorflow -Fc -f backup.dump

# Dump with compression
pg_dump -U postgres -d vendorflow -Fc -Z 9 -f backup.dump

# Dump only specific tables
pg_dump -U postgres -d vendorflow -t customers -t orders -f tables.sql

# Dump only schema (no data)
pg_dump -U postgres -d vendorflow -s -f schema_only.sql

# Dump only data (no schema)
pg_dump -U postgres -d vendorflow -a -f data_only.sql

# Restore from SQL file
psql -U postgres -d vendorflow_new -f backup.sql

# Restore from custom format
pg_restore -U postgres -d vendorflow_new -Fc backup.dump

# Restore specific table
pg_restore -U postgres -d vendorflow_new -t customers backup.dump

# Dump all databases
pg_dumpall -U postgres -f all_databases.sql

# Schedule daily backup (crontab)
# 0 2 * * * pg_dump -U postgres -d vendorflow -Fc -f /backups/vendorflow_$(date +\%Y\%m\%d).dump
CH 14
Application Integration
Node.js with Prisma, Python, parameterised queries, connection pooling
Node.js with Prisma ORM
prisma/schema.prisma
prisma
datasource db {
  provider = "postgresql"
  url      = env("DATABASE_URL")
}

generator client {
  provider = "prisma-client-js"
}

model Customer {
  id        String   @id @default(dbgenerated("gen_random_uuid()")) @db.Uuid
  name      String   @db.VarChar(100)
  email     String   @unique
  status    String   @default("active")
  metadata  Json     @default("{}")
  createdAt DateTime @default(now()) @map("created_at") @db.Timestamptz
  updatedAt DateTime @updatedAt @map("updated_at") @db.Timestamptz
  orders    Order[]

  @@map("customers")
}

model Order {
  id          String      @id @default(dbgenerated("gen_random_uuid()")) @db.Uuid
  customerId  String      @map("customer_id") @db.Uuid
  totalAmount Decimal     @map("total_amount") @db.Decimal(12, 2)
  status      String      @default("pending")
  createdAt   DateTime    @default(now()) @map("created_at") @db.Timestamptz
  customer    Customer    @relation(fields: [customerId], references: [id])
  items       OrderItem[]

  @@map("orders")
}
src/repositories/customerRepository.js
js
const { PrismaClient } = require('@prisma/client');
const prisma = new PrismaClient();

// ── PRISMA CRUD ──────────────────────────────────────────
const findAll = async ({ page = 1, limit = 10, status }) => {
  const [customers, total] = await prisma.$transaction([
    prisma.customer.findMany({
      where: { status, deletedAt: null },
      select: { id: true, name: true, email: true, status: true, createdAt: true },
      orderBy: { createdAt: 'desc' },
      skip:  (page - 1) * limit,
      take:  limit,
    }),
    prisma.customer.count({ where: { status, deletedAt: null } }),
  ]);
  return { customers, total, pages: Math.ceil(total / limit) };
};

// ── RAW SQL for complex queries ──────────────────────────
const getTopCustomers = async (limit = 10) => {
  return prisma.$queryRaw`
    SELECT
      c.id, c.name, c.email,
      COUNT(o.id)::int         AS order_count,
      SUM(o.total_amount)      AS total_spent
    FROM      customers  c
    INNER JOIN orders    o ON o.customer_id = c.id
    WHERE     o.status = 'completed'
    GROUP BY  c.id, c.name, c.email
    ORDER BY  total_spent DESC
    LIMIT     ${limit}
  `;
};

// ── PARAMETERISED for security ───────────────────────────
const findByEmail = async (email) => {
  return prisma.$queryRaw`
    SELECT * FROM customers WHERE email = ${email}
  `;
  // Prisma automatically parameterises template literals — safe!
};

module.exports = { findAll, getTopCustomers, findByEmail };
Python with psycopg2 & SQLAlchemy
db/repository.py
python
import psycopg2
from psycopg2.extras import RealDictCursor
from contextlib import contextmanager
import os

DATABASE_URL = os.getenv("DATABASE_URL")

@contextmanager
def get_db():
    """Context manager — auto-closes connection."""
    conn = psycopg2.connect(DATABASE_URL)
    try:
        yield conn
        conn.commit()
    except Exception:
        conn.rollback()
        raise
    finally:
        conn.close()

def get_customers(status: str = "active", limit: int = 10) -> list:
    """Fetch customers — parameterised query prevents SQL injection."""
    with get_db() as conn:
        with conn.cursor(cursor_factory=RealDictCursor) as cur:
            cur.execute(
                """
                SELECT id, name, email, status, created_at
                FROM   customers
                WHERE  status = %s
                ORDER BY created_at DESC
                LIMIT  %s
                """,
                (status, limit)  # ← NEVER use f-strings for user input!
            )
            return cur.fetchall()

def create_order(customer_id: str, items: list, total: float):
    """Create order and items atomically."""
    with get_db() as conn:
        with conn.cursor() as cur:
            # Insert order
            cur.execute(
                """
                INSERT INTO orders (customer_id, total_amount, status)
                VALUES (%s, %s, 'pending')
                RETURNING id
                """,
                (customer_id, total)
            )
            order_id = cur.fetchone()[0]

            # Insert items
            cur.executemany(
                """
                INSERT INTO order_items (order_id, product_id, quantity, unit_price)
                VALUES (%s, %s, %s, %s)
                """,
                [(order_id, i["product_id"], i["qty"], i["price"]) for i in items]
            )
            return order_id
CH 15
Mini Projects
From student database to full fintech schema with VendorFlow
Project Overview
Beginner
Student Database

Students, courses, enrolments, grades. Basic CRUD, joins, and report queries.

CREATE TABLE JOINS GROUP BY
Intermediate
Inventory System

Products, warehouses, stock movements, low-stock alerts, and supplier management.

Constraints Triggers Views Indexes
Advanced
VendorFlow Fintech

Multi-vendor payments, wallets, ledger, fraud detection, and transaction analytics.

Transactions RLS Window Fns CTEs JSONB
VendorFlow — Complete Fintech Schema
migrations/vendorflow_schema.sql
sql
-- VendorFlow: multi-vendor payment platform schema

CREATE TYPE user_role       AS ENUM ('customer', 'vendor', 'admin');
CREATE TYPE payment_status  AS ENUM ('pending', 'processing', 'success', 'failed', 'refunded');
CREATE TYPE tx_type         AS ENUM ('debit', 'credit', 'fee', 'refund');

CREATE TABLE users (
  id           UUID        PRIMARY KEY DEFAULT gen_random_uuid(),
  email        TEXT        NOT NULL UNIQUE,
  phone        VARCHAR(20) UNIQUE,
  role         user_role   NOT NULL DEFAULT 'customer',
  kyc_status   TEXT        NOT NULL DEFAULT 'pending',
  metadata     JSONB       NOT NULL DEFAULT '{}',
  created_at   TIMESTAMPTZ NOT NULL DEFAULT NOW()
);

CREATE TABLE wallets (
  id          UUID           PRIMARY KEY DEFAULT gen_random_uuid(),
  user_id     UUID           NOT NULL UNIQUE REFERENCES users(id),
  balance     NUMERIC(15, 2) NOT NULL DEFAULT 0 CHECK (balance >= 0),
  currency    CHAR(3)        NOT NULL DEFAULT 'RWF',
  is_frozen   BOOLEAN        NOT NULL DEFAULT false,
  updated_at  TIMESTAMPTZ    NOT NULL DEFAULT NOW()
);

CREATE TABLE payments (
  id           UUID           PRIMARY KEY DEFAULT gen_random_uuid(),
  reference    TEXT           NOT NULL UNIQUE,
  sender_id    UUID           NOT NULL REFERENCES users(id),
  recipient_id UUID           NOT NULL REFERENCES users(id),
  amount       NUMERIC(15, 2) NOT NULL CHECK (amount > 0),
  fee          NUMERIC(10, 2) NOT NULL DEFAULT 0,
  currency     CHAR(3)        NOT NULL DEFAULT 'RWF',
  status       payment_status  NOT NULL DEFAULT 'pending',
  provider     TEXT           NOT NULL,
  metadata     JSONB          NOT NULL DEFAULT '{}',
  created_at   TIMESTAMPTZ    NOT NULL DEFAULT NOW(),
  CONSTRAINT   chk_different_parties CHECK (sender_id != recipient_id)
);

CREATE TABLE ledger (
  id          BIGSERIAL      PRIMARY KEY,
  payment_id  UUID           NOT NULL REFERENCES payments(id),
  wallet_id   UUID           NOT NULL REFERENCES wallets(id),
  type        tx_type        NOT NULL,
  amount      NUMERIC(15, 2) NOT NULL,
  balance_before NUMERIC(15, 2) NOT NULL,
  balance_after  NUMERIC(15, 2) NOT NULL,
  created_at  TIMESTAMPTZ    NOT NULL DEFAULT NOW()
);

-- Performance indexes
CREATE INDEX idx_payments_sender     ON payments (sender_id,    created_at DESC);
CREATE INDEX idx_payments_recipient  ON payments (recipient_id, created_at DESC);
CREATE INDEX idx_payments_status     ON payments (status)  WHERE status = 'pending';
CREATE INDEX idx_ledger_wallet       ON ledger   (wallet_id, created_at DESC);
CREATE INDEX idx_payments_metadata   ON payments USING GIN (metadata);

-- Fraud detection query: high-velocity transactions
SELECT
  sender_id,
  COUNT(*)                                  AS tx_count,
  SUM(amount)                               AS total_sent,
  MAX(amount)                               AS largest_tx
FROM   payments
WHERE  created_at >= NOW() - INTERVAL '1 hour'
  AND  status != 'failed'
GROUP BY sender_id
HAVING
  COUNT(*) > 10           -- more than 10 transactions per hour
  OR SUM(amount) > 5000000 -- or over 5M RWF sent
ORDER BY total_sent DESC;
✅ Production-Ready Checklist
Always use TIMESTAMPTZ (not TIMESTAMP) · UUID primary keys for distributed systems · NUMERIC(p,s) for money — never FLOAT · Parameterised queries everywhere · RETURNING clauses on INSERT/UPDATE/DELETE · Partial indexes for status filtering · pg_dump scheduled backups · Row-Level Security for multi-tenant data · EXPLAIN ANALYZE before deploying new queries · CONCURRENTLY for production index creation