🗄️ Module 5 of 6
Advanced SQL
Window functions, CTEs, query optimisation, and the patterns every senior analyst uses daily. This is what separates a junior analyst from someone who can answer any data question.
1. Why Advanced SQL Matters
📊Window Functions
Running totals, rankings, period-over-period comparisons — without GROUP BY collapsing your rows. The most interview-tested advanced SQL topic.
🏗️CTEs
Break complex queries into named, readable steps. The difference between a query that takes 10 minutes to understand and one that's instantly clear.
⚡Optimisation
Write SQL that doesn't destroy performance on 100M-row tables. What gets you respected by the data engineering team.
🔁Patterns
Deduplication, sessionisation, funnel analysis — recurring patterns that appear in nearly every analytics job.
2. Window Functions — The Anatomy Must Know
A window function performs a calculation across rows related to the current row — without collapsing those rows like GROUP BY does.
SQL — Window function anatomyFUNCTION_NAME(column) OVER (
PARTITION BY group_column -- reset window for each group (optional)
ORDER BY sort_column -- order rows within window (required for ranking)
ROWS BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW -- frame (optional)
)
📌 PARTITION BY vs GROUP BY
GROUP BY collapses all rows in a group into one result row. PARTITION BY keeps ALL rows but computes the function within each partition. To see every order AND its customer's total spend — use a window function.
3. Ranking Functions Must Know
SQL — Rank customers by lifetime valueSELECT customer_id, SUM(amount) AS ltv,
ROW_NUMBER() OVER (ORDER BY SUM(amount) DESC) AS row_num, -- always unique
RANK() OVER (ORDER BY SUM(amount) DESC) AS rank_pos, -- gaps after ties: 1,1,3
DENSE_RANK() OVER (ORDER BY SUM(amount) DESC) AS dense_pos -- no gaps: 1,1,2
FROM orders WHERE status = 'Delivered' GROUP BY customer_id;
| Function | Ties? | Use When |
| ROW_NUMBER() | Always unique — no ties possible | Deduplication, picking exactly top N rows |
| RANK() | Same rank, then gaps (1,1,3) | Sports-style leaderboards |
| DENSE_RANK() | Same rank, no gaps (1,1,2) | Most analyst use cases |
| NTILE(n) | Divides into n equal buckets | Quartile/decile analysis |
4. Aggregate Window Functions Must Know
SQL — Running total + 3-day moving averageSELECT order_date, SUM(amount) AS daily_rev,
SUM(SUM(amount)) OVER (ORDER BY order_date ROWS BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW) AS running_total,
AVG(SUM(amount)) OVER (ORDER BY order_date ROWS BETWEEN 2 PRECEDING AND CURRENT ROW) AS moving_avg_3d
FROM orders WHERE status = 'Delivered'
GROUP BY order_date ORDER BY order_date;
5. LAG & LEAD — Period-Over-Period Must Know
SQL — Month-over-month revenue growthWITH monthly AS (
SELECT DATE_TRUNC('month', order_date) AS month, SUM(amount) AS revenue
FROM orders WHERE status = 'Delivered' GROUP BY 1
)
SELECT month, revenue,
LAG(revenue) OVER (ORDER BY month) AS prev_month,
ROUND(100.0*(revenue - LAG(revenue) OVER (ORDER BY month))
/ NULLIF(LAG(revenue) OVER (ORDER BY month), 0), 1) AS mom_growth_pct
FROM monthly ORDER BY month;
📌 NULLIF() to Avoid Division by Zero
NULLIF(value, 0) returns NULL instead of 0 — dividing by NULL returns NULL (not an error), which is correct for a missing prior period.
6. CTEs — Deep Dive & Chained Patterns Must Know
SQL — Full analytical pipeline: LTV with MoM growthWITH
monthly_spend AS (
SELECT customer_id, DATE_TRUNC('month', order_date) AS month, SUM(amount) AS rev
FROM orders WHERE status = 'Delivered' GROUP BY 1,2
),
with_growth AS (
SELECT *, ROUND(100.0*(rev - LAG(rev) OVER (PARTITION BY customer_id ORDER BY month))
/ NULLIF(LAG(rev) OVER (PARTITION BY customer_id ORDER BY month), 0), 1) AS mom_pct
FROM monthly_spend
),
ltv AS (
SELECT customer_id, SUM(rev) AS total_ltv,
DENSE_RANK() OVER (ORDER BY SUM(rev) DESC) AS ltv_rank
FROM monthly_spend GROUP BY 1
)
SELECT c.name, c.city, l.total_ltv, l.ltv_rank, g.month, g.rev, g.mom_pct
FROM with_growth g
JOIN ltv l ON g.customer_id = l.customer_id
JOIN customers c ON g.customer_id = c.customer_id
ORDER BY l.ltv_rank, g.month;
7. Recursive CTEs Pro Level
SQL — Generate every date in Q1 2024WITH RECURSIVE date_series AS (
SELECT DATE('2024-01-01') AS dt
UNION ALL
SELECT dt + INTERVAL 1 DAY FROM date_series WHERE dt < '2024-03-31'
)
SELECT ds.dt, COALESCE(SUM(o.amount),0) AS revenue
FROM date_series ds
LEFT JOIN orders o ON ds.dt = o.order_date AND o.status = 'Delivered'
GROUP BY ds.dt ORDER BY ds.dt;
8. Query Optimisation
❌ SLOW — Correlated Subquery
SELECT customer_id, amount
FROM orders o1
WHERE amount > (
SELECT AVG(amount) FROM orders o2
WHERE o2.customer_id = o1.customer_id
);
-- Runs once per row. 1M rows = 1M executions
✅ FAST — CTE Pre-aggregation
WITH avg_c AS (
SELECT customer_id, AVG(amount) AS avg_amt
FROM orders GROUP BY 1
)
SELECT o.customer_id, o.amount
FROM orders o JOIN avg_c a
ON o.customer_id = a.customer_id
WHERE o.amount > a.avg_amt;
-- CTE runs ONCE. Then fast JOIN
| Rule | Why It Matters |
| Filter early — WHERE before JOIN | Join smaller filtered tables, not full tables |
| Never SELECT * in production | Columnar storage scans every column — very expensive |
| Replace correlated subqueries with CTE+JOIN | Subqueries run once per row; CTEs run once total |
| Use date partition filters | On warehouses, skips entire file partitions |
| Use EXPLAIN / EXPLAIN ANALYZE | See what the database is actually doing |
9. Common SQL Patterns
Deduplication — Keep Only the Latest Record
SQLWITH deduped AS (
SELECT *, ROW_NUMBER() OVER (PARTITION BY order_id ORDER BY updated_at DESC) AS rn
FROM order_events
)
SELECT * FROM deduped WHERE rn = 1;
Funnel Analysis
SQL — E-commerce funnelSELECT
COUNT(DISTINCT CASE WHEN event='product_viewed' THEN user_id END) AS viewers,
COUNT(DISTINCT CASE WHEN event='add_to_cart' THEN user_id END) AS cart_adders,
COUNT(DISTINCT CASE WHEN event='order_placed' THEN user_id END) AS purchasers
FROM user_events WHERE event_date >= '2024-01-01';
10. Quiz
Question 1 of 4
What's the key difference between RANK() and DENSE_RANK() with ties?
A
RANK() is faster to compute
B
RANK() leaves gaps after ties (1,1,3); DENSE_RANK() doesn't (1,1,2)
C
DENSE_RANK() requires ORDER BY; RANK() doesn't
D
They produce identical results
✅ Correct! RANK() skips the next rank after ties: (1,1,3). DENSE_RANK() continues consecutively: (1,1,2). Use DENSE_RANK() in most analyst contexts.
❌ RANK() leaves gaps (1,1,3). DENSE_RANK() doesn't (1,1,2). Both require ORDER BY in the OVER clause.
Question 2 of 4
You want each order's amount as % of that customer's total lifetime spend, while keeping individual order rows. Which approach?
A
GROUP BY customer_id and divide each amount by SUM(amount)
B
SUM(amount) OVER (PARTITION BY customer_id) to get total, then divide each row's amount
C
A correlated subquery in the SELECT clause
D
This is not possible in SQL
✅ Correct! SUM() OVER (PARTITION BY customer_id) computes the total per customer without collapsing rows. GROUP BY would lose individual order data.
❌ GROUP BY collapses all orders per customer — losing individual rows. Use a window function: SUM(amount) OVER (PARTITION BY customer_id).
Question 3 of 4
Most efficient pattern to deduplicate a table, keeping only the latest record per order_id?
A
GROUP BY order_id and MAX(updated_at)
B
ROW_NUMBER() OVER (PARTITION BY order_id ORDER BY updated_at DESC), then WHERE rn = 1
D
Delete all rows except those with the latest date
✅ Correct! ROW_NUMBER() PARTITION BY order_id ORDER BY updated_at DESC assigns rank 1 to the latest row per order. WHERE rn = 1 keeps only that row.
❌ GROUP BY with MAX() only returns the date — you lose all other columns. ROW_NUMBER() + WHERE rn = 1 is the standard deduplication pattern.
Question 4 of 4
A correlated subquery on a 1M row table takes 30 minutes. Best fix?
A
Add more RAM to the database server
B
Replace with a CTE that pre-aggregates once, then JOIN
C
Use SELECT * to bring in all columns first
D
Add ORDER BY to speed up the query
✅ Correct! A correlated subquery runs once per row. On 1M rows = 1M executions. A CTE pre-aggregates once, then a JOIN finds the matching value — orders of magnitude faster.
❌ Pre-aggregate in a CTE and JOIN to it. SELECT * makes things worse. ORDER BY doesn't speed up queries. Add RAM treats the symptom, not the cause.
🎯 Module 5 — What You've Learned
- Window functions: Calculations across related rows without GROUP BY collapsing them.
- Ranking: ROW_NUMBER() for deduplication, DENSE_RANK() for leaderboards, NTILE() for segmentation.
- Running totals & moving averages: SUM/AVG OVER with ROWS BETWEEN.
- LAG & LEAD: Access previous/next row values — essential for period-over-period analysis.
- Chained CTEs: Break complex logic into named readable steps.
- Optimisation: Filter early, avoid SELECT *, replace correlated subqueries with CTE+JOIN.
- Deduplication: ROW_NUMBER() PARTITION BY + WHERE rn = 1 — the universal pattern.
Up next — Module 6
Data Visualisation — From Charts to Decisions
→