Data Engineering · Episode 04 · 30-second cut · 0:37
SCD Type 2: stop rewriting history
Type 1 turned Q2 East revenue into $2,500. Type 2 kept the real $6,700.
Key moments
What you’ll needPython 3.10+, DuckDB
What you’ll need
- python.org/downloads →
Python 3.10 or newerRuns the code. Check yours with python3 --version. - code.visualstudio.com →
VS Code or CursorEither works; so does any editor you like. - git-scm.com/downloads →
gitGets the code from GitHub. 
DuckDB 1.0+SQL on files and tables, no database serverpip install "duckdb>=1.0"
Set it up
- 1Get the code
git clone https://github.com/DayanEbrar0X/data-anatomy.ai.git cd data-anatomy.ai
- 2Make a virtual environmentIt keeps this project's packages separate from the rest of your computer.
python3 -m venv .venv source .venv/bin/activate
- 3Install the packagesrequirements.txt installs every lesson's packages. This one needs only: DuckDB.
pip install -r requirements.txt # or just this lesson: pip install "duckdb>=1.0"
- 4Run it
cd data-engineering/18-scd-type-2 python3 src/scd.py
Project structure18-scd-type-2/
Project structure
data-engineering/18-scd-type-2/Open on GitHub →
- data/
- customers.csv3 customers: id, name, region
- sales.csv5 sales in Q2 and Q3 2026: customer, day, usd
- short/
- scd.pyrunthe code from the video
- shop.pyloads data/ into DuckDB tables
- src/
- scd.pymainthe code from the video
- shop.pyloads data/ into DuckDB tables
- README.mdthe idea, how to run it, a line-by-line walkthrough, things to try
data-engineering/18-scd-type-2/short/scd.pyOpen on GitHub →
from shop import setup, by_region # same data
db = setup() # t1: customers; t2: + valid dates
db.sql("UPDATE t1 SET region='West' WHERE id=1")
db.sql("UPDATE t2 SET valid_to='2026-07-01', "
"is_current=false WHERE id=1")
db.sql("INSERT INTO t2 VALUES (1, 'Ana', 'West', "
"'2026-07-01', '9999-12-31', true)")
print("Q2 type 1:", by_region(db, "t1"))
print("Q2 type 2:", by_region(db, "t2"))Transcript
Your customer moved, and your report rewrote history. Ana lived East. On July 1, she moved West. Same customers, same sales, two dimension tables.
Type 1 overwrites Ana's row. Type 2 closes the old row with a valid-to date, and inserts a new one: West, from July 1. Run it. Type 1 says East made $2,500 in Q2.
Ana's spring sales moved West with her. Type 2 says $6,700. History kept. Overwriting rewrites the past.
Add a row instead.
Read the lesson on GitHub →- #scdtype2
- #duckdb
- #datawarehouse
- #dataengineering
