Data Engineering · Episode 07 · 30-second cut · 0:38
Data contracts stop silent $0 days
Without a contract, a renamed column turned Oct 8 into $0.00. With one, the bad batch is rejected before load.
Key moments
What you’ll needPython 3.10+, DuckDB, Pydantic
What you’ll need
- python.org/downloads →
Python 3.10 or newerRuns the code. Check yours with python3 --version. - code.visualstudio.com →
VS Code or CursorEither works; so does any editor you like. - git-scm.com/downloads →
gitGets the code from GitHub. 
DuckDB 1.0+SQL on files and tables, no database serverpip install "duckdb>=1.0"- Pydantic 2+Checks each record against a data contract
pip install "pydantic>=2"
Set it up
- 1Get the code
git clone https://github.com/DayanEbrar0X/data-anatomy.ai.git cd data-anatomy.ai
- 2Make a virtual environmentIt keeps this project's packages separate from the rest of your computer.
python3 -m venv .venv source .venv/bin/activate
- 3Install the packagesrequirements.txt installs every lesson's packages. This one needs only: DuckDB, Pydantic.
pip install -r requirements.txt # or just this lesson: pip install "duckdb>=1.0" "pydantic>=2"
- 4Run it
cd data-engineering/21-data-contracts cd src python3 contract.py
Project structure21-data-contracts/
Project structure
data-engineering/21-data-contracts/Open on GitHub →
- short/
- contract.pymainthe code from the video
- oct7.jsonOctober 7: 5 orders in the agreed shape
- oct8.jsonOctober 8: 5 orders, amount renamed to amount_usd, order_id sent as text
- shop.pythe consumer side: DuckDB warehouse, a loader with no checks, the revenue report
- src/
- contract.pyrunthe code from the video
- oct7.jsonOctober 7: 5 orders in the agreed shape
- oct8.jsonOctober 8: 5 orders, amount renamed to amount_usd, order_id sent as text
- shop.pythe consumer side: DuckDB warehouse, a loader with no checks, the revenue report
- README.mdthe idea, how to run it, a line-by-line walkthrough, things to try
data-engineering/21-data-contracts/short/contract.pyOpen on GitHub →
from typing import Literal
from pydantic import BaseModel, StrictInt
from shop import oct8, run
class Order(BaseModel): # the contract
order_id: StrictInt
amount: float # required, never null
status: Literal["paid", "refunded"]
run(oct8) # no contract
run(oct8, check=Order.model_validate)Transcript
Upstream renamed one column. Your dashboard died silently. Five real orders. Zero revenue.
No error. The fix: a data contract. order_id must be a real integer, amount is required, and status is paid or refunded. Load it twice: once without, once checking every row first.
No contract: October 8 shows zero dollars. With it, October 8 is rejected before load: bad order_id, missing amount. The dashboard keeps $412.40.
A contract turns silent zeros into loud errors.
Read the lesson on GitHub →- #datacontracts
- #pydantic
- #dataengineering
- #python
