Model Anatomy · Episode 09 · 30-second cut · 0:39
Why one neuron can't solve XOR, and two layers can
A single neuron draws one straight line and XOR needs two. A 2-4-1 network trained with backprop in 9 lines of NumPy takes the loss from 0.795 to 0.001.
What you’ll needPython 3.10+, NumPy
What you’ll need
- python.org/downloads →
Python 3.10 or newerRuns the code. Check yours with python3 --version. - code.visualstudio.com →
VS Code or CursorEither works; so does any editor you like. - git-scm.com/downloads →
gitGets the code from GitHub. 
NumPy 1.26+Math on many numbers at oncepip install "numpy>=1.26"
Set it up
- 1Get the code
git clone https://github.com/DayanEbrar0X/data-anatomy.ai.git cd data-anatomy.ai
- 2Make a virtual environmentIt keeps this project's packages separate from the rest of your computer.
python3 -m venv .venv source .venv/bin/activate
- 3Install the packagesrequirements.txt installs every lesson's packages. This one needs only: NumPy.
pip install -r requirements.txt # or just this lesson: pip install "numpy>=1.26"
- 4Run it
cd machine-learning/09-neural-network-from-scratch python3 src/xor.py
Project structure09-neural-network-from-scratch/
Project structure
machine-learning/09-neural-network-from-scratch/Open on GitHub →
- src/
- xor.pymainthe 22 lines from the video
- README.mdthe idea, how to run it, a line-by-line walkthrough, things to try
xor.pyOpen on GitHub →
from net import np, X, y, W1, b1, W2, b2
for step in range(2000):
h = np.tanh(X @ W1 + b1)
p = 1 / (1 + np.exp(-(h @ W2 + b2)))
d2 = (p - y) * p * (1 - p)
d1 = d2 @ W2.T * (1 - h ** 2)
W2 -= 0.5 * h.T @ d2; b2 -= 0.5 * d2.sum(0)
W1 -= 0.5 * X.T @ d1; b1 -= 0.5 * d1.sum(0)
print("predictions:", *p.round().astype(int).flat)Transcript
One neuron can't solve this four-point puzzle. Two layers can. XOR: output 1 when the inputs differ. One neuron is one line.
It always misses a point. Add a hidden layer: four neurons. Each bends the inputs with tanh, and a sigmoid makes the guess. Backprop sends the error back, and every weight steps downhill.
Two thousand times. Loss: 0.795 down to 0.001.
Predictions: 0, 1, 1, 0. Solved. One line can't. Two layers and backprop can.
Read the lesson on GitHub →- #neuralnetwork
- #deeplearning
- #python
- #machinelearning
