43. Backpropagation Intuition (No Heavy Maths)
43.3 Chain Rule in One Sentence
Why. One maths phrase is enough: the chain rule multiplies local slopes along the path.
What. If loss depends on ŷ, and ŷ depends on weight w, then d(loss)/dw = d(loss)/dŷ × dŷ/dw. Deep nets chain many such multiplications. Frameworks automate it.
How.
Tiny numeric chain
- Let ŷ = 2w, loss = (ŷ − 4)**2.
- At w=1: ŷ=2, loss=4.
- d(loss)/dŷ = 2(ŷ−4) = −4; dŷ/dw = 2; product = −8.
- That −8 is the gradient for w at that point.
w = 1.0
y_hat = 2 * w
loss = (y_hat - 4) ** 2
dloss_dyhat = 2 * (y_hat - 4)
dyhat_dw = 2
grad_w = dloss_dyhat * dyhat_dw
print("loss", loss, "grad_w", grad_w)
No heavy maths exam
You need the idea, not proving theorems. PyTorch applies the chain rule for you.