43. Backpropagation Intuition (No Heavy Maths)
43.4 Beginner Backprop Mistakes
Why. Beginners forget zero_grad, call backward twice without clearing, or expect backprop without a differentiable loss.
What. Gradients accumulate by default in PyTorch. Always optimizer.zero_grad() before backward in a normal loop. Also: vanishing/exploding gradients are advanced topics — start with short nets and ReLU.
How.
Checklist
- zero_grad → backward → step (order matters).
- Do not detach tensors if you still need gradients.
- If loss is NaN, lower lr and check inputs for NaNs (Chapter 40).
print("OK order: zero_grad → backward → step")
print("BAD: backward → backward (grads stack up)")
Ravindra Bagale's Tip
zero_grad visarle ki gradients stack hotat — weird training. Habit banva.