42. Activation Functions, Loss, and Gradient Descent (Plain English)
42.2 What Is Loss
Why. Training needs a score for “how wrong.” That score is the loss.
What. Regression often uses mean squared error. Classification often uses cross-entropy (you will call it from PyTorch later). Lower loss on training data — carefully checked on test data — is the goal.
How.
Compute a tiny MSE by hand
- True y = 10, prediction = 7 → error 3, squared 9.
- Average squared errors across rows = MSE.
- Training tries to reduce that average.
y_true = [10, 12, 8]
y_pred = [9, 11, 10]
mse = sum((a-b)**2 for a,b in zip(y_true, y_pred)) / len(y_true)
print("MSE:", round(mse, 3))