31. Train Test Split in Machine Learning
31.1 Why Holding Data Back Matters
Why. If you study the exam paper and then sit the same paper, your marks lie. Machine learning has the same trap.
What. A train set is the rows the model is allowed to learn from. A test set is rows you hide until the end to check honest performance. Holding data back means keeping the test set untouched during training.
How.
Mental model
- Shuffle (or carefully split) your clean table.
- Give most rows to train (often ~80%).
- Lock the rest as test (~20%).
- Fit only on train. Score on test once you are ready.
Ravindra Bagale's Tip
Test set = final exam. Training accuracy hashtag nahi. Honest number test varun. Lakshat theva.