39. End-to-End Machine Learning Project (India Dataset)
39.4 Evaluate and Choose
Why. Choose with metrics + a quick overfit sanity check, then speak the result in business words.
What. Compare precision/recall (and accuracy) on the test set; also peek at train accuracy to spot silly memorisation.
How.
Evaluate and decide
- Print classification_report for both models on test.
- Print train vs test accuracy for the forest.
- Pick a primary model for this toy story and say why.
- List one next improvement (more rows, city feature encoded, better threshold).
from sklearn.metrics import classification_report, accuracy_score, confusion_matrix
for name, model, pred in [
("logistic", log_reg, pred_lr),
("forest", forest, pred_rf),
]:
print("====", name, "TEST ====")
print(classification_report(y_test, pred, zero_division=0))
print("confusion:\n", confusion_matrix(y_test, pred))
print("forest train acc", accuracy_score(y_train, forest.predict(X_train)))
print("forest test acc", accuracy_score(y_test, forest.predict(X_test)))
What you should see. Reports that may favour one model on this tiny sample. On 20 rows, treat the winner as a teaching choice — real Mauli data would need more shops and a time-aware split.
Ravindra Bagale's Tip
Tiny data var champion declare karu naka. Process barobar aahe — scale later.