40. Data Preprocessing for Machine Learning (Missing Values, Scaling, Encoding)
40.3 StandardScaler
Why. litres might be 50–300 while distance_km is 5–60. Distance can look “small” to distance-based models even when it matters. Scaling puts columns on a comparable footing.
What. StandardScaler transforms each column to roughly mean 0 and std 1 using train statistics: (x - mean) / std.
How.
Scale after the split
scaler.fit(X_train)only.transformboth train and test.- Never
fiton the full dataset before splitting.
import pandas as pd
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import StandardScaler
df = pd.DataFrame({
"litres": [120, 80, 200, 60, 180, 90, 220, 70],
"distance_km": [12, 40, 8, 55, 10, 35, 6, 48],
"reorder": [1, 0, 1, 0, 1, 0, 1, 0],
})
X = df[["litres", "distance_km"]]
y = df["reorder"]
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.25, random_state=1)
scaler = StandardScaler()
X_train_s = scaler.fit_transform(X_train)
X_test_s = scaler.transform(X_test)
print("train scaled sample:\n", X_train_s.round(2))
print("mean_ used:", scaler.mean_.round(1), "scale_:", scaler.scale_.round(1))
What you should see. Scaled values centred near 0. Tree models often cope without scaling; logistic regression, K-Means, and many nets benefit from it.
Ravindra Bagale's Tip
Scaler fit = train chya numbers. Test la fakt transform — fit karu naka.