23. Common pandas Mistakes and Fixes
23.4 Timezone-Naive vs IST Mistakes
Why. Timestamps without timezone look fine until you subtract two times across systems, or until a UTC export lands in IST India. A “10-minute delivery” becomes −330 minutes of nonsense.
What.
- Naive datetime: no timezone attached.
- Aware datetime: knows IST (
Asia/Kolkata) or UTC. - Classroom rule: store dates as dates; for timestamps, localise to IST explicitly when you need duration math.
How.
IST-safe steps
- Parse with
dayfirst=Truefor Indiandd-mm-yyyy. - If the column is date-only, keep it at midnight naive or normalise on purpose.
- For timestamps:
dt.tz_localize("Asia/Kolkata")if they are local wall times. - If the feed is UTC:
tz_localize("UTC").tz_convert("Asia/Kolkata"). - Document in the notebook: “All wall times IST”.
Worked example A — Localise fictional dispatch times.
import pandas as pd
ts = pd.DataFrame({
"order_id": ["BLK-30", "BLK-31"],
"city": ["Pune", "Kolhapur"],
"dispatched_at": ["22-10-2026 21:05:00", "22-10-2026 21:20:00"],
"delivered_at": ["22-10-2026 21:18:00", "22-10-2026 21:31:00"],
})
for c in ["dispatched_at", "delivered_at"]:
ts[c] = pd.to_datetime(ts[c], dayfirst=True)
ts[c] = ts[c].dt.tz_localize("Asia/Kolkata")
ts["mins"] = (ts["delivered_at"] - ts["dispatched_at"]).dt.total_seconds() / 60
print(ts[["order_id", "mins"]])
Worked example B — UTC feed mistaken as local.
utc_feed = pd.Series(["2026-10-22 15:35:00"]) # actually UTC
aware_utc = pd.to_datetime(utc_feed, utc=True)
ist = aware_utc.dt.tz_convert("Asia/Kolkata")
print("UTC", aware_utc.iloc[0])
print("IST", ist.iloc[0]) # evening in India
If Salman treats 15:35 UTC as 15:35 IST, the “evening rush” chart shifts by 5 hours 30 minutes.
Worked example C — Date-only filters stay simple.
days = pd.to_datetime(
pd.Series(["15-10-2026", "20-10-2026", "25-10-2026"]), dayfirst=True
)
diwali_window = days.to_series().between("2026-10-15", "2026-10-22")
print(list(zip(days.dt.strftime("%d-%m-%Y"), diwali_window)))
For day-level EDA you often do not need tz — but say so in the README.