Ravindra BagaleCourses & study guides मराठी Track your progress

23. Common pandas Mistakes and Fixes

23.6 Debugging Checklist

In short: A fixed checklist finds 90% of classroom bugs in five minutes.

Why. When GMV “looks wrong”, random plot tweaks waste time. A fixed checklist finds 90% of classroom bugs in five minutes.

What. Always keep these prints near merges and exports:

  • shape
  • dtypes
  • isna().sum()
  • duplicated().sum() on business keys
  • merge(..., indicator=True)["_merge"].value_counts()
  • control totals: amount.sum() before vs after

How.

Five-minute debug

  1. Print df.shape after every major step.
  2. print(df.isna().sum().sort_values(ascending=False).head()).
  3. print(df["order_id"].duplicated().sum()).
  4. On merges: indicator=True and value-counts _merge.
  5. Compare a hand total for one city (Pune) in Excel vs pandas.
  6. Only then touch charts.

Worked example A — Helper function.

def peek(df, key="order_id", label="df"):
    print(f"== {label} ==")
    print("shape", df.shape)
    print("dtypes\n", df.dtypes)
    print("na\n", df.isna().sum())
    if key in df.columns:
        print("dup keys", df[key].duplicated().sum())
    if "amount" in df.columns and pd.api.types.is_numeric_dtype(df["amount"]):
        print("amount sum", df["amount"].sum())

peek(df, label="orders")

Worked example B — Merge audit.

left = pd.DataFrame({"store_id": ["PUN-01", "NAS-01"], "amount": [10, 20]})
right = pd.DataFrame({"store_id": ["PUN-01"], "city": ["Pune"]})
m = left.merge(right, on="store_id", how="left", indicator=True)
print(m["_merge"].value_counts())
print(m.loc[m["_merge"] != "both"])

Worked example C — Story from Rani (Nagpur).

Rani’s cancel rate hit 40% after a “quick filter”. Checklist showed status had mixed case (cancelled vs Cancelled) after a bad export round-trip. str.title() fixed it — not a new chart theme.

Ravindra Bagale's Tip

Debug order: shape → na → dups → merge indicator → one-city total. Chart shevati.