5. Python for Pipelines (pandas to files/DB)

5.2 Read → Clean → Write

By Ravindra Bagale · Last updated: 24 October 2026

Why. Every pipeline is some version of read → clean → write.

import pandas as pd
from pathlib import Path

raw = Path("data/raw/orders_nagpur.csv")
clean = Path("data/clean/orders_nagpur.parquet")

df = pd.read_csv(raw)
df["city"] = df["city"].str.strip().str.title()
df = df.drop_duplicates(subset=["order_id"])
df.to_parquet(clean, index=False)
print("wrote", clean, "rows", len(df))