Ravindra BagaleCourses & study guides मराठी Track your progress

26. How to Read APIs, JSON and Zipped CSVs into pandas

26.7 Practice, Tip and Close

By , trainer LinkedIn Last updated:

Why. File, zip, and JSON paths should feel interchangeable before project week.

What. One lab: write JSON + zip, load both to DataFrames, compare shapes, export a tiny CSV.

How.

Chapter practice lab

  1. Recreate data/raw/status_sample.json with ≥8 fictional orders across six Maharashtra cities (mix Open/Delivered).
  2. Load with json_normalize into df_api.
  3. Build orders_mini.csv.zip and load into df_zip.
  4. Harmonize columns; concat or compare city counts.
  5. Export outputs/api_zip_city_counts.csv.
  6. Write three bullets: timeout habit, which JSON key held rows, how you opened the zip.
import json
import zipfile
from pathlib import Path
import pandas as pd

Path("data/raw").mkdir(parents=True, exist_ok=True)
Path("outputs").mkdir(exist_ok=True)

payload = {
    "orders": [
        {"order_id": "BLK-61", "city": "Pune", "status": "Delivered", "amount_inr": 189},
        {"order_id": "BLK-62", "city": "Nashik", "status": "Open", "amount_inr": 95},
        {"order_id": "AMN-63", "city": "Kolhapur", "status": "Delivered", "amount_inr": 210},
        {"order_id": "BLK-64", "city": "Solapur", "status": "Delivered", "amount_inr": 70},
        {"order_id": "BLK-65", "city": "Nagpur", "status": "Open", "amount_inr": 150},
        {"order_id": "BLK-66", "city": "Sambhaji Nagar", "status": "Delivered", "amount_inr": 120},
        {"order_id": "BLK-67", "city": "Pune", "status": "Delivered", "amount_inr": 99},
        {"order_id": "AMN-68", "city": "Nashik", "status": "Delivered", "amount_inr": 130},
    ]
}
Path("data/raw/status_sample.json").write_text(json.dumps(payload), encoding="utf-8")
df_api = pd.json_normalize(payload["orders"])

csv_path = Path("data/raw/orders_mini.csv")
df_api.to_csv(csv_path, index=False)
zip_path = Path("data/raw/orders_mini.csv.zip")
with zipfile.ZipFile(zip_path, "w", compression=zipfile.ZIP_DEFLATED) as z:
    z.write(csv_path, arcname="orders_mini.csv")
df_zip = pd.read_csv(zip_path)

counts = df_zip.groupby("city", as_index=False).size().rename(columns={"size": "orders"})
counts.to_csv("outputs/api_zip_city_counts.csv", index=False)
print(df_api.shape, df_zip.shape)
print(counts)

Practice task

Zoya: “Nested customer.city in JSON + a zip of flat CSV. City counts must match after normalize.” No real production API key.

Ravindra Bagale's Tip

JSON ne tap, zip ne bottle, pandas ne story. Secret Git madhe nahi. Samajla ka?

Thodkyaat sangaycha tar (quick recap)

  • APIs and zips still land in the same DataFrame kitchen
  • requests: timeout, status, env secrets
  • Find the list of records; use json_normalize
  • read_csv often reads .csv.zip directly
  • Wrong grain = DataFrame(wrapper_dict)
  • Save raw files under data/raw/ for replay

Samajla ka? Aata API JSON ani zip CSV donhi DataFrame madhe yenar — pudhe practice raw folder neat theva.