26. How to Read APIs, JSON and Zipped CSVs into pandas
26.6 Common Mistakes
Why. Two classroom disasters: (1) API key in Git; (2) downloading a 2 GB extract into RAM on a 8 GB laptop without filters.
What.
- No
timeout→ hung kernel. - Ignoring
status_code→ parsing HTML as JSON. DataFrame(payload)on a wrapped dict → wrong grain.- Reading zip path wrong (
orders.csvinside but you pointed at folder). - Logging full response text that contains tokens.
How.
Safer pull habit
- Env var for secrets; never commit
.envif it has real keys. - Always
timeout=andraise_for_status(). - Save raw JSON/CSV to disk before heavy cleaning.
- Prefer filtered API query params (city, date) — same idea as SQL Chapter 24.
- Check
df.shapeimmediately. - For huge files, read columns subset / chunksize later — start small in class.
Worked example A — Status check.
import requests
r = requests.get("https://httpbin.org/status/404", timeout=30)
print(r.status_code)
try:
r.raise_for_status()
except requests.HTTPError as e:
print("caught", type(e).__name__)
Worked example B — Wrong grain reminder.
import pandas as pd
payload = {"count": 2, "orders": [{"order_id": "BLK-1"}, {"order_id": "BLK-2"}]}
print(pd.DataFrame(payload))
print(pd.DataFrame(payload["orders"]))
Worked example C — Zip member pick.
Shraddha’s zip held orders.csv and README.txt. She listed names, then pd.read_csv(z.open("orders.csv")) pattern — or extracted one file first.
Ravindra Bagale's Tip
Key, timeout, status, shape — char checks. Galicha JSON parse = aadhi HTML error asel.