26. How to Read APIs, JSON and Zipped CSVs into pandas
26.1 Why APIs, JSON and Zips
Why. Shraddha’s morning CSV is on Google Drive. The ops status board updates every 15 minutes through an internal HTTP JSON API. Amir emails a .csv.zip because the raw extract is 80 MB. You still want a pandas DataFrame for groupby and charts (Chapters 7–15).
What.
- API — a URL that returns data (often JSON) when you send an HTTP request.
- JSON — text structured as objects
{...}and arrays[...]. - Zipped CSV — a CSV compressed so email and disks stay smaller; pandas can read many zips directly.
How.
Mental model
- Ask where the source of truth lives (file, zip, HTTP JSON).
- Pull bytes or text into Python.
- Parse to a table-shaped structure.
- Land in a DataFrame — then clean / analyse as usual.
- Cache a local copy for class so you are not hammering a live server.
Worked example A — When CSV is enough.
Raja downloads last month’s closed orders once. Plain read_csv (Chapter 5) is fine.
Worked example B — When JSON API is better (fictional).
At 4 pm IST Ruhi needs “open orders right now for Pune dark stores”. The CSV is from 9 am. A small GET to a practice status API returns fresh JSON.
Worked example C — Stack sketch.
| Source | You pull with | You analyse in |
|---|---|---|
| Drive CSV | pd.read_csv |
DataFrame |
Email .csv.zip |
pd.read_csv + compression / zip |
same |
| HTTP JSON | requests + json_normalize |
same |
Ravindra Bagale's Tip
API = live tap. Zip = compressed bottle. pandas = kitchen counter. Pehle shape bagha, mag chart.