Ravindra BagaleCourses & study guides मराठी Track your progress

24. How to Load SQL Query Results into a pandas DataFrame

24.3 pd.read_sql and read_sql_query

Why. You already trust read_csv. For SQL, pandas gives read_sql (and the clearer twin read_sql_query). One call: query in, DataFrame out.

What.

  • pd.read_sql(sql, con) — works with a DB-API connection or a SQLAlchemy connectable.
  • pd.read_sql_query(sql, con) — same idea; name stresses “this is a query”.
  • pd.read_sql_table(table_name, con) — loads a whole table (dangerous on huge tables — see 24.6).
  • Result is a normal DataFrame: .head(), .dtypes, .groupby() all work.

How.

First successful pull (SQLite)

  1. Create a tiny practice table (or use your lab .db).
  2. import pandas as pd and open a connection.
  3. Write a short SELECT.
  4. df = pd.read_sql_query(sql, con).
  5. print(df.shape), df.head(), df.dtypes.
  6. Close the connection when done exploring.

Worked example A — Seed + read (copy-paste lab).

import sqlite3
import pandas as pd
from pathlib import Path

Path("data").mkdir(exist_ok=True)
con = sqlite3.connect("data/blinkit_lab.db")

con.execute("DROP TABLE IF EXISTS orders")
con.execute(
    """
    CREATE TABLE orders (
        order_id TEXT,
        city TEXT,
        status TEXT,
        amount_inr REAL,
        customer TEXT
    )
    """
)
rows = [
    ("BLK-261024-001", "Pune", "Delivered", 189.0, "Ruhi Bagale"),
    ("BLK-261024-002", "Pune", "Cancelled", 90.0, "Amir"),
    ("BLK-261024-003", "Nashik", "Delivered", 95.0, "Zoya"),
    ("AMN-261024-004", "Kolhapur", "Delivered", 210.0, "Ravina"),
    ("BLK-261024-005", "Nagpur", "Returned", 150.0, "Raja"),
    ("BLK-261024-006", "Solapur", "Delivered", 120.0, "Salman"),
    ("BLK-261024-007", "Sambhaji Nagar", "Delivered", 340.0, "Rani"),
    ("AMN-261024-008", "Pune", "Delivered", 72.0, "Shraddha Bagale"),
]
con.executemany("INSERT INTO orders VALUES (?,?,?,?,?)", rows)
con.commit()

df = pd.read_sql_query(
    "SELECT order_id, city, status, amount_inr, customer FROM orders",
    con,
)
print(df.shape)
print(df.head())
con.close()

What you should see. Eight rows, five columns. amount_inr is numeric. You can now df.loc[df["city"] == "Pune"] like any other chapter.

Worked example B — read_sql vs read_sql_query.

import sqlite3
import pandas as pd

con = sqlite3.connect("data/blinkit_lab.db")
sql = "SELECT city, COUNT(*) AS n FROM orders GROUP BY city ORDER BY n DESC"
a = pd.read_sql(sql, con)
b = pd.read_sql_query(sql, con)
print(a.equals(b))
con.close()

For everyday SELECT work, either name is fine. Prefer read_sql_query when you want the code to shout “I am not loading a whole table blindly”.

Worked example C — After the pull, same pandas you know.

import sqlite3
import pandas as pd

con = sqlite3.connect("data/blinkit_lab.db")
df = pd.read_sql_query(
    "SELECT city, status, amount_inr FROM orders WHERE status = 'Delivered'",
    con,
)
con.close()

g = (
    df.groupby("city", as_index=False)["amount_inr"]
    .agg(orders="count", gmv="sum")
    .sort_values("gmv", ascending=False)
)
print(g)

Ravindra Bagale's Tip

shape ani head — connection success chi pehli proof. Chart nantar.