Ravindra Bagale · Data Scienceसर्व coursesया course चे lessonsशोधाEnglish

Data Science · मराठी आवृत्ती

24.6 चुका: huge tables आणि string-format

By Ravindra Bagale

या course चे सर्व chapters (py + ch)

हे का शिकायचं?

SELECT * , password in notebook, आणि f-string SQL — तीन मोठे धोके.

हे काय आहे?

Columns सूची, secrets बाहेर, bind params, chunksize / date windows.

कसं करायचं?

Step 1. Column list लिहा.

Step 2. Secret स्क्रीनशॉट/commit तर नाही ना.

Step 3. Bind params.

Step 4. Chunk किंवा महिन्यानुसार windows.

उदाहरणं: copy करा आणि चालवा

उदाहरण A — Fragile vs parameterized (SQLite ?).

import sqlite3
import pandas as pd

con = sqlite3.connect("data/blinkit_lab.db")
city = "Pune"  # in real apps this might come from a form — still use params

# Fragile habit — do not use for external input
bad = f"SELECT * FROM orders WHERE city = '{city}'"

# Safer — values travel separately from SQL text
good = "SELECT order_id, city, amount_inr FROM orders WHERE city = ?"
df = pd.read_sql_query(good, con, params=("Pune",))
print(df)

con.close()

उदाहरण B — Named style sketch (SQLAlchemy / many servers).

import pandas as pd
from sqlalchemy import create_engine, text

engine = create_engine("sqlite:///data/blinkit_lab.db")
sql = text(
    """
    SELECT city, status, amount_inr
    FROM orders
    WHERE status = :status AND amount_inr >= :min_amt
    """
)
df = pd.read_sql_query(
    sql,
    engine,
    params={"status": "Delivered", "min_amt": 100},
)
print(df)
engine.dispose()

उदाहरण C — Huge-table checklist (fictional Monday).

Shraddha almost ran pd.read_sql_table("orders", engine) on production. Checklist stopped her:

  1. Row estimate from DBA / COUNT(*) on a filtered subset.
  2. Date filter for “today IST”.
  3. Column list without receipt JSON blobs.
  4. LIMIT 1000 smoke test first.
  5. Then remove LIMIT only if the filtered shape is sane.