7. First Spark Job (PySpark) on a Small CSV
7.2 Read CSV and groupBy
Why. groupBy on a small CSV proves the verbs.
# Tiny local sketch — may need pyspark installed; not a live cluster claim
from pyspark.sql import SparkSession
spark = SparkSession.builder.appName("raja-orders-lab").master("local[*]").getOrCreate()
df = spark.read.option("header", True).csv("data/raw/orders_nagpur.csv")
summary = df.groupBy("city").count()
summary.show()
# summary.write.mode("overwrite").parquet("data/clean/orders_by_city")
spark.stop()Fictional data
Retail tables and city names here are fictional teaching data. Not real company metrics.