Chapter 57: Docker Projects Part 1: Websites, Databases, LAMP, LEMP, WordPress, 3-Tier and Every Language with MySQL and MongoDB

57.12 Project 12: Jupyter Lab and a FastAPI model server (CPU, then GPU)

By Ravindra Bagale · Last updated: 25 October 2026

Why: the problem

Suppose we work in the data team at Swiggy and we must predict how many minutes a food delivery will take. The data scientist trains the model in a Jupyter notebook on her laptop with Python 3.12, scikit-learn 1.5 and pandas 2.2. The backend team must call this model from the app, but their servers have Python 3.9 and an older scikit-learn, so the saved model fails to load. Every new team member spends a day installing the same notebook tools, and the results differ from laptop to laptop. With Docker we put Jupyter in one container and the model server in another, both built from the same pinned requirements file. The notebook saves model.joblib into a shared volume, the FastAPI container loads it, and the app calls one HTTP URL. Later the same image runs on a GPU server with one extra line in Compose.

What: every term in plain English

• Jupyter Lab — A web notebook where we write Python, run cells and see charts.
• Model — A file that learned a pattern from data, like delivery minutes from distance and items.
• joblib — A Python library that saves and loads scikit-learn models to a file.
• Model server — A small web API that loads the model once and answers predictions over HTTP.
• FastAPI — A fast Python web framework. It checks input types and makes /docs automatically.
• Shared volume — One named volume mounted in two containers, so the notebook writes and the server reads.
• Pinned versions — Exact versions like scikit-learn==1.5.2, so training and serving use the same library.
• GPU passthrough — Giving a container access to the NVIDIA GPU with the NVIDIA Container Toolkit.

How: step by step

1. Run Jupyter in a container. Token login, notebooks saved in a volume.
2. Train a model in the notebook. Save model.joblib in /models.
3. Build the model server image. FastAPI loads /models/model.joblib.
4. Join both in one Compose file. Shared models volume, health check.
5. Call the API. curl POST /predict.
6. Load test and reload a new model. Retrain, then restart only the server.
7. Run on a GPU. deploy.resources.reservations.devices.
8. Clean up. down -v.

Architecture

Folder layout

ml-serve/
  compose.yaml
  compose.gpu.yaml
  .env
  requirements.txt
  train.py
  server/
    app.py
    Dockerfile

Example 1: Jupyter Lab in one docker run command

Goal. Jupyter Lab in one docker run command.

jupyter commands

docker volume create notebooks
docker run -d --name lab \
  -p 127.0.0.1:8888:8888 \
  -v notebooks:/home/jovyan/work \
  -e JUPYTER_TOKEN=change-me-token \
  quay.io/jupyter/scipy-notebook:2024-10-07
docker logs lab --tail 3

What each line does

• quay.io/jupyter/scipy-notebook — The official Jupyter Docker Stacks image with pandas, numpy, scikit-learn and matplotlib already installed.
• 2024-10-07 — A date tag. It never changes, so every laptop gets the same tools.
• jovyan — The default non-root user inside Jupyter images.
• /home/jovyan/work — Where notebooks are saved. We mount a named volume here so they survive container removal.
• JUPYTER_TOKEN — The login token. Use a placeholder here and a strong value in real use.
• 127.0.0.1:8888 — Only this machine can open the notebook. Use an SSH tunnel from a laptop.

Green output

$ docker logs lab --tail 2
[I ServerApp] Jupyter Server 2.14.2 is running at:
[I ServerApp] http://127.0.0.1:8888/lab?token=...
# Browser: http://localhost:8888/lab?token=change-me-token
# File > New > Notebook > Python 3 > run: import sklearn; sklearn.__version__
'1.5.2'
$ docker rm -f lab && docker run -d --name lab -p 127.0.0.1:8888:8888 -v notebooks:/home/jovyan/work -e JUPYTER_TOKEN=change-me-token quay.io/jupyter/scipy-notebook:2024-10-07
# The notebook you saved in work/ is still there.

Expected result. A full data science lab in one command, with notebooks safe in a volume.

Example 2: Train the delivery-time model and save it

Goal. Train the delivery-time model and save it.

requirements.txt

scikit-learn==1.5.2
joblib==1.4.2
numpy==2.1.2
pandas==2.2.3
fastapi==0.115.2
uvicorn[standard]==0.32.0

train.py (also pasted in a notebook cell)

import numpy as np
import pandas as pd
import joblib
from sklearn.linear_model import LinearRegression
from sklearn.model_selection import train_test_split
from sklearn.metrics import mean_absolute_error

rng = np.random.default_rng(42)
n = 2000
df = pd.DataFrame({
    "distance_km": rng.uniform(0.5, 12, n),
    "items": rng.integers(1, 8, n),
    "rain": rng.integers(0, 2, n),
})
df["minutes"] = 8 + 3.2 * df.distance_km + 1.5 * df["items"] + 7 * df.rain + rng.normal(0, 2, n)

X = df[["distance_km", "items", "rain"]]
y = df["minutes"]
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=1)

model = LinearRegression().fit(X_train, y_train)
print("MAE minutes:", round(mean_absolute_error(y_test, model.predict(X_test)), 2))

joblib.dump({"model": model, "features": list(X.columns), "version": "1.0"}, "/models/model.joblib")
print("saved /models/model.joblib")

What each line does

• requirements.txt — One pinned list used by both the notebook image and the server image.
• default_rng(42) — Fixed random seed, so everyone gets the same fake data and the same result.
• LinearRegression — A simple model: minutes = a + bdistance + citems + d*rain.
• mean_absolute_error — On average, how many minutes the prediction is wrong.
• joblib.dump(dict) — We save the model with its feature names and a version, so the server can check them.
• /models — The shared volume. The notebook writes here; the server reads from here.

Green output

$ docker compose exec lab python /home/jovyan/work/train.py
MAE minutes: 1.61
saved /models/model.joblib
$ docker compose exec lab ls -l /models
-rw-r--r-- 1 jovyan users 1123 Oct 25 10:12 model.joblib

Expected result. The model is wrong by about 1.6 minutes on average, and the file is ready for the server.

Example 3: Model server image and one Compose file for both

Goal. Model server image and one Compose file for both.

server/app.py

import os
import joblib
from fastapi import FastAPI, HTTPException
from pydantic import BaseModel, Field

MODEL_PATH = os.getenv("MODEL_PATH", "/models/model.joblib")
app = FastAPI(title="Delivery time model")
state = {}

def load():
    bundle = joblib.load(MODEL_PATH)
    state.update(bundle)
    return bundle["version"]

class Order(BaseModel):
    distance_km: float = Field(gt=0, le=50)
    items: int = Field(ge=1, le=50)
    rain: int = Field(ge=0, le=1)

@app.on_event("startup")
def startup():
    try:
        print("loaded model version", load())
    except FileNotFoundError:
        print("no model yet at", MODEL_PATH)

@app.get("/health")
def health():
    if "model" not in state:
        raise HTTPException(503, "model not loaded")
    return {"status": "ok", "version": state["version"]}

@app.post("/predict")
def predict(o: Order):
    if "model" not in state:
        raise HTTPException(503, "model not loaded")
    row = [[o.distance_km, o.items, o.rain]]
    return {"minutes": round(float(state["model"].predict(row)[0]), 1), "version": state["version"]}

@app.post("/reload")
def reload():
    return {"loaded": load()}

server/Dockerfile

FROM python:3.12-slim
ENV PYTHONDONTWRITEBYTECODE=1 PYTHONUNBUFFERED=1
WORKDIR /app
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt
COPY server/app.py .
RUN useradd -u 10001 model
USER model
EXPOSE 8000
HEALTHCHECK --interval=10s --timeout=3s --start-period=10s --retries=3 \
  CMD python -c "import urllib.request; urllib.request.urlopen('http://localhost:8000/health')"
CMD ["uvicorn", "app:app", "--host", "0.0.0.0", "--port", "8000", "--workers", "2"]

compose.yaml

services:
  lab:
    image: quay.io/jupyter/scipy-notebook:2024-10-07
    ports: ["127.0.0.1:8888:8888"]
    volumes:
      - notebooks:/home/jovyan/work
      - ./train.py:/home/jovyan/work/train.py:ro
      - models:/models
    user: root
    command: ["start-notebook.py", "--ServerApp.root_dir=/home/jovyan/work"]
    environment:
      JUPYTER_TOKEN: ${JUPYTER_TOKEN}
      CHOWN_EXTRA: /models
  api:
    build: { context: ., dockerfile: server/Dockerfile }
    image: swiggy-eta:1.0
    ports: ["8000:8000"]
    volumes: ["models:/models:ro"]
    mem_limit: 512m
    restart: unless-stopped
volumes:
  notebooks:
  models:

.env

JUPYTER_TOKEN=change-me-token

What each line does

• Field(gt=0, le=50) — FastAPI rejects bad input like a negative distance with a 422 error, before the model sees it.
• state dict — The model is loaded once at start, not on every request. Loading per request is slow.
• /reload — Loads a new model file without restarting the container.
• models:/models:ro — The server can only read the model. Only the notebook can write it.
• user: root + CHOWN_EXTRA — The Jupyter start script runs as root only to give the jovyan user ownership of /models, then drops to jovyan.
• --workers 2 — Two uvicorn processes so two predictions can run at the same time.
• HEALTHCHECK — Docker marks the server unhealthy until a model is loaded, because /health returns 503.

Green output

$ docker compose up -d --build
 Container ml-serve-lab-1  Started
 Container ml-serve-api-1  Started
$ docker compose ps api
NAME             STATUS
ml-serve-api-1   Up 20 seconds (unhealthy)
$ docker compose exec lab python /home/jovyan/work/train.py
saved /models/model.joblib
$ curl -s -X POST localhost:8000/reload
{"loaded":"1.0"}
$ curl -s -X POST localhost:8000/predict -H 'Content-Type: application/json' -d '{"distance_km":4.2,"items":3,"rain":1}'
{"minutes":32.0,"version":"1.0"}
$ docker compose ps api
ml-serve-api-1   Up 1 minute (healthy)

Expected result. Training and serving use the same pinned libraries, so the model loads without version errors.

Example 4: Run the model server on an NVIDIA GPU

Goal. Run the model server on an NVIDIA GPU.

install the NVIDIA Container Toolkit (Ubuntu)

nvidia-smi
# First add the NVIDIA apt repository as shown in the NVIDIA Container Toolkit install guide.
sudo apt-get update && sudo apt-get install -y nvidia-container-toolkit
sudo nvidia-ctk runtime configure --runtime=docker
sudo systemctl restart docker
docker run --rm --gpus all nvidia/cuda:12.6.2-base-ubuntu24.04 nvidia-smi

compose.gpu.yaml

services:
  api:
    environment:
      DEVICE: cuda
    deploy:
      resources:
        reservations:
          devices:
            - driver: nvidia
              count: 1
              capabilities: [gpu]

What each line does

• nvidia-smi — Shows the GPU and driver on the host. If this fails, Docker cannot use the GPU either.
• nvidia-container-toolkit — Lets Docker pass the GPU driver into containers.
• nvidia-ctk runtime configure — Adds the nvidia runtime to /etc/docker/daemon.json.
• --gpus all — docker run flag that gives all GPUs to the container.
• deploy.resources.reservations.devices — The Compose way to ask for GPUs.
• count: 1 — Only one GPU. On a server with 4 GPUs, other containers can use the rest.
• Second compose file — compose.gpu.yaml only adds GPU settings on top of compose.yaml; the CPU laptop never needs it.

Green output

$ docker run --rm --gpus all nvidia/cuda:12.6.2-base-ubuntu24.04 nvidia-smi
| NVIDIA-SMI 560.35.03   Driver Version: 560.35.03   CUDA Version: 12.6 |
|   0  Tesla T4            Off |   00000000:00:1E.0 Off |   0MiB / 15360MiB |
$ docker compose -f compose.yaml -f compose.gpu.yaml up -d api
$ docker compose exec api nvidia-smi --query-gpu=name --format=csv,noheader
Tesla T4

Expected result. The same image now sees the GPU on an AWS g4dn.xlarge instance. Deep learning models (PyTorch) use it; small scikit-learn models stay on CPU.

Every command, step by step (plain docker and Compose)

1. Make folders

   mkdir -p ml-serve/server && cd ml-serve
   

   What it does: Project folder with the server code inside server/.

   (no output)
   

2. Add files

   ls -R
   

   What it does: Copy requirements.txt, train.py, server/app.py, server/Dockerfile, compose.yaml and .env from the examples.

   compose.yaml  requirements.txt  train.py  .env
   ./server:
   app.py  Dockerfile
   

3. Build and start

   docker compose up -d --build
   

   What it does: Builds swiggy-eta:1.0 and starts Jupyter and the API.

   Container ml-serve-api-1 Started
   

4. Open Jupyter

   echo http://localhost:8888/lab?token=change-me-token
   

   What it does: Open in the browser and see train.py in the file list.

   http://localhost:8888/lab?token=change-me-token
   

5. Check the API before training

   curl -s localhost:8000/health
   

   What it does: No model yet, so it says 503.

   {"detail":"model not loaded"}
   

6. Train

   docker compose exec lab python /home/jovyan/work/train.py
   

   What it does: Writes model.joblib into the models volume.

   MAE minutes: 1.61
   saved /models/model.joblib
   

7. Load the model

   curl -s -X POST localhost:8000/reload
   

   What it does: The server reads the new file.

   {"loaded":"1.0"}
   

8. Predict

   curl -s -X POST localhost:8000/predict -H 'Content-Type: application/json' -d '{"distance_km":4.2,"items":3,"rain":1}'
   

   What it does: A 4.2 km order with 3 items in rain.

   {"minutes":32.0,"version":"1.0"}
   

9. Send bad input

   curl -s -o /dev/null -w '%{http_code}\n' -X POST localhost:8000/predict -H 'Content-Type: application/json' -d '{"distance_km":-1,"items":3,"rain":1}'
   

   What it does: FastAPI validation blocks a negative distance.

   422
   

10. Load test

    docker run --rm --network host williamyeh/hey -n 2000 -c 20 -m POST -T application/json -d '{"distance_km":3,"items":2,"rain":0}' http://localhost:8000/predict
    

    What it does: 2000 requests, 20 at a time.

    Requests/sec: 1450.32
    [200] 2000 responses
    

11. See resources

    docker stats --no-stream ml-serve-api-1
    

    What it does: The server stays far below the 512 MB limit.

    ml-serve-api-1  1.2%  148MiB / 512MiB
    

12. Open the API docs

    echo http://localhost:8000/docs
    

    What it does: FastAPI makes a test page for every endpoint.

    http://localhost:8000/docs
    

13. Clean up

    docker compose down -v
    docker image rm swiggy-eta:1.0
    

    What it does: Removes containers, both volumes and the image.

    Volume ml-serve_models Removed
    

When it is red 1

InconsistentVersionWarning: Trying to unpickle estimator LinearRegression from version 1.5.2 when using version 1.3.0

Cause. The server image has a different scikit-learn than the notebook.

Fix, step by step

1. Install both from the same requirements.txt with pinned versions.
2. Rebuild: docker compose build --no-cache api.

When it is red 2

PermissionError: [Errno 13] Permission denied: '/models/model.joblib'

Cause. The models volume is owned by root, so jovyan cannot write.

Fix, step by step

1. Keep user: root and CHOWN_EXTRA: /models on the lab service.
2. Recreate it: docker compose up -d --force-recreate lab.

When it is red 3

{"detail":"model not loaded"}  (status 503)

Cause. The API started before any model was trained.

Fix, step by step

1. Run train.py, then POST /reload.
2. Or restart the api container after training.

When it is red 4

could not select device driver "nvidia" with capabilities: [[gpu]]

Cause. The NVIDIA Container Toolkit is missing or Docker was not restarted.

Fix, step by step

1. Install nvidia-container-toolkit and run nvidia-ctk runtime configure --runtime=docker.
2. sudo systemctl restart docker, then test with nvidia/cuda ... nvidia-smi.

When it is red 5

Killed  (exit code 137)

Cause. The model is too big for mem_limit 512m.

Fix, step by step

1. Check with docker stats.
2. Raise mem_limit or use a smaller model; do not use more workers than RAM allows.

Practice tasks

docker run -d --name lab -p 127.0.0.1:8888:8888 -e JUPYTER_TOKEN=change-me-token quay.io/jupyter/scipy-notebook:2024-10-07

docker rm -f lab
docker run -d --name lab -p 127.0.0.1:8888:8888 -v notebooks:/home/jovyan/work quay.io/jupyter/scipy-notebook:2024-10-07

docker compose exec lab python /home/jovyan/work/train.py
docker compose exec lab ls -l /models

curl -s -X POST localhost:8000/predict -H 'Content-Type: application/json' -d '{"distance_km":2,"items":1,"rain":0}'

curl -s -X POST localhost:8000/predict -H 'Content-Type: application/json' -d '{"distance_km":2,"items":0,"rain":0}'

docker compose exec lab python /home/jovyan/work/train.py
curl -s -X POST localhost:8000/reload

docker compose ps api
docker inspect --format '{{.State.Health.Status}}' ml-serve-api-1

docker run --rm --network host williamyeh/hey -n 5000 -c 50 -m POST -T application/json -d '{"distance_km":3,"items":2,"rain":0}' http://localhost:8000/predict

docker compose -f compose.yaml -f compose.gpu.yaml up -d api
docker compose exec api nvidia-smi

docker compose down -v
docker image rm swiggy-eta:1.0

Recap

• Jupyter and the model server are two containers from pinned versions.
• A shared named volume moves the model file from notebook to server.
• The server loads the model once and answers over HTTP.
• FastAPI validates input before the model sees it.
• A health check stays red until a model is loaded.
• A second Compose file adds a GPU without changing the CPU setup.

FAQ

Q. Can I use PyTorch instead of scikit-learn?

Yes. Use a pytorch/pytorch CUDA image or install torch in requirements.txt; the serving pattern is the same.

Q. Where should models live in production?

In S3 or a model registry like MLflow; the container downloads the version it needs at start.

Q. Is Jupyter safe to expose on the internet?

No. Keep it on 127.0.0.1 with a strong token and use an SSH tunnel or VPN.

Q. Do I need a GPU for this project?

No. The scikit-learn model runs fast on CPU. The GPU part is for deep learning.

Q. How big are these images?

scipy-notebook is about 1.5 GB; the slim model server is about 250 MB.

Q. Can I use the same image on Kubernetes?

Yes. The image and health URL work unchanged in a Deployment.

Interview questions

1. How do you deploy an ML model with Docker?

   Model answer: Put the model and a small API (FastAPI) in an image with pinned libraries, load the model at startup, expose /predict and /health, and run it with resource limits.

2. What causes 'model works in notebook but fails in production'?

   Model answer: Different library versions or features; pin versions and save feature names with the model.

3. How do you give a container a GPU?

   Model answer: Install the NVIDIA Container Toolkit, then use --gpus all or Compose device reservations.

4. How do you update a model without downtime?

   Model answer: Load the new file with a reload endpoint, or start a new container version and switch traffic after its health check passes.

5. Why not put training and serving in one container?

   Model answer: They have different needs: training needs tools and lots of RAM, serving needs a small, fast, secure image.

Samajla ka?

Ravi Sir's Cards for this lesson

Revision cardInterview card

मराठी cards