50. Tiny Text Classifier with a Transformer Intro in PyTorch

50.2 Token Ids as Numbers

By Ravindra Bagale · Last updated: 24 October 2026

Why. Build the id tensors students will train on.

What. Map words to ids; pad with 0; stack a batch.

How.

import torch

vocab = {"pad": 0, "pedha": 1, "tasty": 2, "love": 3, "stale": 4, "bad": 5, "not": 6, "good": 7}

def encode(words, max_len=4):
 ids = [vocab.get(w, 0) for w in words][:max_len]
 ids += [0] * (max_len - len(ids))
 return ids

reviews = [
 (["pedha", "tasty", "love"], 1),
 (["stale", "bad"], 0),
 (["not", "good"], 0),
 (["good", "pedha"], 1),
 (["bad", "stale"], 0),
 (["love", "good"], 1),
]
X = torch.tensor([encode(r) for r, _ in reviews], dtype=torch.long)
y = torch.tensor([lab for _, lab in reviews], dtype=torch.float32).unsqueeze(1)
print(X)
print(y.view(-1))

What you should see. A (6, 4) LongTensor of ids and six 0/1 labels.