50. Tiny Text Classifier with a Transformer Intro in PyTorch
50.4 Train on Fictional Reviews
Why. Train the tiny classifier and predict.
What. BCEWithLogitsLoss + Adam for a few dozen epochs on the 6-row toy — API practice, not a Kaggle win.
How.
Fit and infer
- Build X, y from 50.2.
- Train 50 epochs; print loss sometimes.
- Sigmoid probs; threshold 0.5.
import torch.optim as optim
torch.manual_seed(0)
model = MeanEmbedClassifier()
opt = optim.Adam(model.parameters(), lr=0.05)
crit = nn.BCEWithLogitsLoss()
for epoch in range(60):
opt.zero_grad()
logits = model(X)
loss = crit(logits, y)
loss.backward()
opt.step()
if epoch % 15 == 0:
print(epoch, float(loss))
model.eval()
with torch.no_grad():
probs = torch.sigmoid(model(X))
print(probs.round(decimals=3).view(-1))
What you should see. Loss falling; probs tending toward the toy labels. With 6 rows this is a learning sandbox — Chapter 38 still applies.