49. What Transformers Are (Text and Sequences)
49.1 Why Sequences Need Attention
Why. Text and sequences are not grids like images. Meaning depends on order and long-range links: “not good” ≠ “good.” Transformers handle that with attention.
What. A transformer block lets each token look at other tokens and decide what matters (“attention”). Stacked blocks power modern NLP and are the backbone idea behind ChatGPT-style models (full apps live in the Gen AI course).
How.
Spot a sequence problem
- Reviews, SMS, support tickets, Marathi/English mix captions.
- Order matters.
- You need context beyond a single word bag (TF-IDF still useful — Chapter 52).
print("bag-of-words ignores order; attention can use order + context")
Ravindra Bagale's Tip
Transformer = “kontya words kade baghayche?” smart focus. Magic nahi — learned attention weights.