Ravi Sir's Cards › Generative AI
What is a token, and why does it matter?
Interview Q&A card: What is a token, and why does it matter? A token is a small piece of text that an LLM (Large Language Model, like ChatGPT) reads and writes: a word, part of a word, or a sign like “.”. A common English word is often 1 token; about 100 tokens ≈ 75 words .
Read card text
What this card says
Interview Q&A card: What is a token, and why does it matter? A token is a small piece of text that an LLM (Large Language Model, like ChatGPT) reads and writes: a word, part of a word, or a sign like “.”. A common English word is often 1 token
about 100 tokens ≈ 75 words .
Learn this in the lesson: Tokens and Context Window Explained: Why AI Forgets Long Chats