Interview Q&A · 0 · Foundations
What is a token, and why does the same prompt cost different amounts on different models?
Reveal the answer
A token is the atomic unit an LLM reads and generates — roughly a common
character sequence like a word, subword, or punctuation. It's not a word:
"GPT" is one token, " GPT" (with a leading space) is another, and
"aiengineering" is usually several. Each provider trains its own tokenizer
(Anthropic, OpenAI, Google all differ) so the same English sentence
encodes to different token counts across models — that, plus different
per-token prices, is why the same prompt costs different amounts.
Common variants
- How would you count tokens in Python before making a call?
- Why is output token count almost always the dominant cost?
- What's the tokenizer difference between OpenAI's cl100k and Anthropic's?
Track this card
Stored in this browser only — no signup, no sync. Clearing site data clears it.