LLM Token Counter
Count tokens with OpenAI's o200k and cl100k encodings, estimate Claude, Gemini and Llama, and see context window use and input cost. Text stays in your browser.
The interactive tool loads here when JavaScript runs.
About this toolWhat is the LLM Token Counter?
Paste any text and the LLM Token Counter shows how many tokens it uses for current GPT, Claude, Gemini and Llama models, what share of each model's context window that is, and what it costs as input. OpenAI counts are exact: the tool runs the same byte-pair encodings as OpenAI's tiktoken library, o200k_base for GPT-4o, GPT-4.1, the o-series and the GPT-5 family, and cl100k_base for GPT-4 and GPT-3.5 Turbo.
A colourised view splits the text into its tokens, with spaces and line breaks made visible and optional token IDs, so you can see why a word costs one token and a product code costs five. The tokenizer runs in your browser; your text is not uploaded.
- Exact OpenAI counts with o200k_base and cl100k_base
- Labelled estimates for Claude, Gemini and Llama with the method shown
- Context window share and input cost per model
- Colourised token view with visible spaces and optional IDs
- Characters, words, UTF-8 bytes and characters per token
- Runs in the browser; text is never uploaded
Updated
How to use the LLM Token Counter
- Paste a prompt, document or transcript, or keep the example.
- Read the exact o200k_base and cl100k_base counts, characters, words and characters per token.
- Check the model table for each model's tokens, whether the number is exact or estimated, context window share and input cost.
- Use the token view to see where the tokens go; switch encodings or show token IDs.
Worked examples
A classic pangram
Input
The quick brown fox jumps over the lazy dog.
Output
10 tokens in both o200k_base and cl100k_base, 44 characters
Each word plus its leading space is one token, and the full stop is the tenth.
Non-English text
Input
こんにちは世界 ("Hello world" in Japanese)Output
2 tokens in o200k_base, 4 in cl100k_base
The larger o200k vocabulary has more multi-character entries for non-Latin scripts, so the same text is cheaper on newer OpenAI models.
Known tiktoken IDs
Input
hello world
Output
cl100k_base [15339, 1917], o200k_base [24912, 2375]
These match tiktoken exactly, which is how the counter is tested.
Background and insight
Language models read text as tokens: frequent words and word pieces from a fixed vocabulary, built with byte-pair encoding. Common English words are usually one token including their leading space ("·the"), while rare words, numbers, code and many non-English scripts split into several. OpenAI's o200k_base vocabulary has about 200,000 entries against cl100k_base's 100,000, which is why the same Japanese sentence "こんにちは世界" is 2 tokens in o200k_base and 4 in cl100k_base. tiktoken maps every gpt-5 model name to o200k_base; OpenAI has not published the encoding for GPT-6, so the tool counts it with o200k_base and labels the result as assumed.
Anthropic and Google do not publish their tokenizers, so the Claude and Gemini rows are estimates from the vendors' own ratios. Anthropic's models overview says 1M tokens is about 750,000 English words on models before Claude Opus 4.7 and about 555,000 words on the newer tokenizer, which its pricing page says produces roughly 30 percent more tokens for the same text. Google's token guide says a Gemini token is about 4 characters. Llama 3 builds its 128,000-token vocabulary on tiktoken's 100,000 tokens plus 28,000 more for other languages, according to Meta's Llama 3 paper, so the cl100k_base count is a close English proxy. For billing-grade Claude or Gemini numbers, call their count-tokens API endpoints.
Common use cases
Fitting a context window
Check that a long document plus your instructions fits before sending it.
Estimating input cost
See what one request costs on each model before you run it at scale.
Trimming prompts
Spot the parts of a prompt that cost the most tokens, such as pasted HTML or long IDs.
Setting max token limits
Size truncation limits and chunk sizes in tokens instead of guessing from characters.
Tips and limitations
- Chat API requests add a few tokens per message for role markers and formatting, so a request costs slightly more than its text.
- Images, audio and tool definitions are billed as tokens too, and are not counted here.
- Claude estimates are rough; real counts can differ by 20 percent or more on code, tables or non-English text.
- Reasoning models also bill hidden reasoning tokens as output. Input token counts do not include them.
- Special-token text such as <|endoftext|> in your input is counted as ordinary text, the way the API treats user content.
- The tokenizer library loads on first use (about 2 MB) and then works offline.
Frequently asked questions
Which tokenizer does GPT-4o use?
o200k_base, the same encoding as GPT-4.1, o1, o3, o4-mini and the GPT-5 family. GPT-4 and GPT-3.5 Turbo use cl100k_base.
How many tokens is 1,000 English words?
Roughly 1,300 tokens on OpenAI's encodings and earlier Claude models, and about 1,800 on Claude models from Opus 4.7, based on Anthropic's published ratios. Paste your own text for an exact OpenAI count.
Are the Claude and Gemini token counts exact?
No. Their tokenizers are not published, so the tool estimates from vendor ratios and says so. Use Anthropic's or Google's count-tokens endpoints for exact numbers.
Why does the same text have different token counts on different models?
Each model family has its own vocabulary. Larger vocabularies cover more whole words and more non-English text, so they need fewer tokens for the same input.
Is my text sent to OpenAI to count tokens?
No. The open-source gpt-tokenizer library, a JavaScript port of tiktoken, runs in your browser.
Why does an emoji use several tokens?
Tokenizers work on UTF-8 bytes. An emoji is 4 bytes, and uncommon ones are not in the vocabulary as a single entry, so they split into two or three byte-level tokens.
Is there an in-depth guide to the LLM Token Counter?
Yes. “How to Reduce LLM Token Usage and API Cost” covers the background, how it works, common pitfalls and worked examples. Read the full guide.
Related Tools You May Like
Free, in-browser tools that pair well with this one.