Token Counter

No file loaded
Supported: .txt, .md, .vtt, .srt, .json, .csv | Max: 400,000 characters | Max file size: 10 MB
Truncated to 400,000 characters
Token count error
Characters: 0 | Words: 0 | Sentences: 0 | Paragraphs: 0 | Tokens OpenAI: 0 | Tokens legacy: 0
Tokens are exact counts using OpenAI tiktoken encodings: OpenAI = o200k_base (newer), legacy = cl100k_base (older).
About this Token Counter (FAQ)

Token Counter helps you measure how much text fits into a GPT model’s context window. This OpenAI token counter uses exact tiktoken encodings to count tokens — not rough estimates. Paste text or drop a file to instantly see characters, words, sentences, paragraphs, plus two token totals: Tokens OpenAI (o200k_base) and Tokens legacy (cl100k_base).

Token counts matter for prompts, RAG chunks, pricing, and avoiding “context length exceeded” errors. If you’re comparing model limits or preparing long transcripts, use this GPT token counter to validate your content before sending it to an API. Supported file types: .txt, .md, .vtt, .srt, .json, .csv. The tool reads plain text and shows exact tokenization results for both encodings.

How this token counter works

  • Characters include spaces.
  • Words are whitespace-separated.
  • Sentences split on . ! ?.
  • Paragraphs are blocks separated by blank lines.
  • Tokens are exact counts using OpenAI tiktoken encodings (o200k_base and cl100k_base).

FAQ

1) What is a token in GPT / OpenAI models?

A token is a chunk of text used by GPT models (often a few characters or part of a word). Token counts determine how much text fits into a model’s context window and can affect API cost and truncation.

2) Why do token counts differ from word counts?

Words are human-readable units, but tokenization splits text differently based on vocabulary rules and byte-level encoding. The same sentence can produce different token totals depending on punctuation, spacing, and language.

3) What’s the difference between o200k_base and cl100k_base?

Both are OpenAI tiktoken encodings. o200k_base is a newer encoding, while cl100k_base is a legacy encoding widely used by earlier GPT setups. This tool shows both so you can compare.

4) Is this an exact token counter or an estimate?

It is exact for the selected encodings. The backend uses OpenAI’s tiktoken tokenizer to compute the token counts precisely.

5) Which file formats can I upload and how are they handled?

Supported formats: .txt, .md, .vtt, .srt, .json, .csv. Files are treated as plain text: the content is loaded into the textarea and then token counts are computed immediately.

Related guides