What Are Tokens in AI? A Guide for SMEs

A token is a small piece of text - a word, part of a word, or a punctuation mark - that an AI model splits language into before it can process it. Language models like ChatGPT or Claude don't read sentences the way people do; they see a sequence of tokens, each with its own numerical code. Tokens are the computational unit behind everything a model does: reading, reasoning, and generating answers.

How does tokenization work?

Before a language model can process a prompt, a tool called a tokenizer chops the text into pieces. This doesn't necessarily happen word by word. A short, common word gets its own token, but a longer or unusual word can split into several. The word \"darkness\", for example, is commonly split into \"dark\" and \"ness\" - two tokens for one word.

Punctuation, spaces, and numbers often count as separate tokens too. A sentence with heavy punctuation or abbreviations therefore uses relatively more tokens than a sentence of the same length written in plain words.

Rule of thumb: words to tokens

There's no exact, language-independent conversion factor, but for English text this rule of thumb is commonly used:

• 1 token ≈ 4 characters
• 1 token ≈ ¾ of a word
• 100 tokens ≈ 75 words

For other languages, including Dutch, the ratio is often somewhat less favorable, since compound words and inflections tend to split into more tokens than in English. As a rough guide, expect around 1.3 to 1.5 tokens per Dutch word.

Why does this matter for SMEs?

As a business owner, tokens show up in two places in practice:

1. Cost. API access to models like GPT or Claude is almost always billed per token, separately for input (your prompt) and output (the response). The longer your prompts and documents, the higher the bill at scale.
2. Context window. Every model has a maximum number of tokens it can process at once - the so-called context window. If your document, conversation, or set of instructions exceeds that limit, the model has to \"forget\" or truncate older information.

For most SME use cases - a chatbot on your website, an internal knowledge assistant, automated email processing - this mainly means thinking deliberately about how much text you're asking a model to process. If you're using a consumer app (like the ChatGPT app) for one-off questions, this barely matters; you pay a flat subscription. Once you integrate via an AI agent or process large volumes of documents yourself, token usage becomes a concrete cost line.

A concrete example

Say you build an AI agent that summarizes incoming customer emails and drafts a reply. An average email of around 150 words translates to roughly 200-250 tokens. Add context - prior correspondence, a customer profile, company guidelines - and the actual input can quickly grow to around 1,000 tokens per processed email.

At hundreds of emails a week, that adds up, but per-token cost for most modern models runs in fractions of a cent. For a small or mid-sized business, token usage is rarely the bottleneck; how you set up the task - what information you provide, how clearly you define the job - usually matters far more for the result than the raw token count.

When should you actually worry about this?

Should you worry? Usually not, if you're just getting started. For a first AI agent or pilot project, token usage is rarely what goes wrong - unclear prompts, missing data, or the wrong model choice play a much bigger role.

It does become relevant once you:

• Structurally process large documents or entire knowledge bases.
• Build an application that calls a model hundreds or thousands of times a day.
• Notice that conversations or documents exceed a model's context window, making responses less accurate.

In those cases, it's worth having the real, structural cost of a solution calculated up front, and choosing an architecture (for example, summarization or targeted retrieval instead of feeding in whole documents) that keeps token usage under control without sacrificing quality.

Related concepts

Tokens are closely tied to a few other foundational concepts:

• Context window - the maximum number of tokens a model can process at once.
• LLM (large language model) - the underlying model that reads and generates tokens.
• Prompt engineering - how you phrase a task, which directly affects how many tokens are needed.
• Cost of AI usage - largely driven by token consumption in API-based applications.

Want to know how this plays out for your organization in practice - in terms of cost, architecture, and which use cases actually deliver value? Take the free AI scan or see what our AI consultancy can do for you.