Why Does Vietnamese Use More AI Tokens Than English?

October 15, 2026

Type the same sentence in Vietnamese and in English, and the AI will usually break the Vietnamese one into more tokens, sometimes two or three times as many. The main reason is Vietnamese's tone marks and vowel diacritics: the tokenizer that most AI models use was trained mostly on English text, so it does not recognize Vietnamese words as familiar chunks and has to cut them into smaller pieces. Understanding this explains why using AI in Vietnamese can sometimes cost more, run into limits faster, and get cut off sooner than the same conversation in English.

Quick Everyday Analogy

Think of tokens as Lego bricks the AI uses to build sentences. That brick set was manufactured by a factory tuned for English: it already has bricks shaped exactly like common English word pieces such as "the," "ing," or "tion." When you feed the AI an English word, it usually just snaps together one or two ready made bricks.

Vietnamese, though, has many shapes that factory never molded. A word like "được" carries an acute accent and a hook above the vowel; "nghiêng" stacks three vowels together with a tone mark. Since no single brick matches, the AI has to break the word into very small pieces, sometimes down to individual letters or bytes, then snap together many tiny bricks to rebuild one word. More pieces means more tokens, like building a wall out of rubble instead of whole bricks.

How It Actually Works

AI does not read letters, it reads tokens. Before answering anything, every generative AI model first splits your text into tokens, chunks of characters treated as a single unit. This happens before the model does any "thinking."
The token vocabulary is learned from training data. Major AI labs train their tokenizers with an algorithm called BPE (byte pair encoding). It scans huge amounts of text and merges character sequences that repeat often into their own token. Since most training data is English (and Chinese for some models), common English chunks get merged into whole-word tokens very early.
Vietnamese has too many accented variations to merge them all. Vietnamese is a tonal, isolating language: the syllable "ma" alone has six different meanings depending on the tone mark, "ma, má, mà, mả, mã, mạ." Each vowel-plus-tone combination is a different Unicode sequence. Giving the tokenizer enough "bricks" for every combination would require a huge Vietnamese-specific vocabulary, but most popular models do not reserve enough room for that.
The result: Vietnamese words get chopped up. According to recent measurements, compared to English, GPT-2 or RoBERTa style tokenizers can need more than four times as many tokens for the same Vietnamese passage, while more modern multilingual tokenizers still cost roughly 1.4 to nearly 2 times more depending on the model. This gap is sometimes jokingly called the "token tax" paid by speakers of less common languages.
Models trained specifically for Vietnamese do much better. Tokenizers built specifically for Vietnamese, similar to PhoBERT's approach, reach token counts close to English because they learn the language's actual syllable and tone structure from the start, instead of forcing Vietnamese into a vocabulary built for a different language.

What This Means For You

Writing in Vietnamese with AI can cost more than English for the same content. Since API plans charge by the token, a 200 word Vietnamese passage can translate into roughly the same token count as a 400 to 600 word English passage. If you are building a service that processes large volumes of Vietnamese text with AI, such as a customer support chatbot or document summarizer, budget your costs using real Vietnamese token counts, not an English based estimate.
The context window fills up faster when you write in Vietnamese. If a model has a 32,000 token limit, a Vietnamese document will hit that limit sooner than an English document of similar sentence length. If you paste a long document and the AI says it is "too long," remember part of the reason may simply be that Vietnamese is naturally more token-heavy, not that your document is actually too long.
A practical tip: if you need AI to process a very long Vietnamese document in one go, such as pasting an entire contract to summarize, try a model advertised as well optimized for Vietnamese or Asian languages. These usually use more efficient tokenizers for tone marks, letting you fit more content within the same token limit.

Common Misconceptions

"A token is a word." Not true. A token can be a whole word, a piece of a word, or just a few characters. For Vietnamese, a single accented word is often split into two or three tokens instead of one.
"Every AI wastes the same number of tokens on Vietnamese." Not true either. The gap depends on each company's tokenizer: some models only cost about 1.4 times more than English, while older models can cost four times more. Models trained with more Vietnamese data and better designed tokenizers tend to waste fewer tokens.
"Typing without diacritics saves tokens." Technically this can be partly true, since unaccented text looks closer to English letters, but you should not do this: the AI will understand your intent less accurately, since many unaccented words collide with each other. Every article on vibeaicoder.xyz recommends typing Vietnamese with full diacritics so the AI understands and answers correctly.

FAQ

How many times more tokens does Vietnamese use compared to English? There is no single fixed number for every AI. Depending on the model and tokenizer, the gap ranges from roughly 1.4 times to more than 4 times compared to an English passage of similar sentence length.

Why hasn't every AI company fixed this? Rebuilding a tokenizer for a language requires retraining almost the entire model from scratch, which takes significant time and cost, so companies typically only improve this gradually with each new model version.

Does this affect how smart the answers are? Not directly. A higher token count mainly affects cost and the amount of text you can send or receive in one turn. It does not make the AI understand Vietnamese worse, as long as it was trained on enough quality Vietnamese data.

How can I check how many tokens a piece of Vietnamese text uses? Some AI companies offer online token counting tools for their models. You can paste a Vietnamese passage and an equivalent English passage to compare directly.

Should I avoid using Vietnamese with AI to save costs? Not necessary for everyday users. This difference matters more if you are building a service that processes large volumes of Vietnamese text through an API. For normal daily conversations or writing help, just keep typing Vietnamese with full diacritics so the AI understands you correctly.

To understand how AI pricing works through tokens in more depth, read how AI pricing works across subscriptions, APIs, and input and output tokens. If you are not yet clear on how AI "predicts the next word" using tokens, the article on what a large language model (LLM) is will give you the foundation first. How well a model handles Vietnamese also depends on its training data, which you can compare further in open source versus closed source AI models. And if you want to learn how to write Vietnamese prompts the AI understands correctly, see the guide on what a prompt is and how to write effective ones.

#AI #Vietnamese #tokens #AIfoundations