--- title: "The Token Tax: Systematic Bias in Multilingual Tokenization (2025)" description: "Subword tokenizers split text in morphologically rich, low-resource languages into far more pieces than… 7 references, cited by 10 papers here." authors: ["Jessica M. Lundin", "Ada Zhang", "Nihal Karim", "Hamza Louzan", "Victor Wei", "David Adelani", "Cody Carroll"] year: 2025 venue: "arXiv:2509.05486 · Proceedings of the 7th Workshop on African Natural Language Processing (AfricaNLP 2026)" source: "https://arxiv.org/abs/2509.05486" doi: "10.18653/v1/2026.africanlp-main.10" arxiv: "2509.05486" licence: "CC BY 4.0 (arXiv)" licence_url: "http://creativecommons.org/licenses/by/4.0/" summary_licence: "Elamur Cosmic License" canonical: "https://research.elamur.ai/free/en/papers/lundin-2025-token-tax-systematic-bias-multilingual/" --- [Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/lundin-2025-token-tax-systematic-bias-multilingual/) [home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2025](https://md.research.elamur.ai/free/en/papers/2025/page/7/) / The Token Tax: Systematic Bias in Multilingual Tokenization # The Token Tax: Systematic Bias in Multilingual Tokenization **Authors:** [Jessica M. Lundin](https://md.research.elamur.ai/free/en/authors/jessica-lundin/), Ada Zhang, Nihal Karim, Hamza Louzan, Victor Wei, [David Adelani](https://md.research.elamur.ai/free/en/authors/david-ifeoluwa-adelani/) and Cody Carroll · **Year:** 2025 · **Venue:** [arXiv:2509.05486 · Proceedings of the 7th Workshop on African Natural Language Processing (AfricaNLP 2026)](https://md.research.elamur.ai/free/en/venues/africanlp/) ([Workshop on African Natural Language Processing 2025](https://md.research.elamur.ai/free/en/venues/africanlp/2025/)) · **Licence:** [CC BY 4.0 (arXiv)](http://creativecommons.org/licenses/by/4.0/) **Links:** [Publisher's page](https://doi.org/10.18653/v1/2026.africanlp-main.10) · [PDF](https://arxiv.org/pdf/2509.05486) · [DOI 10.18653/v1/2026.africanlp-main.10](https://doi.org/10.18653/v1/2026.africanlp-main.10) · [arXiv 2509.05486](https://arxiv.org/abs/2509.05486) · [OpenAlex](https://openalex.org/W4415059154) **Citations:** 2 citations (OpenAlex, 2026-10-14) · [10](https://md.research.elamur.ai/free/en/papers/lundin-2025-token-tax-systematic-bias-multilingual/cited-by/) in our library, of the 149,439 papers whose bibliographies we read ## Summary Subword tokenizers split text in morphologically rich, low-resource languages into far more pieces than English. This paper measures what that costs in accuracy and money. The authors evaluate 10 LLMs on AfriMMLU, a multiple-choice benchmark of 9,000 items across five subjects in 16 African languages. For each model and language, they compute fertility (average tokens per word) with that model's own tokenizer and record accuracy. They then fit a linear regression of accuracy on fertility for each model–subject pair. A mixed-effects model selected by AIC adds language-specific intercepts and slopes, which suggests the fertility effect differs by language. Across every model and subject, higher fertility goes with lower accuracy, and fertility explains a substantial share of the variation. Reasoning models such as DeepSeek and o1 narrow the gap to English but do not close it. Because attention cost grows quadratically with sequence length, doubling the token count roughly quadruples training cost and time. The authors treat tokenization as a structural equity problem. They call for morphology-aware tokenizers, fairer pricing and wider multilingual benchmarks. ## Key points - Regression slopes range from −0.08 to −0.18, so each extra token per word costs about 8–18 accuracy points. Fertility explains 20–50% of accuracy variance. Examples include Llama-3.1-405B on economics (−0.185, p = 0.002; the table labels it "Microeconomics", although the AfriMMLU subject is macroeconomics) and Qwen-2.5-32B on Geography (−0.155, p = 0.006). - On average, African languages score about 25 points below English, and French falls in between. DeepSeek and o1 add 8–12 points on African languages. On Global Facts, they shrink the English–African gap from 25 to 12–14 points. - Estimated cost of 2× fertility: - Training Llama-3.1-405B rises from $105M to $420M. - GPT-4o inference per 1M English-equivalent tokens rises from $5–20 to $10–40. - Latency goes from about 2 s to 4 s. The conclusion's claim of a 4× latency increase does not match this figure. - The mixed-effects analysis leaves out Amharic because it is less similar to the other languages. The accuracy figures count 17 languages, including Amharic. - The authors release public data: AfriMMLU results for all models, including the reasoning models, and MMLU token metrics. *Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.* ## Institutions - [McGill University](https://md.research.elamur.ai/free/en/universities/mcgill-university/) — University, Montreal, Canada · 87 works · 11,027 citations - [Mila - Quebec Artificial Intelligence Institute](https://md.research.elamur.ai/free/en/labs/mila-quebec-artificial-intelligence-institute/) — Research institute, Montreal, Canada · 54 works · 13,565 citations - [Canadian Institute for Advanced Research](https://md.research.elamur.ai/free/en/labs/canadian-institute-for-advanced-research/) — Research institute, Toronto, Canada · 28 works · 16,725 citations - [Gates Foundation](https://md.research.elamur.ai/free/en/ngos/gates-foundation/) — Nonprofit, Seattle, United States · 2 works · 85 citations Also on the byline: [University of San Francisco](https://ror.org/029m7xn54) · [Institute for Disease Modeling](https://ror.org/0532gcg76). 📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 16,039 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 15,891 works 🧭 Topics (2): [🌍 Multilingual & cross-lingual NLP](https://md.research.elamur.ai/free/en/topics/multilingual-nlp/) · 828 works · [✂️ Tokenization & segmentation](https://md.research.elamur.ai/free/en/topics/tokenization/) · 119 works ## Cite Cite as: Jessica M. Lundin, Ada Zhang, Nihal Karim, Hamza Louzan, Victor Wei, David Adelani and Cody Carroll. 2025. The Token Tax: Systematic Bias in Multilingual Tokenization. arXiv:2509.05486 · Proceedings of the 7th Workshop on African Natural Language Processing (AfricaNLP 2026). The paper itself, not this page. BibTeX from the library's record of the work. ```bibtex @misc{lundin2025token, title = {{The Token Tax: Systematic Bias in Multilingual Tokenization}}, author = {Jessica M. Lundin, Ada Zhang, Nihal Karim, Hamza Louzan, Victor Wei, David Adelani and Cody Carroll}, year = {2025}, doi = {10.18653/v1/2026.africanlp-main.10}, eprint = {2509.05486}, archivePrefix = {arXiv}, url = {https://doi.org/10.18653/v1/2026.africanlp-main.10} } ``` - [← previous The Impact of Token Granularity on the Predictive Power of Language Model Surprisal](https://md.research.elamur.ai/free/en/papers/oh-2025-impact-token-granularity-predictive-power/) - [next → ToDi: Token-wise Distillation via Fine-Grained Divergence Control](https://md.research.elamur.ai/free/en/papers/jung-2025-todi-token-wise-distillation-fine/) - [↑ back to papers of 2025](https://md.research.elamur.ai/free/en/papers/2025/page/7/) ## References 7 references, 4 of them with a card of ours. ↗ leads to the work. 1. David Ifeoluwa Adelani, Jessica Ojo, Israel Abebe Azime, Jian Yun Zhuang, Jesujoba Oluwadara Alabi, Xuanli He, Millicent Ochieng, Sara Hooker, Andiswa Bukula, En-Shiun Annie Lee, Chiamaka Ijeoma Chukwuneke, Happy Buzaaba, Blessing Kudzaishe Sibanda, Godson Koffi Kalipe, Jonathan Mukiibi, Salomon Kab… — [IrokoBench: A New Benchmark for African Languages in the Age of Large Language Models](https://md.research.elamur.ai/free/en/papers/adelani-2025-irokobench-new-benchmark-african-languages/) 2. Mehdi Ali, Michael Fromm, Klaudia Thellmann, Richard Rutmann, Max Lübbering, Johannes Leveling, Katrin Klug, Jan Ebert, Niclas Doll, Jasper Buschhoff, Charvi Jain, Alexander Weber, Lena Jurkschat, Hammam Abdelwahab, Chelsea John, Pedro Ortiz Suarez, Malte Ostendorff, Samuel Weinbach, Rafet Sifa, Ste… — [Tokenizer Choice For LLM Training: Negligible or Crucial?](https://md.research.elamur.ai/free/en/papers/ali-2024-tokenizer-choice-llm-training-negligible/) 3. Feyza Duman Keles, Pruthuvi Mahesakya Wijewardena, and Chinmay Hegde. On the computational complexity of self-attention, 2022. URL https://arxiv.org/abs/2209.04881. — [arXiv:2209.04881](https://arxiv.org/abs/2209.04881) [↗](https://arxiv.org/abs/2209.04881) 4. Aleksandar Petrov, Emanuele La Malfa, Philip Torr, and Adel Bibi. [Language model tokenizers introduce unfairness between languages](https://md.research.elamur.ai/free/en/papers/petrov-2023-language-model-tokenizers-introduce-unfairness/). In Thirty-seventh Conference on Neural Information Processing Systems, 2023a. URL [https://openreview.net/forum?id=78yDLKi95p](https://openreview.net/forum?id=78yDLKi95p). [↗](https://openreview.net/forum?id=78yDLKi95p) 5. Aleksandar Petrov, Emanuele La Malfa, Philip H. S. Torr, and Adel Bibi. [Language model tokenizers introduce unfairness between languages](https://md.research.elamur.ai/free/en/papers/petrov-2023-language-model-tokenizers-introduce-unfairness/), 2023b. URL https://arxiv.org/abs/2305.15425. [↗](https://arxiv.org/abs/2305.15425) 6. Makesh Narsimhan Sreedhar, Xiangpeng Wan, Yu Cheng, and Junjie Hu. Local byte fusion for neural machine translation, 2023. URL https://arxiv.org/abs/2205.11490. [↗](https://arxiv.org/abs/2205.11490) 7. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need, 2023. URL https://arxiv.org/abs/ 1706.03762. [↗](https://arxiv.org/abs/) *Updated 1 November 2026*