---
title: "The Curse of Verbalization — paper by Zhou et al. (2026)"
description: "It proposes that one hidden cause is a mismatch between the order in which facts appear in a prompt and the order in which a… 18 references."
authors: ["Yue Zhou", "Henry Peng Zou", "Barbara Di Eugenio", "Yang Zhang"]
year: 2026
venue: "Findings of the Association for Computational Linguistics: EACL 2026"
source: "https://aclanthology.org/2026.findings-eacl.218/"
doi: "10.18653/v1/2026.findings-eacl.218"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/zhou-2026-curse-verbalization-how-presentation-order/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/zhou-2026-curse-verbalization-how-presentation-order/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2026](https://md.research.elamur.ai/free/en/papers/2026/page/52/) / The Curse of Verbalization: How Presentation Order Constrains LLM Reasoning

# The Curse of Verbalization: How Presentation Order Constrains LLM Reasoning

**Authors:** [Yue Zhou](https://md.research.elamur.ai/free/en/authors/yue-zhou/), [Henry Peng Zou](https://md.research.elamur.ai/free/en/authors/henry-peng-zou/), [Barbara Di Eugenio](https://md.research.elamur.ai/free/en/authors/barbara-di-eugenio/) and [Yang Zhang](https://md.research.elamur.ai/free/en/authors/yang-zhang/) · **Year:** 2026 · **Venue:** [Findings of the Association for Computational Linguistics: EACL 2026](https://md.research.elamur.ai/free/en/venues/findings/) ([Findings of the Association for Computational Linguistics 2026](https://md.research.elamur.ai/free/en/venues/findings/2026/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2026.findings-eacl.218/) · [PDF](https://aclanthology.org/2026.findings-eacl.218.pdf) · [DOI 10.18653/v1/2026.findings-eacl.218](https://doi.org/10.18653/v1/2026.findings-eacl.218) · [OpenAlex](https://openalex.org/W7140112950)

**Citations:** 0 citations (OpenAlex, 2026-10-14)

## Summary
The paper asks why certain problems are hard for LLMs. It proposes that one hidden cause is a mismatch between the order in which facts appear in a prompt and the order in which a solution needs them. The authors first test GPT-4o on tiny one-dimensional Befunge programs. Befunge is an esoteric language that is rare online, so contamination is unlikely. Accuracy drops sharply when execution starts from the right or reverses direction.

They then rewrite GSM8K, MATH and AIME problems so that conditions follow the order used in reference solutions. A reversed version serves as a control. Aligned versions improve results for GPT-3.5-turbo, GPT-4o, Claude 3.7 and DeepSeek-R1, while reversal usually hurts.

Their explanation, called the "Curse of Verbalization", is that training on human text teaches models to work through inputs in the order given. Pseudo-math problems built from invented words show that a model's flexibility in departing from that order tracks its general reasoning strength. This offers a contamination-resistant way to diagnose what makes problems difficult.

## Key points
- Pilot test: 200 randomly generated 1-D Befunge programs per execution pattern (five patterns, e.g. "→ only", "← only", "→ then ←"), printing at most five characters. GPT-4o was evaluated with 0, 8, 32 and 128 in-context examples.
- "→ only" was the easiest pattern, followed by "→ then ←". Programs starting at the rightmost cell or needing multiple traversals were much harder. Adding more in-context examples raised accuracy.
- Because math problems have no single true utilization order, the order in the reference solution served as a proxy. Aligned reordering consistently raised pass@k and maj@k, including for DeepSeek-R1 under several token budgets.
- Pseudo-mathematical problems with nonsense terms (e.g. "plumbus", "fleeb", "chumble") revealed that weaker models verbalize terms in their given order, while stronger ones restructure more freely.
- Across twelve LLMs, verbalization flexibility correlated with LMArena reasoning rankings at ρ = 0.87. The authors report that mitigating this bias is difficult.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [IBM](https://md.research.elamur.ai/free/en/companies/ibm/) — Company, Armonk, United States · 154 works · 72,606 citations
- [University of Illinois Chicago](https://md.research.elamur.ai/free/en/universities/university-of-illinois-chicago/) — University, Chicago, United States · 70 works · 20,363 citations

📚 Shelves (5): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 18,614 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 17,965 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 17,450 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 17,296 works · [👁️ Computer Vision and Pattern Recognition](https://md.research.elamur.ai/free/en/shelves/computer-science/computer-vision-and-pattern-recognition/) · 3,486 works

🧭 Topics (2): [🧠 Language models & pretraining](https://md.research.elamur.ai/free/en/topics/language-models/) · 3,894 works · [🧮 Reasoning](https://md.research.elamur.ai/free/en/topics/reasoning/) · 1,685 works

## Cite

Cite as: Yue Zhou, Henry Peng Zou, Barbara Di Eugenio and Yang Zhang. 2026. The Curse of Verbalization: How Presentation Order Constrains LLM Reasoning. Findings of the Association for Computational Linguistics: EACL 2026.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{zhou2026curse,
  title = {{The Curse of Verbalization: How Presentation Order Constrains LLM Reasoning}},
  author = {Yue Zhou, Henry Peng Zou, Barbara {Di Eugenio} and Yang Zhang},
  booktitle = {Findings of the Association for Computational Linguistics: EACL 2026},
  year = {2026},
  doi = {10.18653/v1/2026.findings-eacl.218},
  url = {https://doi.org/10.18653/v1/2026.findings-eacl.218}
}
```

- [← previous The Cost of Perfect English: Pragmatic Flattening and the Erasure of Authorial Voice in L2 Writing Supported by GenAI](https://md.research.elamur.ai/free/en/papers/liu-2026-cost-perfect-english-pragmatic-flattening/)
- [next → The Data Frontier for Large Language Models: Selection, Synthesis, and Tools](https://md.research.elamur.ai/free/en/papers/wu-2026-data-frontier-large-language-models/)
- [↑ back to papers of 2026](https://md.research.elamur.ai/free/en/papers/2026/page/52/)

## References

18 references, 11 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Anthropic. 2025. Claude 3.7 sonnet system card. [find](https://scholar.google.com/scholar?q=Anthropic%2E+2025%2E+Claude+3%2E7+sonnet+system+card%2E)
2. Maciej Besta, Julia Barth, Eric Schreiber, Ales Kubicek, Afonso Catarino, Robert Gerstenberger, Piotr Nyczyk, Patrick Iff, Yueling Li, Sam Houliston, and 1 others. 2025. Reasoning language models: A blueprint. arXiv preprint arXiv:2501.11223. — [arXiv:2501.11223](https://arxiv.org/abs/2501.11223) [find](https://scholar.google.com/scholar?q=Maciej+Besta%2C+Julia+Barth%2C+Eric+Schreiber%2C+Ales+Kubicek%2C+Afonso+Catarino%2C+Robert+Gerstenberger%2C+Piotr+Nyczyk%2C+Patrick+Iff%2C+Yueling+Li%2C+Sam+Houliston%2C+and+1+others%2E+2025%2E+Reasoning+language+models%3A+A+b)
3. Maciej Besta, Nils Blach, Ales Kubicek, Robert Gerstenberger, Lukas Gianinazzi, Joanna Gajda, Tomasz Lehmann, Michal Podstawski, Hubert Niewiadomski, Piotr Nyczyk, and Torsten Hoefler. 2023. Graph of thoughts: Solving elaborate problems with large language models. Preprint, arXiv:2308.09687. [find](https://scholar.google.com/scholar?q=Maciej+Besta%2C+Nils+Blach%2C+Ales+Kubicek%2C+Robert+Gerstenberger%2C+Lukas+Gianinazzi%2C+Joanna+Gajda%2C+Tomasz+Lehmann%2C+Michal+Podstawski%2C+Hubert+Niewiadomski%2C+Piotr+Nyczyk%2C+and+Torsten+Hoefler%2E+2023%2E+Graph+of+)
4. Yixin Cao, Shibo Hong, Xinze Li, Jiahao Ying, Yubo Ma, Haiyuan Liang, Yantao Liu, Zijun Yao, Xiaozhi Wang, Dan Huang, and 1 others. 2025. Toward generalizable evaluation in the llm era: A survey beyond benchmarks. arXiv preprint arXiv:2504.18838. — [arXiv:2504.18838](https://arxiv.org/abs/2504.18838) [find](https://scholar.google.com/scholar?q=Yixin+Cao%2C+Shibo+Hong%2C+Xinze+Li%2C+Jiahao+Ying%2C+Yubo+Ma%2C+Haiyuan+Liang%2C+Yantao+Liu%2C+Zijun+Yao%2C+Xiaozhi+Wang%2C+Dan+Huang%2C+and+1+others%2E+2025%2E+Toward+generalizable+evaluation+in+the+llm+era%3A+A+survey+beyon)
5. Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, and 1 others. 2021b. [Evaluating large language models trained on code](https://md.research.elamur.ai/free/en/papers/chen-2021-evaluating-large-language-models-trained/). arXiv preprint arXiv:2107.03374.
6. Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, and 1 others. 2021a. [Training verifiers to solve math word problems](https://md.research.elamur.ai/free/en/papers/cobbe-2021-training-verifiers-solve-math-word/). arXiv preprint arXiv:2110.14168.
7. Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, and 1 others. 2021b. [Training verifiers to solve math word problems](https://md.research.elamur.ai/free/en/papers/cobbe-2021-training-verifiers-solve-math-word/). arXiv preprint arXiv:2110.14168.
8. Yihong Dong, Xue Jiang, Huanyu Liu, Zhi Jin, Bin Gu, Mengfei Yang, and Ge Li. 2024. Generalization or memorization: Data contamination and trustworthy evaluation for large language models. In Findings of the Association for Computational Linguistics: ACL 2024, pages 12039–12050, Bangkok, Thailand. A… [find](https://scholar.google.com/scholar?q=Yihong+Dong%2C+Xue+Jiang%2C+Huanyu+Liu%2C+Zhi+Jin%2C+Bin+Gu%2C+Mengfei+Yang%2C+and+Ge+Li%2E+2024%2E+Generalization+or+memorization%3A+Data+contamination+and+trustworthy+evaluation+for+large+language+models%2E+In+Findings)
9. Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, and 1 others. 2025. [Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning](https://md.research.elamur.ai/free/en/papers/deepseek-ai-2025-deepseek-r1-incentivizing-reasoning-capability/). arXiv preprint arXiv:2501.12948.
10. Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardt. [Measuring mathematical problem solving with the math dataset](https://md.research.elamur.ai/free/en/papers/hendrycks-2021-measuring-mathematical-problem-solving-math/). In Thirty-fifth Conference on Neural Information Processing Systems Datasets and Benchmarks Track (Round 2).
11. Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardt. 2021. [Measuring mathematical problem solving with the math dataset](https://md.research.elamur.ai/free/en/papers/hendrycks-2021-measuring-mathematical-problem-solving-math/). NeurIPS.
12. Jie Huang and Kevin Chen-Chuan Chang. 2023. [Towards reasoning in large language models: A survey](https://md.research.elamur.ai/free/en/papers/huang-2023-reasoning-large-language-models-survey/). In Findings of the Association for Computational Linguistics: ACL 2023, pages 1049–1065.
13. Naman Jain, King Han, Alex Gu, Wen-Ding Li, Fanjia Yan, Tianjun Zhang, Sida Wang, Armando Solar-Lezama, Koushik Sen, and Ion Stoica. 2024. [Livecodebench: Holistic and contamination free evaluation of large language models for code](https://md.research.elamur.ai/free/en/papers/jain-2024-livecodebench-holistic-contamination-free-evaluation/). Preprint, arXiv:2403.07974.
14. Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde de Oliveira Pinto, and 1 others. 2021a. [Evaluating large language models trained on code](https://md.research.elamur.ai/free/en/papers/chen-2021-evaluating-large-language-models-trained/). Preprint, arXiv:2107.03374.
15. Nelson F. Liu, Kevin Lin, John Hewitt, Ashwin Paranjape, Michele Bevilacqua, Fabio Petroni, and Percy Liang. 2024. [Lost in the middle: How language models use long contexts](https://md.research.elamur.ai/free/en/papers/liu-2023-lost-middle-language-models-use/). Transactions of the Association for Computational Linguistics, 12:157–173.
16. Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde De Oliveira Pinto, Jared Kaplan, OpenAI. 2024. Gpt-4o — [GPT-4o System Card](https://md.research.elamur.ai/free/en/papers/openai-2024-gpt-4o-system-card/) [https://arxiv.org/pdf/2410.21276](https://arxiv.org/pdf/2410.21276). system card. [↗](https://arxiv.org/pdf/2410.21276)
17. Zhenting Qi, Hongyin Luo, Xuliang Huang, Zhuokai Zhao, Yibo Jiang, Xiangjun Fan, Himabindu Lakkaraju, and James R. Glass. 2025. Quantifying generalization complexity for large language models. In The Thirteenth International Conference on Learning Representations. [find](https://scholar.google.com/scholar?q=Zhenting+Qi%2C+Hongyin+Luo%2C+Xuliang+Huang%2C+Zhuokai+Zhao%2C+Yibo+Jiang%2C+Xiangjun+Fan%2C+Himabindu+Lakkaraju%2C+and+James+R%2E+Glass%2E+2025%2E+Quantifying+generalization+complexity+for+large+language+models%2E+In+The+)
18. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), pages 2793–2804, Mexico City, Mexico. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=In+Proceedings+of+the+2024+Conference+of+the+North+American+Chapter+of+the+Association+for+Computational+Linguistics%3A+Human+Language+Technologies+%28Volume+1%3A+Long+Papers%29%2C+pages+2793%E2%80%932804%2C+Mexico+City%2C)

*Updated 2 November 2026*