--- title: "Lost in the Prompt Order — paper by Ok and Lee (2026)" description: "This paper explains why decoder-only language models are sensitive to how a multiple-choice prompt is arranged. 14.7 points. 2 references." authors: ["Hyunjong Ok", "Jaeho Lee"] year: 2026 venue: "Findings of the Association for Computational Linguistics: ACL 2026" source: "https://aclanthology.org/2026.findings-acl.1921/" doi: "10.18653/v1/2026.findings-acl.1921" licence: "CC BY 4.0 (ACL Anthology)" licence_url: "https://creativecommons.org/licenses/by/4.0/" summary_licence: "Elamur Cosmic License" canonical: "https://research.elamur.ai/free/en/papers/ok-2026-lost-prompt-order-revealing-limitations/" --- [Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/ok-2026-lost-prompt-order-revealing-limitations/) [home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2026](https://md.research.elamur.ai/free/en/papers/2026/page/40/) / Lost in the Prompt Order: Revealing the Limitations of Causal Attention in Language Models # Lost in the Prompt Order: Revealing the Limitations of Causal Attention in Language Models **Authors:** [Hyunjong Ok](https://md.research.elamur.ai/free/en/authors/hyunjong-ok/) and [Jaeho Lee](https://md.research.elamur.ai/free/en/authors/jaeho-lee/) · **Year:** 2026 · **Venue:** [Findings of the Association for Computational Linguistics: ACL 2026](https://md.research.elamur.ai/free/en/venues/findings/) ([Findings of the Association for Computational Linguistics 2026](https://md.research.elamur.ai/free/en/venues/findings/2026/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/) **Links:** [Publisher's page](https://aclanthology.org/2026.findings-acl.1921/) · [PDF](https://aclanthology.org/2026.findings-acl.1921.pdf) · [DOI 10.18653/v1/2026.findings-acl.1921](https://doi.org/10.18653/v1/2026.findings-acl.1921) · [OpenAlex](https://openalex.org/W7166801060) **Citations:** 0 citations (OpenAlex, 2026-10-14) ## Summary This paper explains why decoder-only language models are sensitive to how a multiple-choice prompt is arranged. When the passage comes first, followed by the question and options (CQO), models do much better than when the passage comes last (QOC). The authors tested 21 decoder-only LLMs from the LLaMA 3, Qwen 2.5/3 and Gemma 2 families (0.5B–9B parameters, base and instruct variants) on LogiQA, RACE-H, RACE-M and SciQ. They then checked three possible explanations through controlled experiments. Two explanations failed. Unfamiliarity with the format from training data did not account for the gap, and neither did the models forgetting options placed mid-prompt. The evidence instead points to the causal attention mask. In QOC, option tokens are encoded before the context appears, so their representations carry no information from it. Several experiments support this account: - Bidirectional architectures show almost no gap. - QOC accuracy roughly equals accuracy with the context removed entirely. - Targeted interventions on the option–context pathway move accuracy in the predicted directions. The work turns a known prompt-sensitivity effect into a mechanistic explanation. It also suggests cheap fixes for inference time. ## Key points - **Size of the gap:** averaged over 21 models, accuracy falls from 69.26% (CQO) to 54.54% (QOC), a gap of about 14.7 points. - **Architecture comparison:** Flan-T5 (encoder-decoder) shows a 2.30-point gap, and BERT, RoBERTa and ALBERT (encoder-only) show 0.02. - **Rejected explanations:** - Base and instruct models have nearly identical gaps. - Five-shot ICL lifts QOC by only about 3.1 points. - Option recall under QOC is as good as or better than under CQO. - **Attribution and modulating factors:** Gradient×Input attribution gives the context 0.797 of the total attribution in CQO but only 0.335 in QOC. Longer contexts widen the gap (24.8% on RACE-M vs. 6.2% on LogiQA). Correct answers in position A suffer more (22.4%) than those in position D (9.9%). - **Interventions:** - Blocking option-to-context attention in CQO costs 26.8 points. - Activation patching improves QOC by 6.0 points. - Repeating the options after the context (QOCO) improves QOC by 8.2 points. - CoT prompting shrinks the gap from 14.72 to 7.47 points. *Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.* 📚 Shelves (5): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 18,614 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 17,965 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 17,450 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 17,296 works · [👁️ Computer Vision and Pattern Recognition](https://md.research.elamur.ai/free/en/shelves/computer-science/computer-vision-and-pattern-recognition/) · 3,486 works 🧭 Topics (1): [🧠 Language models & pretraining](https://md.research.elamur.ai/free/en/topics/language-models/) · 3,894 works ## Cite Cite as: Hyunjong Ok and Jaeho Lee. 2026. Lost in the Prompt Order: Revealing the Limitations of Causal Attention in Language Models. Findings of the Association for Computational Linguistics: ACL 2026. The paper itself, not this page. BibTeX from the library's record of the work. ```bibtex @inproceedings{ok2026lost, title = {{Lost in the Prompt Order: Revealing the Limitations of Causal Attention in Language Models}}, author = {Hyunjong Ok and Jaeho Lee}, booktitle = {Findings of the Association for Computational Linguistics: ACL 2026}, year = {2026}, doi = {10.18653/v1/2026.findings-acl.1921}, url = {https://doi.org/10.18653/v1/2026.findings-acl.1921} } ``` - [← previous Lost in Quantization: Activation Outliers Explain Language-Specific FP8 Sensitivity in Llama-3](https://md.research.elamur.ai/free/en/papers/silva-2026-lost-quantization-activation-outliers-explain/) - [next → Lost in Translation, and Found: Detecting and Interpreting Translation Effects](https://md.research.elamur.ai/free/en/papers/wein-2026-lost-translation-found-detecting-interpreting/) - [↑ back to papers of 2026](https://md.research.elamur.ai/free/en/papers/2026/page/40/) ## References 2 references, 2 of them with a card of ours. ↗ leads to the work. 1. Samira Abnar and Willem Zuidema. 2020. [Quantifying attention flow in transformers](https://md.research.elamur.ai/free/en/papers/abnar-2020-quantifying-attention-flow-transformers/). In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 4190–4197. 2. Shengnan An, Zexiong Ma, Zeqi Lin, Nanning Zheng, Jian-Guang Lou, and Weizhu Chen. 2024. [Make your llm fully utilize the context](https://md.research.elamur.ai/free/en/papers/an-2024-make-llm-fully-utilize-context/). Advances in Neural Information Processing Systems, 37:62160–62188. \#\# Conclusion *Updated 2 November 2026*