---
title: "“My Answer is C” — paper by Wang et al. (2024)"
description: "This study asks whether that shortcut reflects what the model actually writes. 99% accuracy. 27 references, cited by 74 papers here."
authors: ["Xinpeng Wang", "Bolei Ma", "Chengzhi Hu", "Leon Weber-Genzel", "Paul Röttger", "Frauke Kreuter", "Dirk Hovy", "Barbara Plank"]
year: 2024
venue: "Findings of the Association for Computational Linguistics: ACL 2024"
source: "https://aclanthology.org/2024.findings-acl.441/"
doi: "10.18653/v1/2024.findings-acl.441"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/wang-2024-my-answer-is-c-first/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/wang-2024-my-answer-is-c-first/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2024](https://md.research.elamur.ai/free/en/papers/2024/page/4/) / “My Answer is C”: First-Token Probabilities Do Not Match Text Answers in Instruction-Tuned Language Models

# “My Answer is C”: First-Token Probabilities Do Not Match Text Answers in Instruction-Tuned Language Models

**Authors:** Xinpeng Wang, [Bolei Ma](https://md.research.elamur.ai/free/en/authors/bolei-ma/), Chengzhi Hu, [Leon Weber-Genzel](https://md.research.elamur.ai/free/en/authors/leon-weber-genzel/), [Paul Röttger](https://md.research.elamur.ai/free/en/authors/paul-rottger/), [Frauke Kreuter](https://md.research.elamur.ai/free/en/authors/frauke-kreuter/), [Dirk Hovy](https://md.research.elamur.ai/free/en/authors/dirk-hovy/) and [Barbara Plank](https://md.research.elamur.ai/free/en/authors/barbara-plank/) · **Year:** 2024 · **Venue:** [Findings of the Association for Computational Linguistics: ACL 2024](https://md.research.elamur.ai/free/en/venues/findings/) ([Findings of the Association for Computational Linguistics 2024](https://md.research.elamur.ai/free/en/venues/findings/2024/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2024.findings-acl.441/) · [PDF](https://aclanthology.org/2024.findings-acl.441.pdf) · [DOI 10.18653/v1/2024.findings-acl.441](https://doi.org/10.18653/v1/2024.findings-acl.441) · [OpenAlex](https://openalex.org/W4402669926)

**Citations:** 15 citations (OpenAlex, 2026-10-14) · [74](https://md.research.elamur.ai/free/en/papers/wang-2024-my-answer-is-c-first/cited-by/) in our library, of the 141,840 papers whose bibliographies we read

## Summary
Multiple-choice benchmarks often score instruction-tuned LLMs by checking which option letter receives the highest log probability as the first generated token. This study asks whether that shortcut reflects what the model actually writes. Chat-style models often open with a polite preamble such as "Sure" or decline to answer sensitive questions. The authors tested six instruction-tuned models (Llama2-Chat 7b/13b/70b, Mistral-Instruct-v0.1/v0.2, Mixtral-8x7b-Instruct-v0.1) on MMLU and on a 414-question public-opinion subset of OpinionQA. They used four instruction prompts with increasing constraint and shuffled the answer order ten times per question. To extract the chosen option from free text, they trained a QLoRA-tuned Mistral-7b classifier on manually labelled responses. First-token predictions and text answers diverged sharply on several measures: the chosen option, how often the model refused, the distribution of choices, and stability under reordering. The divergence persisted even with strict format instructions. The authors conclude that evaluations should inspect generated text rather than rely only on first-token scoring.

## Key points
- **Answer mismatch:** first-token and text answers disagree in up to 66.2% of OpinionQA cases for Llama2-7b-Chat, falling to 13.3% for the 70b model.
- **Refusals:** refusal is a major source of mismatch, reaching 51.4% for Llama2-7b-Chat. On MMLU, this model refuses every "moral scenario" question in its text output, a failure that first-token scoring hides.
- **Prompt-format effects:** the Example Template prompt pushes Llama2-70b-Chat's first-token choice to "C" about 85% of the time. Its text answers stay far more evenly spread.
- **Consistency:** text answers are more consistent across shuffled option orders (lower entropy) than first-token answers for every model except Mixtral 8x7b.
- **Answer-extraction classifier:** the classifier was trained on 2070 annotated responses and reaches 99% accuracy, versus 55% for string matching. The classifiers and experimental results are released at github.com/mainlp/MCQ-Mismatch.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Ludwig-Maximilians-Universität München](https://md.research.elamur.ai/free/en/universities/ludwig-maximilians-universitat-munchen/) — University, Munich, Germany · 44 works · 55,193 citations
- [Bocconi University](https://md.research.elamur.ai/free/en/universities/bocconi-university/) — University, Milan, Italy · 16 works · 2,850 citations
- [Munich Center for Machine Learning](https://md.research.elamur.ai/free/en/universities/munich-center-for-machine-learning/) — University, Munich, Germany · 11 works · 655 citations

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 9,086 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 8,889 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 9,296 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 9,206 works

🧭 Topics (1): [🧠 Language models & pretraining](https://md.research.elamur.ai/free/en/topics/language-models/) · 1,343 works

## Cite

Cite as: Xinpeng Wang, Bolei Ma, Chengzhi Hu, Leon Weber-Genzel, Paul Röttger, Frauke Kreuter, Dirk Hovy and Barbara Plank. 2024. “My Answer is C”: First-Token Probabilities Do Not Match Text Answers in Instruction-Tuned Language Models. Findings of the Association for Computational Linguistics: ACL 2024.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{wang2024my,
  title = {{“My Answer is C”: First-Token Probabilities Do Not Match Text Answers in Instruction-Tuned Language Models}},
  author = {Xinpeng Wang, Bolei Ma, Chengzhi Hu, Leon Weber-Genzel, Paul Röttger, Frauke Kreuter, Dirk Hovy and Barbara Plank},
  booktitle = {Findings of the Association for Computational Linguistics: ACL 2024},
  year = {2024},
  doi = {10.18653/v1/2024.findings-acl.441},
  url = {https://doi.org/10.18653/v1/2024.findings-acl.441}
}
```

- [← previous WildTeaming at Scale: From In-the-Wild Jailbreaks to (Adversarially) Safer Language Models](https://md.research.elamur.ai/free/en/papers/jiang-2024-wildteaming-scale-wild-jailbreaks-adversarially/)
- [next → CultureBank: An Online Community-Driven Knowledge Base Towards Culturally Aware Language Technologies](https://md.research.elamur.ai/free/en/papers/shi-2024-culturebank-online-community-driven-knowledge/)
- [↑ back to papers of 2024](https://md.research.elamur.ai/free/en/papers/2024/page/4/)

## References

27 references, 10 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. BIG bench authors. 2023. [Beyond the imitation game: Quantifying and extrapolating the capabilities of language models](https://md.research.elamur.ai/free/en/papers/srivastava-2022-imitation-game-quantifying-extrapolating-capabilities/). Transactions on Machine Learning Research.
2. Yonatan Bisk, Rowan Zellers, Jianfeng Gao, Yejin Choi, et al. 2020. [Piqa: Reasoning about physical commonsense in natural language](https://md.research.elamur.ai/free/en/papers/bisk-2019-piqa-reasoning-about-physical-commonsense/). In Proceedings of the AAAI conference on artificial intelligence, volume 34, pages 7432–7439.
3. Peter Clark, Isaac Cowhey, Oren Etzioni, Tushar Khot, Ashish Sabharwal, Carissa Schoenick, and Oyvind Tafjord. 2018. Think you have solved question answering? try arc, the ai2 reasoning challenge. arXiv preprint arXiv:1803.05457. [find](https://scholar.google.com/scholar?q=Peter+Clark%2C+Isaac+Cowhey%2C+Oren+Etzioni%2C+Tushar+Khot%2C+Ashish+Sabharwal%2C+Carissa+Schoenick%2C+and+Oyvind+Tafjord%2E+2018%2E+Think+you+have+solved+question+answering%3F+try+arc%2C+the+ai2+reasoning+challenge%2E+arX)
4. Tim Dettmers, Artidoro Pagnoni, Ari Holtzman, and Luke Zettlemoyer. 2024. Qlora: Efficient finetuning of quantized llms. Advances in Neural Information Processing Systems, 36. [find](https://scholar.google.com/scholar?q=Tim+Dettmers%2C+Artidoro+Pagnoni%2C+Ari+Holtzman%2C+and+Luke+Zettlemoyer%2E+2024%2E+Qlora%3A+Efficient+finetuning+of+quantized+llms%2E+Advances+in+Neural+Information+Processing+Systems%2C+36%2E)
5. Ricardo Dominguez-Olmedo, Moritz Hardt, and Celestine Mendler-Dünner. 2023. Questioning the survey responses of large language models. [find](https://scholar.google.com/scholar?q=Ricardo+Dominguez%2DOlmedo%2C+Moritz+Hardt%2C+and+Celestine+Mendler%2DD%C3%BCnner%2E+2023%2E+Questioning+the+survey+responses+of+large+language+models%2E)
6. Esin Durmus, Karina Nyugen, Thomas I Liao, Nicholas Schiefer, Amanda Askell, Anton Bakhtin, Carol Chen, Zac Hatfield-Dodds, Danny Hernandez, Nicholas Joseph, et al. 2023. [Towards measuring the representation of subjective global opinions in language models](https://md.research.elamur.ai/free/en/papers/durmus-2023-towards-measuring-representation-subjective-global/). arXiv preprint arXiv:2306.16388.
7. Clémentine Fourrier, Nathan Habib, Julien Launay, and Julien Wolf. 2023. What’s going on with the open LLM leaderboard? [https://huggingface.co/blog/](https://huggingface.co/blog/) evaluating-mmlu-leaderboard. Accessed: 2024-2-10. [↗](https://huggingface.co/blog/)
8. Leo Gao, Jonathan Tow, Baber Abbasi, Stella Biderman, Sid Black, Anthony DiPofi, Charles Foster, Laurence Golding, Jeffrey Hsu, Alain Le Noac’h, Haonan Li, Kyle McDonell, Niklas Muennighoff, Chris Ociepa, Jason Phang, Laria Reynolds, Hailey Schoelkopf, Aviya Skowron, Lintang Sutawika, Eric Tang, Ani… [find](https://scholar.google.com/scholar?q=Leo+Gao%2C+Jonathan+Tow%2C+Baber+Abbasi%2C+Stella+Biderman%2C+Sid+Black%2C+Anthony+DiPofi%2C+Charles+Foster%2C+Laurence+Golding%2C+Jeffrey+Hsu%2C+Alain+Le+Noac%E2%80%99h%2C+Haonan+Li%2C+Kyle+McDonell%2C+Niklas+Muennighoff%2C+Chris+Oci)
9. Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. 2021. [Measuring massive multitask language understanding](https://md.research.elamur.ai/free/en/papers/hendrycks-2020-measuring-massive-multitask-language-understanding/). Proceedings of the International Conference on Learning Representations (ICLR).
10. Jennifer Hu and Roger Levy. 2023. [Prompting is not a substitute for probability measurements in large language models](https://md.research.elamur.ai/free/en/papers/hu-2023-prompting-is-not-substitute-probability/). In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 5040–5060, Singapore. Association for Computational Linguistics.
11. Albert Q Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, et al. 2023. Mistral 7b. arXiv preprint arXiv:2310.06825. [find](https://scholar.google.com/scholar?q=Albert+Q+Jiang%2C+Alexandre+Sablayrolles%2C+Arthur+Mensch%2C+Chris+Bamford%2C+Devendra+Singh+Chaplot%2C+Diego+de+las+Casas%2C+Florian+Bressand%2C+Gianna+Lengyel%2C+Guillaume+Lample%2C+Lucile+Saulnier%2C+et+al%2E+2023%2E+Mist)
12. Albert Q Jiang, Alexandre Sablayrolles, Antoine Roux, Arthur Mensch, Blanche Savary, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Emma Bou Hanna, Florian Bressand, et al. 2024. Mixtral of experts. arXiv preprint arXiv:2401.04088. [find](https://scholar.google.com/scholar?q=Albert+Q+Jiang%2C+Alexandre+Sablayrolles%2C+Antoine+Roux%2C+Arthur+Mensch%2C+Blanche+Savary%2C+Chris+Bamford%2C+Devendra+Singh+Chaplot%2C+Diego+de+las+Casas%2C+Emma+Bou+Hanna%2C+Florian+Bressand%2C+et+al%2E+2024%2E+Mixtral+o)
13. Percy Liang, Rishi Bommasani, Tony Lee, Dimitris Tsipras, Dilara Soylu, Michihiro Yasunaga, Yian Zhang, Deepak Narayanan, Yuhuai Wu, Ananya Kumar, et al. 2022. Holistic evaluation of language models. arXiv preprint arXiv:2211.09110. [find](https://scholar.google.com/scholar?q=Percy+Liang%2C+Rishi+Bommasani%2C+Tony+Lee%2C+Dimitris+Tsipras%2C+Dilara+Soylu%2C+Michihiro+Yasunaga%2C+Yian+Zhang%2C+Deepak+Narayanan%2C+Yuhuai+Wu%2C+Ananya+Kumar%2C+et+al%2E+2022%2E+Holistic+evaluation+of+language+models%2E+)
14. Stephanie Lin, Jacob Hilton, and Owain Evans. 2022. [TruthfulQA: Measuring how models mimic human falsehoods](https://md.research.elamur.ai/free/en/papers/lin-2022-truthfulqa-measuring-how-models-mimic/). In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 3214–3252, Dublin, Ireland. Association for Computational Linguistics…
15. Chenyang Lyu, Minghao Wu, and Alham Fikri Aji. 2024. Beyond probabilities: Unveiling the misalignment in evaluating large language models. arXiv preprint arXiv:2402.13887. [find](https://scholar.google.com/scholar?q=Chenyang+Lyu%2C+Minghao+Wu%2C+and+Alham+Fikri+Aji%2E+2024%2E+Beyond+probabilities%3A+Unveiling+the+misalignment+in+evaluating+large+language+models%2E+arXiv+preprint+arXiv%3A2402%2E13887%2E)
16. Sourab Mangrulkar, Sylvain Gugger, Lysandre Debut, Younes Belkada, Sayak Paul, and Benjamin Bossan. 2022. Peft: State-of-the-art parameter-efficient fine-tuning methods. [https://github.com/huggingface/peft](https://github.com/huggingface/peft). [↗](https://github.com/huggingface/peft)
17. Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. 2022. [Training language models to follow instructions with human feedback](https://md.research.elamur.ai/free/en/papers/ouyang-2022-training-language-models-follow-instructions/). Advances in Neural Information Processing Systems, 35:27730–27744.
18. Keisuke Sakaguchi, Ronan Le Bras, Chandra Bhagavatula, and Yejin Choi. 2021. Winogrande: An adversarial winograd schema challenge at scale. Communications of the ACM, 64(9):99–106. [find](https://scholar.google.com/scholar?q=Keisuke+Sakaguchi%2C+Ronan+Le+Bras%2C+Chandra+Bhagavatula%2C+and+Yejin+Choi%2E+2021%2E+Winogrande%3A+An+adversarial+winograd+schema+challenge+at+scale%2E+Communications+of+the+ACM%2C+64%289%29%3A99%E2%80%93106%2E)
19. Shibani Santurkar, Esin Durmus, Faisal Ladhak, Cinoo Lee, Percy Liang, and Tatsunori Hashimoto. 2023. [Whose opinions do language models reflect?](https://md.research.elamur.ai/free/en/papers/santurkar-2023-whose-opinions-language-models-reflect/) ArXiv, abs/2303.17548.
20. Maarten Sap, Hannah Rashkin, Derek Chen, Ronan Le Bras, and Yejin Choi. 2019. Social IQa: Commonsense reasoning about social interactions. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processi… [find](https://scholar.google.com/scholar?q=Maarten+Sap%2C+Hannah+Rashkin%2C+Derek+Chen%2C+Ronan+Le+Bras%2C+and+Yejin+Choi%2E+2019%2E+Social+IQa%3A+Commonsense+reasoning+about+social+interactions%2E+In+Proceedings+of+the+2019+Conference+on+Empirical+Methods+in)
21. Nino Scherrer, Claudia Shi, Amir Feder, and David Blei. 2023. Evaluating the moral beliefs encoded in llms. In Thirty-seventh Conference on Neural Information Processing Systems. [find](https://scholar.google.com/scholar?q=Nino+Scherrer%2C+Claudia+Shi%2C+Amir+Feder%2C+and+David+Blei%2E+2023%2E+Evaluating+the+moral+beliefs+encoded+in+llms%2E+In+Thirty%2Dseventh+Conference+on+Neural+Information+Processing+Systems%2E)
22. Alon Talmor, Jonathan Herzig, Nicholas Lourie, and Jonathan Berant. 2019. [CommonsenseQA: A question answering challenge targeting commonsense knowledge](https://md.research.elamur.ai/free/en/papers/talmor-2019-commonsenseqa-question-answering-challenge-targeting/). In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies,…
23. Lindia Tjuatja, Valerie Chen, Sherry Tongshuang Wu, Ameet Talwalkar, and Graham Neubig. 2023. [Do LLMs exhibit human-like response biases? a case study in survey design](https://md.research.elamur.ai/free/en/papers/tjuatja-2024-do-llms-exhibit-human-like/). arXiv.
24. Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al. 2023. Llama 2: Open foundation and fine-tuned chat models. arXiv preprint arXiv:2307.09288. [find](https://scholar.google.com/scholar?q=Hugo+Touvron%2C+Louis+Martin%2C+Kevin+Stone%2C+Peter+Albert%2C+Amjad+Almahairi%2C+Yasmine+Babaei%2C+Nikolay+Bashlykov%2C+Soumya+Batra%2C+Prajjwal+Bhargava%2C+Shruti+Bhosale%2C+et+al%2E+2023%2E+Llama+2%3A+Open+foundation+and+fi)
25. Xinpeng Wang, Chengzhi Hu, Bolei Ma, Paul Röttger, and Barbara Plank. 2024. Look at the text: Instruction-tuned language models are more robust multiple choice selectors than you think. arXiv preprint arXiv:2404.08382. — [arXiv:2404.08382](https://arxiv.org/abs/2404.08382) [find](https://scholar.google.com/scholar?q=Xinpeng+Wang%2C+Chengzhi+Hu%2C+Bolei+Ma%2C+Paul+R%C3%B6ttger%2C+and+Barbara+Plank%2E+2024%2E+Look+at+the+text%3A+Instruction%2Dtuned+language+models+are+more+robust+multiple+choice+selectors+than+you+think%2E+arXiv+preprint)
26. Rowan Zellers, Ari Holtzman, Yonatan Bisk, Ali Farhadi, and Yejin Choi. 2019. HellaSwag: Can a machine really finish your sentence? In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 4791–4800, Florence, Italy. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Rowan+Zellers%2C+Ari+Holtzman%2C+Yonatan+Bisk%2C+Ali+Farhadi%2C+and+Yejin+Choi%2E+2019%2E+HellaSwag%3A+Can+a+machine+really+finish+your+sentence%3F+In+Proceedings+of+the+57th+Annual+Meeting+of+the+Association+for+Com)
27. Chujie Zheng, Hao Zhou, Fandong Meng, Jie Zhou, and Minlie Huang. 2023. Large language models are not robust multiple choice selectors. ArXiv, abs/2309.03882. [find](https://scholar.google.com/scholar?q=Chujie+Zheng%2C+Hao+Zhou%2C+Fandong+Meng%2C+Jie+Zhou%2C+and+Minlie+Huang%2E+2023%2E+Large+language+models+are+not+robust+multiple+choice+selectors%2E+ArXiv%2C+abs%2F2309%2E03882%2E)

*Updated 24 October 2026*