---
title: "Revisiting Non-Verbatim Memorization in Large Language"
description: "Benchmarks that use entity-based QA to probe which facts LLMs have stored usually refer to each entity by a single canonical… 31 references."
authors: ["Yuto Nishida", "Naoki Shikoda", "Yosuke Kishinami", "Ryo Fujii", "Makoto Morishita", "Hidetaka Kamigaito", "Taro Watanabe"]
year: 2026
venue: "Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)"
source: "https://aclanthology.org/2026.acl-long.2178/"
doi: "10.18653/v1/2026.acl-long.2178"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/nishida-2026-revisiting-non-verbatim-memorization-large/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/nishida-2026-revisiting-non-verbatim-memorization-large/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2026](https://md.research.elamur.ai/free/en/papers/2026/page/41/) / Revisiting Non-Verbatim Memorization in Large Language Models: The Role of Entity Surface Forms

# Revisiting Non-Verbatim Memorization in Large Language Models: The Role of Entity Surface Forms

**Authors:** Yuto Nishida, Naoki Shikoda, Yosuke Kishinami, Ryo Fujii, Makoto Morishita, [Hidetaka Kamigaito](https://md.research.elamur.ai/free/en/authors/hidetaka-kamigaito/) and [Taro Watanabe](https://md.research.elamur.ai/free/en/authors/taro-watanabe/) · **Year:** 2026 · **Venue:** [Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)](https://md.research.elamur.ai/free/en/venues/acl/) ([ACL 2026](https://md.research.elamur.ai/free/en/venues/acl/2026/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2026.acl-long.2178/) · [PDF](https://aclanthology.org/2026.acl-long.2178.pdf) · [DOI 10.18653/v1/2026.acl-long.2178](https://doi.org/10.18653/v1/2026.acl-long.2178) · [OpenAlex](https://openalex.org/W7166867035)

**Citations:** 0 citations (OpenAlex, 2026-10-14)

## Summary
Benchmarks that use entity-based QA to probe which facts LLMs have stored usually refer to each entity by a single canonical name. A wrong answer is then ambiguous: the model may lack the fact, or it may simply fail to connect that particular name to the entity. Nishida et al. separate these two explanations with RedirectQA, an English dataset that holds each Wikidata triple constant and changes only how the subject entity is written. The alternative names come from Wikipedia redirect pages and are labelled by redirect category, such as aliases, abbreviations, spelling variants or common misspellings. Questions are built from relation-specific templates and answered without external evidence. Across 13 LLMs, renaming the subject alone often turns a correct answer into a wrong one, or the reverse. Models handle small orthographic changes well but struggle with lexical substitutes like aliases and abbreviations. Accuracy tracks both the frequency of the specific name and the overall frequency of the entity, and entity frequency often adds information beyond name frequency. Factual recall is therefore partly name-dependent, so evaluations should test diverse surface forms.

## Key points
- In a pilot study, Pythia-12B on a redirect-augmented version of PopQA gave inconsistent predictions for 23.7% of canonical–redirect question pairs.
- RedirectQA was built as follows:
  - 500k triples covering 16 relation types (following PopQA) were sampled from a Wikidata dump.
  - Triples were filtered by 2024 Wikimedia pageviews.
  - 33 manually selected redirect categories were grouped into three types: Alternative Names and Abbreviations, Spelling Variants, and Typical Errors.
  - After balancing, the dataset holds about 30k surface instances.
- Consistency is measured over canonical–redirect pairs. Robustness is highest for diacritics, punctuation and spelling changes, and lowest for aliases and abbreviations.
- The frequency analyses use only training-transparent models. They point to cross-surface coupling, meaning access to a fact through one name is not fully independent of the other names for the same entity.
- The dataset is released on Hugging Face as naist-nlp/RedirectQA under CC BY-SA 4.0. Its scope is limited to English and to variation of the subject entity only.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Nara Institute of Science and Technology](https://md.research.elamur.ai/free/en/universities/nara-institute-of-science-and-technology/) — University, Ikoma, Japan · 46 works · 4,565 citations

📚 Shelves (5): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 16,039 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 15,891 works · [🗄️ Information Systems](https://md.research.elamur.ai/free/en/shelves/computer-science/information-systems/) · 1,437 works

🧭 Topics (1): [🧠 Language models & pretraining](https://md.research.elamur.ai/free/en/topics/language-models/) · 3,320 works

## Cite

Cite as: Yuto Nishida, Naoki Shikoda, Yosuke Kishinami, Ryo Fujii, Makoto Morishita, Hidetaka Kamigaito and Taro Watanabe. 2026. Revisiting Non-Verbatim Memorization in Large Language Models: The Role of Entity Surface Forms. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers).

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{nishida2026revisiting,
  title = {{Revisiting Non-Verbatim Memorization in Large Language Models: The Role of Entity Surface Forms}},
  author = {Yuto Nishida, Naoki Shikoda, Yosuke Kishinami, Ryo Fujii, Makoto Morishita, Hidetaka Kamigaito and Taro Watanabe},
  booktitle = {Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2026},
  doi = {10.18653/v1/2026.acl-long.2178},
  url = {https://doi.org/10.18653/v1/2026.acl-long.2178}
}
```

- [← previous Revisiting German Complex Word Identification: Contextualized LLMs and Feature Injection](https://md.research.elamur.ai/free/en/papers/schomacker-2026-revisiting-german-complex-word-identification/)
- [next → Revisiting the syntax of imperatives in Yemeni Arabic: An Agree across phases approach](https://md.research.elamur.ai/free/en/papers/shormani-2026-revisiting-syntax-imperatives-yemeni-arabic/)
- [↑ back to papers of 2026](https://md.research.elamur.ai/free/en/papers/2026/page/41/)

## References

31 references, 11 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Maksym Andriushchenko and Nicolas Flammarion. 2025. Does refusal training in LLMs generalize to the past tense? In The Thirteenth International Conference on Learning Representations. [find](https://scholar.google.com/scholar?q=Maksym+Andriushchenko+and+Nicolas+Flammarion%2E+2025%2E+Does+refusal+training+in+LLMs+generalize+to+the+past+tense%3F+In+The+Thirteenth+International+Conference+on+Learning+Representations%2E)
2. Stella Biderman, Hailey Schoelkopf, Quentin Anthony, Herbie Bradley, Kyle O’Brien, Eric Hallahan, Mohammad Aflah Khan, Shivanshu Purohit, USVSN Sai Prashanth, Edward Raff, Aviya Skowron, Lintang Sutawika, and Oskar Van Der Wal. 2023. Pythia: a suite for analyzing large language models across trainin… [find](https://scholar.google.com/scholar?q=Stella+Biderman%2C+Hailey+Schoelkopf%2C+Quentin+Anthony%2C+Herbie+Bradley%2C+Kyle+O%E2%80%99Brien%2C+Eric+Hallahan%2C+Mohammad+Aflah+Khan%2C+Shivanshu+Purohit%2C+USVSN+Sai+Prashanth%2C+Edward+Raff%2C+Aviya+Skowron%2C+Lintang+Sutaw)
3. Piotr Bojanowski, Edouard Grave, Armand Joulin, and Tomas Mikolov. 2017. [Enriching word vectors with subword information](https://md.research.elamur.ai/free/en/papers/bojanowski-2017-enriching-word-vectors-subword-information/). Transactions of the Association for Computational Linguistics, 5:135–146.
4. Nicholas Carlini, Daphne Ippolito, Matthew Jagielski, Katherine Lee, Florian Tramer, and Chiyuan Zhang. 2023. Quantifying memorization across neural language models. In Proceedings of the Eleventh International Conference on Learning Representations. [find](https://scholar.google.com/scholar?q=Nicholas+Carlini%2C+Daphne+Ippolito%2C+Matthew+Jagielski%2C+Katherine+Lee%2C+Florian+Tramer%2C+and+Chiyuan+Zhang%2E+2023%2E+Quantifying+memorization+across+neural+language+models%2E+In+Proceedings+of+the+Eleventh+Int)
5. Nicholas Carlini, Florian Tramèr, Eric Wallace, Matthew Jagielski, Ariel Herbert-Voss, Katherine Lee, Adam Roberts, Tom Brown, Dawn Song, Úlfar Erlingsson, Alina Oprea, and Colin Raffel. 2021. [Extracting training data from large language models](https://md.research.elamur.ai/free/en/papers/carlini-2020-extracting-training-data-large-language/). In Proceedings of the 30th USENIX Security Symposium,…
6. Bowen Chen, Namgi Han, and Yusuke Miyao. 2024. A multi-perspective analysis of memorization in large language models. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 11190–11209. [find](https://scholar.google.com/scholar?q=Bowen+Chen%2C+Namgi+Han%2C+and+Yusuke+Miyao%2E+2024%2E+A+multi%2Dperspective+analysis+of+memorization+in+large+language+models%2E+In+Proceedings+of+the+2024+Conference+on+Empirical+Methods+in+Natural+Language+Pro)
7. Yanai Elazar, Nora Kassner, Shauli Ravfogel, Amir Feder, Abhilasha Ravichander, Marius Mosbach, Yonatan Belinkov, Hinrich Schütze, and Yoav Goldberg. 2023. Measuring causal effects of data statistics on language model’s ‘factual’ predictions. Preprint, arXiv:2207.14251. — [arXiv:2207.14251](https://arxiv.org/abs/2207.14251) [find](https://scholar.google.com/scholar?q=Yanai+Elazar%2C+Nora+Kassner%2C+Shauli+Ravfogel%2C+Amir+Feder%2C+Abhilasha+Ravichander%2C+Marius+Mosbach%2C+Yonatan+Belinkov%2C+Hinrich+Sch%C3%BCtze%2C+and+Yoav+Goldberg%2E+2023%2E+Measuring+causal+effects+of+data+statistics+)
8. Yanai Elazar, Nora Kassner, Shauli Ravfogel, Abhilasha Ravichander, Eduard Hovy, Hinrich Schütze, and Yoav Goldberg. 2021. Measuring and improving consistency in pretrained language models. Transactions of the Association for Computational Linguistics, 9:1012–1031. [find](https://scholar.google.com/scholar?q=Yanai+Elazar%2C+Nora+Kassner%2C+Shauli+Ravfogel%2C+Abhilasha+Ravichander%2C+Eduard+Hovy%2C+Hinrich+Sch%C3%BCtze%2C+and+Yoav+Goldberg%2E+2021%2E+Measuring+and+improving+consistency+in+pretrained+language+models%2E+Transactio)
9. Leo Gao, Stella Biderman, Sid Black, Laurence Golding, Travis Hoppe, Charles Foster, Jason Phang, Horace He, Anish Thite, Noa Nabeshima, Shawn Presser, and Connor Leahy. 2020. The pile: An 800gb dataset of diverse text for language modeling. Preprint, arXiv:2101.00027. [find](https://scholar.google.com/scholar?q=Leo+Gao%2C+Stella+Biderman%2C+Sid+Black%2C+Laurence+Golding%2C+Travis+Hoppe%2C+Charles+Foster%2C+Jason+Phang%2C+Horace+He%2C+Anish+Thite%2C+Noa+Nabeshima%2C+Shawn+Presser%2C+and+Connor+Leahy%2E+2020%2E+The+pile%3A+An+800gb+datas)
10. Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, Amy Yang, Angela Fan, Anirudh Goyal, Anthony Hartshorn, Aobo Yang, Archi Mitra, Archie Sravankumar, Artem Korenev, Arthur Hinsvark, and 542 o… — [The Llama 3 Herd of Models](https://md.research.elamur.ai/free/en/papers/grattafiori-2024-llama-3-herd-models/)
11. M. Hoerger. 2013. Zh: An updated version of steiger’s z and web-based calculator for testing the statistical significance of the difference between dependent correlations. [find](https://scholar.google.com/scholar?q=M%2E+Hoerger%2E+2013%2E+Zh%3A+An+updated+version+of+steiger%E2%80%99s+z+and+web%2Dbased+calculator+for+testing+the+statistical+significance+of+the+difference+between+dependent+correlations%2E)
12. Mandar Joshi, Eunsol Choi, Daniel Weld, and Luke Zettlemoyer. 2017. [TriviaQA: A large scale distantly supervised challenge dataset for reading comprehension](https://md.research.elamur.ai/free/en/papers/joshi-2017-triviaqa-large-scale-distantly-supervised/). In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1601–1611.
13. Nikhil Kandpal, Haikang Deng, Adam Roberts, Eric Wallace, and Colin Raffel. 2023. [Large language models struggle to learn long-tail knowledge](https://md.research.elamur.ai/free/en/papers/kandpal-2022-large-language-models-struggle-learn/). In Proceedings of the 40th International Conference on Machine Learning, pages 15696–15707.
14. Tom Kwiatkowski, Jennimaria Palomaki, Olivia Redfield, Michael Collins, Ankur Parikh, Chris Alberti, Danielle Epstein, Illia Polosukhin, Jacob Devlin, Kenton Lee, Kristina Toutanova, Llion Jones, Matthew Kelcey, Ming-Wei Chang, Andrew M. Dai, Jakob Uszkoreit, Quoc Le, and Slav Petrov. 2019. [Natural…](https://md.research.elamur.ai/free/en/papers/kwiatkowski-2019-natural-questions-benchmark-question-answering/)
15. David Atkinson, Faeze Brahman, Christopher Clark, Pradeep Dasigi, Nouha Dziri, and 21 others. 2024. 2 olmo 2 furious. Preprint, arXiv:2501.00656. [find](https://scholar.google.com/scholar?q=David+Atkinson%2C+Faeze+Brahman%2C+Christopher+Clark%2C+Pradeep+Dasigi%2C+Nouha+Dziri%2C+and+21+others%2E+2024%2E+2+olmo+2+furious%2E+Preprint%2C+arXiv%3A2501%2E00656%2E)
16. OpenAI. 2024a. Gpt-4o mini: advancing cost-efficient intelligence. [find](https://scholar.google.com/scholar?q=OpenAI%2E+2024a%2E+Gpt%2D4o+mini%3A+advancing+cost%2Defficient+intelligence%2E)
17. OpenAI. 2024b. [Gpt-4o system card](https://md.research.elamur.ai/free/en/papers/openai-2024-gpt-4o-system-card/).
18. Fabio Petroni, Tim Rocktäschel, Sebastian Riedel, Patrick Lewis, Anton Bakhtin, Yuxiang Wu, and Alexander Miller. 2019. [Language models as knowledge bases?](https://md.research.elamur.ai/free/en/papers/petroni-2019-language-models-as-knowledge-bases/) In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural…
19. Marco Tulio Ribeiro, Carlos Guestrin, and Sameer Singh. 2019. Are red roses red? evaluating consistency of question-answering models. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 6174–6184, Florence, Italy. Association for Computational Linguistic… [find](https://scholar.google.com/scholar?q=Marco+Tulio+Ribeiro%2C+Carlos+Guestrin%2C+and+Sameer+Singh%2E+2019%2E+Are+red+roses+red%3F+evaluating+consistency+of+question%2Danswering+models%2E+In+Proceedings+of+the+57th+Annual+Meeting+of+the+Association+for+C)
20. Adam Roberts, Colin Raffel, and Noam Shazeer. 2020. How much knowledge can you pack into the parameters of a language model? In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, pages 5418–5426. [find](https://scholar.google.com/scholar?q=Adam+Roberts%2C+Colin+Raffel%2C+and+Noam+Shazeer%2E+2020%2E+How+much+knowledge+can+you+pack+into+the+parameters+of+a+language+model%3F+In+Proceedings+of+the+2020+Conference+on+Empirical+Methods+in+Natural+Langu)
21. Seiji Maekawa, Hayate Iso, Sairam Gurajada, and Nikita Bhutani. 2024. Retrieval helps or hurts? a deeper dive into the efficacy of retrieval augmentation to language models. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human La… [find](https://scholar.google.com/scholar?q=Seiji+Maekawa%2C+Hayate+Iso%2C+Sairam+Gurajada%2C+and+Nikita+Bhutani%2E+2024%2E+Retrieval+helps+or+hurts%3F+a+deeper+dive+into+the+efficacy+of+retrieval+augmentation+to+language+models%2E+In+Proceedings+of+the+2024)
22. Yusuke Sakai, Adam Nohejl, Jiangnan Hang, Hidetaka Kamigaito, and Taro Watanabe. 2024. Toward the evaluation of large language models considering score variance across instruction templates. In Proceedings of the 7th BlackboxNLP Workshop: Analyzing and Interpreting Neural Networks for NLP, pages 499… [find](https://scholar.google.com/scholar?q=Yusuke+Sakai%2C+Adam+Nohejl%2C+Jiangnan+Hang%2C+Hidetaka+Kamigaito%2C+and+Taro+Watanabe%2E+2024%2E+Toward+the+evaluation+of+large+language+models+considering+score+variance+across+instruction+templates%2E+In+Procee)
23. Alex Mallen, Akari Asai, Victor Zhong, Rajarshi Das, Daniel Khashabi, and Hannaneh Hajishirzi. 2023. When not to trust language models: Investigating effectiveness of parametric and non-parametric memories. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (V… [find](https://scholar.google.com/scholar?q=Alex+Mallen%2C+Akari+Asai%2C+Victor+Zhong%2C+Rajarshi+Das%2C+Daniel+Khashabi%2C+and+Hannaneh+Hajishirzi%2E+2023%2E+When+not+to+trust+language+models%3A+Investigating+effectiveness+of+parametric+and+non%2Dparametric+mem)
24. Christopher Sciavolino, Zexuan Zhong, Jinhyuk Lee, and Danqi Chen. 2021. [Simple entity-centric questions challenge dense retrievers](https://md.research.elamur.ai/free/en/papers/sciavolino-2021-simple-entity-centric-questions-challenge/). In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 6138–6148.
25. Pablo N. Mendes, Max Jakob, Andrés García-Silva, and Christian Bizer. 2011. Dbpedia spotlight: shedding light on the web of documents. In Proceedings of the 7th International Conference on Semantic Systems, I-Semantics ’11, page 1–8, New York, NY, USA. Association for Computing Machinery. [find](https://scholar.google.com/scholar?q=Pablo+N%2E+Mendes%2C+Max+Jakob%2C+Andr%C3%A9s+Garc%C3%ADa%2DSilva%2C+and+Christian+Bizer%2E+2011%2E+Dbpedia+spotlight%3A+shedding+light+on+the+web+of+documents%2E+In+Proceedings+of+the+7th+International+Conference+on+Semantic+Sy)
26. Marianna Nezhurina, Jörg Franke, Taishi Nakamura, Timur Carstensen, Niccolò Ajroldi, Ville Komulainen, David Salinas, and Jenia Jitsev. 2025. Open-sci-ref-0.01: open and reproducible reference baselines for language model and dataset comparison. Preprint, arXiv:2509.09009. — [arXiv:2509.09009](https://arxiv.org/abs/2509.09009) [find](https://scholar.google.com/scholar?q=Marianna+Nezhurina%2C+J%C3%B6rg+Franke%2C+Taishi+Nakamura%2C+Timur+Carstensen%2C+Niccol%C3%B2+Ajroldi%2C+Ville+Komulainen%2C+David+Salinas%2C+and+Jenia+Jitsev%2E+2025%2E+Open%2Dsci%2Dref%2D0%2E01%3A+open+and+reproducible+reference+baselin)
27. Team OLMo, Pete Walsh, Luca Soldaini, Dirk Groeneveld, Kyle Lo, Shane Arora, Akshita Bhagia, Yuling Gu, Shengyi Huang, Matt Jordan, Nathan Lambert, Dustin Schwenk, Oyvind Tafjord, Taira Anderson, Adi Simhi, Jonathan Herzig, Idan Szpektor, and Yonatan Belinkov. 2024. Distinguishing ignorance from err… — [arXiv:2410.22071](https://arxiv.org/abs/2410.22071) [find](https://scholar.google.com/scholar?q=Team+OLMo%2C+Pete+Walsh%2C+Luca+Soldaini%2C+Dirk+Groeneveld%2C+Kyle+Lo%2C+Shane+Arora%2C+Akshita+Bhagia%2C+Yuling+Gu%2C+Shengyi+Huang%2C+Matt+Jordan%2C+Nathan+Lambert%2C+Dustin+Schwenk%2C+Oyvind+Tafjord%2C+Taira+Anderson%2C+Adi+)
28. J. H. Steiger. 1980. Tests for comparing elements of a correlation matrix. Psychological Bulletin, 87:245– 251. [find](https://scholar.google.com/scholar?q=J%2E+H%2E+Steiger%2E+1980%2E+Tests+for+comparing+elements+of+a+correlation+matrix%2E+Psychological+Bulletin%2C+87%3A245%E2%80%93+251%2E)
29. An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, Chujie Zheng, Dayiheng Liu, Fan Zhou, Fei Huang, Feng Hu, Hao Ge, Haoran Wei, Huan Lin, Jialong Tang, and 41 others. 2025. [Qwen3 technical report](https://md.research.elamur.ai/free/en/papers/yang-2025-qwen3-technical-report/). Preprint, arXiv:2505.09388.
30. Wenhao Yu, Dan Iter, Shuohang Wang, Yichong Xu, Mingxuan Ju, Soumya Sanyal, Chenguang Zhu, Michael Zeng, and Meng Jiang. 2023. [Generate rather than retrieve: Large language models are strong context generators](https://md.research.elamur.ai/free/en/papers/yu-2022-generate-rather-than-retrieve-large/). In Proceedings of the Eleventh International Conference on Learning Representations.
31. Chujie Zheng, Hao Zhou, Fandong Meng, Jie Zhou, and Minlie Huang. 2024. Large language models are not robust multiple choice selectors. In The Twelfth International Conference on Learning Representations. [find](https://scholar.google.com/scholar?q=Chujie+Zheng%2C+Hao+Zhou%2C+Fandong+Meng%2C+Jie+Zhou%2C+and+Minlie+Huang%2E+2024%2E+Large+language+models+are+not+robust+multiple+choice+selectors%2E+In+The+Twelfth+International+Conference+on+Learning+Representati)

*Updated 1 November 2026*