---
title: "Native Design Bias — paper by Reusens et al. (2025)"
description: "This study adds a third group: speakers from countries where English has no official status. 36 references, cited by 2 papers here."
authors: ["Manon Reusens", "Philipp Borchert", "Jochen De Weerdt", "Bart Baesens"]
year: 2025
venue: "Proceedings of the 14th International Joint Conference on Natural Language Processing and the 4th Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics"
source: "https://aclanthology.org/2025.findings-ijcnlp.73/"
doi: "10.18653/v1/2025.findings-ijcnlp.73"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/reusens-2025-native-design-bias-studying-impact/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/reusens-2025-native-design-bias-studying-impact/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2025](https://md.research.elamur.ai/free/en/papers/2025/page/12/) / Native Design Bias: Studying the Impact of English Nativeness on Language Model Performance

# Native Design Bias: Studying the Impact of English Nativeness on Language Model Performance

**Authors:** Manon Reusens, Philipp Borchert, Jochen De Weerdt and Bart Baesens · **Year:** 2025 · **Venue:** [Proceedings of the 14th International Joint Conference on Natural Language Processing and the 4th Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics](https://md.research.elamur.ai/free/en/venues/findings/) ([Findings of the Association for Computational Linguistics 2025](https://md.research.elamur.ai/free/en/venues/findings/2025/)) · **Publisher:** [The Asian Federation of Natural Language Processing and The Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/the-asian-federation-of-natural-language-processing-and-the-association-for/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2025.findings-ijcnlp.73/) · [PDF](https://aclanthology.org/2025.findings-ijcnlp.73.pdf) · [DOI 10.18653/v1/2025.findings-ijcnlp.73](https://doi.org/10.18653/v1/2025.findings-ijcnlp.73) · [OpenAlex](https://openalex.org/W7138169506)

**Citations:** 1 citations (OpenAlex, 2026-10-14) · [2](https://md.research.elamur.ai/free/en/papers/reusens-2025-native-design-bias-studying-impact/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
The paper asks whether large language models serve people worse when their English is not native. Earlier work compared native dialects. This study adds a third group: speakers from countries where English has no official status. Participants thus fall into three categories: native speakers from the West (WN), native speakers from outside the West (NWN), and speakers for whom English is not a first language (NN). The authors had 124 annotators, each reporting their first language and English level, write prompts for ten tasks taken from the natural-instructions collection. This produced over 12,000 unique prompts, released with translations into eight languages. Five LLMs, including GPT4o and Sonnet, answered these prompts on objective classification, subjective classification and generation tasks. On objective classification, WN prompts led to the most accurate answers. The NWN–NN ordering depended on the model. Generation stayed largely unaffected. When the prompt also stated the writer's nativeness, models favoured whichever group was labelled native, even when the label was false. This is an anchoring effect. The work shows that a user's phrasing alone, even with the same meaning, can quietly lower answer quality for non-native users.

## Key points
- The dataset has over 12,000 unique prompts from 124 paid volunteer annotators, with metadata on native language and self-reported English level (mostly C1/C2).
- Tasks come from ten natural-instructions datasets. They include paraphrasing, article writing from a summary or title, sentiment, NLU, multiple choice, and Amazon food review rating (the subjective task). Some sources, such as CNN Dailymail, TweetQA and McTaco, have a Western topical slant.
- Objective classification showed the largest gaps, with WN prompts scoring best. Both GPT models did slightly better on NN than on NWN prompts. The other models treated the two native groups about equally.
- Telling the model about nativeness caused strong anchoring on objective tasks. GPT4o resisted this best. Sonnet sometimes even switched its response language based on the stated background.
- Generative outputs were scored by an LLM judge from a different model family than those evaluated. A manually checked sample showed high correlation and Cohen's kappa with the judge.

## Interesting ideas
- Have writers from different groups phrase the same underlying item, then compare system outputs on these meaning-equivalent variants to isolate phrasing effects from content.
- Report results separately for objective classification, subjective classification and generation, because a robustness gap can be hidden in one task type and absent in another.
- Probe for anchoring by adding metadata about the author, including deliberately wrong labels, and check whether the decision follows the label rather than the text.
- Choose the automatic judge from a different model family to avoid self-preference, and calibrate it against a hand-labelled sample using correlation and kappa.
- Vary how much freedom annotators have across tasks, from tightly constrained to open-ended, to see where variation in phrasing actually matters.

## For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

- **I1** (tree): Build a small UD test set of non-native rephrasings of the same sentences. Then check whether parses and LAS degrade on learner-style constructions such as literal translations.
- **I2** (answer): Break down school-test and fable-QA scores by question type, so phrasing sensitivity in multiple-choice items is not averaged away.
- **I3** (learn): Add negative controls that inject irrelevant or false context tags, such as source or author origin, and require the frame and answer outputs to stay unchanged.
- **I4** (learn): When LLM teacher tiers grade silver data, use a teacher unrelated to the system being tuned. Also keep a human-checked subset with agreement scores.
- **I5** (frames): Test whether role and frame extraction stays stable on short constrained inputs as well as on freer, longer texts.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [KU Leuven](https://md.research.elamur.ai/free/en/universities/ku-leuven/) — University, Leuven, Belgium · 42 works · 5,839 citations
- [University of Southampton](https://md.research.elamur.ai/free/en/universities/university-of-southampton/) — University, Southampton, United Kingdom · 10 works · 2,495 citations

Also on the byline: [Institut d'Economie Scientifique Et de Gestion](https://ror.org/04jqeag92).

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 16,039 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 15,891 works

🧭 Topics (2): [🧠 Language models & pretraining](https://md.research.elamur.ai/free/en/topics/language-models/) · 3,320 works · [English](https://md.research.elamur.ai/free/en/topics/english/) · 2,263 works

## Cite

Cite as: Manon Reusens, Philipp Borchert, Jochen De Weerdt and Bart Baesens. 2025. Native Design Bias: Studying the Impact of English Nativeness on Language Model Performance. Proceedings of the 14th International Joint Conference on Natural Language Processing and the 4th Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{reusens2025native,
  title = {{Native Design Bias: Studying the Impact of English Nativeness on Language Model Performance}},
  author = {Manon Reusens, Philipp Borchert, Jochen {De Weerdt} and Bart Baesens},
  booktitle = {Proceedings of the 14th International Joint Conference on Natural Language Processing and the 4th Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics},
  year = {2025},
  doi = {10.18653/v1/2025.findings-ijcnlp.73},
  url = {https://doi.org/10.18653/v1/2025.findings-ijcnlp.73}
}
```

- [← previous My LLM might Mimic AAE - But When Should It?](https://md.research.elamur.ai/free/en/papers/sandoval-2025-my-llm-might-mimic-aae/)
- [next → Natural Language Processing for Tigrinya: Current State and Future Directions](https://md.research.elamur.ai/free/en/papers/gaim-2025-natural-language-processing-tigrinya-current/)
- [↑ back to papers of 2025](https://md.research.elamur.ai/free/en/papers/2025/page/12/)

## References

36 references, 16 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. AI Anthropic. 2024. The claude 3 model family: Opus, sonnet, haiku. Claude-3 Model Card. [find](https://scholar.google.com/scholar?q=AI+Anthropic%2E+2024%2E+The+claude+3+model+family%3A+Opus%2C+sonnet%2C+haiku%2E+Claude%2D3+Model+Card%2E)
2. Jinze Bai, Shuai Bai, Yunfei Chu, Zeyu Cui, Kai Dang, Xiaodong Deng, Yang Fan, Wenbin Ge, Yu Han, Fei Huang, Binyuan Hui, Luo Ji, Mei Li, Junyang Lin, Runji Lin, Dayiheng Liu, Gao Liu, Chengqiang Lu, Keming Lu, Jianxin Ma, Rui Men, Xingzhang Ren, Xuancheng Ren, Chuanqi Tan, Sinan Tan, Jianhong Tu, Peng Wang, Shijie Wang, Wei Wang, Shengguang Wu, Benfeng Xu, Jin Xu, An Yang, Hao Yang, Jian Yang, Shusheng Yang, Yang Yao, Bowen Yu, Hongyi Yuan, Zheng Yuan, Jianwei Zhang, Xingxuan Zhang, Yichang Zhang, Zhenru Zhang, Chang Zhou, Jingren Zhou, Xiaohuan Zhou, and Tianhang Zhu. 2023. [Qwen technical report](https://md.research.elamur.ai/free/en/papers/bai-2023-qwen-technical-report/). Preprint, arXiv:2309.16609. [↗](https://arxiv.org/abs/2309.16609)
3. Yejin Bang, Samuel Cahyawijaya, Nayeon Lee, Wenliang Dai, Dan Su, Bryan Wilie, Holy Lovenia, Ziwei Ji, Tiezheng Yu, Willy Chung, Quyet V. Do, Yan Xu, and Pascale Fung. 2023. [A multitask, multilingual, multimodal evaluation of ChatGPT on reasoning, hallucination, and interactivity](https://md.research.elamur.ai/free/en/papers/bang-2023-multitask-multilingual-multimodal-evaluation-chatgpt/). In Proceedings of the 13th International Joint Conference on Natural Language Processing and the 3rd Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics (Volume 1: Long Papers), pages 675–718, Nusa Dua, Bali. Association for Computational Linguistics. [↗](https://doi.org/10.48550/arxiv.2302.04023)
4. Soumya Barikeri, Anne Lauscher, Ivan Vulić, and Goran Glavaš. 2021. RedditBias: A real-world resource for bias evaluation and debiasing of conversational language models. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 1941–1955, Online. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Soumya+Barikeri%2C+Anne+Lauscher%2C+Ivan+Vulic%CC%81%2C+and+Goran+Glava%C5%A1%2E+2021%2E+RedditBias%3A+A+real%2Dworld+resource+for+bias+evaluation+and+debiasing+of+conversational+language+models%2E+In+Proceedings+of+the+59th+A)
5. Anna Bavaresco, Raffaella Bernardi, Leonardo Bertolazzi, Desmond Elliott, Raquel Fernández, Albert Gatt, E. Ghaleb, Mario Giulianelli, Michael Hanna, Alexander Koller, André F. T. Martins, Philipp Mondorf, Vera Neplenbroek, Sandro Pezzelle, Barbara Plank, David Schlangen, Alessandro Suglia, Aditya K Surikuchi, Ece Takmaz, and Alberto Testoni. 2024. [Llms instead of human judges? a large scale empirical study across 20 nlp evaluation tasks](https://md.research.elamur.ai/free/en/papers/bavaresco-2024-llms-instead-human-judges-large/).
6. Tilman Beck, Hendrik Schuff, Anne Lauscher, and Iryna Gurevych. 2024. [Sensitivity, performance, robustness: Deconstructing the effect of sociodemographic prompting](https://md.research.elamur.ai/free/en/papers/beck-2024-sensitivity-performance-robustness-deconstructing-effect/). In Proceedings of the 18th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers), pages 2589–2615, St. Julian’s, Malta. Association for Computational Linguistics.
7. Damian Blasi, Antonios Anastasopoulos, and Graham Neubig. 2022. [Systematic inequalities in language technology performance across the world’s languages](https://md.research.elamur.ai/free/en/papers/blasi-2022-systematic-inequalities-language-technology-performance/). In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 5486–5505, Dublin, Ireland. Association for Computational Linguistics.
8. Su Lin Blodgett, Solon Barocas, Hal Daumé III, and Hanna Wallach. 2020. [Language (technology) is power: A critical survey of “bias” in NLP](https://md.research.elamur.ai/free/en/papers/blodgett-2020-language-technology-is-power-critical/). In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 5454– 5476, Online. Association for Computational Linguistics. [↗](https://doi.org/10.18653/v1/2020.acl-main.485)
9. Maarten Buyl, Alexander Rogiers, Sander Noels, Guillaume Bied, Iris Dominguez-Catena, Edith Heiter, Iman Johary, Alexandru-Cristian Mara, Raphaël Romero, Jefrey Lijffijt, et al. 2024. Large language models reflect the ideology of their creators. arXiv preprint arXiv:2410.18417. [↗](https://arxiv.org/abs/2410.18417)
10. Scott Allen Cambo and Darren Gergle. 2022. Model positionality and computational reflexivity: Promoting reflexivity in data science. In Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems, pages 1–19. [↗](https://doi.org/10.48550/arxiv.2203.07031)
11. Mohna Chakraborty, Adithya Kulkarni, and Qi Li. 2023. Zero-shot approach to overcome perturbation sensitivity of prompts. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 5698–5711, Toronto, Canada. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Mohna+Chakraborty%2C+Adithya+Kulkarni%2C+and+Qi+Li%2E+2023%2E+Zero%2Dshot+approach+to+overcome+perturbation+sensitivity+of+prompts%2E+In+Proceedings+of+the+61st+Annual+Meeting+of+the+Association+for+Computational)
12. Myra Cheng, Esin Durmus, and Dan Jurafsky. 2023. [Marked personas: Using natural language prompts to measure stereotypes in language models](https://md.research.elamur.ai/free/en/papers/cheng-2023-marked-personas-natural-language-prompts/). In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1504–1532, Toronto, Canada. Association for Computational Linguistics.
13. Ameet Deshpande, Vishvak Murahari, Tanmay Rajpurohit, Ashwin Kalyan, and Karthik Narasimhan. 2023. [Toxicity in chatgpt: Analyzing persona-assigned language models](https://md.research.elamur.ai/free/en/papers/deshpande-2023-toxicity-chatgpt-analyzing-persona-assigned/). In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 1236–1270, Singapore. Association for Computational Linguistics.
14. Esin Durmus, Karina Nyugen, Thomas I Liao, Nicholas Schiefer, Amanda Askell, Anton Bakhtin, Carol Chen, Zac Hatfield-Dodds, Danny Hernandez, Nicholas Joseph, et al. 2023. [Towards measuring the representation of subjective global opinions in language models](https://md.research.elamur.ai/free/en/papers/durmus-2023-towards-measuring-representation-subjective-global/). arXiv preprint arXiv:2306.16388. [↗](https://arxiv.org/abs/2306.16388)
15. Jessica Maria Echterhoff, Yao Liu, Abeer Alessa, Julian McAuley, and Zexue He. 2024. Cognitive bias in decision-making with LLMs. In Findings of the Association for Computational Linguistics: EMNLP 2024, pages 12640–12653, Miami, Florida, USA. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Jessica+Maria+Echterhoff%2C+Yao+Liu%2C+Abeer+Alessa%2C+Julian+McAuley%2C+and+Zexue+He%2E+2024%2E+Cognitive+bias+in+decision%2Dmaking+with+LLMs%2E+In+Findings+of+the+Association+for+Computational+Linguistics%3A+EMNLP+20)
16. Shashank Gupta, Vaishnavi Shrivastava, Ameet Deshpande, Ashwin Kalyan, Peter Clark, Ashish Sabharwal, and Tushar Khot. 2023. Bias runs deep: Implicit reasoning biases in persona-assigned llms. arXiv preprint arXiv:2311.04892. [↗](https://arxiv.org/abs/2311.04892)
17. Suchin Gururangan, Dallas Card, Sarah Dreier, Emily Gade, Leroy Wang, Zeyu Wang, Luke Zettlemoyer, and Noah A. Smith. 2022. Whose language counts as high quality? measuring language ideologies in text data selection. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 2562–2580, Abu Dhabi, United Arab Emirates. Association for Computational Linguistics. [↗](https://doi.org/10.48550/arxiv.2201.10474)
18. Melissa Hall, Laurens van der Maaten, Laura Gustafson, Maxwell Jones, and Aaron Adcock. 2022. A systematic study of bias amplification. arXiv preprint arXiv:2201.11706. [↗](https://arxiv.org/abs/2201.11706)
19. Valentin Hofmann, Pratyusha Ria Kalluri, Dan Jurafsky, and Sharese King. 2024. Ai generates covertly racist decisions about people based on their dialect. Nature, 633(8028):147–154. [↗](https://doi.org/10.1038/s41586-024-07856-5)
20. Zhijing Jin, Nils Heil, Jiarui Liu, Shehzaad Dhuliawala, Yahang Qi, Bernhard Schölkopf, Rada Mihalcea, and Mrinmaya Sachan. 2024. Implicit personalization in language models: A systematic study. In Findings of the Association for Computational Linguistics: EMNLP 2024, pages 12309–12325, Miami, Florida, USA. Association for Computational Linguistics. [↗](https://doi.org/10.48550/arxiv.2405.14808)
21. Erik Jones and Jacob Steinhardt. 2022. Capturing failures of large language models via human cognitive biases. Advances in Neural Information Processing Systems, 35:11785–11799. [find](https://scholar.google.com/scholar?q=Erik+Jones+and+Jacob+Steinhardt%2E+2022%2E+Capturing+failures+of+large+language+models+via+human+cognitive+biases%2E+Advances+in+Neural+Information+Processing+Systems%2C+35%3A11785%E2%80%9311799%2E)
22. Pratik Joshi, Sebastin Santy, Amar Budhiraja, Kalika Bali, and Monojit Choudhury. 2020. [The state and fate of linguistic diversity and inclusion in the NLP world](https://md.research.elamur.ai/free/en/papers/joshi-2020-state-fate-linguistic-diversity-inclusion/). In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 6282–6293, Online. Association for Computational Linguistics.
23. Anjali Kantharuban, Jeremiah Milbauer, Emma Strubell, and Graham Neubig. 2024. Stereotype or personalization? user identity biases chatbot recommendations. arXiv preprint arXiv:2410.05613. [↗](https://arxiv.org/abs/2410.05613)
24. Viet Lai, Nghia Ngo, Amir Pouran Ben Veyseh, Hieu Man, Franck Dernoncourt, Trung Bui, and Thien Nguyen. 2023. [ChatGPT beyond English: Towards a comprehensive evaluation of large language models in multilingual learning](https://md.research.elamur.ai/free/en/papers/lai-2023-chatgpt-beyond-english-comprehensive-evaluation/). In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 13171–13189, Singapore. Association for Computational Linguistics.
25. Swaroop Mishra, Daniel Khashabi, Chitta Baral, and Hannaneh Hajishirzi. 2022. Cross-task generalization via natural language crowdsourcing instructions. In ACL. [find](https://scholar.google.com/scholar?q=Swaroop+Mishra%2C+Daniel+Khashabi%2C+Chitta+Baral%2C+and+Hannaneh+Hajishirzi%2E+2022%2E+Cross%2Dtask+generalization+via+natural+language+crowdsourcing+instructions%2E+In+ACL%2E)
26. Jeremy K. Nguyen. 2024. Human bias in ai models? anchoring effects and mitigation strategies in large language models. Journal of Behavioral and Experimental Finance, 43:100971. [↗](https://doi.org/10.1016/j.jbef.2024.100971)
27. Chengwei Qin, Aston Zhang, Zhuosheng Zhang, Jiaao Chen, Michihiro Yasunaga, and Diyi Yang. 2023. [Is ChatGPT a general-purpose natural language processing task solver?](https://md.research.elamur.ai/free/en/papers/qin-2023-chatgpt-general-purpose-natural-language/) In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 1339–1384, Singapore. Association for Computational Linguistics.
28. Michael J Ryan, William Held, and Diyi Yang. 2024. [Unintended impacts of llm alignment on global representation](https://md.research.elamur.ai/free/en/papers/ryan-2024-unintended-impacts-llm-alignment-global/). arXiv preprint arXiv:2402.15018. [↗](https://arxiv.org/abs/2402.15018)
29. Sebastin Santy, Jenny Liang, Ronan Le Bras, Katharina Reinecke, and Maarten Sap. 2023. [NLPositionality: Characterizing design biases of datasets and models](https://md.research.elamur.ai/free/en/papers/santy-2023-nlpositionality-characterizing-design-biases-datasets/). In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 9080–9102, Toronto, Canada. Association for Computational Linguistics.
30. Melanie Sclar, Yejin Choi, Yulia Tsvetkov, and Alane Suhr. 2023. Quantifying language models’ sensitivity to spurious features in prompt design or: How i learned to start worrying about prompt formatting. arXiv preprint arXiv:2310.11324. [↗](https://arxiv.org/abs/2310.11324)
31. Yarden Tal, Inbal Magar, and Roy Schwartz. 2022. Fewer errors, but more stereotypes? the effect of model size on gender bias. In Proceedings of the 4th Workshop on Gender Bias in Natural Language Processing (GeBNLP), pages 112–120, Seattle, Washington. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Yarden+Tal%2C+Inbal+Magar%2C+and+Roy+Schwartz%2E+2022%2E+Fewer+errors%2C+but+more+stereotypes%3F+the+effect+of+model+size+on+gender+bias%2E+In+Proceedings+of+the+4th+Workshop+on+Gender+Bias+in+Natural+Language+Proc)
32. Zeerak Talat, Aurélie Névéol, Stella Biderman, Miruna Clinciu, Manan Dey, Shayne Longpre, Sasha Luccioni, Maraim Masoud, Margaret Mitchell, Dragomir Radev, Shanya Sharma, Arjun Subramonian, Jaesung Tae, Samson Tan, Deepak Tunuguntla, and Oskar Van Der Wal. 2022. [You reap what you sow: On the challenges of bias evaluation under multilingual settings](https://md.research.elamur.ai/free/en/papers/talat-2022-you-reap-what-you-sow/). In Proceedings of BigScience Episode #5 – Workshop on Challenges & Perspectives in Creating Large Language Models, pages 26–41, virtual+Dublin. Association for Computational Linguistics.
33. Amos Tversky and Daniel Kahneman. 1974. Judgment under uncertainty: Heuristics and biases: Biases in judgments reveal some heuristics of thinking under uncertainty. science, 185(4157):1124–1131. [find](https://scholar.google.com/scholar?q=Amos+Tversky+and+Daniel+Kahneman%2E+1974%2E+Judgment+under+uncertainty%3A+Heuristics+and+biases%3A+Biases+in+judgments+reveal+some+heuristics+of+thinking+under+uncertainty%2E+science%2C+185%284157%29%3A1124%E2%80%931131%2E)
34. Yizhong Wang, Swaroop Mishra, Pegah Alipoormolabashi, Yeganeh Kordi, Amirreza Mirzaei, Anjana Arunkumar, Arjun Ashok, Arut Selvan Dhanasekaran, Atharva Naik, David Stap, et al. 2022. Super-naturalinstructions:generalization via declarative instructions on 1600+ tasks. In EMNLP. [find](https://scholar.google.com/scholar?q=Yizhong+Wang%2C+Swaroop+Mishra%2C+Pegah+Alipoormolabashi%2C+Yeganeh+Kordi%2C+Amirreza+Mirzaei%2C+Anjana+Arunkumar%2C+Arjun+Ashok%2C+Arut+Selvan+Dhanasekaran%2C+Atharva+Naik%2C+David+Stap%2C+et+al%2E+2022%2E+Super%2Dnaturalinst)
35. Xiang Zhang, Senyu Li, Bradley Hauer, Ning Shi, and Grzegorz Kondrak. 2023. Don’t trust ChatGPT when your question is not in English: A study of multilingual abilities and types of LLMs. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 7915–7927, Singapore. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Xiang+Zhang%2C+Senyu+Li%2C+Bradley+Hauer%2C+Ning+Shi%2C+and+Grzegorz+Kondrak%2E+2023%2E+Don%E2%80%99t+trust+ChatGPT+when+your+question+is+not+in+English%3A+A+study+of+multilingual+abilities+and+types+of+LLMs%2E+In+Proceeding)
36. Caleb Ziems, William Held, Jingfeng Yang, Jwala Dhamala, Rahul Gupta, and Diyi Yang. 2023. [Multi-VALUE: A framework for cross-dialectal English NLP](https://md.research.elamur.ai/free/en/papers/ziems-2023-multi-value-framework-cross-dialectal/). In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 744–768, Toronto, Canada. Association for Computational Linguistics.

*Updated 1 November 2026*