---
title: "Bias Runs Deep — paper by Gupta et al. (2023)"
description: "The paper asks whether telling an LLM to adopt a socio-demographic identity changes its ability to… 38 references, cited by 79 papers here."
authors: ["Shashank Gupta", "Vaishnavi Shrivastava", "Ameet Deshpande", "Ashwin Kalyan", "Peter Clark", "Ashish Sabharwal", "Tushar Khot"]
year: 2023
venue: "arXiv:2311.04892"
source: "https://arxiv.org/abs/2311.04892"
doi: "10.48550/arxiv.2311.04892"
arxiv: "2311.04892"
licence: "CC BY 4.0 (arXiv)"
licence_url: "http://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/gupta-2023-bias-runs-deep-implicit-reasoning/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/gupta-2023-bias-runs-deep-implicit-reasoning/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2023](https://md.research.elamur.ai/free/en/papers/2023/page/8/) / Bias Runs Deep: Implicit Reasoning Biases in Persona-Assigned LLMs

# Bias Runs Deep: Implicit Reasoning Biases in Persona-Assigned LLMs

**Authors:** [Shashank Gupta](https://md.research.elamur.ai/free/en/authors/shashank-gupta/), Vaishnavi Shrivastava, [Ameet Deshpande](https://md.research.elamur.ai/free/en/authors/ameet-deshpande/), [Ashwin Kalyan](https://md.research.elamur.ai/free/en/authors/ashwin-kalyan/), [Peter Clark](https://md.research.elamur.ai/free/en/authors/peter-clark/), [Ashish Sabharwal](https://md.research.elamur.ai/free/en/authors/ashish-sabharwal/) and [Tushar Khot](https://md.research.elamur.ai/free/en/authors/tushar-khot/) · **Year:** 2023 · **Venue:** arXiv:2311.04892 · **Licence:** [CC BY 4.0 (arXiv)](http://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://arxiv.org/abs/2311.04892) · [PDF](https://arxiv.org/pdf/2311.04892) · [DOI 10.48550/arxiv.2311.04892](https://doi.org/10.48550/arxiv.2311.04892) · [arXiv 2311.04892](https://arxiv.org/abs/2311.04892) · [OpenAlex](https://openalex.org/W4388555895)

**Citations:** 12 citations (OpenAlex, 2026-10-14) · [79](https://md.research.elamur.ai/free/en/papers/gupta-2023-bias-runs-deep-implicit-reasoning/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
The paper asks whether telling an LLM to adopt a socio-demographic identity changes its ability to solve reasoning problems that have nothing to do with that identity. The authors placed persona instructions in the system prompt, using three wordings, and evaluated the models zero-shot. Their 19 personas were drawn from five categories, spanning political affiliation, religion, disability, gender and race. The test set had 24 datasets: 22 subsets of MMLU, Sports Understanding from Big-Bench-Hard, and MBPP. Four models were tested: gpt-3.5-turbo-0613, gpt-3.5-turbo-1106, gpt-4-turbo-1106 and Llama-2-70b-chat.

When asked directly, the models deny stereotypes. Yet with a persona assigned, they often refuse tasks based on false assumptions about that group, or they quietly make more mistakes, and accuracy falls noticeably as a result. Asking the model in the prompt to avoid stereotypes barely helped. The work matters because persona prompting is spreading quickly, and it can expose hidden bias that is difficult to notice or fix.

## Key points
- With ChatGPT-3.5 (June 2023), 80% of personas performed worse on at least one dataset. Some datasets showed relative accuracy losses above 70%, and certain personas lost significantly on more than 80% of datasets.
- The "physically-disabled person" persona scored on average 33% lower than the able-bodied persona and was affected on 96% of datasets. About 58% of its errors were explicit refusals.
- The share of biased personas differed by model: 100% for the November 2023 ChatGPT-3.5, 80% for Llama-2-70b-chat, and 42% for GPT-4-Turbo, the lowest but still substantial.
- Bias also appeared inside a single group. For example, the Atheist persona beat the Christian one on science subsets, and the Obama Supporter persona beat the Trump Supporter one on ethics.
- The authors release their code and roughly 1.5 million model outputs. The paper was accepted at ICLR 2024.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Stanford University](https://md.research.elamur.ai/free/en/universities/stanford-university/) — University, Stanford, United States · 506 works · 330,053 citations
- [Allen Institute for Artificial Intelligence](https://md.research.elamur.ai/free/en/labs/allen-institute-for-artificial-intelligence/) — Research institute, Seattle, United States · 304 works · 106,137 citations
- [Princeton University](https://md.research.elamur.ai/free/en/universities/princeton-university/) — University, Princeton, United States · 113 works · 48,612 citations

📚 Shelves (7): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 18,614 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 17,965 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 17,450 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 17,296 works · [🤝 Social Sciences](https://md.research.elamur.ai/free/en/shelves/social-sciences/) · 1,954 works · [🤝 Safety Research](https://md.research.elamur.ai/free/en/shelves/social-sciences/safety-research/) · 626 works · [🤝 Sociology and Political Science](https://md.research.elamur.ai/free/en/shelves/social-sciences/sociology-and-political-science/) · 626 works

🧭 Topics (2): [🧠 Language models & pretraining](https://md.research.elamur.ai/free/en/topics/language-models/) · 3,894 works · [🧮 Reasoning](https://md.research.elamur.ai/free/en/topics/reasoning/) · 1,685 works

## Cite

Cite as: Shashank Gupta, Vaishnavi Shrivastava, Ameet Deshpande, Ashwin Kalyan, Peter Clark, Ashish Sabharwal and Tushar Khot. 2023. Bias Runs Deep: Implicit Reasoning Biases in Persona-Assigned LLMs. arXiv:2311.04892.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@misc{gupta2023bias,
  title = {{Bias Runs Deep: Implicit Reasoning Biases in Persona-Assigned LLMs}},
  author = {Shashank Gupta, Vaishnavi Shrivastava, Ameet Deshpande, Ashwin Kalyan, Peter Clark, Ashish Sabharwal and Tushar Khot},
  year = {2023},
  doi = {10.48550/arxiv.2311.04892},
  eprint = {2311.04892},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2311.04892}
}
```

- [← previous What's In My Big Data?](https://md.research.elamur.ai/free/en/papers/elazar-2023-s-my-big-data/)
- [next → Language Modeling Is Compression](https://md.research.elamur.ai/free/en/papers/deletang-2023-language-modeling-compression/)
- [↑ back to papers of 2023](https://md.research.elamur.ai/free/en/papers/2023/page/8/)

## References

38 references, 18 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Gati Aher, Rosa I. Arriaga, and Adam T. Kalai. Using large language models to simulate multiple humans and replicate human subject studies. In ICML, 2023. [find](https://scholar.google.com/scholar?q=Gati+Aher%2C+Rosa+I%2E+Arriaga%2C+and+Adam+T%2E+Kalai%2E+Using+large+language+models+to+simulate+multiple+humans+and+replicate+human+subject+studies%2E+In+ICML%2C+2023%2E)
2. Lisa P. Argyle, Ethan C. Busby, Nancy Fulda, Joshua R. Gubler, Christopher Rytting, and David Wingate. [Out of one, many: Using language models to simulate human samples](https://md.research.elamur.ai/free/en/papers/argyle-2022-out-one-many-using-language/). Political Analysis, 31(3):337–351, 2023.
3. Jacob Austin, Augustus Odena, Maxwell Nye, Maarten Bosma, Henryk Michalewski, David Dohan, Ellen Jiang, Carrie J. Cai, Michael Terry, Quoc V. Le, and Charles Sutton. [Program synthesis with large language models](https://md.research.elamur.ai/free/en/papers/austin-2021-program-synthesis-large-language-models/). ArXiv, abs/2108.07732, 2021.
4. Solon Barocas, Kate Crawford, Aaron Shapiro, and Hanna Wallach. The problem with bias: From allocative to representational harms in machine learning. In SIGCIS conference paper, 2017. [find](https://scholar.google.com/scholar?q=Solon+Barocas%2C+Kate+Crawford%2C+Aaron+Shapiro%2C+and+Hanna+Wallach%2E+The+problem+with+bias%3A+From+allocative+to+representational+harms+in+machine+learning%2E+In+SIGCIS+conference+paper%2C+2017%2E)
5. Tolga Bolukbasi, Kai-Wei Chang, James Y. Zou, Venkatesh Saligrama, and Adam T. Kalai. [Man is to computer programmer as woman is to homemaker? Debiasing word embeddings](https://md.research.elamur.ai/free/en/papers/bolukbasi-2016-man-computer-programmer-woman-homemaker/). In NeurIPS, 2016.
6. Sébastien Bubeck, Varun Chandrasekaran, Ronen Eldan, John A. Gehrke, Eric Horvitz, Ece Kamar, Peter Lee, Yin Tat Lee, Yuan-Fang Li, Scott M. Lundberg, Harsha Nori, Hamid Palangi, Marco Tulio Ribeiro, and Yi Zhang. [Sparks of artificial general intelligence: Early experiments with GPT-4](https://md.research.elamur.ai/free/en/papers/bubeck-2023-sparks-artificial-general-intelligence-early/). ArXiv, abs/2…
7. Myra Cheng, Esin Durmus, and Dan Jurafsky. [Marked personas: Using natural language prompts to measure stereotypes in language models](https://md.research.elamur.ai/free/en/papers/cheng-2023-marked-personas-natural-language-prompts/). ArXiv, abs/2305.18189, 2023.
8. David Danks and Alex John London. Algorithmic bias in autonomous systems. In IJCAI, 2017. [find](https://scholar.google.com/scholar?q=David+Danks+and+Alex+John+London%2E+Algorithmic+bias+in+autonomous+systems%2E+In+IJCAI%2C+2017%2E)
9. Ameet Deshpande, Vishvak Murahari, Tanmay Rajpurohit, Ashwin Kalyan, and Karthik Narasimhan. [Toxicity in chatgpt: Analyzing persona-assigned language models](https://md.research.elamur.ai/free/en/papers/deshpande-2023-toxicity-chatgpt-analyzing-persona-assigned/). ArXiv, abs/2304.05335, 2023.
10. Sunipa Dev, Tao Li, J. M. Phillips, and Vivek Srikumar. On measuring and mitigating biased inferences of word embeddings. In AAAI, 2019. [find](https://scholar.google.com/scholar?q=Sunipa+Dev%2C+Tao+Li%2C+J%2E+M%2E+Phillips%2C+and+Vivek+Srikumar%2E+On+measuring+and+mitigating+biased+inferences+of+word+embeddings%2E+In+AAAI%2C+2019%2E)
11. Jonas Freiknecht and Wolfgang Effelsberg. Procedural generation of interactive stories using language models. In Proceedings of the 15th International Conference on the Foundations of Digital Games, pp. 1–8, 2020. [find](https://scholar.google.com/scholar?q=Jonas+Freiknecht+and+Wolfgang+Effelsberg%2E+Procedural+generation+of+interactive+stories+using+language+models%2E+In+Proceedings+of+the+15th+International+Conference+on+the+Foundations+of+Digital+Games%2C+p)
12. Perttu Hämäläinen, Mikke Tavast, and Anton Kunnari. Evaluating large language models in generating synthetic hci research data: a case study. In CHI, 2023. [find](https://scholar.google.com/scholar?q=Perttu+Ha%CC%88ma%CC%88la%CC%88inen%2C+Mikke+Tavast%2C+and+Anton+Kunnari%2E+Evaluating+large+language+models+in+generating+synthetic+hci+research+data%3A+a+case+study%2E+In+CHI%2C+2023%2E)
13. Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Xiaodong Song, and Jacob Steinhardt. [Measuring massive multitask language understanding](https://md.research.elamur.ai/free/en/papers/hendrycks-2020-measuring-massive-multitask-language-understanding/). ArXiv, abs/2009.03300, 2020.
14. Joseph Henrich, Steven J. Heine, and Ara Norenzayan. Most people are not weird. Nature, 466: 29–29, 2010. [find](https://scholar.google.com/scholar?q=Joseph+Henrich%2C+Steven+J%2E+Heine%2C+and+Ara+Norenzayan%2E+Most+people+are+not+weird%2E+Nature%2C+466%3A+29%E2%80%9329%2C+2010%2E)
15. John J. Horton. Large language models as simulated economic agents: What can we learn from homo silicus? Technical report, National Bureau of Economic Research, 2023. [find](https://scholar.google.com/scholar?q=John+J%2E+Horton%2E+Large+language+models+as+simulated+economic+agents%3A+What+can+we+learn+from+homo+silicus%3F+Technical+report%2C+National+Bureau+of+Economic+Research%2C+2023%2E)
16. Peter A. Jansen. From words to wires: Generating functioning electronic devices from natural language descriptions. ArXiv, abs/2305.14874, 2023. [find](https://scholar.google.com/scholar?q=Peter+A%2E+Jansen%2E+From+words+to+wires%3A+Generating+functioning+electronic+devices+from+natural+language+descriptions%2E+ArXiv%2C+abs%2F2305%2E14874%2C+2023%2E)
17. Takeshi Kojima, Shixiang Shane Gu, Machel Reid, Yutaka Matsuo, and Yusuke Iwasawa. [Large language models are zero-shot reasoners](https://md.research.elamur.ai/free/en/papers/kojima-2022-large-language-models-zero-shot/). In NeurIPS, 2022.
18. Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph E. Gonzalez, Hao Zhang, and Ion Stoica. [Efficient memory management for large language model serving with pagedattention](https://md.research.elamur.ai/free/en/papers/kwon-2023-efficient-memory-management-large-language/). In Proceedings of the ACM SIGOPS 29th Symposium on Operating Systems Principles, 2023.
19. Tao Li, Daniel Khashabi, Tushar Khot, Ashish Sabharwal, and Vivek Srikumar. Unqovering stereotypical biases via underspecified questions. In Findings@EMNLP, 2020. [find](https://scholar.google.com/scholar?q=Tao+Li%2C+Daniel+Khashabi%2C+Tushar+Khot%2C+Ashish+Sabharwal%2C+and+Vivek+Srikumar%2E+Unqovering+stereotypical+biases+via+underspecified+questions%2E+In+Findings%40EMNLP%2C+2020%2E)
20. Yingji Li, Mengnan Du, Rui Song, Xin Wang, and Y. Wang. A survey on fairness in large language models. ArXiv, abs/2308.10149, 2023. — [arXiv:2308.10149](https://arxiv.org/abs/2308.10149) [find](https://scholar.google.com/scholar?q=Yingji+Li%2C+Mengnan+Du%2C+Rui+Song%2C+Xin+Wang%2C+and+Y%2E+Wang%2E+A+survey+on+fairness+in+large+language+models%2E+ArXiv%2C+abs%2F2308%2E10149%2C+2023%2E+%E2%80%94+%5BarXiv%3A2308%2E10149%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2308%2E10149%29)
21. Ji Lin, Jiaming Tang, Haotian Tang, Shang Yang, Xingyu Dang, Chuang Gan, and Song Han. Awq: Activation-aware weight quantization for llm compression and acceleration, 2023. [find](https://scholar.google.com/scholar?q=Ji+Lin%2C+Jiaming+Tang%2C+Haotian+Tang%2C+Shang+Yang%2C+Xingyu+Dang%2C+Chuang+Gan%2C+and+Song+Han%2E+Awq%3A+Activation%2Daware+weight+quantization+for+llm+compression+and+acceleration%2C+2023%2E)
22. OpenAI. Custom instructions for chatgpt, 2023a. [https://openai.com/blog/custom-instructions-for-chatgpt](https://openai.com/blog/custom-instructions-for-chatgpt). [↗](https://openai.com/blog/custom-instructions-for-chatgpt)
23. OpenAI. [Gpt-4 technical report](https://md.research.elamur.ai/free/en/papers/openai-2023-gpt-4-technical-report/). ArXiv, abs/2303.08774, 2023b.
24. Jeongeon Park and Daeun Choi. Audilens: Configurable llm-generated audiences for public speech practice. Adjunct Proceedings of the 36th Annual ACM Symposium on User Interface Software and Technology, 2023. [find](https://scholar.google.com/scholar?q=Jeongeon+Park+and+Daeun+Choi%2E+Audilens%3A+Configurable+llm%2Dgenerated+audiences+for+public+speech+practice%2E+Adjunct+Proceedings+of+the+36th+Annual+ACM+Symposium+on+User+Interface+Software+and+Technology%2C)
25. Joon Sung Park, Lindsay Popowski, Carrie Cai, Meredith Ringel Morris, Percy Liang, and Michael S. Bernstein. [Social simulacra: Creating populated prototypes for social computing systems](https://md.research.elamur.ai/free/en/papers/park-2022-social-simulacra-creating-populated-prototypes/). In Proceedings of the 35th Annual ACM Symposium on User Interface Software and Technology, pp. 1–18, 2022.
26. Joon Sung Park, Joseph C. O’Brien, Carrie J. Cai, Meredith Ringel Morris, Percy Liang, and Michael S. Bernstein. Generative agents: Interactive simulacra of human behavior. In UIST, 2023. [find](https://scholar.google.com/scholar?q=Joon+Sung+Park%2C+Joseph+C%2E+O%E2%80%99Brien%2C+Carrie+J%2E+Cai%2C+Meredith+Ringel+Morris%2C+Percy+Liang%2C+and+Michael+S%2E+Bernstein%2E+Generative+agents%3A+Interactive+simulacra+of+human+behavior%2E+In+UIST%2C+2023%2E)
27. Chen Qian, Xin Cong, Cheng Yang, Weize Chen, Yusheng Su, Juyuan Xu, Zhiyuan Liu, and Maosong Sun. Communicative agents for software development. ArXiv, abs/2307.[07924, 2023](https://md.research.elamur.ai/free/en/papers/qian-2023-chatdev-communicative-agents-software/).
28. Chengwei Qin, Aston Zhang, Zhuosheng Zhang, Jiaao Chen, Michihiro Yasunaga, and Diyi Yang. [Is chatgpt a general-purpose natural language processing task solver?](https://md.research.elamur.ai/free/en/papers/qin-2023-chatgpt-general-purpose-natural-language/) ArXiv, abs/2302.06476, 2023.
29. Rachel Rudinger, Jason Naradowsky, Brian Leonard, and Benjamin Van Durme. [Gender bias in coreference resolution](https://md.research.elamur.ai/free/en/papers/rudinger-2018-gender-bias-coreference-resolution/). In NAACL, 2018.
30. Leonard Salewski, Stephan Alaniz, Isabel Rio-Torto, Eric Schulz, and Zeynep Akata. In-context impersonation reveals large language models’ strengths and biases. ArXiv, abs/2305.14930, 2023. — [arXiv:2305.14930](https://arxiv.org/abs/2305.14930) [find](https://scholar.google.com/scholar?q=Leonard+Salewski%2C+Stephan+Alaniz%2C+Isabel+Rio%2DTorto%2C+Eric+Schulz%2C+and+Zeynep+Akata%2E+In%2Dcontext+impersonation+reveals+large+language+models%E2%80%99+strengths+and+biases%2E+ArXiv%2C+abs%2F2305%2E14930%2C+2023%2E+%E2%80%94+%5BarXiv%3A2)
31. Emily Sheng, Josh Arnold, Zhou Yu, Kai-Wei Chang, and Nanyun Peng. Revealing persona biases in dialogue systems. ArXiv, abs/2104.08728, 2021. — [arXiv:2104.08728](https://arxiv.org/abs/2104.08728) [find](https://scholar.google.com/scholar?q=Emily+Sheng%2C+Josh+Arnold%2C+Zhou+Yu%2C+Kai%2DWei+Chang%2C+and+Nanyun+Peng%2E+Revealing+persona+biases+in+dialogue+systems%2E+ArXiv%2C+abs%2F2104%2E08728%2C+2021%2E+%E2%80%94+%5BarXiv%3A2104%2E08728%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2104%2E08728%29)
32. Mirac Suzgun, Nathan Scales, Nathanael Scharli, Sebastian Gehrmann, Yi Tay, Hyung Won Chung, Aakanksha Chowdhery, Quoc V. Le, Ed Huai hsin Chi, Denny Zhou, and Jason Wei. [Challenging big-bench tasks and whether chain-of-thought can solve them](https://md.research.elamur.ai/free/en/papers/suzgun-2023-challenging-big-bench-tasks-whether/). In Annual Meeting of the Association for Computational L…
33. Hugo Touvron, Louis Martin, Kevin R. Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, Daniel M. Bikel, Lukas Blecher, Cristian Cantón Ferrer, Moya Chen, Guillem Cucurull, David Esiobu, Jude Fernandes, Jeremy Fu, Wenyin Fu, Bri… — [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://md.research.elamur.ai/free/en/papers/touvron-2023-llama-2-open-foundation-fine/)
34. Yixin Wan, Jieyu Zhao, Nanyun Peng, Kai-Wei Chang, and Aman Chadha. Are personalized stochastic parrots more dangerous? Evaluating persona biases in dialogue systems. ArXiv, abs/2310.05280, 2023. [find](https://scholar.google.com/scholar?q=Yixin+Wan%2C+Jieyu+Zhao%2C+Nanyun+Peng%2C+Kai%2DWei+Chang%2C+and+Aman+Chadha%2E+Are+personalized+stochastic+parrots+more+dangerous%3F+Evaluating+persona+biases+in+dialogue+systems%2E+ArXiv%2C+abs%2F2310%2E05280%2C+2023%2E)
35. Zhenhailong Wang, Shaoguang Mao, Wenshan Wu, Tao Ge, Furu Wei, and Heng Ji. Unleashing cognitive synergy in large language models: A task-solving agent through multi-persona self-collaboration. ArXiv, abs/2307.[05300, 2023](https://md.research.elamur.ai/free/en/papers/wang-2023-unleashing-emergent-cognitive-synergy-large/).
36. Edwin B Wilson. Probable inference, the law of succession, and statistical inference. Journal of the American Statistical Association, 1927. [find](https://scholar.google.com/scholar?q=Edwin+B+Wilson%2E+Probable+inference%2C+the+law+of+succession%2C+and+statistical+inference%2E+Journal+of+the+American+Statistical+Association%2C+1927%2E)
37. Jieyu Zhao, Tianlu Wang, Mark Yatskar, Vicente Ordonez, and Kai-Wei Chang. [Gender bias in coreference resolution: Evaluation and debiasing methods](https://md.research.elamur.ai/free/en/papers/zhao-2018-gender-bias-coreference-resolution-evaluation/). In NAACL, 2018.
38. Jieyu Zhao, Daniel Khashabi, Tushar Khot, Ashish Sabharwal, and Kai-Wei Chang. Ethical-advice taker: Do language models understand natural language interventions? In Findings@ACL, 2021. [find](https://scholar.google.com/scholar?q=Jieyu+Zhao%2C+Daniel+Khashabi%2C+Tushar+Khot%2C+Ashish+Sabharwal%2C+and+Kai%2DWei+Chang%2E+Ethical%2Dadvice+taker%3A+Do+language+models+understand+natural+language+interventions%3F+In+Findings%40ACL%2C+2021%2E)

*Updated 2 November 2026*