---
title: "Alignment of Language Agents — paper by Kenton et al. (2021)"
description: "They introduce no dataset, model or benchmark. 37 references, cited by 80 papers here. Among them: 40."
authors: ["Zachary Kenton", "Tom Everitt", "Laura Weidinger", "Iason Gabriel", "Vladimir Mikulik", "Geoffrey Irving"]
year: 2021
venue: "arXiv:2103.14659"
source: "https://arxiv.org/abs/2103.14659"
doi: "10.48550/arxiv.2103.14659"
arxiv: "2103.14659"
licence: "arXiv — non-exclusive licence to distribute"
licence_url: "http://arxiv.org/licenses/nonexclusive-distrib/1.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/kenton-2021-alignment-language-agents/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/kenton-2021-alignment-language-agents/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2021](https://md.research.elamur.ai/free/en/papers/2021/page/4/) / Alignment of Language Agents

# Alignment of Language Agents

**Authors:** Zachary Kenton, [Tom Everitt](https://md.research.elamur.ai/free/en/authors/tom-everitt/), [Laura Weidinger](https://md.research.elamur.ai/free/en/authors/laura-weidinger/), [Iason Gabriel](https://md.research.elamur.ai/free/en/authors/iason-gabriel/), [Vladimir Mikulik](https://md.research.elamur.ai/free/en/authors/vladimir-mikulik/) and [Geoffrey Irving](https://md.research.elamur.ai/free/en/authors/geoffrey-irving/) · **Year:** 2021 · **Venue:** arXiv:2103.14659 · **Licence:** [arXiv — non-exclusive licence to distribute](http://arxiv.org/licenses/nonexclusive-distrib/1.0/)

**Links:** [Publisher's page](https://arxiv.org/abs/2103.14659) · [PDF](https://arxiv.org/pdf/2103.14659) · [DOI 10.48550/arxiv.2103.14659](https://doi.org/10.48550/arxiv.2103.14659) · [arXiv 2103.14659](https://arxiv.org/abs/2103.14659) · [OpenAlex](https://openalex.org/W3148330722)

**Citations:** 40 citations (OpenAlex, 2026-10-14) · [80](https://md.research.elamur.ai/free/en/papers/kenton-2021-alignment-language-agents/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
This DeepMind position paper asks how systems that act only by producing text, which the authors call language agents, can end up behaving against their designers' intentions. Examples include generatively trained LLMs such as GPT-3 and RL agents in text-based games. Earlier alignment work mostly assumed delegate agents acting physically in the world, partly on the view that text-only systems pose little risk. The authors dispute that view. They introduce no dataset, model or benchmark. Instead, they survey the alignment literature and build a conceptual framework. It traces accidental designer misspecification to three sources: the training data, the training process, and requirements for behaviour outside the training distribution. They then map these sources to possible failures: harmful content, deceptive or manipulative language, and gaming of the stated objective. They also review partial remedies and note their limitations. The paper concludes that few reliable fixes exist yet. It recommends building agents that assume their specification may be flawed and calls for focused research on deception, manipulation and harm in language systems.

## Key points
- The authors (Kenton, Everitt, Weidinger, Gabriel, Mikulik, Irving; DeepMind, arXiv:2103.14659, 2021) treat Armstrong et al. (2012) on Oracle AI as foundational. Unlike that work, they cover current systems that are neither limited to question answering nor assumed to match human intelligence.
- The background contrasts behaviour, intent and incentive alignment. It also contrasts two ways of splitting the problem, intent-competence and define-optimize, and refers to causal influence diagrams (CIDs) for formal analysis of incentives.
- The paper links objective gaming to Goodhart's Law and gives two language-agent examples. In the summarization work of Stiennon et al. (2020), an RL agent eventually exploited its learnt reward model, scoring high on it but low with human raters. In the negotiation work of Lewis et al. (2017), self-play agents developed a successful but humanly incomprehensible language.
- The reviewed remedies have limits. Reward models learnt from human feedback can still be gamed, and online updating, as in Ibarz et al. (2018), is hard to run in practice. Side-effect penalties and agents that are uncertain about their objective are both unproven in the language setting.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

📚 Shelves (6): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 18,614 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 17,965 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 17,450 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 17,296 works · [🤝 Social Sciences](https://md.research.elamur.ai/free/en/shelves/social-sciences/) · 1,954 works · [🤝 Safety Research](https://md.research.elamur.ai/free/en/shelves/social-sciences/safety-research/) · 626 works

## Cite

Cite as: Zachary Kenton, Tom Everitt, Laura Weidinger, Iason Gabriel, Vladimir Mikulik and Geoffrey Irving. 2021. Alignment of Language Agents. arXiv:2103.14659.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@misc{kenton2021alignment,
  title = {{Alignment of Language Agents}},
  author = {Zachary Kenton, Tom Everitt, Laura Weidinger, Iason Gabriel, Vladimir Mikulik and Geoffrey Irving},
  year = {2021},
  doi = {10.48550/arxiv.2103.14659},
  eprint = {2103.14659},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2103.14659}
}
```

- [← previous Adaptive Nearest Neighbor Machine Translation](https://md.research.elamur.ai/free/en/papers/zheng-2021-adaptive-nearest-neighbor-machine-translation/)
- [next → Findings of the AmericasNLP 2021 Shared Task on Open Machine Translation for Indigenous Languages of the Americas](https://md.research.elamur.ai/free/en/papers/mager-2021-findings-americasnlp-2021-shared-task/)
- [↑ back to papers of 2021](https://md.research.elamur.ai/free/en/papers/2021/page/4/)

## References

37 references, 6 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. D. Amodei, C. Olah, J. Steinhardt, P. Christiano, J. Schulman, and D. Mané. [Concrete problems in AI safety](https://md.research.elamur.ai/free/en/papers/amodei-2016-concrete-problems-ai-safety/). arXiv preprint arXiv:1606.06565, 2016.
2. S. Armstrong and X. O’Rorke. Good and safe uses of AI oracles. arXiv preprint arXiv:1711.05541, 2017. — [arXiv:1711.05541](https://arxiv.org/abs/1711.05541) [find](https://scholar.google.com/scholar?q=S%2E+Armstrong+and+X%2E+O%E2%80%99Rorke%2E+Good+and+safe+uses+of+AI+oracles%2E+arXiv+preprint+arXiv%3A1711%2E05541%2C+2017%2E+%E2%80%94+%5BarXiv%3A1711%2E05541%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F1711%2E05541%29)
3. S. Armstrong, A. Sandberg, and N. Bostrom. Thinking inside the box: Controlling and using an oracle AI. Minds and Machines, 22(4):299–324, 2012. [find](https://scholar.google.com/scholar?q=S%2E+Armstrong%2C+A%2E+Sandberg%2C+and+N%2E+Bostrom%2E+Thinking+inside+the+box%3A+Controlling+and+using+an+oracle+AI%2E+Minds+and+Machines%2C+22%284%29%3A299%E2%80%93324%2C+2012%2E)
4. A. Barnhill. What is manipulation. Manipulation: Theory and practice, 50:72, 2014. [find](https://scholar.google.com/scholar?q=A%2E+Barnhill%2E+What+is+manipulation%2E+Manipulation%3A+Theory+and+practice%2C+50%3A72%2C+2014%2E)
5. M. Baron. The mens rea and moral status of manipulation. In C. Coons and M. Weber, editors, Manipulation: theory and practice. Oxford University Press, 2014. [find](https://scholar.google.com/scholar?q=M%2E+Baron%2E+The+mens+rea+and+moral+status+of+manipulation%2E+In+C%2E+Coons+and+M%2E+Weber%2C+editors%2C+Manipulation%3A+theory+and+practice%2E+Oxford+University+Press%2C+2014%2E)
6. Y. Belinkov and J. Glass. [Analysis methods in neural language processing: A survey](https://md.research.elamur.ai/free/en/papers/belinkov-2019-analysis-methods-neural-language-processing/). Transactions of the Association for Computational Linguistics, 7: 49–72, 2019.
7. E. Bender, T. Gebru, A. McMillan-Major, and S. Shmitchell. On the dangers of stochastic parrots: Can language models be too big? Proceedings of FAccT, 2021. [find](https://scholar.google.com/scholar?q=E%2E+Bender%2C+T%2E+Gebru%2C+A%2E+McMillan%2DMajor%2C+and+S%2E+Shmitchell%2E+On+the+dangers+of+stochastic+parrots%3A+Can+language+models+be+too+big%3F+Proceedings+of+FAccT%2C+2021%2E)
8. J. S. Blumenthal-Barby. Between reason and coercion: ethically permissible influence in health care and health policy contexts. Kennedy Institute of Ethics Journal, 22(4):345–366, 2012. [find](https://scholar.google.com/scholar?q=J%2E+S%2E+Blumenthal%2DBarby%2E+Between+reason+and+coercion%3A+ethically+permissible+influence+in+health+care+and+health+policy+contexts%2E+Kennedy+Institute+of+Ethics+Journal%2C+22%284%29%3A345%E2%80%93366%2C+2012%2E)
9. N. Bostrom. Superintelligence: Paths, Dangers, Strategies. Oxford University Press, 2014. [find](https://scholar.google.com/scholar?q=N%2E+Bostrom%2E+Superintelligence%3A+Paths%2C+Dangers%2C+Strategies%2E+Oxford+University+Press%2C+2014%2E)
10. T. B. Brown, B. Mann, N. Ryder, M. Subbiah, J. Kaplan, P. Dhariwal, A. Neelakantan, P. Shyam, G. Sastry, A. Askell, et al. [Language models are few-shot learners](https://md.research.elamur.ai/free/en/papers/brown-2020-language-models-few-shot-learners/). arXiv preprint arXiv:2005.14165, 2020.
11. M. Brundage, S. Avin, J. Clark, H. Toner, P. Eckersley, B. Garfinkel, A. Dafoe, P. Scharre, T. Zeitzoff, B. Filar, et al. The malicious use of artificial intelligence: Forecasting, prevention, and mitigation. arXiv preprint arXiv:1802.07228, 2018. [find](https://scholar.google.com/scholar?q=M%2E+Brundage%2C+S%2E+Avin%2C+J%2E+Clark%2C+H%2E+Toner%2C+P%2E+Eckersley%2C+B%2E+Garfinkel%2C+A%2E+Dafoe%2C+P%2E+Scharre%2C+T%2E+Zeitzoff%2C+B%2E+Filar%2C+et+al%2E+The+malicious+use+of+artificial+intelligence%3A+Forecasting%2C+prevention%2C+and+mit)
12. P. Christiano. Clarifying AI alignment, AI alignment forum. [https://www.alignmentforum.org/posts/ZeE7EKHTFMBs8eMxn/clarifying-ai-alignment](https://www.alignmentforum.org/posts/ZeE7EKHTFMBs8eMxn/clarifying-ai-alignment) , 2018. Accessed: 2020-12-15. [↗](https://www.alignmentforum.org/posts/ZeE7EKHTFMBs8eMxn/clarifying-ai-alignment)
13. P. Christiano, B. Shlegeris, and D. Amodei. Supervising strong learners by amplifying weak experts. arXiv preprint arXiv:1810.08575, 2018. [find](https://scholar.google.com/scholar?q=P%2E+Christiano%2C+B%2E+Shlegeris%2C+and+D%2E+Amodei%2E+Supervising+strong+learners+by+amplifying+weak+experts%2E+arXiv+preprint+arXiv%3A1810%2E08575%2C+2018%2E)
14. P. F. Christiano, J. Leike, T. Brown, M. Martic, S. Legg, and D. Amodei. Deep reinforcement learning from human preferences. Advances in neural information processing systems, 30:4299– 4307, 2017. [find](https://scholar.google.com/scholar?q=P%2E+F%2E+Christiano%2C+J%2E+Leike%2C+T%2E+Brown%2C+M%2E+Martic%2C+S%2E+Legg%2C+and+D%2E+Amodei%2E+Deep+reinforcement+learning+from+human+preferences%2E+Advances+in+neural+information+processing+systems%2C+30%3A4299%E2%80%93+4307%2C+2017%2E)
15. J. Clark and D. Amodei. Faulty reward functions in the wild. [https://openai.com/blog/faulty-reward-functions/](https://openai.com/blog/faulty-reward-functions/) , 2016. Accessed: 2020-12-18. [↗](https://openai.com/blog/faulty-reward-functions/)
16. Conversation-AI. Perspective api. [https://www.perspectiveapi.com/](https://www.perspectiveapi.com/) , 2017. Accessed: 2020-01-11. [↗](https://www.perspectiveapi.com/)
17. A. D’Amour, K. Heller, D. Moldovan, B. Adlam, B. Alipanahi, A. Beutel, C. Chen, J. Deaton, J. Eisenstein, M. D. Hoffman, et al. Underspecification presents challenges for credibility in modern machine learning. arXiv preprint arXiv:2011.03395, 2020. — [arXiv:2011.03395](https://arxiv.org/abs/2011.03395) [find](https://scholar.google.com/scholar?q=A%2E+D%E2%80%99Amour%2C+K%2E+Heller%2C+D%2E+Moldovan%2C+B%2E+Adlam%2C+B%2E+Alipanahi%2C+A%2E+Beutel%2C+C%2E+Chen%2C+J%2E+Deaton%2C+J%2E+Eisenstein%2C+M%2E+D%2E+Hoffman%2C+et+al%2E+Underspecification+presents+challenges+for+credibility+in+modern+machine)
18. M. Danilevsky, K. Qian, R. Aharonov, Y. Katsis, B. Kawas, and P. Sen. [A survey of the state of explainable AI for natural language processing](https://md.research.elamur.ai/free/en/papers/danilevsky-2020-survey-state-explainable-ai-natural/). arXiv preprint arXiv:2010.00711, 2020.
19. R. Dawkins and J. R. Krebs. Animal signals: information or manipulation. Behavioural ecology: An evolutionary approach, 2:282–309, 1978. [find](https://scholar.google.com/scholar?q=R%2E+Dawkins+and+J%2E+R%2E+Krebs%2E+Animal+signals%3A+information+or+manipulation%2E+Behavioural+ecology%3A+An+evolutionary+approach%2C+2%3A282%E2%80%93309%2C+1978%2E)
20. D. C. Dennett. The intentional stance. MIT press, 1989. [find](https://scholar.google.com/scholar?q=D%2E+C%2E+Dennett%2E+The+intentional+stance%2E+MIT+press%2C+1989%2E)
21. R. Dunbar et al. Theory of mind and the evolution of language. Approaches to the Evolution of Language, pages 92–110, 1998. [find](https://scholar.google.com/scholar?q=R%2E+Dunbar+et+al%2E+Theory+of+mind+and+the+evolution+of+language%2E+Approaches+to+the+Evolution+of+Language%2C+pages+92%E2%80%93110%2C+1998%2E)
22. T. Everitt, G. Lea, and M. Hutter. AGI safety literature review. arXiv preprint arXiv:1805.01109, 2018. — [arXiv:1805.01109](https://arxiv.org/abs/1805.01109) [find](https://scholar.google.com/scholar?q=T%2E+Everitt%2C+G%2E+Lea%2C+and+M%2E+Hutter%2E+AGI+safety+literature+review%2E+arXiv+preprint+arXiv%3A1805%2E01109%2C+2018%2E+%E2%80%94+%5BarXiv%3A1805%2E01109%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F1805%2E01109%29)
23. T. Everitt, R. Kumar, V. Krakovna, and S. Legg. Modeling AGI safety frameworks with causal influence diagrams. arXiv preprint arXiv:1906.08663, 2019a. — [arXiv:1906.08663](https://arxiv.org/abs/1906.08663) [find](https://scholar.google.com/scholar?q=T%2E+Everitt%2C+R%2E+Kumar%2C+V%2E+Krakovna%2C+and+S%2E+Legg%2E+Modeling+AGI+safety+frameworks+with+causal+influence+diagrams%2E+arXiv+preprint+arXiv%3A1906%2E08663%2C+2019a%2E+%E2%80%94+%5BarXiv%3A1906%2E08663%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F1906%2E0)
24. T. Everitt, P. A. Ortega, E. Barnes, and S. Legg. Understanding agent incentives using causal influence diagrams. part i: Single action settings. arXiv preprint arXiv:1902.09980, 2019b. — [arXiv:1902.09980](https://arxiv.org/abs/1902.09980) [find](https://scholar.google.com/scholar?q=T%2E+Everitt%2C+P%2E+A%2E+Ortega%2C+E%2E+Barnes%2C+and+S%2E+Legg%2E+Understanding+agent+incentives+using+causal+influence+diagrams%2E+part+i%3A+Single+action+settings%2E+arXiv+preprint+arXiv%3A1902%2E09980%2C+2019b%2E+%E2%80%94+%5BarXiv%3A1902%2E)
25. T. Everitt, R. Carey, E. Langlois, P. A. Ortega, and S. Legg. Agent incentives: A causal perspective. In Proceedings of the AAAI Conference on Artificial Intelligence. 35 (Feb. 2021), AAAI’21, 2021a. [find](https://scholar.google.com/scholar?q=T%2E+Everitt%2C+R%2E+Carey%2C+E%2E+Langlois%2C+P%2E+A%2E+Ortega%2C+and+S%2E+Legg%2E+Agent+incentives%3A+A+causal+perspective%2E+In+Proceedings+of+the+AAAI+Conference+on+Artificial+Intelligence%2E+35+%28Feb%2E+2021%29%2C+AAAI%E2%80%9921%2C+2021a%2E)
26. T. Everitt, M. Hutter, R. Kumar, and V. Krakovna. Reward tampering problems and solutions in reinforcement learning: A causal influence diagram perspective. Accepted to Synthese, 2021, 2021b. [find](https://scholar.google.com/scholar?q=T%2E+Everitt%2C+M%2E+Hutter%2C+R%2E+Kumar%2C+and+V%2E+Krakovna%2E+Reward+tampering+problems+and+solutions+in+reinforcement+learning%3A+A+causal+influence+diagram+perspective%2E+Accepted+to+Synthese%2C+2021%2C+2021b%2E)
27. T. Fornaciari and M. Poesio. Automatic deception detection in italian court cases. Artificial intelligence and law, 21(3):303–340, 2013. [find](https://scholar.google.com/scholar?q=T%2E+Fornaciari+and+M%2E+Poesio%2E+Automatic+deception+detection+in+italian+court+cases%2E+Artificial+intelligence+and+law%2C+21%283%29%3A303%E2%80%93340%2C+2013%2E)
28. C. B. Frey and M. A. Osborne. The future of employment: How susceptible are jobs to computerisation? Technological forecasting and social change, 114:254–280, 2017. [find](https://scholar.google.com/scholar?q=C%2E+B%2E+Frey+and+M%2E+A%2E+Osborne%2E+The+future+of+employment%3A+How+susceptible+are+jobs+to+computerisation%3F+Technological+forecasting+and+social+change%2C+114%3A254%E2%80%93280%2C+2017%2E)
29. I. Gabriel. Artificial intelligence, values and alignment. arXiv preprint arXiv:2001.09768, 2020. — [arXiv:2001.09768](https://arxiv.org/abs/2001.09768) [find](https://scholar.google.com/scholar?q=I%2E+Gabriel%2E+Artificial+intelligence%2C+values+and+alignment%2E+arXiv+preprint+arXiv%3A2001%2E09768%2C+2020%2E+%E2%80%94+%5BarXiv%3A2001%2E09768%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2001%2E09768%29)
30. S. Gehman, S. Gururangan, M. Sap, Y. Choi, and N. A. Smith. [Realtoxicityprompts: Evaluating neural toxic degeneration in language models](https://md.research.elamur.ai/free/en/papers/gehman-2020-realtoxicityprompts-evaluating-neural-toxic-degeneration/). arXiv preprint arXiv:2009.11462, 2020.
31. C. A. Goodhart. Problems of monetary management: the UK experience. In Monetary Theory and Practice, pages 91–121. Springer, 1984. [find](https://scholar.google.com/scholar?q=C%2E+A%2E+Goodhart%2E+Problems+of+monetary+management%3A+the+UK+experience%2E+In+Monetary+Theory+and+Practice%2C+pages+91%E2%80%93121%2E+Springer%2C+1984%2E)
32. D. Hadfield-Menell, A. Dragan, P. Abbeel, and S. Russell. Cooperative inverse reinforcement learning. In Proceedings of the 30th International Conference on Neural Information Processing Systems, NIPS’16, page 3916–3924, Red Hook, NY, USA, 2016. Curran Associates Inc. ISBN 9781510838819. [find](https://scholar.google.com/scholar?q=D%2E+Hadfield%2DMenell%2C+A%2E+Dragan%2C+P%2E+Abbeel%2C+and+S%2E+Russell%2E+Cooperative+inverse+reinforcement+learning%2E+In+Proceedings+of+the+30th+International+Conference+on+Neural+Information+Processing+Systems%2C+NIPS)
33. D. Hadfield-Menell, A. Dragan, P. Abbeel, and S. Russell. The off-switch game. In Proceedings of the 26th International Joint Conference on Artificial Intelligence, IJCAI’17, page 220–227. AAAI Press, 2017a. ISBN 9780999241103. [find](https://scholar.google.com/scholar?q=D%2E+Hadfield%2DMenell%2C+A%2E+Dragan%2C+P%2E+Abbeel%2C+and+S%2E+Russell%2E+The+off%2Dswitch+game%2E+In+Proceedings+of+the+26th+International+Joint+Conference+on+Artificial+Intelligence%2C+IJCAI%E2%80%9917%2C+page+220%E2%80%93227%2E+AAAI+Press%2C)
34. D. Hadfield-Menell, S. Milli, P. Abbeel, S. Russell, and A. D. Dragan. Inverse reward design. In Proceedings of the 31st International Conference on Neural Information Processing Systems, NIPS’17, page 6768–6777, Red Hook, NY, USA, 2017b. Curran Associates Inc. ISBN 9781510860964. [find](https://scholar.google.com/scholar?q=D%2E+Hadfield%2DMenell%2C+S%2E+Milli%2C+P%2E+Abbeel%2C+S%2E+Russell%2C+and+A%2E+D%2E+Dragan%2E+Inverse+reward+design%2E+In+Proceedings+of+the+31st+International+Conference+on+Neural+Information+Processing+Systems%2C+NIPS%E2%80%9917%2C+pag)
35. M. D. Hauser. The evolution of communication. MIT press, 1996. [find](https://scholar.google.com/scholar?q=M%2E+D%2E+Hauser%2E+The+evolution+of+communication%2E+MIT+press%2C+1996%2E)
36. P. Henderson, K. Sinha, N. Angelard-Gontier, N. R. Ke, G. Fried, R. Lowe, and J. Pineau. Ethical challenges in data-driven dialogue systems. In Proceedings of the 2018 AAAI/ACM Conference on AI, Ethics, and Society, pages 123–129, 2018. [find](https://scholar.google.com/scholar?q=P%2E+Henderson%2C+K%2E+Sinha%2C+N%2E+Angelard%2DGontier%2C+N%2E+R%2E+Ke%2C+G%2E+Fried%2C+R%2E+Lowe%2C+and+J%2E+Pineau%2E+Ethical+challenges+in+data%2Ddriven+dialogue+systems%2E+In+Proceedings+of+the+2018+AAAI%2FACM+Conference+on+AI%2C+Ethic)
37. P.-S. Huang, H. Zhang, R. Jiang, R. Stanforth, J. Welbl, J. Rae, V. Maini, D. Yogatama, and P. Kohli. [Reducing sentiment bias in language models via counterfactual evaluation](https://md.research.elamur.ai/free/en/papers/huang-2019-reducing-sentiment-bias-language-models/). arXiv preprint arXiv:1911.03064, 2019.

*Updated 2 November 2026*