---
title: "Can Machines Learn Morality? The Delphi Experiment (2021)"
description: "The authors present Delphi, a model that predicts how people would typically judge ordinary… 128 references, cited by 110 papers here."
authors: ["Liwei Jiang", "Jena D. Hwang", "Chandra Bhagavatula", "Ronan Le Bras", "Jenny Liang", "Jesse Dodge", "Keisuke Sakaguchi", "Maxwell Forbes", "Jon Borchardt", "Saadia Gabriel", "Yulia Tsvetkov", "Oren Etzioni", "Maarten Sap", "Regina Rini", "Yejin Choi"]
year: 2021
venue: "arXiv:2110.07574"
source: "https://arxiv.org/abs/2110.07574"
doi: "10.48550/arxiv.2110.07574"
arxiv: "2110.07574"
licence: "arXiv — non-exclusive license to distribute"
licence_url: "http://arxiv.org/licenses/nonexclusive-distrib/1.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/jiang-2021-machines-learn-morality-delphi-experiment/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/jiang-2021-machines-learn-morality-delphi-experiment/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2021](https://md.research.elamur.ai/free/en/papers/2021/page/4/) / Can Machines Learn Morality? The Delphi Experiment

# Can Machines Learn Morality? The Delphi Experiment

**Authors:** [Liwei Jiang](https://md.research.elamur.ai/free/en/authors/liwei-jiang/), [Jena D. Hwang](https://md.research.elamur.ai/free/en/authors/jena-d-hwang/), [Chandra Bhagavatula](https://md.research.elamur.ai/free/en/authors/chandra-bhagavatula/), [Ronan Le Bras](https://md.research.elamur.ai/free/en/authors/ronan-le-bras/), [Jenny Liang](https://md.research.elamur.ai/free/en/authors/jenny-liang/), [Jesse Dodge](https://md.research.elamur.ai/free/en/authors/jesse-dodge/), [Keisuke Sakaguchi](https://md.research.elamur.ai/free/en/authors/keisuke-sakaguchi/), [Maxwell Forbes](https://md.research.elamur.ai/free/en/authors/maxwell-forbes/), [Jon Borchardt](https://md.research.elamur.ai/free/en/authors/jon-borchardt/), [Saadia Gabriel](https://md.research.elamur.ai/free/en/authors/saadia-gabriel/), [Yulia Tsvetkov](https://md.research.elamur.ai/free/en/authors/yulia-tsvetkov/), [Oren Etzioni](https://md.research.elamur.ai/free/en/authors/oren-etzioni/), [Maarten Sap](https://md.research.elamur.ai/free/en/authors/maarten-sap/), Regina Rini and [Yejin Choi](https://md.research.elamur.ai/free/en/authors/yejin-choi/) · **Year:** 2021 · **Venue:** arXiv:2110.07574 · **Licence:** [arXiv — non-exclusive license to distribute](http://arxiv.org/licenses/nonexclusive-distrib/1.0/)

**Links:** [Publisher's page](https://arxiv.org/abs/2110.07574) · [PDF](https://arxiv.org/pdf/2110.07574) · [DOI 10.48550/arxiv.2110.07574](https://doi.org/10.48550/arxiv.2110.07574) · [arXiv 2110.07574](https://arxiv.org/abs/2110.07574)

**Citations:** [110](https://md.research.elamur.ai/free/en/papers/jiang-2021-machines-learn-morality-delphi-experiment/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
AI systems increasingly make choices with ethical consequences, yet large language models trained on web text have no explicit grasp of everyday moral norms. The authors present Delphi, a model that predicts how people would typically judge ordinary situations. It is framed as descriptive ethics and loosely grounded in Rawls's bottom-up "decision procedure" and reflective equilibrium. Delphi starts from UNICORN, a T5-11B model tuned for commonsense QA. It is then trained on COMMONSENSE NORM BANK, which merges five existing crowdsourced resources into one format. Given a situation, the model returns either a yes/no verdict or a short free-text judgment. On held-out data it clearly beats GPT-3 and remains sensitive to small changes in context. The paper is equally frank about weaknesses: Delphi inherits social biases and makes inconsistent calls, and its annotators were mostly white and US-based. The authors still show practical value, using Delphi as a component in other systems. The work matters as an early, openly tested resource for computational moral reasoning that also maps its own failure modes.

## Key points
- COMMONSENSE NORM BANK holds about 1.7M crowdsourced ethical judgments drawn from SOCIAL CHEM, SOCIAL BIAS FRAMES, MORAL STORIES, ETHICS and SCRUPLES.
- On unseen NORM BANK examples, Delphi is correct 92.8% of the time, compared with 60.2% for GPT-3.
- The public demo (delphi.allenai.org, now running the improved Delphi+) has received more than four million queries.
- Downstream uses include:
  - a few-shot hate speech detector,
  - morally guided story generation,
  - transfer to other moral frameworks.
- Bias was probed with Universal Declaration of Human Rights (UDHR) items across 213 identities grouped into 12 categories. The authors added identity-related training data to make Delphi+ more robust against these biases.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Cite

Cite as: Liwei Jiang, Jena D. Hwang, Chandra Bhagavatula, Ronan Le Bras, Jenny Liang, Jesse Dodge, Keisuke Sakaguchi, Maxwell Forbes, Jon Borchardt, Saadia Gabriel, Yulia Tsvetkov, Oren Etzioni, Maarten Sap, Regina Rini and Yejin Choi. 2021. Can Machines Learn Morality? The Delphi Experiment. arXiv:2110.07574.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@misc{jiang2021can,
  title = {{Can Machines Learn Morality? The Delphi Experiment}},
  author = {Liwei Jiang, Jena D. Hwang, Chandra Bhagavatula, Ronan {Le Bras}, Jenny Liang, Jesse Dodge, Keisuke Sakaguchi, Maxwell Forbes, Jon Borchardt, Saadia Gabriel, Yulia Tsvetkov, Oren Etzioni, Maarten Sap, Regina Rini and Yejin Choi},
  year = {2021},
  doi = {10.48550/arxiv.2110.07574},
  eprint = {2110.07574},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2110.07574}
}
```

- [← previous What's in the Box? An Analysis of Undesirable Content in the Common Crawl Corpus](https://md.research.elamur.ai/free/en/papers/luccioni-2021-what-s-box-analysis-undesirable/)
- [next → NeuroLogic Decoding: (Un)supervised Neural Text Generation with Predicate Logic Constraints](https://md.research.elamur.ai/free/en/papers/lu-2021-neurologic-decoding-un-supervised-neural/)
- [↑ back to papers of 2021](https://md.research.elamur.ai/free/en/papers/2021/page/4/)

## References

128 references, 19 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Saleema Amershi, Maya Cakmak, W. Knox, and Todd Kulesza. Power to the people: The role of humans in interactive machine learning. AI Magazine, 35:105–120, 12 2014. doi: 10.1609/aima g.v35i4.2513. — [doi:10.1609/aima](https://doi.org/10.1609/aima) [find](https://scholar.google.com/scholar?q=Saleema+Amershi%2C+Maya+Cakmak%2C+W%2E+Knox%2C+and+Todd+Kulesza%2E+Power+to+the+people%3A+The+role+of+humans+in+interactive+machine+learning%2E+AI+Magazine%2C+35%3A105%E2%80%93120%2C+12+2014%2E+doi%3A+10%2E1609%2Faima+g%2Ev35i4%2E2513%2E+%E2%80%94+%5Bd)
2. Saleema Amershi, Dan Weld, Mihaela Vorvoreanu, Adam Fourney, Besmira Nushi, Penny Collisson, Jina Suh, Shamsi Iqbal, Paul N. Bennett, Kori Inkpen, Jaime Teevan, Ruth Kikin-Gil, and Eric Horvitz. Guidelines for human-ai interaction. In Proceedings of the 2019 CHI Conference on Human Factors in Comput… — [doi:10.1145/3290605.3300233](https://doi.org/10.1145/3290605.3300233) [find](https://scholar.google.com/scholar?q=Saleema+Amershi%2C+Dan+Weld%2C+Mihaela+Vorvoreanu%2C+Adam+Fourney%2C+Besmira+Nushi%2C+Penny+Collisson%2C+Jina+Suh%2C+Shamsi+Iqbal%2C+Paul+N%2E+Bennett%2C+Kori+Inkpen%2C+Jaime+Teevan%2C+Ruth+Kikin%2DGil%2C+and+Eric+Horvitz%2E+Guide)
3. Prithviraj Ammanabrolu, Liwei Jiang, Maarten Sap, Hanna Hajishirzi, and Yejin Choi. Aligning to social norms and values in interactive narratives. In NAACL, 2022. [find](https://scholar.google.com/scholar?q=Prithviraj+Ammanabrolu%2C+Liwei+Jiang%2C+Maarten+Sap%2C+Hanna+Hajishirzi%2C+and+Yejin+Choi%2E+Aligning+to+social+norms+and+values+in+interactive+narratives%2E+In+NAACL%2C+2022%2E)
4. Susan Leigh Anderson. Asimov’s “three laws of robotics” and machine metaethics. Ai & Society, 22(4):477–493, 2008. [find](https://scholar.google.com/scholar?q=Susan+Leigh+Anderson%2E+Asimov%E2%80%99s+%E2%80%9Cthree+laws+of+robotics%E2%80%9D+and+machine+metaethics%2E+Ai+%26+Society%2C+22%284%29%3A477%E2%80%93493%2C+2008%2E)
5. Alex Andonian, Quentin Anthony, Stella Biderman, Sid Black, Preetham Gali, Leo Gao, Eric Hallahan, Josh Levy-Kramer, Connor Leahy, Lucas Nestler, Kip Parker, Michael Pieler, Shivanshu Purohit, Tri Songz, Phil Wang, and Samuel Weinbach. GPT-NeoX: Large scale autoregressive language modeling in pytorc… [find](https://scholar.google.com/scholar?q=Alex+Andonian%2C+Quentin+Anthony%2C+Stella+Biderman%2C+Sid+Black%2C+Preetham+Gali%2C+Leo+Gao%2C+Eric+Hallahan%2C+Josh+Levy%2DKramer%2C+Connor+Leahy%2C+Lucas+Nestler%2C+Kip+Parker%2C+Michael+Pieler%2C+Shivanshu+Purohit%2C+Tri+Son)
6. Edmond Awad, Sohan Dsouza, Richard Kim, Jonathan Schulz, Joseph Henrich, Azim Shariff, Jean-François Bonnefon, and Iyad Rahwan. The Moral Machine experiment. Nature, 2018. [find](https://scholar.google.com/scholar?q=Edmond+Awad%2C+Sohan+Dsouza%2C+Richard+Kim%2C+Jonathan+Schulz%2C+Joseph+Henrich%2C+Azim+Shariff%2C+Jean%2DFran%C3%A7ois+Bonnefon%2C+and+Iyad+Rahwan%2E+The+Moral+Machine+experiment%2E+Nature%2C+2018%2E)
7. Edmond Awad, Sydney Levine, Michael Anderson, Susan Leigh Anderson, Vincent Conitzer, M.J. Crockett, Jim A.C. Everett, Theodoros Evgeniou, Alison Gopnik, Julian C. Jamison, Tae Wan Kim, S. Matthew Liao, Michelle N. Meyer, John Mikhail, Kweku Opoku-Agyemang, Jana Schaich Borg, Juliana Schroeder, Walt… — [doi:10.1016/j.tics.2022.02.009](https://doi.org/10.1016/j.tics.2022.02.009) [find](https://scholar.google.com/scholar?q=Edmond+Awad%2C+Sydney+Levine%2C+Michael+Anderson%2C+Susan+Leigh+Anderson%2C+Vincent+Conitzer%2C+M%2EJ%2E+Crockett%2C+Jim+A%2EC%2E+Everett%2C+Theodoros+Evgeniou%2C+Alison+Gopnik%2C+Julian+C%2E+Jamison%2C+Tae+Wan+Kim%2C+S%2E+Matthew+Lia)
8. Yejin Bang, Nayeon Lee, Tiezheng Yu, Leila Khalatbari, Yan Xu, Dan Su, Elham J. Barezi, Andrea Madotto, Hayden Kee, and Pascale Fung. Aisocrates: Towards answering ethical quandary questions, 2022. URL https://arxiv.org/abs/2205.05989. — [arXiv:2205.05989](https://arxiv.org/abs/2205.05989) [↗](https://arxiv.org/abs/2205.05989)
9. Solon Barocas, Kate Crawford, Aaron Shapiro, and Hanna Wallach. The problem with bias: Allocative versus representational harms in machine learning. In SIGCIS, 2017. URL [http://meetings.sigcis.org/uploads/6/3/6/8/6368912/program.pdf](http://meetings.sigcis.org/uploads/6/3/6/8/6368912/program.pdf). [↗](http://meetings.sigcis.org/uploads/6/3/6/8/6368912/program.pdf)
10. Emily M. Bender, Timnit Gebru, Angelina McMillan-Major, and Shmargaret Shmitchell. On the dangers of stochastic parrots: Can language models be too big? In Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency, FAccT ’21, pp. 610–623, New York, NY, USA, 2021. Associati… — [doi:10.1145/34](https://doi.org/10.1145/34) [find](https://scholar.google.com/scholar?q=Emily+M%2E+Bender%2C+Timnit+Gebru%2C+Angelina+McMillan%2DMajor%2C+and+Shmargaret+Shmitchell%2E+On+the+dangers+of+stochastic+parrots%3A+Can+language+models+be+too+big%3F+In+Proceedings+of+the+2021+ACM+Conference+on+Fa)
11. Ruha Benjamin. Race After Technology: Abolitionist Tools for the New Jim Code. John Wiley & Sons, 2019. [find](https://scholar.google.com/scholar?q=Ruha+Benjamin%2E+Race+After+Technology%3A+Abolitionist+Tools+for+the+New+Jim+Code%2E+John+Wiley+%26+Sons%2C+2019%2E)
12. Fiona Berreby, Gauvain Bourgne, and Jean-Gabriel Ganascia. Modelling moral reasoning and ethical responsibility with logic programming. In Logic for programming, artificial intelligence, and reasoning, pp. 532–548. Springer, 2015. [find](https://scholar.google.com/scholar?q=Fiona+Berreby%2C+Gauvain+Bourgne%2C+and+Jean%2DGabriel+Ganascia%2E+Modelling+moral+reasoning+and+ethical+responsibility+with+logic+programming%2E+In+Logic+for+programming%2C+artificial+intelligence%2C+and+reasoning)
13. Chandra Bhagavatula, Ronan Le Bras, Chaitanya Malaviya, Keisuke Sakaguchi, Ari Holtzman, Hannah Rashkin, Doug Downey, Wen tau Yih, and Yejin Choi. [Abductive commonsense reasoning](https://md.research.elamur.ai/free/en/papers/bhagavatula-2019-abductive-commonsense-reasoning/). In International Conference on Learning Representations, 2020. URL [https://openreview.net/forum?id=Byg1v1HKDB](https://openreview.net/forum?id=Byg1v1HKDB). [↗](https://openreview.net/forum?id=Byg1v1HKDB)
14. Christina Bicchieri. Norms in the Wild, How to Diagnose, Measure and Change Social Norms. Oxford University Press, 2016. [find](https://scholar.google.com/scholar?q=Christina+Bicchieri%2E+Norms+in+the+Wild%2C+How+to+Diagnose%2C+Measure+and+Change+Social+Norms%2E+Oxford+University+Press%2C+2016%2E)
15. Yochanan E. Bigman and Kurt Gray. People are averse to machines making moral decisions. Cognition, 181:21–34, 2018. ISSN 0010-0277. doi: https://doi.org/10.1016/j.cognition.2018.08.003. URL https://www.sciencedirect.com/science/article/pii/S001002771 8302087. — [doi:10.1016/j.cognition.2018.08.003](https://doi.org/10.1016/j.cognition.2018.08.003) [↗](https://doi.org/10.1016/j.cognition.2018.08.003)
16. Yonatan Bisk, Rowan Zellers, Ronan Le Bras, Jianfeng Gao, and Yejin Choi. [Piqa: Reasoning about physical commonsense in natural language](https://md.research.elamur.ai/free/en/papers/bisk-2019-piqa-reasoning-about-physical-commonsense/). In Thirty-Fourth AAAI Conference on Artificial Intelligence, 2020.
17. Su Lin Blodgett, Solon Barocas, Hal Daumé III, and Hanna Wallach. Language (technology) is power: A critical survey of “bias” in NLP. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pp. 5454[–5476, Online, July 2020](https://md.research.elamur.ai/free/en/papers/blodgett-2020-language-technology-is-power-critical/). Association for Computational Linguistic…
18. Nicholas Botzer, Shawn Gu, and Tim Weninger. Analysis of moral judgement on reddit, 2021. [find](https://scholar.google.com/scholar?q=Nicholas+Botzer%2C+Shawn+Gu%2C+and+Tim+Weninger%2E+Analysis+of+moral+judgement+on+reddit%2C+2021%2E)
19. Richard Boyd. Finite beings, finite goods: The semantics, metaphysics and ethics of naturalist consequentialism, part i. Philosophy and Phenomenological Research, 66(3):505–553, 2003. doi: 10.1111/j.1933-1592.2003.tb00278.x. — [doi:10.1111/j.1933-1592.2003.tb00278.x](https://doi.org/10.1111/j.1933-1592.2003.tb00278.x) [find](https://scholar.google.com/scholar?q=Richard+Boyd%2E+Finite+beings%2C+finite+goods%3A+The+semantics%2C+metaphysics+and+ethics+of+naturalist+consequentialism%2C+part+i%2E+Philosophy+and+Phenomenological+Research%2C+66%283%29%3A505%E2%80%93553%2C+2003%2E+doi%3A+10%2E1111%2Fj%2E1)
20. Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel Ziegler, Jeffrey Wu, Clemens Winter, Chri… [find](https://scholar.google.com/scholar?q=Tom+Brown%2C+Benjamin+Mann%2C+Nick+Ryder%2C+Melanie+Subbiah%2C+Jared+D+Kaplan%2C+Prafulla+Dhariwal%2C+Arvind+Neelakantan%2C+Pranav+Shyam%2C+Girish+Sastry%2C+Amanda+Askell%2C+Sandhini+Agarwal%2C+Ariel+Herbert%2DVoss%2C+Gretchen)
21. Curran Associates, Inc., 2020. URL [https://proceedings.neurips.cc/paper/2020/file/1457c0d6bfcb4967418bfb8ac142f64a-Pape](https://proceedings.neurips.cc/paper/2020/file/1457c0d6bfcb4967418bfb8ac142f64a-Pape) r.pdf. [↗](https://proceedings.neurips.cc/paper/2020/file/1457c0d6bfcb4967418bfb8ac142f64a-Pape)
22. Miles Brundage, Shahar Avin, Jack Clark, Helen Toner, Peter Eckersley, Ben Garfinkel, Allan Dafoe, Paul Scharre, Thomas Zeitzoff, Bobby Filar, Hyrum Anderson, Heather Roff, Gregory C. Allen, Jacob Steinhardt, Carrick Flynn, Seán Ó hÉigeartaigh, Simon Beard, Haydn Belfield, Sebastian Farquhar, Clare… [find](https://scholar.google.com/scholar?q=Miles+Brundage%2C+Shahar+Avin%2C+Jack+Clark%2C+Helen+Toner%2C+Peter+Eckersley%2C+Ben+Garfinkel%2C+Allan+Dafoe%2C+Paul+Scharre%2C+Thomas+Zeitzoff%2C+Bobby+Filar%2C+Hyrum+Anderson%2C+Heather+Roff%2C+Gregory+C%2E+Allen%2C+Jacob+Ste)
23. Nicholas J. Bryan, Gautham J. Mysore, and Ge Wang. Isse: An interactive source separation editor. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI ’14, pp. 257–266, New York, NY, USA, 2014. Association for Computing Machinery. ISBN 9781450324731. doi: 10.1145/255628… — [doi:10.1145/2556288.2557253](https://doi.org/10.1145/2556288.2557253) [find](https://scholar.google.com/scholar?q=Nicholas+J%2E+Bryan%2C+Gautham+J%2E+Mysore%2C+and+Ge+Wang%2E+Isse%3A+An+interactive+source+separation+editor%2E+In+Proceedings+of+the+SIGCHI+Conference+on+Human+Factors+in+Computing+Systems%2C+CHI+%E2%80%9914%2C+pp%2E+257%E2%80%93266%2C+N)
24. Boxi Cao, Hongyu Lin, Xianpei Han, Fangchao Liu, and Le Sun. Can prompt probe pretrained language models? understanding the invisible risks from a causal view. In ACL, March 2022. URL http://arxiv.org/abs/2203.12258. [↗](http://arxiv.org/abs/2203.12258)
25. Dallas Card and Noah A. Smith. On consequentialism and fairness. Frontiers in Artificial Intelligence, 3:34, 2020. ISSN 2624-8212. doi: 10.3389/frai.2020.00034. URL [https://www.frontiersin.org/article/10.3389/frai.2020.00034](https://www.frontiersin.org/article/10.3389/frai.2020.00034). — [doi:10.3389/frai.2020.00034](https://doi.org/10.3389/frai.2020.00034) [↗](https://www.frontiersin.org/article/10.3389/frai.2020.00034)
26. Tuhin Chakrabarty, Yejin Choi, and Vered Shwartz. It’s not rocket science : Interpreting figurative language in narratives. TACL, 2022. — [It's not Rocket Science: Interpreting Figurative Language in Narratives](https://md.research.elamur.ai/free/en/papers/chakrabarty-2022-it-s-not-rocket-science/)
27. China AI Report. China AI report 2020, 2020. URL [http://www.cioall.com/uploads/f](http://www.cioall.com/uploads/f) 2021020114221175046.pdf. [↗](http://www.cioall.com/uploads/f)
28. Brian Christian. The Alignment Problem: Machine Learning and Human Values. W.W. Norton, 2020. [find](https://scholar.google.com/scholar?q=Brian+Christian%2E+The+Alignment+Problem%3A+Machine+Learning+and+Human+Values%2E+W%2EW%2E+Norton%2C+2020%2E)
29. Jennifer Chubb, Sondess Missaoui, Shauna Concannon, Liam Maloney, and James Alfred Walker. Interactive storytelling for children: A case-study of design and development considerations for ethical conversational ai, 2021. [find](https://scholar.google.com/scholar?q=Jennifer+Chubb%2C+Sondess+Missaoui%2C+Shauna+Concannon%2C+Liam+Maloney%2C+and+James+Alfred+Walker%2E+Interactive+storytelling+for+children%3A+A+case%2Dstudy+of+design+and+development+considerations+for+ethical+conv)
30. Mark Coeckelbergh. AI Ethics. The MIT Press, 2020. [find](https://scholar.google.com/scholar?q=Mark+Coeckelbergh%2E+AI+Ethics%2E+The+MIT+Press%2C+2020%2E)
31. European Commission. In Proposal for a regulation of the european parliament and of the council laying down harmonised rules on artificial intelligence (artificial intelligence act) and amending certain union legislative acts, 2021. [find](https://scholar.google.com/scholar?q=European+Commission%2E+In+Proposal+for+a+regulation+of+the+european+parliament+and+of+the+council+laying+down+harmonised+rules+on+artificial+intelligence+%28artificial+intelligence+act%29+and+amending+certa)
32. Florian Cova, Brent Strickland, Angela Gaia Felicita Abatista, Aurélien Allard, James Andow, Mario Attie, James R. Beebe, Renatas Berniūnas, Jordane Boudesseul, Matteo Colombo, Fiery Andrews Cushman, Rodrigo Díaz, Noah N’Djaye Nikolai van Dongen, Vilius Dranseika, Brian D. Earp, Antonio Gaitán Torr… [find](https://scholar.google.com/scholar?q=Florian+Cova%2C+Brent+Strickland%2C+Angela+Gaia+Felicita+Abatista%2C+Aur%C3%A9lien+Allard%2C+James+Andow%2C+Mario+Attie%2C+James+R%2E+Beebe%2C+Renatas+Berniu%CC%84nas%2C+Jordane+Boudesseul%2C+Matteo+Colombo%2C+Fiery+Andrews+Cushman%2C)
33. Kate Crawford. Atlas of AI. Yale University Press, March 2021. URL [https://www.degruy](https://www.degruy) ter.com/document/doi/10.12987/9780300252392/html. — [doi:10.12987/9780300252392/html](https://doi.org/10.12987/9780300252392/html) [↗](https://www.degruy)
34. Cultural Atlas. Indian culture etiquette, 2022a. URL [https://culturalatlas.sbs.com.au/indian-culture/indian-culture-etiquette](https://culturalatlas.sbs.com.au/indian-culture/indian-culture-etiquette). [↗](https://culturalatlas.sbs.com.au/indian-culture/indian-culture-etiquette)
35. Cultural Atlas. Sri lankan culture etiquette, 2022b. URL [https://culturalatlas.sbs.com.au/sri-lankan-culture/sri-lankan-culture-etiquette](https://culturalatlas.sbs.com.au/sri-lankan-culture/sri-lankan-culture-etiquette). [↗](https://culturalatlas.sbs.com.au/sri-lankan-culture/sri-lankan-culture-etiquette)
36. Norman Daniels. Wide reflective equilibrium and theory acceptance in ethics. The Journal of Philosophy, 76(5):256–282, 1979. ISSN 0022362X. URL [http://www.jstor.org/stab](http://www.jstor.org/stab) le/2025881. [↗](http://www.jstor.org/stab)
37. David Dobolyi. Moral foundation theory, 2021. URL [https://moralfoundations.org](https://moralfoundations.org). [↗](https://moralfoundations.org)
38. Lucas Dixon, John Li, Jeffrey Sorensen, Nithum Thain, and Lucy Vasserman. Measuring and mitigating unintended bias in text classification. In Proceedings of the 2018 AAAI/ACM Conference on AI, Ethics, and Society, AIES ’18, pp. 67–73, New York, NY, USA, December 2018. Association for Computing Machi… [find](https://scholar.google.com/scholar?q=Lucas+Dixon%2C+John+Li%2C+Jeffrey+Sorensen%2C+Nithum+Thain%2C+and+Lucy+Vasserman%2E+Measuring+and+mitigating+unintended+bias+in+text+classification%2E+In+Proceedings+of+the+2018+AAAI%2FACM+Conference+on+AI%2C+Ethics%2C)
39. Jesse Dodge, Maarten Sap, Ana Marasović, William Agnew, Gabriel Ilharco, Dirk Groeneveld, Margaret Mitchell, and Matt Gardner. Documenting large webtext corpora: A case study on the colossal clean crawled corpus. In EMNLP, 2021. [find](https://scholar.google.com/scholar?q=Jesse+Dodge%2C+Maarten+Sap%2C+Ana+Marasovic%CC%81%2C+William+Agnew%2C+Gabriel+Ilharco%2C+Dirk+Groeneveld%2C+Margaret+Mitchell%2C+and+Matt+Gardner%2E+Documenting+large+webtext+corpora%3A+A+case+study+on+the+colossal+clean+cr)
40. Serena Does, Belle Derks, and Naomi Ellemers. Thou shalt not discriminate: How emphasizing moral ideals rather than obligations increases whites’ support for social equality. Journal of Experimental Social Psychology, 47(3):562–571, 2011. [find](https://scholar.google.com/scholar?q=Serena+Does%2C+Belle+Derks%2C+and+Naomi+Ellemers%2E+Thou+shalt+not+discriminate%3A+How+emphasizing+moral+ideals+rather+than+obligations+increases+whites%E2%80%99+support+for+social+equality%2E+Journal+of+Experimental+S)
41. Mai ElSherief, Caleb Ziems, David Muchlinski, Vaishnavi Anupindi, Jordyn Seybolt, Munmun De Choudhury, and Diyi Yang. [Latent hatred: A benchmark for understanding implicit hate speech](https://md.research.elamur.ai/free/en/papers/elsherief-2021-latent-hatred-benchmark-understanding-implicit/). In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pp. 345–363, Online and…
42. Denis Emelin, Ronan Le Bras, Jena D. Hwang, Maxwell Forbes, and Yejin Choi. Moral stories: Situated reasoning about norms, intents, actions, and their consequences. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pp. 698–718, Online and Punta Cana, Dominica… [find](https://scholar.google.com/scholar?q=Denis+Emelin%2C+Ronan+Le+Bras%2C+Jena+D%2E+Hwang%2C+Maxwell+Forbes%2C+and+Yejin+Choi%2E+Moral+stories%3A+Situated+reasoning+about+norms%2C+intents%2C+actions%2C+and+their+consequences%2E+In+Proceedings+of+the+2021+Conferen)
43. Oren Etzioni. Point: Should ai technology be regulated? yes, and here’s how. Commun. ACM, 61(12):30–32, November 2018. ISSN 0001-0782. doi: 10.1145/3197382. URL https://doi.org/10.1145/3197382. [↗](https://doi.org/10.1145/3197382)
44. European Commission. Ethics guidelines for trustworthy artificial intelligence, 2019. URL [https://digital-strategy.ec.europa.eu/en/library/ethics-guidelines-tru](https://digital-strategy.ec.europa.eu/en/library/ethics-guidelines-tru) stworthy-ai. [↗](https://digital-strategy.ec.europa.eu/en/library/ethics-guidelines-tru)
45. Maxwell Forbes, Jena D Hwang, Vered Shwartz, Maarten Sap, and Yejin Choi. Social chemistry 101: Learning to reason about social and moral norms. In EMNLP, 2020. URL https://ww w.aclweb.org/anthology/2020.emnlp-main.48. [find](https://scholar.google.com/scholar?q=Maxwell+Forbes%2C+Jena+D+Hwang%2C+Vered+Shwartz%2C+Maarten+Sap%2C+and+Yejin+Choi%2E+Social+chemistry+101%3A+Learning+to+reason+about+social+and+moral+norms%2E+In+EMNLP%2C+2020%2E+URL+https%3A%2F%2Fww+w%2Eaclweb%2Eorg%2Fanthology%2F2)
46. Kathleen C. Fraser, Svetlana Kiritchenko, and Esma Balkir. Does moral code have a moral code? probing delphi’s moral philosophy. 2022. [find](https://scholar.google.com/scholar?q=Kathleen+C%2E+Fraser%2C+Svetlana+Kiritchenko%2C+and+Esma+Balkir%2E+Does+moral+code+have+a+moral+code%3F+probing+delphi%E2%80%99s+moral+philosophy%2E+2022%2E)
47. Sam Gehman, Suchin Gururangan, Maarten Sap, Yejin Choi, and Noah A Smith. [Realtoxicityprompts: Evaluating neural toxic degeneration in language models](https://md.research.elamur.ai/free/en/papers/gehman-2020-realtoxicityprompts-evaluating-neural-toxic-degeneration/). In Findings of EMNLP, 2020. URL [https://www.aclweb.org/anthology/2020.findings-emnlp.301/](https://www.aclweb.org/anthology/2020.findings-emnlp.301/). [↗](https://www.aclweb.org/anthology/2020.findings-emnlp.301/)
48. Barbara J. Grosz and Candace L. Sidner. [Attention, intentions, and the structure of discourse](https://md.research.elamur.ai/free/en/papers/grosz-1986-attention-intentions-structure-discourse/). Comput. Linguist., 12(3):175–204, jul 1986. ISSN 0891-2017.
49. John Haugeland. Artificial Intelligence: The Very Idea. Cambridge: MIT Press, 1985. [find](https://scholar.google.com/scholar?q=John+Haugeland%2E+Artificial+Intelligence%3A+The+Very+Idea%2E+Cambridge%3A+MIT+Press%2C+1985%2E)
50. Marc Hauser, Fiery Cushman, Liane Young, J. I. N. Kang-Xing, and John Mikhail. A dissociation between moral judgments and justifications. Mind and Language, 22(1):1–21, 2007. doi: 10.111 1/j.1468-0017.2006.00297.x. [find](https://scholar.google.com/scholar?q=Marc+Hauser%2C+Fiery+Cushman%2C+Liane+Young%2C+J%2E+I%2E+N%2E+Kang%2DXing%2C+and+John+Mikhail%2E+A+dissociation+between+moral+judgments+and+justifications%2E+Mind+and+Language%2C+22%281%29%3A1%E2%80%9321%2C+2007%2E+doi%3A+10%2E111+1%2Fj%2E1468%2D0017)
51. Dan Hendrycks, Collin Burns, Steven Basart, Andrew Critch, Jerry Li, Dawn Song, and Jacob Steinhardt. [Aligning AI with shared human values](https://md.research.elamur.ai/free/en/papers/hendrycks-2020-aligning-ai-shared-human-values/). In International Conference on Learning Representations, 2021a. URL [https://openreview.net/forum?id=dNy_RKzJacY](https://openreview.net/forum?id=dNy_RKzJacY). [↗](https://openreview.net/forum?id=dNy_RKzJacY)
52. Dan Hendrycks, Mantas Mazeika, Andy Zou, Sahil Patel, Christine Zhu, Jesus Navarro, Dawn Song, Bo Li, and Jacob Steinhardt. What would jiminy cricket do? towards agents that behave morally. In Thirty-fifth Conference on Neural Information Processing Systems Datasets and Benchmarks Track (Round 2), 2… [find](https://scholar.google.com/scholar?q=Dan+Hendrycks%2C+Mantas+Mazeika%2C+Andy+Zou%2C+Sahil+Patel%2C+Christine+Zhu%2C+Jesus+Navarro%2C+Dawn+Song%2C+Bo+Li%2C+and+Jacob+Steinhardt%2E+What+would+jiminy+cricket+do%3F+towards+agents+that+behave+morally%2E+In+Thirty%2D)
53. Joseph Hoover, Mohammad Atari, Aida Mostafazadeh Davani, Brendan Kennedy, Gwenyth Portillo-Wightman, Leigh Yeh, Drew Kogon, and Morteza Dehghani. Bound in hatred: The role of group-based morality in acts of hate. 2019. [find](https://scholar.google.com/scholar?q=Joseph+Hoover%2C+Mohammad+Atari%2C+Aida+Mostafazadeh+Davani%2C+Brendan+Kennedy%2C+Gwenyth+Portillo%2DWightman%2C+Leigh+Yeh%2C+Drew+Kogon%2C+and+Morteza+Dehghani%2E+Bound+in+hatred%3A+The+role+of+group%2Dbased+morality+in+a)
54. Lifu Huang, Ronan Le Bras, Chandra Bhagavatula, and Yejin Choi. [Cosmos qa: Machine reading comprehension with contextual commonsense reasoning](https://md.research.elamur.ai/free/en/papers/huang-2019-cosmos-qa-machine-reading-comprehension/). In EMNLP/IJCNLP, 2019.
55. Jialun Aaron Jiang, Morgan Klaus Scheuerman, Casey Fiesler, and Jed R Brubaker. Understanding international perceptions of the severity of harmful content online. PloS one, 16(8), 2021a. [find](https://scholar.google.com/scholar?q=Jialun+Aaron+Jiang%2C+Morgan+Klaus+Scheuerman%2C+Casey+Fiesler%2C+and+Jed+R+Brubaker%2E+Understanding+international+perceptions+of+the+severity+of+harmful+content+online%2E+PloS+one%2C+16%288%29%2C+2021a%2E)
56. Liwei Jiang, Jena D Hwang, Chandra Bhagavatula, Ronan Le Bras, Maxwell Forbes, Jon Borchardt, Jenny Liang, Oren Etzioni, Maarten Sap, and Yejin Choi. Delphi: Towards machine ethics and norms. arXiv preprint arXiv:2110.07574, 2021b. — [arXiv:2110.07574](https://arxiv.org/abs/2110.07574) [find](https://scholar.google.com/scholar?q=Liwei+Jiang%2C+Jena+D+Hwang%2C+Chandra+Bhagavatula%2C+Ronan+Le+Bras%2C+Maxwell+Forbes%2C+Jon+Borchardt%2C+Jenny+Liang%2C+Oren+Etzioni%2C+Maarten+Sap%2C+and+Yejin+Choi%2E+Delphi%3A+Towards+machine+ethics+and+norms%2E+arXiv+pr)
57. Immanuel Kant. Groundwork for the Metaphysics of Morals. Yale University Press, 1785/2002. [find](https://scholar.google.com/scholar?q=Immanuel+Kant%2E+Groundwork+for+the+Metaphysics+of+Morals%2E+Yale+University+Press%2C+1785%2F2002%2E)
58. Hyunwoo Kim, Youngjae Yu, Liwei Jiang, Ximing Lu, Daniel Khashabi, Gunhee Kim, Yejin Choi, and Maarten Sap. [Prosocialdialog: A prosocial backbone for conversational agents](https://md.research.elamur.ai/free/en/papers/kim-2022-prosocialdialog-prosocial-backbone-conversational-agents/), 2022. URL https://arxiv.org/abs/2205.12688. [↗](https://arxiv.org/abs/2205.12688)
59. Richard Kim, Max Kleiman-Weiner, Andres Abeliuk, Edmond Awad, Sohan Dsouza, Joshua Tenenbaum, and Iyad Rahwan. A computational model of commonsense moral decision making. pp. 197–203, 12 2018. doi: 10.1145/3278721.3278770. — [doi:10.1145/3278721.3278770](https://doi.org/10.1145/3278721.3278770) [find](https://scholar.google.com/scholar?q=Richard+Kim%2C+Max+Kleiman%2DWeiner%2C+Andres+Abeliuk%2C+Edmond+Awad%2C+Sohan+Dsouza%2C+Joshua+Tenenbaum%2C+and+Iyad+Rahwan%2E+A+computational+model+of+commonsense+moral+decision+making%2E+pp%2E+197%E2%80%93203%2C+12+2018%2E+doi%3A+10)
60. Will Knight. This program can give AI a sense of Ethics—Sometimes. Wired, October 2021. URL [https://www.wired.com/story/program-give-ai-ethics-sometimes/](https://www.wired.com/story/program-give-ai-ethics-sometimes/). [↗](https://www.wired.com/story/program-give-ai-ethics-sometimes/)
61. Joshua Knobe. Philosophical intuitions are surprisingly stable across both demographic groups and situations. Filozofia Nauki, 2021. [find](https://scholar.google.com/scholar?q=Joshua+Knobe%2E+Philosophical+intuitions+are+surprisingly+stable+across+both+demographic+groups+and+situations%2E+Filozofia+Nauki%2C+2021%2E)
62. Christine M. Korsgaard. The Sources of Normativity. Cambridge University Press, 1996. [find](https://scholar.google.com/scholar?q=Christine+M%2E+Korsgaard%2E+The+Sources+of+Normativity%2E+Cambridge+University+Press%2C+1996%2E)
63. Kobi Leins, Jey Han Lau, and Timothy Baldwin. Give me convenience and give her death: Who should decide what uses of NLP are appropriate, and on what basis? In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pp. 2908–2913, Online, July 2020. Association for C… [find](https://scholar.google.com/scholar?q=Kobi+Leins%2C+Jey+Han+Lau%2C+and+Timothy+Baldwin%2E+Give+me+convenience+and+give+her+death%3A+Who+should+decide+what+uses+of+NLP+are+appropriate%2C+and+on+what+basis%3F+In+Proceedings+of+the+58th+Annual+Meeting+o)
64. S. Matthew Liao. Ethics of Artificial Intelligence. Oxford University Press, 2020. [find](https://scholar.google.com/scholar?q=S%2E+Matthew+Liao%2E+Ethics+of+Artificial+Intelligence%2E+Oxford+University+Press%2C+2020%2E)
65. Nicholas Lourie, Ronan Le Bras, Chandra Bhagavatula, and Yejin Choi. Unicorn on rainbow: A universal commonsense reasoning model on a new multitask benchmark. In AAAI, 2021a. [find](https://scholar.google.com/scholar?q=Nicholas+Lourie%2C+Ronan+Le+Bras%2C+Chandra+Bhagavatula%2C+and+Yejin+Choi%2E+Unicorn+on+rainbow%3A+A+universal+commonsense+reasoning+model+on+a+new+multitask+benchmark%2E+In+AAAI%2C+2021a%2E)
66. Nicholas Lourie, Ronan Le Bras, and Yejin Choi. Scruples: A corpus of community ethical judgments on 32, 000 real-life anecdotes. In AAAI, 2021b. [find](https://scholar.google.com/scholar?q=Nicholas+Lourie%2C+Ronan+Le+Bras%2C+and+Yejin+Choi%2E+Scruples%3A+A+corpus+of+community+ethical+judgments+on+32%2C+000+real%2Dlife+anecdotes%2E+In+AAAI%2C+2021b%2E)
67. Li Lucy and David Bamman. [Gender and representation bias in gpt-3 generated stories](https://md.research.elamur.ai/free/en/papers/lucy-2021-gender-representation-bias-gpt-3/). In Proceedings of the Third Workshop on Narrative Understanding, pp. 48–55, 2021.
68. Steven Lukes. Moral relativism. Picador, 2008. [find](https://scholar.google.com/scholar?q=Steven+Lukes%2E+Moral+relativism%2E+Picador%2C+2008%2E)
69. John Leslie Mackie. Ethics: Inventing Right and Wrong. Penguin Books, 1977. [find](https://scholar.google.com/scholar?q=John+Leslie+Mackie%2E+Ethics%3A+Inventing+Right+and+Wrong%2E+Penguin+Books%2C+1977%2E)
70. Nikolay Malkin, Sameera Lanka, Pranav Goel, Sudha Rao, and Nebojsa Jojic. GPT perdetry test: Generating new meanings for new words. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. Association for Compu… [find](https://scholar.google.com/scholar?q=Nikolay+Malkin%2C+Sameera+Lanka%2C+Pranav+Goel%2C+Sudha+Rao%2C+and+Nebojsa+Jojic%2E+GPT+perdetry+test%3A+Generating+new+meanings+for+new+words%2E+In+Proceedings+of+the+2021+Conference+of+the+North+American+Chapter+)
71. Gary Marcus and Ernest Davis. In Rebooting AI: Building Artificial Intelligence We Can Trust, 2019. [find](https://scholar.google.com/scholar?q=Gary+Marcus+and+Ernest+Davis%2E+In+Rebooting+AI%3A+Building+Artificial+Intelligence+We+Can+Trust%2C+2019%2E)
72. Cade Metz. Self-driving cars will teach themselves to save lives—but also take them \| wired. [https://www.wired.com/2016/06/self-driving-cars-will-power-kill-wont](https://www.wired.com/2016/06/self-driving-cars-will-power-kill-wont) -conscience/, 09 2016. [↗](https://www.wired.com/2016/06/self-driving-cars-will-power-kill-wont)
73. Cade Metz. Can a machine learn morality? The New York Times, November 2021. URL [https://www.nytimes.com/2021/11/19/technology/can-a-machine-learn-mora](https://www.nytimes.com/2021/11/19/technology/can-a-machine-learn-mora) lity.html. [↗](https://www.nytimes.com/2021/11/19/technology/can-a-machine-learn-mora)
74. John Mikhail. Universal moral grammar: theory, evidence and the future. Trends in Cognitive Sciences, 11(4):143–152, 2007. ISSN 1364-6613. doi: https://doi.org/10.1016/j.tics.2006.12.007. URL https://www.sciencedirect.com/science/article/pii/S136466130 7000496. — [doi:10.1016/j.tics.2006.12.007](https://doi.org/10.1016/j.tics.2006.12.007) [↗](https://doi.org/10.1016/j.tics.2006.12.007)
75. Margaret Mitchell, Simone Wu, Andrew Zaldivar, Parker Barnes, Lucy Vasserman, Ben Hutchinson, Elena Spitzer, Inioluwa Deborah Raji, and Timnit Gebru. Model cards for model reporting. In Proceedings of the conference on fairness, accountability, and transparency, pp. 220–229, 2019. [find](https://scholar.google.com/scholar?q=Margaret+Mitchell%2C+Simone+Wu%2C+Andrew+Zaldivar%2C+Parker+Barnes%2C+Lucy+Vasserman%2C+Ben+Hutchinson%2C+Elena+Spitzer%2C+Inioluwa+Deborah+Raji%2C+and+Timnit+Gebru%2E+Model+cards+for+model+reporting%2E+In+Proceedings+of)
76. James Moor. The nature, importance, and difficulty of machine ethics. IEEE Intelligent Systems, 21:18–21, 08 2006. doi: 10.1109/MIS.2006.80. — [doi:10.1109/mis.2006.80](https://doi.org/10.1109/mis.2006.80) [find](https://scholar.google.com/scholar?q=James+Moor%2E+The+nature%2C+importance%2C+and+difficulty+of+machine+ethics%2E+IEEE+Intelligent+Systems%2C+21%3A18%E2%80%9321%2C+08+2006%2E+doi%3A+10%2E1109%2FMIS%2E2006%2E80%2E+%E2%80%94+%5Bdoi%3A10%2E1109%2Fmis%2E2006%2E80%5D%28https%3A%2F%2Fdoi%2Eorg%2F10%2E1109%2Fmis%2E200)
77. Nasrin Mostafazadeh, Nathanael Chambers, Xiaodong He, Devi Parikh, Dhruv Batra, Lucy Vanderwende, Pushmeet Kohli, and James F. Allen. A corpus and evaluation framework for deeper understanding of commonsense stories. CoRR, abs/1604.01696, 2016. URL http://arxiv.org/abs/1604.01696. [↗](http://arxiv.org/abs/1604.01696)
78. Thomas Nagel. The View From Nowhere. Oxford University Press, 1986. [find](https://scholar.google.com/scholar?q=Thomas+Nagel%2E+The+View+From+Nowhere%2E+Oxford+University+Press%2C+1986%2E)
79. New York Times. Résumé-writing tips to help you get past the a.i. gatekeepers, 2021. URL [https://www.nytimes.com/2021/03/19/business/resume-filter-articial-int](https://www.nytimes.com/2021/03/19/business/resume-filter-articial-int) elligence.html. [↗](https://www.nytimes.com/2021/03/19/business/resume-filter-articial-int)
80. Tuan Dung Nguyen, Georgiana Lyall, Alasdair Tran, Minjeong Shin, Nicholas George Carroll, Colin Klein, and Lexing Xie. Mapping topics in 100,000 real-life moral dilemmas. Proceedings of the International AAAI Conference on Web and Social Media, 16(1):699–710, May 2022. URL [https://ojs.aaai.org/index…](https://ojs.aaai.org/index…) [↗](https://ojs.aaai.org/index…)
81. John T. Nockleby. Hate speech. In Encyclopedia of the American Constitution, 2000. [find](https://scholar.google.com/scholar?q=John+T%2E+Nockleby%2E+Hate+speech%2E+In+Encyclopedia+of+the+American+Constitution%2C+2000%2E)
82. Poppy Noor. ‘is it OK to ...’: the bot that gives you an instant moral judgment. The Guardian, November 2021. URL [https://www.theguardian.com/technology/2021/nov/02/delphi-online-ai-bot-philosophy](https://www.theguardian.com/technology/2021/nov/02/delphi-online-ai-bot-philosophy). [↗](https://www.theguardian.com/technology/2021/nov/02/delphi-online-ai-bot-philosophy)
83. Derek Parfit. On What Matters: Volume One. Oxford Scholarship Online, 2011. [find](https://scholar.google.com/scholar?q=Derek+Parfit%2E+On+What+Matters%3A+Volume+One%2E+Oxford+Scholarship+Online%2C+2011%2E)
84. Gonçalo Pereira, Rui Prada, and Pedro A. Santos. Integrating social power into the decision-making of cognitive agents. Artificial Intelligence, 241:1–44, 2016. ISSN 0004-3702. doi: https://doi.org/10.1016/j.artint.2016.08.003. URL https://www.sciencedirect.com/science/article/pii/S0004370216300868… — [doi:10.1016/j.artint.2016.08.003](https://doi.org/10.1016/j.artint.2016.08.003) [↗](https://doi.org/10.1016/j.artint.2016.08.003)
85. Luís Moniz Pereira and Ari Saptawijaya. Modelling morality with prospective logic. In Portuguese Conference on Artificial Intelligence, pp. 99–111. Springer, 2007. [find](https://scholar.google.com/scholar?q=Lu%C3%ADs+Moniz+Pereira+and+Ari+Saptawijaya%2E+Modelling+morality+with+prospective+logic%2E+In+Portuguese+Conference+on+Artificial+Intelligence%2C+pp%2E+99%E2%80%93111%2E+Springer%2C+2007%2E)
86. Shrimai Prabhumoye, Brendon Boldt, Ruslan Salakhutdinov, and Alan W Black. Case study: Deontological ethics in nlp, 2021. [find](https://scholar.google.com/scholar?q=Shrimai+Prabhumoye%2C+Brendon+Boldt%2C+Ruslan+Salakhutdinov%2C+and+Alan+W+Black%2E+Case+study%3A+Deontological+ethics+in+nlp%2C+2021%2E)
87. Jack W. Rae, Sebastian Borgeaud, Trevor Cai, Katie Millican, Jordan Hoffmann, Francis Song, John Aslanides, Sarah Henderson, Roman Ring, Susannah Young, Eliza Rutherford, Tom Hennigan, Jacob Menick, Albin Cassirer, Richard Powell, George van den Driessche, Lisa Anne Hendricks, Maribeth Rauh, Po-Sen… [find](https://scholar.google.com/scholar?q=Jack+W%2E+Rae%2C+Sebastian+Borgeaud%2C+Trevor+Cai%2C+Katie+Millican%2C+Jordan+Hoffmann%2C+Francis+Song%2C+John+Aslanides%2C+Sarah+Henderson%2C+Roman+Ring%2C+Susannah+Young%2C+Eliza+Rutherford%2C+Tom+Hennigan%2C+Jacob+Menick%2C+A)
88. Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. [Exploring the limits of transfer learning with a unified text-to-text transformer](https://md.research.elamur.ai/free/en/papers/raffel-2019-exploring-limits-transfer-learning-unified/). Journal of Machine Learning Research, 21(140):1–67, 2020. URL [http://jmlr.org/papers/v21/2…](http://jmlr.org/papers/v21/2…) [↗](http://jmlr.org/papers/v21/2…)
89. Peter Railton. Ethical learning, natural and artificial. In Ethics of Artificial Intelligence, 2020. [find](https://scholar.google.com/scholar?q=Peter+Railton%2E+Ethical+learning%2C+natural+and+artificial%2E+In+Ethics+of+Artificial+Intelligence%2C+2020%2E)
90. John Rawls. Outline of a decision procedure for ethics. Philosophical Review, 60(2):177–197, 1951. doi: 10.2307/2181696. — [doi:10.2307/2181696](https://doi.org/10.2307/2181696) [find](https://scholar.google.com/scholar?q=John+Rawls%2E+Outline+of+a+decision+procedure+for+ethics%2E+Philosophical+Review%2C+60%282%29%3A177%E2%80%93197%2C+1951%2E+doi%3A+10%2E2307%2F2181696%2E+%E2%80%94+%5Bdoi%3A10%2E2307%2F2181696%5D%28https%3A%2F%2Fdoi%2Eorg%2F10%2E2307%2F2181696%29)
91. John Rawls. A Theory of Justice. Belknap Press of Harvard University Press, Cambridge, Mas-sachussets, 1 edition, 1971. ISBN 0-674-88014-5. [find](https://scholar.google.com/scholar?q=John+Rawls%2E+A+Theory+of+Justice%2E+Belknap+Press+of+Harvard+University+Press%2C+Cambridge%2C+Mas%2Dsachussets%2C+1+edition%2C+1971%2E+ISBN+0%2D674%2D88014%2D5%2E)
92. Rob Reich, Mehran Sahami, and Jeremy M Weinstein. System error: Where big tech went wrong and how we can reboot. Hodder & Stoughton, 2021. [find](https://scholar.google.com/scholar?q=Rob+Reich%2C+Mehran+Sahami%2C+and+Jeremy+M+Weinstein%2E+System+error%3A+Where+big+tech+went+wrong+and+how+we+can+reboot%2E+Hodder+%26+Stoughton%2C+2021%2E)
93. Reuters. Amazon scraps secret ai recruiting tool that showed bias against women, 2018. [find](https://scholar.google.com/scholar?q=Reuters%2E+Amazon+scraps+secret+ai+recruiting+tool+that+showed+bias+against+women%2C+2018%2E)
94. Francesca Rossi. Building trust in artificial intelligence. Journal of International Affairs, 72(1): 127–134, 2018. ISSN 0022197X. URL [https://www.jstor.org/stable/26588348](https://www.jstor.org/stable/26588348). [↗](https://www.jstor.org/stable/26588348)
95. Roy Furchgott. Public streets are the lab for self-driving experiments, 2021. URL https://ww w.nytimes.com/2021/12/23/business/tesla-self-driving-regulations .html. [find](https://scholar.google.com/scholar?q=Roy+Furchgott%2E+Public+streets+are+the+lab+for+self%2Ddriving+experiments%2C+2021%2E+URL+https%3A%2F%2Fww+w%2Enytimes%2Ecom%2F2021%2F12%2F23%2Fbusiness%2Ftesla%2Dself%2Ddriving%2Dregulations+%2Ehtml%2E)
96. Rachel Rudinger, Vered Shwartz, Jena D Hwang, Chandra Bhagavatula, Maxwell Forbes, Ronan Le Bras, Noah A Smith, and Yejin Choi. Thinking like a skeptic: Defeasible inference in natural language. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: Findings, pp.… [find](https://scholar.google.com/scholar?q=Rachel+Rudinger%2C+Vered+Shwartz%2C+Jena+D+Hwang%2C+Chandra+Bhagavatula%2C+Maxwell+Forbes%2C+Ronan+Le+Bras%2C+Noah+A+Smith%2C+and+Yejin+Choi%2E+Thinking+like+a+skeptic%3A+Defeasible+inference+in+natural+language%2E+In+Pr)
97. Keisuke Sakaguchi, Ronan Le Bras, Chandra Bhagavatula, and Yejin Choi. Winogrande: An adversarial winograd schema challenge at scale. In AAAI, 2020. [find](https://scholar.google.com/scholar?q=Keisuke+Sakaguchi%2C+Ronan+Le+Bras%2C+Chandra+Bhagavatula%2C+and+Yejin+Choi%2E+Winogrande%3A+An+adversarial+winograd+schema+challenge+at+scale%2E+In+AAAI%2C+2020%2E)
98. Maarten Sap, Hannah Rashkin, Derek Chen, Ronan Le Bras, and Yejin Choi. Social iqa: Commonsense reasoning about social interactions. In EMNLP 2019, 2019. [find](https://scholar.google.com/scholar?q=Maarten+Sap%2C+Hannah+Rashkin%2C+Derek+Chen%2C+Ronan+Le+Bras%2C+and+Yejin+Choi%2E+Social+iqa%3A+Commonsense+reasoning+about+social+interactions%2E+In+EMNLP+2019%2C+2019%2E)
99. Maarten Sap, Saadia Gabriel, Lianhui Qin, Dan Jurafsky, Noah A Smith, and Yejin Choi. [Social bias frames: Reasoning about social and power implications of language](https://md.research.elamur.ai/free/en/papers/sap-2020-social-bias-frames-reasoning-about/). In ACL, 2020. URL [https://www.aclweb.org/anthology/2020.acl-main.486](https://www.aclweb.org/anthology/2020.acl-main.486). [↗](https://www.aclweb.org/anthology/2020.acl-main.486)
100. Maarten Sap, Swabha Swayamdipta, Laura Vianna, Xuhui Zhou, Yejin Choi, and Noah A. Smith. [Annotators with attitudes: How annotator beliefs and identities bias toxic language detection](https://md.research.elamur.ai/free/en/papers/sap-2022-annotators-attitudes-how-annotator-beliefs/). In NAACL, 2022. URL https://arxiv.org/abs/2111.07997. [↗](https://arxiv.org/abs/2111.07997)
101. Timo Schick and Hinrich Schütze. It’s not just size that matters: Small language models are also few-shot learners. arXiv preprint arXiv:2009.[07118, 2020](https://md.research.elamur.ai/free/en/papers/schick-2021-it-s-not-just-size/).
102. Patrick Schramowski, Cigdem Turan, Sophie Jentzsch, Constantin Rothkopf, and Kristian Kersting. The moral choice machine. Frontiers in artificial intelligence, 3:36, 2020. [find](https://scholar.google.com/scholar?q=Patrick+Schramowski%2C+Cigdem+Turan%2C+Sophie+Jentzsch%2C+Constantin+Rothkopf%2C+and+Kristian+Kersting%2E+The+moral+choice+machine%2E+Frontiers+in+artificial+intelligence%2C+3%3A36%2C+2020%2E)
103. Patrick Schramowski, Cigdem Turan, Nico Andersen, Constantin Rothkopf, and Kristian Kersting. Language models have a moral dimension, 2021. [find](https://scholar.google.com/scholar?q=Patrick+Schramowski%2C+Cigdem+Turan%2C+Nico+Andersen%2C+Constantin+Rothkopf%2C+and+Kristian+Kersting%2E+Language+models+have+a+moral+dimension%2C+2021%2E)
104. Patrick Schramowski, Cigdem Turan, Nico Andersen, Constantin Rothkopf, and Kristian Kersting. Large pre-trained language models contain human-like biases of what is right and wrong to do. Nature Machine Intelligence, 2022. [find](https://scholar.google.com/scholar?q=Patrick+Schramowski%2C+Cigdem+Turan%2C+Nico+Andersen%2C+Constantin+Rothkopf%2C+and+Kristian+Kersting%2E+Large+pre%2Dtrained+language+models+contain+human%2Dlike+biases+of+what+is+right+and+wrong+to+do%2E+Nature+Machi)
105. Eric Schwitzgebel and Mara Garza. Designing ai with rights, consciousness, self-respect, and freedom. In Ethics of Artificial Intelligence, pp. 459–479. 2020. [find](https://scholar.google.com/scholar?q=Eric+Schwitzgebel+and+Mara+Garza%2E+Designing+ai+with+rights%2C+consciousness%2C+self%2Drespect%2C+and+freedom%2E+In+Ethics+of+Artificial+Intelligence%2C+pp%2E+459%E2%80%93479%2E+2020%2E)
106. Emily Sheng, Kai-Wei Chang, Prem Natarajan, and Nanyun Peng. [The woman worked as a babysitter: On biases in language generation](https://md.research.elamur.ai/free/en/papers/sheng-2019-woman-worked-as-babysitter-biases/). In EMNLP, pp. 3407–3412, 2019.
107. Adam Smith. The Theory of Moral Sentiments. Project Gutenberg, 1759/2022. [find](https://scholar.google.com/scholar?q=Adam+Smith%2E+The+Theory+of+Moral+Sentiments%2E+Project+Gutenberg%2C+1759%2F2022%2E)
108. Sophie Pettit. To kiss or not to kiss? greeting customs around the world, 2022. URL [https://www.expatica.com/living/integration/greeting-customs-around-th](https://www.expatica.com/living/integration/greeting-customs-around-th) e-world-11731/. [↗](https://www.expatica.com/living/integration/greeting-customs-around-th)
109. Sharon Street. Coming to terms with contingency : Humean constructivism about practical reason. In Jimmy Lenman and Yonatan Shemmer (eds.), Constructivism in Practical Philosophy. Oxford University Press, 2012. [find](https://scholar.google.com/scholar?q=Sharon+Street%2E+Coming+to+terms+with+contingency+%3A+Humean+constructivism+about+practical+reason%2E+In+Jimmy+Lenman+and+Yonatan+Shemmer+%28eds%2E%29%2C+Constructivism+in+Practical+Philosophy%2E+Oxford+University+Pr)
110. Zeerak Talat, Hagen Blix, Josef Valvoda, Maya Indira Ganesh, Ryan Cotterell, and Adina Williams. A word on machine ethics: A response to jiang et al. (2021). ArXiv, abs/2111.04158, 2021. — [arXiv:2111.04158](https://arxiv.org/abs/2111.04158) [find](https://scholar.google.com/scholar?q=Zeerak+Talat%2C+Hagen+Blix%2C+Josef+Valvoda%2C+Maya+Indira+Ganesh%2C+Ryan+Cotterell%2C+and+Adina+Williams%2E+A+word+on+machine+ethics%3A+A+response+to+jiang+et+al%2E+%282021%29%2E+ArXiv%2C+abs%2F2111%2E04158%2C+2021%2E+%E2%80%94+%5BarXiv%3A2111)
111. Alon Talmor, Ori Yoran, Ronan Le Bras, Chandra Bhagavatula, Yoav Goldberg, Yejin Choi, and Jonathan Berant. [CommonsenseQA 2.0: Exposing the limits of AI through gamification](https://md.research.elamur.ai/free/en/papers/talmor-2022-commonsenseqa-2-0-exposing-limits/). In Thirty-fifth Conference on Neural Information Processing Systems Datasets and Benchmarks Track (Round 1), 2021. URL https:…
112. Dimitrios Tsarapatsanis and Nikolaos Aletras. On the ethical limits of natural language processing on legal text, 2021. [find](https://scholar.google.com/scholar?q=Dimitrios+Tsarapatsanis+and+Nikolaos+Aletras%2E+On+the+ethical+limits+of+natural+language+processing+on+legal+text%2C+2021%2E)
113. Mark Ungar. State violence and lesbian, gay, bisexual and transgender (lgbt) rights. New Political Science, 22(1):61–75, 2000. [find](https://scholar.google.com/scholar?q=Mark+Ungar%2E+State+violence+and+lesbian%2C+gay%2C+bisexual+and+transgender+%28lgbt%29+rights%2E+New+Political+Science%2C+22%281%29%3A61%E2%80%9375%2C+2000%2E)
114. United Nations. Universal declaration of human rights, 2021. URL [https://www.un.org/e](https://www.un.org/e) n/about-us/universal-declaration-of-human-rights. [↗](https://www.un.org/e)
115. Bertie Vidgen, Tristan Thrush, Zeerak Waseem, and Douwe Kiela. [Learning from the worst: Dynamically generated datasets to improve online hate detection](https://md.research.elamur.ai/free/en/papers/vidgen-2021-learning-worst-dynamically-generated-datasets/). In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural L…
116. Wendell Wallach and Colin Allen. Moral Machines: Teaching Machines Right from Wrong. Oxford University Press, 2010. [find](https://scholar.google.com/scholar?q=Wendell+Wallach+and+Colin+Allen%2E+Moral+Machines%3A+Teaching+Machines+Right+from+Wrong%2E+Oxford+University+Press%2C+2010%2E)
117. Daniel Weld and Oren Etzioni. The first law of robotics (a call to arms). In Proceedings of the Twelfth AAAI National Conference on Artificial Intelligence, AAAI’94, pp. 1042–1047. AAAI Press, 1994. [find](https://scholar.google.com/scholar?q=Daniel+Weld+and+Oren+Etzioni%2E+The+first+law+of+robotics+%28a+call+to+arms%29%2E+In+Proceedings+of+the+Twelfth+AAAI+National+Conference+on+Artificial+Intelligence%2C+AAAI%E2%80%9994%2C+pp%2E+1042%E2%80%931047%2E+AAAI+Press%2C+1994%2E)
118. White House. Big data: A report on algorithmic systems, opportunity, and civil rights, 2016. URL [https://obamawhitehouse.archives.gov/sites/default/files/microsi](https://obamawhitehouse.archives.gov/sites/default/files/microsi) tes/ostp/2016_0504_data_discrimination.pdf. [↗](https://obamawhitehouse.archives.gov/sites/default/files/microsi)
119. Thomas Wischmeyer and Timo Rademacher (eds.). Regulating Artificial Intelligence. Springer, Cham, 2020. URL [https://link.springer.com/book/10.1007/978-3-030-3](https://link.springer.com/book/10.1007/978-3-030-3) 2361-5. — [doi:10.1007/978-3-030-3](https://doi.org/10.1007/978-3-030-3) [↗](https://link.springer.com/book/10.1007/978-3-030-3)
120. David B. Wong. Natural Moralities:A Defense of Pluralistic Relativism: A Defense of Pluralistic Relativism. Oxford University Press, 2006. [find](https://scholar.google.com/scholar?q=David+B%2E+Wong%2E+Natural+Moralities%3AA+Defense+of+Pluralistic+Relativism%3A+A+Defense+of+Pluralistic+Relativism%2E+Oxford+University+Press%2C+2006%2E)
121. World Value Survey. World value survey, 2022. URL [https://www.worldvaluessurvey.org/wvs.jsp](https://www.worldvaluessurvey.org/wvs.jsp). [↗](https://www.worldvaluessurvey.org/wvs.jsp)
122. Rowan Zellers, Ari Holtzman, Yonatan Bisk, Ali Farhadi, and Yejin Choi. Hellaswag: Can a machine really finish your sentence? In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, 2019. [find](https://scholar.google.com/scholar?q=Rowan+Zellers%2C+Ari+Holtzman%2C+Yonatan+Bisk%2C+Ali+Farhadi%2C+and+Yejin+Choi%2E+Hellaswag%3A+Can+a+machine+really+finish+your+sentence%3F+In+Proceedings+of+the+57th+Annual+Meeting+of+the+Association+for+Computati)
123. Rowan Zellers, Ari Holtzman, Elizabeth Clark, Lianhui Qin, Ali Farhadi, and Yejin Choi. TuringAdvice: A generative and dynamic evaluation of language use. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies… [find](https://scholar.google.com/scholar?q=Rowan+Zellers%2C+Ari+Holtzman%2C+Elizabeth+Clark%2C+Lianhui+Qin%2C+Ali+Farhadi%2C+and+Yejin+Choi%2E+TuringAdvice%3A+A+generative+and+dynamic+evaluation+of+language+use%2E+In+Proceedings+of+the+2021+Conference+of+the+)
124. Eviatar Zerubavel. The marked and the unmarked. In Taken for Granted: The Remarkable Power of the Unremarkable. Princeton University Press, 2018. URL [http://assets.press.princ](http://assets.press.princ) eton.edu/chapters/s11226.pdf. [↗](http://assets.press.princ)
125. Susan Zhang, Stephen Roller, Naman Goyal, Mikel Artetxe, Moya Chen, Shuohui Chen, Christopher Dewan, Mona Diab, Xian Li, Xi Victoria Lin, Todor Mihaylov, Myle Ott, Sam Shleifer, Kurt Shuster, Daniel Simig, Punit Singh Koura, Anjali Sridhar, Tianlu Wang, and Luke Zettlemoyer. Opt: Open pre-trained tr… — [OPT: Open Pre-trained Transformer Language Models](https://md.research.elamur.ai/free/en/papers/zhang-2022-opt-open-pre-trained-transformer/)
126. Jieyu Zhao, Daniel Khashabi, Tushar Khot, Ashish Sabharwal, and Kai-Wei Chang. Ethical-advice taker: Do language models understand natural language interventions? In Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021, pp. 4158–4164, Online, August 2021a. Association for Compu… [find](https://scholar.google.com/scholar?q=Jieyu+Zhao%2C+Daniel+Khashabi%2C+Tushar+Khot%2C+Ashish+Sabharwal%2C+and+Kai%2DWei+Chang%2E+Ethical%2Dadvice+taker%3A+Do+language+models+understand+natural+language+interventions%3F+In+Findings+of+the+Association+for+Co)
127. Jieyu Zhao, Daniel Khashabi, Tushar Khot, Ashish Sabharwal, and Kai-Wei Chang. Ethical-advice taker: Do language models understand natural language interventions?, 2021b. [find](https://scholar.google.com/scholar?q=Jieyu+Zhao%2C+Daniel+Khashabi%2C+Tushar+Khot%2C+Ashish+Sabharwal%2C+and+Kai%2DWei+Chang%2E+Ethical%2Dadvice+taker%3A+Do+language+models+understand+natural+language+interventions%3F%2C+2021b%2E)
128. Karen Zhou, Ana Smith, and Lillian Lee. Assessing cognitive linguistic influences in the assignment of blame. In Proceedings of the Ninth International Workshop on Natural Language Processing for Social Media, pp. 61–69, Online, June 2021. Association for Computational Linguistics. doi: 10.18653/v1/… [find](https://scholar.google.com/scholar?q=Karen+Zhou%2C+Ana+Smith%2C+and+Lillian+Lee%2E+Assessing+cognitive+linguistic+influences+in+the+assignment+of+blame%2E+In+Proceedings+of+the+Ninth+International+Workshop+on+Natural+Language+Processing+for+Soci)

*Updated 2 November 2026*