---
title: "Deliberative Alignment — paper by Guan et al. (2024)"
description: "OpenAI researchers propose Deliberative Alignment, which teaches the model the actual text of the… 38 references, cited by 74 papers here."
authors: ["Melody Y. Guan", "Manas Joglekar", "Eric Wallace", "Saachi Jain", "Boaz Barak", "Alec Helyar", "Rachel Dias", "Andrea Vallone", "Hongyu Ren", "Jason Wei", "Hyung Won Chung", "Sam Toyer", "Johannes Heidecke", "Alex Beutel", "Amelia Glaese"]
year: 2024
venue: "arXiv:2412.16339"
source: "https://arxiv.org/abs/2412.16339"
doi: "10.48550/arxiv.2412.16339"
arxiv: "2412.16339"
licence: "CC BY 4.0 (arXiv)"
licence_url: "http://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/guan-2024-deliberative-alignment-reasoning-enables-safer/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/guan-2024-deliberative-alignment-reasoning-enables-safer/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2024](https://md.research.elamur.ai/free/en/papers/2024/page/3/) / Deliberative Alignment: Reasoning Enables Safer Language Models

# Deliberative Alignment: Reasoning Enables Safer Language Models

**Authors:** Melody Y. Guan, [Manas Joglekar](https://md.research.elamur.ai/free/en/authors/manas-joglekar/), [Eric Wallace](https://md.research.elamur.ai/free/en/authors/eric-wallace/), Saachi Jain, [Boaz Barak](https://md.research.elamur.ai/free/en/authors/boaz-barak/), [Alec Helyar](https://md.research.elamur.ai/free/en/authors/alec-helyar/), Rachel Dias, [Andrea Vallone](https://md.research.elamur.ai/free/en/authors/andrea-vallone/), Hongyu Ren, [Jason Wei](https://md.research.elamur.ai/free/en/authors/jason-wei/), [Hyung Won Chung](https://md.research.elamur.ai/free/en/authors/hyung-won-chung/), Sam Toyer, [Johannes Heidecke](https://md.research.elamur.ai/free/en/authors/johannes-heidecke/), [Alex Beutel](https://md.research.elamur.ai/free/en/authors/alex-beutel/) and [Amelia Glaese](https://md.research.elamur.ai/free/en/authors/amelia-glaese/) · **Year:** 2024 · **Venue:** arXiv:2412.16339 · **Licence:** [CC BY 4.0 (arXiv)](http://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://arxiv.org/abs/2412.16339) · [PDF](https://arxiv.org/pdf/2412.16339) · [DOI 10.48550/arxiv.2412.16339](https://doi.org/10.48550/arxiv.2412.16339) · [arXiv 2412.16339](https://arxiv.org/abs/2412.16339) · [OpenAlex](https://openalex.org/W4412505606)

**Citations:** 35 citations (OpenAlex, 2026-10-14) · [74](https://md.research.elamur.ai/free/en/papers/guan-2024-deliberative-alignment-reasoning-enables-safer/cited-by/) in our library, of the 141,840 papers whose bibliographies we read

## Summary
Conventional safety training (SFT, RLHF) has two weaknesses. LLMs answer instantly without deliberating, and they infer safety rules only indirectly from labeled examples. As a result, they can be jailbroken yet still refuse harmless requests. OpenAI researchers propose Deliberative Alignment, which teaches the model the actual text of the safety specifications and trains it to recall and reason over them in its chain-of-thought before answering.

The training data is entirely synthetic. A reasoning model trained only for helpfulness receives category-specific policy excerpts in its prompt. A policy-aware judge model filters the resulting (prompt, CoT, output) completions. The policies are then removed from the prompt (context distillation), and the remaining examples are used for SFT. A subsequent RL stage uses the same judge to provide reward. No human-written CoTs or answers are required.

Applied to the o-series models, o1 reduces both harmful compliance and overrefusal compared with GPT-4o and other strong LLMs. It also generalizes better to out-of-distribution cases. The approach moves alignment away from expensive human labeling toward scalable, interpretable reasoning over explicit rules.

## Key points
- **Two-stage pipeline:**
  - SFT of the base reasoning model (G_base) on judge-filtered (prompt, CoT, output) data.
  - High-compute RL in which the judge (G_RM), given the spec, supplies the reward.
  - The only label used is the broad safety category of each prompt.
- **Specification contents:**
  - Content policies covering areas that include self-harm, violence, harassment, extremist material, sexually explicit content, unlawful activities, and advice in regulated domains.
  - Each request type is classed as allowed, disallowed, or requiring a safe completion.
  - Style guidelines describe how to comply, refuse, or safe-complete.
  - The spec is partly based on OpenAI's published model spec.
- **Compact prompts:** To keep context short, spec(category) gives a high-level view of all categories but full detail only for the relevant one.
- **Evaluation:**
  - On StrongREJECT (jailbreak prompts) versus XSTest (benign prompts), o1 pushes the Pareto frontier beyond GPT-4o.
  - Error bars come from 1,000 bootstrap trials.
  - o1 saturates many of the hardest internal safety benchmarks.
- **Ablations:**
  - The SFT stage gives a strong reasoning prior, and RL refines it.
  - Building the policies into the model is more reliable than supplying them in context at deployment time.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Stanford University](https://md.research.elamur.ai/free/en/universities/stanford-university/) — University, Stanford, United States · 310 works · 284,480 citations
- [Harvard University](https://md.research.elamur.ai/free/en/universities/harvard-university/) — University, Cambridge, United States · 73 works · 37,309 citations
- [OpenAI](https://md.research.elamur.ai/free/en/companies/openai/) — Company, San Francisco, United States · 13 works · 18,282 citations

📚 Shelves (6): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 9,085 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 8,888 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 9,296 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 9,206 works · [🤝 Social Sciences](https://md.research.elamur.ai/free/en/shelves/social-sciences/) · 744 works · [🤝 Safety Research](https://md.research.elamur.ai/free/en/shelves/social-sciences/safety-research/) · 221 works

🧭 Topics (3): [🧠 Language models & pretraining](https://md.research.elamur.ai/free/en/topics/language-models/) · 1,344 works · [🧮 Reasoning](https://md.research.elamur.ai/free/en/topics/reasoning/) · 510 works · [🛡️ Bias, fairness & safety](https://md.research.elamur.ai/free/en/topics/bias-fairness-safety/) · 209 works

## Cite

Cite as: Melody Y. Guan, Manas Joglekar, Eric Wallace, Saachi Jain, Boaz Barak, Alec Helyar, Rachel Dias, Andrea Vallone, Hongyu Ren, Jason Wei, Hyung Won Chung, Sam Toyer, Johannes Heidecke, Alex Beutel and Amelia Glaese. 2024. Deliberative Alignment: Reasoning Enables Safer Language Models. arXiv:2412.16339.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@misc{guan2024deliberative,
  title = {{Deliberative Alignment: Reasoning Enables Safer Language Models}},
  author = {Melody Y. Guan, Manas Joglekar, Eric Wallace, Saachi Jain, Boaz Barak, Alec Helyar, Rachel Dias, Andrea Vallone, Hongyu Ren, Jason Wei, Hyung Won Chung, Sam Toyer, Johannes Heidecke, Alex Beutel and Amelia Glaese},
  year = {2024},
  doi = {10.48550/arxiv.2412.16339},
  eprint = {2412.16339},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2412.16339}
}
```

- [← previous RAID: A Shared Benchmark for Robust Evaluation of Machine-Generated Text Detectors](https://md.research.elamur.ai/free/en/papers/dugan-2024-raid-shared-benchmark-robust-evaluation/)
- [next → DuoAttention: Efficient Long-Context LLM Inference with Retrieval and Streaming Heads](https://md.research.elamur.ai/free/en/papers/xiao-2024-duoattention-efficient-long-context-llm/)
- [↑ back to papers of 2024](https://md.research.elamur.ai/free/en/papers/2024/page/3/)

## References

38 references, 19 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. OpenAI, Learning to reason with LLMs, 2024. [Online]. Available: [https://openai.com/index/learning-to-reason-with-llms/](https://openai.com/index/learning-to-reason-with-llms/). [↗](https://openai.com/index/learning-to-reason-with-llms/)
2. L. Ouyang, J. Wu, X. Jiang, D. Almeida, C. Wainwright, P. Mishkin, C. Zhang, S. Agarwal, K. Slama, A. Ray, et al., “[Training language models to follow instructions with human feedback](https://md.research.elamur.ai/free/en/papers/ouyang-2022-training-language-models-follow-instructions/),” in NeurIPS, 2022.
3. A. Dubey, A. Jauhri, A. Pandey, A. Kadian, A. Al-Dahle, A. Letman, A. Mathur, A. Schelten, A. Yang, A. Fan, et al., “The LLaMA 3 herd of models,” arXiv preprint arXiv:2407.21783, 2024. [find](https://scholar.google.com/scholar?q=A%2E+Dubey%2C+A%2E+Jauhri%2C+A%2E+Pandey%2C+A%2E+Kadian%2C+A%2E+Al%2DDahle%2C+A%2E+Letman%2C+A%2E+Mathur%2C+A%2E+Schelten%2C+A%2E+Yang%2C+A%2E+Fan%2C+et+al%2E%2C+%E2%80%9CThe+LLaMA+3+herd+of+models%2C%E2%80%9D+arXiv+preprint+arXiv%3A2407%2E21783%2C+2024%2E)
4. M. Reid, N. Savinov, D. Teplyashin, D. Lepikhin, T. Lillicrap, J.-b. Alayrac, R. Soricut, A. Lazaridou, O. Firat, J. Schrittwieser, et al., “[Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context](https://md.research.elamur.ai/free/en/papers/gemini-team-2024-gemini-1-5-unlocking-multimodal/),” arXiv preprint arXiv:2403.05530, 2024.
5. J. Achiam, S. Adler, S. Agarwal, L. Ahmad, I. Akkaya, F. L. Aleman, D. Almeida, J. Altenschmidt, S. Altman, S. Anadkat, et al., “[GPT-4 technical report](https://md.research.elamur.ai/free/en/papers/openai-2023-gpt-4-technical-report/),” arXiv preprint arXiv:2303.08774, 2023.
6. A. Zou, Z. Wang, N. Carlini, M. Nasr, J. Z. Kolter, and M. Fredrikson, “[Universal and transferable adversarial attacks on aligned language models](https://md.research.elamur.ai/free/en/papers/zou-2023-universal-transferable-adversarial-attacks-aligned/),” arXiv preprint arXiv:2307.15043, 2023.
7. A. Wei, N. Haghtalab, and J. Steinhardt, “[Jailbroken: How does llm safety training fail?](https://md.research.elamur.ai/free/en/papers/wei-2023-jailbroken-llm-safety-training-fail/)” NeurIPS, 2024.
8. M. Andriushchenko, F. Croce, and N. Flammarion, “Jailbreaking leading safety-aligned llms with simple adaptive attacks,” arXiv preprint arXiv:2404.02151, 2024. [find](https://scholar.google.com/scholar?q=M%2E+Andriushchenko%2C+F%2E+Croce%2C+and+N%2E+Flammarion%2C+%E2%80%9CJailbreaking+leading+safety%2Daligned+llms+with+simple+adaptive+attacks%2C%E2%80%9D+arXiv+preprint+arXiv%3A2404%2E02151%2C+2024%2E)
9. J. Uesato, N. Kushman, R. Kumar, F. Song, N. Siegel, L. Wang, A. Creswell, G. Irving, and I. Higgins, “Solving math word problems with process-and outcome-based feedback,” arXiv preprint arXiv:2211.[14275, 2022](https://md.research.elamur.ai/free/en/papers/uesato-2022-solving-math-word-problems-process/).
10. C. Snell, D. Klein, and R. Zhong, “Learning by distilling context,” arXiv preprint arXiv:2209.15189, 2022. [find](https://scholar.google.com/scholar?q=C%2E+Snell%2C+D%2E+Klein%2C+and+R%2E+Zhong%2C+%E2%80%9CLearning+by+distilling+context%2C%E2%80%9D+arXiv+preprint+arXiv%3A2209%2E15189%2C+2022%2E)
11. A. Askell, Y. Bai, A. Chen, et al., “A general language assistant as a laboratory for alignment,” arXiv preprint arXiv:2112.00861, 2021. [find](https://scholar.google.com/scholar?q=A%2E+Askell%2C+Y%2E+Bai%2C+A%2E+Chen%2C+et+al%2E%2C+%E2%80%9CA+general+language+assistant+as+a+laboratory+for+alignment%2C%E2%80%9D+arXiv+preprint+arXiv%3A2112%2E00861%2C+2021%2E)
12. A. Souly, Q. Lu, D. Bowen, et al., “A strongreject for empty jailbreaks,” arXiv preprint arXiv:2402.10260, 2024. [find](https://scholar.google.com/scholar?q=A%2E+Souly%2C+Q%2E+Lu%2C+D%2E+Bowen%2C+et+al%2E%2C+%E2%80%9CA+strongreject+for+empty+jailbreaks%2C%E2%80%9D+arXiv+preprint+arXiv%3A2402%2E10260%2C+2024%2E)
13. P. Röttger, H. R. Kirk, B. Vidgen, G. Attanasio, F. Bianchi, and D. Hovy, “[Xstest: A test suite for identifying exaggerated safety behaviours in large language models](https://md.research.elamur.ai/free/en/papers/rottger-2023-xstest-test-suite-identifying-exaggerated/),” arXiv preprint arXiv:2308.01263, 2024.
14. OpenAI, Introducing the model spec, 2024. [Online]. Available: [https://cdn.openai.com/spec/model-spec-2024-05-08.html](https://cdn.openai.com/spec/model-spec-2024-05-08.html). [↗](https://cdn.openai.com/spec/model-spec-2024-05-08.html)
15. W. Zhao, X. Ren, J. Hessel, C. Cardie, Y. Choi, and Y. Deng, “[Wildchat: 1m chatgpt interaction logs in the wild](https://md.research.elamur.ai/free/en/papers/zhao-2024-wildchat-1m-chatgpt-interaction-logs/),” arXiv preprint arXiv:2405.01470, 2024.
16. X. Shen, Z. Chen, M. Backes, Y. Shen, and Y. Zhang, “”do anything now”: Characterizing and evaluating in-the-wild jailbreak prompts on large language models,” arXiv preprint arXiv:2308.[03825, 2024](https://md.research.elamur.ai/free/en/papers/shen-2023-anything-now-characterizing-evaluating-wild/).
17. P. Chao, A. Robey, E. Dobriban, H. Hassani, G. J. Pappas, and E. Wong, “[Jailbreaking black box large language models in twenty queries](https://md.research.elamur.ai/free/en/papers/chao-2023-jailbreaking-black-box-large-language/),” arXiv preprint arXiv:2310.08419, 2024.
18. P. Chao, E. Debenedetti, A. Robey, et al., “[Jailbreakbench: An open robustness benchmark for jailbreaking large language models](https://md.research.elamur.ai/free/en/papers/chao-2024-jailbreakbench-open-robustness-benchmark-jailbreaking/),” arXiv preprint arXiv:2404.01318, 2024.
19. P. Kumar, E. Lau, S. Vijayakumar, et al., “Refusal-trained llms are easily jailbroken as browser agents,” arXiv preprint arXiv:2410.13886, 2024. — [arXiv:2410.13886](https://arxiv.org/abs/2410.13886) [find](https://scholar.google.com/scholar?q=P%2E+Kumar%2C+E%2E+Lau%2C+S%2E+Vijayakumar%2C+et+al%2E%2C+%E2%80%9CRefusal%2Dtrained+llms+are+easily+jailbroken+as+browser+agents%2C%E2%80%9D+arXiv+preprint+arXiv%3A2410%2E13886%2C+2024%2E+%E2%80%94+%5BarXiv%3A2410%2E13886%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2410%2E13886%29)
20. OpenAI, O1 system card, 2024. [Online](https://md.research.elamur.ai/free/en/papers/openai-2024-openai-o1-system-card/). Available: [https://cdn.openai.com/o1-system-card.pdf](https://cdn.openai.com/o1-system-card.pdf). [↗](https://cdn.openai.com/o1-system-card.pdf)
21. OpenAI, [Gpt-4o system card](https://md.research.elamur.ai/free/en/papers/openai-2024-gpt-4o-system-card/), 2024. [Online]. Available: [https://cdn.openai.com/gpt-4o-system-card.pdf](https://cdn.openai.com/gpt-4o-system-card.pdf). [↗](https://cdn.openai.com/gpt-4o-system-card.pdf)
22. Anthropic, Model card addendum: Claude 3.5 haiku and upgraded claude 3.5 sonnet, 2024. [Online]. Available: [https://assets.anthropic.com/m/1cd9d098ac3e6467/original/Claude-3-Model-Card-October-Addendum.pdf](https://assets.anthropic.com/m/1cd9d098ac3e6467/original/Claude-3-Model-Card-October-Addendum.pdf). [↗](https://assets.anthropic.com/m/1cd9d098ac3e6467/original/Claude-3-Model-Card-October-Addendum.pdf)
23. G. Gemini Team, “[Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context](https://md.research.elamur.ai/free/en/papers/gemini-team-2024-gemini-1-5-unlocking-multimodal/),” arXiv preprint arXiv:2403.05530, 2024.
24. J. Wei, N. Karina, H. W. Chung, Y. J. Jiao, S. Papay, A. Glaese, J. Schulman, and W. Fedus, “[Measuring short-form factuality in large language models](https://md.research.elamur.ai/free/en/papers/wei-2024-measuring-short-form-factuality-large/),” arXiv preprint arXiv:2411.04368, 2024.
25. A. Parrish, A. Chen, N. Nangia, V. Padmakumar, J. Phang, J. Thompson, P. M. Htut, and S. R. Bowman, “[BBQ: A hand-built bias benchmark for question answering](https://md.research.elamur.ai/free/en/papers/parrish-2022-bbq-hand-built-bias-benchmark/),” arXiv preprint arXiv:2110.08193, 2021.
26. Y. Bai, S. Kadavath, S. Kundu, et al., “[Constitutional AI: Harmlessness from AI feedback](https://md.research.elamur.ai/free/en/papers/bai-2022-constitutional-ai-harmlessness-ai-feedback/),” arXiv preprint arXiv:2212.08073, 2022.
27. A. Madaan, N. Tandon, P. Gupta, et al., “Self-refine: Iterative refinement with self-feedback,” arXiv preprint arXiv:2303.17651, 2023. [find](https://scholar.google.com/scholar?q=A%2E+Madaan%2C+N%2E+Tandon%2C+P%2E+Gupta%2C+et+al%2E%2C+%E2%80%9CSelf%2Drefine%3A+Iterative+refinement+with+self%2Dfeedback%2C%E2%80%9D+arXiv+preprint+arXiv%3A2303%2E17651%2C+2023%2E)
28. P. F. Christiano, J. Leike, T. Brown, M. Martic, S. Legg, and D. Amodei, “Deep reinforcement learning from human preferences,” in Advances in Neural Information Processing Systems, vol. 30, 2017. [find](https://scholar.google.com/scholar?q=P%2E+F%2E+Christiano%2C+J%2E+Leike%2C+T%2E+Brown%2C+M%2E+Martic%2C+S%2E+Legg%2C+and+D%2E+Amodei%2C+%E2%80%9CDeep+reinforcement+learning+from+human+preferences%2C%E2%80%9D+in+Advances+in+Neural+Information+Processing+Systems%2C+vol%2E+30%2C+2017%2E)
29. R. Rafailov, A. Sharma, E. Mitchell, S. Ermon, C. D. Manning, and C. Finn, “[Direct preference optimization: Your language model is secretly a reward model](https://md.research.elamur.ai/free/en/papers/rafailov-2023-direct-preference-optimization-language-model/),” arXiv preprint arXiv:2305.18290, 2024.
30. L. Pan, M. Saxon, W. Xu, D. Nathani, X. Wang, and W. Y. Wang, “Automatically correcting large language models: Surveying the landscape of diverse self-correction strategies,” arXiv preprint arXiv:2308.03188, 2023. [find](https://scholar.google.com/scholar?q=L%2E+Pan%2C+M%2E+Saxon%2C+W%2E+Xu%2C+D%2E+Nathani%2C+X%2E+Wang%2C+and+W%2E+Y%2E+Wang%2C+%E2%80%9CAutomatically+correcting+large+language+models%3A+Surveying+the+landscape+of+diverse+self%2Dcorrection+strategies%2C%E2%80%9D+arXiv+preprint+arXiv%3A2308)
31. S. Welleck, X. Lu, P. West, F. Brahman, T. Shen, D. Khashabi, and Y. Choi, “[Generating sequences by learning to self-correct](https://md.research.elamur.ai/free/en/papers/welleck-2022-generating-sequences-learning-self-correct/),” in The Eleventh International Conference on Learning Representations, vol. 2, 2023.
32. I. Schlag, S. Sukhbaatar, A. Celikyilmaz, W.-t. Yih, J. Weston, J. Schmidhuber, and X. Li, “Large language model programs,” arXiv preprint arXiv:2305.05364, 2023. [find](https://scholar.google.com/scholar?q=I%2E+Schlag%2C+S%2E+Sukhbaatar%2C+A%2E+Celikyilmaz%2C+W%2E%2Dt%2E+Yih%2C+J%2E+Weston%2C+J%2E+Schmidhuber%2C+and+X%2E+Li%2C+%E2%80%9CLarge+language+model+programs%2C%E2%80%9D+arXiv+preprint+arXiv%3A2305%2E05364%2C+2023%2E)
33. Y. Zhang, J. Chi, H. Nguyen, K. Upasani, D. M. Bikel, J. Weston, and E. M. Smith, “Backtracking improves generation safety,” arXiv preprint arXiv:2409.14586, 2024. — [arXiv:2409.14586](https://arxiv.org/abs/2409.14586) [find](https://scholar.google.com/scholar?q=Y%2E+Zhang%2C+J%2E+Chi%2C+H%2E+Nguyen%2C+K%2E+Upasani%2C+D%2E+M%2E+Bikel%2C+J%2E+Weston%2C+and+E%2E+M%2E+Smith%2C+%E2%80%9CBacktracking+improves+generation+safety%2C%E2%80%9D+arXiv+preprint+arXiv%3A2409%2E14586%2C+2024%2E+%E2%80%94+%5BarXiv%3A2409%2E14586%5D%28https%3A%2F%2Farxiv%2Eo)
34. S. Russell, Human compatible: Artificial intelligence and the problem of control, 1st. USA: Penguin Books, 2019, isbn: 9780525558637. [find](https://scholar.google.com/scholar?q=S%2E+Russell%2C+Human+compatible%3A+Artificial+intelligence+and+the+problem+of+control%2C+1st%2E+USA%3A+Penguin+Books%2C+2019%2C+isbn%3A+9780525558637%2E)
35. N. Bostrom, Superintelligence: Paths, Dangers, Strategies, 1st. USA: Oxford University Press, Inc., 2014, isbn: 0199678111. [find](https://scholar.google.com/scholar?q=N%2E+Bostrom%2C+Superintelligence%3A+Paths%2C+Dangers%2C+Strategies%2C+1st%2E+USA%3A+Oxford+University+Press%2C+Inc%2E%2C+2014%2C+isbn%3A+0199678111%2E)
36. S. M. Omohundro, “The basic ai drives,” in Proceedings of the 2008 Conference on Artificial General Intelligence 2008: Proceedings of the First AGI Conference, NLD: IOS Press, 2008, pp. 483–492, isbn: 9781586038335. [find](https://scholar.google.com/scholar?q=S%2E+M%2E+Omohundro%2C+%E2%80%9CThe+basic+ai+drives%2C%E2%80%9D+in+Proceedings+of+the+2008+Conference+on+Artificial+General+Intelligence+2008%3A+Proceedings+of+the+First+AGI+Conference%2C+NLD%3A+IOS+Press%2C+2008%2C+pp%2E+483%E2%80%93492%2C+isbn%3A)
37. O. Järviniemi and E. Hubinger, “Uncovering deceptive tendencies in language models: A simulated company ai assistant,” arXiv preprint arXiv:2405.01576, 2024. — [arXiv:2405.01576](https://arxiv.org/abs/2405.01576) [find](https://scholar.google.com/scholar?q=O%2E+Ja%CC%88rviniemi+and+E%2E+Hubinger%2C+%E2%80%9CUncovering+deceptive+tendencies+in+language+models%3A+A+simulated+company+ai+assistant%2C%E2%80%9D+arXiv+preprint+arXiv%3A2405%2E01576%2C+2024%2E+%E2%80%94+%5BarXiv%3A2405%2E01576%5D%28https%3A%2F%2Farxiv%2Eorg%2Fab)
38. T. Hagendorff, “Deception abilities emerged in large language models,” Proceedings of the National Academy of Sciences, vol. 121, no. 24, Jun. 2024, issn: 1091-6490. [find](https://scholar.google.com/scholar?q=T%2E+Hagendorff%2C+%E2%80%9CDeception+abilities+emerged+in+large+language+models%2C%E2%80%9D+Proceedings+of+the+National+Academy+of+Sciences%2C+vol%2E+121%2C+no%2E+24%2C+Jun%2E+2024%2C+issn%3A+1091%2D6490%2E)

*Updated 24 October 2026*