---
title: "GPT-4o System Card — OpenAI 2024"
description: "The GPT-4o System Card explains how OpenAI assessed and reduced the risks of GPT-4o."
authors: ["OpenAI", "Aaron Hurst", "Adam Lerer", "Adam P. Goucher", "Adam Perelman", "Aditya Ramesh", "Aidan Clark", "AJ Ostrow", "Akila Welihinda", "Alan Hayes", "Alec Radford", "Aleksander Mądry", "Alex Baker-Whitcomb", "Alex Beutel", "Alex Borzunov", "Alex Carney", "Alex Chow", "Alex Kirillov", "Alex Nichol", "Alex Paino", "Alex Renzin", "Alex Tachard Passos", "Alexander Kirillov", "Alexi Christakis", "Alexis Conneau", "Ali Kamali", "Allan Jabri", "Allison Moyer", "Allison Tam", "Amadou Crookes", "Amin Tootoochian", "Amin Tootoonchian", "Ananya Kumar", "Andrea Vallone", "Andrej Karpathy", "Andrew Braunstein", "Andrew Cann", "Andrew Codispoti", "Andrew Galu", "Andrew Kondrich", "Andrew Tulloch", "Andrey Mishchenko", "Angela Baek", "Angela Jiang", "Antoine Pelisse", "Antonia Woodford", "Anuj Gosalia", "Arka Dhar", "Ashley Pantuliano", "Avi Nayak", "Avital Oliver", "Barret Zoph", "Behrooz Ghorbani", "Ben Leimberger", "Ben Rossen", "Ben Sokolowsky", "Ben Wang", "Benjamin Zweig", "Beth Hoover", "Blake Samic", "Bob McGrew", "Bobby Spero", "Bogo Giertler", "Bowen Cheng", "Brad Lightcap", "Brandon Walkin", "Brendan Quinn", "Brian Guarraci", "Brian Hsu", "Bright Kellogg", "Brydon Eastman", "Camillo Lugaresi", "Carroll Wainwright", "Cary Bassin", "Cary Hudson", "Casey Chu", "Chad Nelson", "Chak Li", "Chan Jun Shern", "Channing Conger", "Charlotte Barette", "Chelsea Voss", "Chen Ding", "Cheng Lu", "Chong Zhang", "Chris Beaumont", "Chris Hallacy", "Chris Koch", "Christian Gibson", "Christina Kim", "Christine Choi", "Christine McLeavey", "Christopher Hesse", "Claudia Fischer", "Clemens Winter", "Coley Czarnecki", "Colin Jarvis", "Colin Wei", "Constantin Koumouzelis", "Dane Sherburn", "Daniel Kappler", "Daniel Levin", "Daniel Levy", "David Carr", "David Farhi", "David Mely", "David Robinson", "David Sasaki", "Denny Jin", "Dev Valladares", "Dimitris Tsipras", "Doug Li", "Duc Phong Nguyen", "Duncan Findlay", "Edede Oiwoh", "Edmund Wong", "Ehsan Asdar", "Elizabeth Proehl", "Elizabeth Yang", "Eric Antonow", "Eric Kramer", "Eric Peterson", "Eric Sigler", "Eric Wallace", "Eugene Brevdo", "Evan Mays", "Farzad Khorasani", "Felipe Petroski Such", "Filippo Raso", "Francis Zhang", "Fred von Lohmann", "Freddie Sulit", "Gabriel Goh", "Gene Oden", "Geoff Salmon", "Giulio Starace", "Greg Brockman", "Hadi Salman", "Haiming Bao", "Haitang Hu", "Hannah Wong", "Haoyu Wang", "Heather Schmidt", "Heather Whitney", "Heewoo Jun", "Hendrik Kirchner", "Henrique Ponde de Oliveira Pinto", "Hongyu Ren", "Huiwen Chang", "Hyung Won Chung", "Ian Kivlichan", "Ian O'Connell", "Ian O'Connell", "Ian Osband", "Ian Silber", "Ian Sohl", "Ibrahim Okuyucu", "Ikai Lan", "Ilya Kostrikov", "Ilya Sutskever", "Ingmar Kanitscheider", "Ishaan Gulrajani", "Jacob Coxon", "Jacob Menick", "Jakub Pachocki", "James Aung", "James Betker", "James Crooks", "James Lennon", "Jamie Kiros", "Jan Leike", "Jane Park", "Jason Kwon", "Jason Phang", "Jason Teplitz", "Jason Wei", "Jason Wolfe", "Jay Chen", "Jeff Harris", "Jenia Varavva", "Jessica Gan Lee", "Jessica Shieh", "Ji Lin", "Jiahui Yu", "Jiayi Weng", "Jie Tang", "Jieqi Yu", "Joanne Jang", "Joaquin Quinonero Candela", "Joe Beutler", "Joe Landers", "Joel Parish", "Johannes Heidecke", "John Schulman", "Jonathan Lachman", "Jonathan McKay", "Jonathan Uesato", "Jonathan Ward", "Jong Wook Kim", "Joost Huizinga", "Jordan Sitkin", "Jos Kraaijeveld", "Josh Gross", "Josh Kaplan", "Josh Snyder", "Joshua Achiam", "Joy Jiao", "Joyce Lee", "Juntang Zhuang", "Justyn Harriman", "Kai Fricke", "Kai Hayashi", "Karan Singhal", "Katy Shi", "Kavin Karthik", "Kayla Wood", "Kendra Rimbach", "Kenny Hsu", "Kenny Nguyen", "Keren Gu-Lemberg", "Kevin Button", "Kevin Liu", "Kiel Howe", "Krithika Muthukumar", "Kyle Luther", "Lama Ahmad", "Larry Kai", "Lauren Itow", "Lauren Workman", "Leher Pathak", "Leo Chen", "Li Jing", "Lia Guy", "Liam Fedus", "Liang Zhou", "Lien Mamitsuka", "Lilian Weng", "Lindsay McCallum", "Lindsey Held", "Long Ouyang", "Louis Feuvrier", "Lu Zhang", "Lukas Kondraciuk", "Lukasz Kaiser", "Luke Hewitt", "Luke Metz", "Lyric Doshi", "Mada Aflak", "Maddie Simens", "Madelaine Boyd", "Madeleine Thompson", "Marat Dukhan", "Mark Chen", "Mark Gray", "Mark Hudnall", "Marvin Zhang", "Marwan Aljubeh", "Mateusz Litwin", "Matthew Zeng", "Max Johnson", "Maya Shetty", "Mayank Gupta", "Meghan Shah", "Mehmet Yatbaz", "Meng Jia Yang", "Mengchao Zhong", "Mia Glaese", "Mianna Chen", "Michael Janner", "Michael Lampe", "Michael Petrov", "Michael Wu", "Michele Wang", "Michelle Fradin", "Michelle Pokrass", "Miguel Castro", "Miguel Oom Temudo de Castro", "Mikhail Pavlov", "Miles Brundage", "Miles Wang", "Minal Khan", "Mira Murati", "Mo Bavarian", "Molly Lin", "Murat Yesildal", "Nacho Soto", "Natalia Gimelshein", "Natalie Cone", "Natalie Staudacher", "Natalie Summers", "Natan LaFontaine", "Neil Chowdhury", "Nick Ryder", "Nick Stathas", "Nick Turley", "Nik Tezak", "Niko Felix", "Nithanth Kudige", "Nitish Keskar", "Noah Deutsch", "Noel Bundick", "Nora Puckett", "Ofir Nachum", "Ola Okelola", "Oleg Boiko", "Oleg Murk", "Oliver Jaffe", "Olivia Watkins", "Olivier Godement", "Owen Campbell-Moore", "Patrick Chao", "Paul McMillan", "Pavel Belov", "Peng Su", "Peter Bak", "Peter Bakkum", "Peter Deng", "Peter Dolan", "Peter Hoeschele", "Peter Welinder", "Phil Tillet", "Philip Pronin", "Philippe Tillet", "Prafulla Dhariwal", "Qiming Yuan", "Rachel Dias", "Rachel Lim", "Rahul Arora", "Rajan Troll", "Randall Lin", "Rapha Gontijo Lopes", "Raul Puri", "Reah Miyara", "Reimar Leike", "Renaud Gaubert", "Reza Zamani", "Ricky Wang", "Rob Donnelly", "Rob Honsby", "Rocky Smith", "Rohan Sahai", "Rohit Ramchandani", "Romain Huet", "Rory Carmichael", "Rowan Zellers", "Roy Chen", "Ruby Chen", "Ruslan Nigmatullin", "Ryan Cheu", "Saachi Jain", "Sam Altman", "Sam Schoenholz", "Sam Toizer", "Samuel Miserendino", "Sandhini Agarwal", "Sara Culver", "Scott Ethersmith", "Scott Gray", "Sean Grove", "Sean Metzger", "Shamez Hermani", "Shantanu Jain", "Shengjia Zhao", "Sherwin Wu", "Shino Jomoto", "Shirong Wu", "Shuaiqi", "Xia", "Sonia Phene", "Spencer Papay", "Srinivas Narayanan", "Steve Coffey", "Steve Lee", "Stewart Hall", "Suchir Balaji", "Tal Broda", "Tal Stramer", "Tao Xu", "Tarun Gogineni", "Taya Christianson", "Ted Sanders", "Tejal Patwardhan", "Thomas Cunninghman", "Thomas Degry", "Thomas Dimson", "Thomas Raoux", "Thomas Shadwell", "Tianhao Zheng", "Todd Underwood", "Todor Markov", "Toki Sherbakov", "Tom Rubin", "Tom Stasi", "Tomer Kaftan", "Tristan Heywood", "Troy Peterson", "Tyce Walters", "Tyna Eloundou", "Valerie Qi", "Veit Moeller", "Vinnie Monaco", "Vishal Kuo", "Vlad Fomenko", "Wayne Chang", "Weiyi Zheng", "Wenda Zhou", "Wesam Manassra", "Will Sheu", "Wojciech Zaremba", "Yash Patil", "Yilei Qian", "Yongjik Kim", "Youlong Cheng", "Yu Zhang", "Yuchen He", "Yuchen Zhang", "Yujia Jin", "Yunxing Dai", "Yury Malkov"]
year: 2024
venue: "arXiv:2410.21276"
source: "https://arxiv.org/abs/2410.21276"
doi: "10.48550/arxiv.2410.21276"
arxiv: "2410.21276"
licence: "CC BY 4.0 (arXiv)"
licence_url: "http://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/openai-2024-gpt-4o-system-card/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/openai-2024-gpt-4o-system-card/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2024](https://md.research.elamur.ai/free/en/papers/2024/) / GPT-4o System Card

# GPT-4o System Card

**Authors:** [OpenAI](https://md.research.elamur.ai/free/en/authors/openai/), Aaron Hurst, [Adam Lerer](https://md.research.elamur.ai/free/en/authors/adam-lerer/), Adam P. Goucher, [Adam Perelman](https://md.research.elamur.ai/free/en/authors/adam-perelman/), [Aditya Ramesh](https://md.research.elamur.ai/free/en/authors/aditya-ramesh/), Aidan Clark, [AJ Ostrow](https://md.research.elamur.ai/free/en/authors/aj-ostrow/), Akila Welihinda, Alan Hayes, [Alec Radford](https://md.research.elamur.ai/free/en/authors/alec-radford/), [Aleksander Mądry](https://md.research.elamur.ai/free/en/authors/aleksander-madry/) et al. (419 authors) · **Year:** 2024 · **Venue:** arXiv:2410.21276 · **Licence:** [CC BY 4.0 (arXiv)](http://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://arxiv.org/abs/2410.21276) · [PDF](https://arxiv.org/pdf/2410.21276) · [DOI 10.48550/arxiv.2410.21276](https://doi.org/10.48550/arxiv.2410.21276) · [arXiv 2410.21276](https://arxiv.org/abs/2410.21276) · [OpenAlex](https://openalex.org/W4404330874)

**Citations:** 174 citations (openalex-snapshot · 2026-10-14) · [2047](#cited-by) in our library, of the 140884 papers whose bibliographies we read

## Summary
The GPT-4o System Card explains how OpenAI assessed and reduced the risks of GPT-4o. GPT-4o is a single end-to-end network that accepts text, audio, image and video and can produce text, audio and image output. Because the model holds real-time spoken conversations, the report focuses on hazards specific to speech-to-speech use, but it also checks text and vision.

Pre-training data (cutoff October 2023) came from public web crawls, ML datasets, code and math, multimodal material and licensed partner content. It was filtered with the Moderation API, safety classifiers, personal-data reduction and fingerprints of DALL-E 3 opt-out images.

Risks were found through staged external red teaming and then turned into measurable evaluations. Existing text benchmarks were converted to audio with Voice Engine (TTS) so they could be reused. Mitigations combine post-training refusals with deployed classifiers, for example:
- limiting output to preset voices,
- refusing speaker identification,
- blocking music,
- hedging on sensitive traits.

Preparedness Framework and third-party results also appear. The card serves as a practical template for safety-testing voice-enabled multimodal models.

## Key points
- Audio response latency is as low as 232 ms, with a 320 ms average, which is close to human conversational timing. The model matches GPT-4 Turbo on English text and code, improves on non-English languages, and costs 50% less in the API.
- Red teaming involved more than 100 external testers covering 45 languages and 29 countries. It ran in four phases (10, 30, 65 and 65 people) from early March to late June 2024, and the final phase used advanced voice mode in the iOS app.
- TTS-based evaluation has known limits. Equations and code convert poorly to speech, and accents, background noise, cross-talk and intonation are not fully represented.
- Moderation classifiers run on transcripts of both audio prompts and generations, and high-severity outputs are blocked. In the limited alpha of Advanced Voice Mode, the model was told not to sing.
- External contributors included METR, Apollo Research and Virtue AI, plus the Uhura Evals group, which is relevant for low-resource languages.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

Fields: [Other](https://md.research.elamur.ai/free/en/shelves/other/)

Topics: [Language models & pretraining](https://md.research.elamur.ai/free/en/topics/language-models/)

## Cited by

Cited by

2047 papers of our library cite it (of the 140884 whose bibliographies we read); 68 of them have a page here, 1979 not yet.

- [gpt-oss-120b & gpt-oss-20b Model Card](https://md.research.elamur.ai/free/en/papers/agarwal-2025-gpt-oss-120b-gpt-oss/)
- [Video-MME: The First-Ever Comprehensive Evaluation Benchmark of Multi-modal LLMs in Video Analysis](https://md.research.elamur.ai/free/en/papers/fu-2024-video-mme-first-ever-comprehensive/)
- [Tulu 3: Pushing Frontiers in Open Language Model Post-Training](https://md.research.elamur.ai/free/en/papers/lambert-2024-tulu-3-pushing-frontiers-open/)
- [Qwen3 Embedding: Advancing Text Embedding and Reranking Through Foundation Models](https://md.research.elamur.ai/free/en/papers/zhang-2025-qwen3-embedding-advancing-text/)
- [Personality Traits in Large Language Models](https://md.research.elamur.ai/free/en/papers/serapio-2023-garcia-personality-traits-large-language-models/)
- [Search-o1: Agentic Search-Enhanced Large Reasoning Models](https://md.research.elamur.ai/free/en/papers/li-2025-search-o1-agentic-search-enhanced/)
- [InternVL3: Exploring Advanced Training and Test-Time Recipes for Open-Source Multimodal Models](https://md.research.elamur.ai/free/en/papers/zhu-2025-internvl3-exploring-advanced-training/)
- [Molmo and PixMo: Open Weights and Open Data for State-of-the-Art Vision-Language Models](https://md.research.elamur.ai/free/en/papers/deitke-2024-molmo-pixmo-open-weights-open/)
- [Global MMLU: Understanding and Addressing Cultural and Linguistic Biases in Multilingual Evaluation](https://md.research.elamur.ai/free/en/papers/singh-2025-global-mmlu-understanding-addressing-cultural/)
- [InternVL3.5: Advancing Open-Source Multimodal Models in Versatility, Reasoning, and Efficiency](https://md.research.elamur.ai/free/en/papers/wang-2025-internvl3-5-advancing-open/)
- [The Lessons of Developing Process Reward Models in Mathematical Reasoning](https://md.research.elamur.ai/free/en/papers/zhang-2025-lessons-developing-process-reward-models/)
- [SimpleRL-Zoo: Investigating and Taming Zero Reinforcement Learning for Open Base Models in the Wild](https://md.research.elamur.ai/free/en/papers/zeng-2025-simplerl-zoo-investigating-taming-zero/)
- [Phi-4-Mini Technical Report: Compact yet Powerful Multimodal Language Models via Mixture-of-LoRAs](https://md.research.elamur.ai/free/en/papers/microsoft-2025-phi-4-mini-technical-report/)
- [Agent Laboratory: Using LLM Agents as Research Assistants](https://md.research.elamur.ai/free/en/papers/schmidgall-2025-agent-laboratory-using-llm/)
- [Mem0: Building Production-Ready AI Agents with Scalable Long-Term Memory](https://md.research.elamur.ai/free/en/papers/chhikara-2025-mem0-building-production-ready-ai/)
- [ProcessBench: Identifying Process Errors in Mathematical Reasoning](https://md.research.elamur.ai/free/en/papers/zheng-2025-processbench-identifying-process-errors-mathematical/)
- [HealthBench: Evaluating Large Language Models Towards Improved Human Health](https://md.research.elamur.ai/free/en/papers/arora-2025-healthbench-evaluating-large-language-models/)
- [DeepResearcher: Scaling Deep Research via Reinforcement Learning in Real-world Environments](https://md.research.elamur.ai/free/en/papers/zheng-2025-deepresearcher-scaling-deep-research-reinforcement/)
- [UI-TARS: Pioneering Automated GUI Interaction with Native Agents](https://md.research.elamur.ai/free/en/papers/qin-2025-ui-tars-pioneering-automated/)
- [SemEval-2025 Task 1: AdMIRe - Advancing Multimodal Idiomaticity Representation](https://md.research.elamur.ai/free/en/papers/pickard-2025-semeval-2025-task-1-admire/)
- [Do Large Language Models Understand Word Senses?](https://md.research.elamur.ai/free/en/papers/lex-2025-meconi-llms-word-senses/)
- [New Trends for Modern Machine Translation with Large Reasoning Models](https://md.research.elamur.ai/free/en/papers/liu-2026-new-trends-modern-machine-translation/)
- [METAL: A Multi-Agent Framework for Chart Generation with Test-Time Scaling](https://md.research.elamur.ai/free/en/papers/li-2025-metal-multi-agent-framework-chart/)
- [GrammaMT: Improving Machine Translation with Grammar-Informed In-Context Learning](https://md.research.elamur.ai/free/en/papers/ramos-2025-grammamt-improving-machine-translation-grammar/)
- [Sacred or Synthetic? Evaluating LLM Reliability and Abstention for Religious Questions](https://md.research.elamur.ai/free/en/papers/atif-2025-sacred-synthetic-evaluating-llm-reliability/)
- [Beyond Literal Mapping: Benchmarking and Improving Non-Literal Translation Evaluation](https://md.research.elamur.ai/free/en/papers/tian-2026-beyond-literal-mapping-benchmarking-improving/)
- [Mechanisms vs. Outcomes: Probing for Syntax Fails to Explain Performance on Targeted Syntactic Evaluations](https://md.research.elamur.ai/free/en/papers/agarwal-2025-mechanisms-vs-outcomes-probing-syntax/)
- [A Multi-Task Benchmark for Abusive Language Detection in Low-Resource Settings](https://md.research.elamur.ai/free/en/papers/gaim-2025-multi-task-benchmark-abusive-language/)
- [Baseer: A Vision-Language Model for Arabic Document-to-Markdown OCR](https://md.research.elamur.ai/free/en/papers/hennara-2025-baseer-vision-language-model-arabic/)
- [Do Diacritics Matter? Evaluating the Impact of Arabic Diacritics on Tokenization and LLM Benchmarks](https://md.research.elamur.ai/free/en/papers/aclbulk-2026-inoue-do-diacritics-matter-evaluating-impact/)
- [Who Watches the Watchmen? Humans Disagree With Translation Metrics on Unseen Domains](https://md.research.elamur.ai/free/en/papers/schmidt-2026-who-watches-watchmen-humans-disagree/)
- [A Critical Study of Automatic Evaluation in Sign Language Translation](https://md.research.elamur.ai/free/en/papers/yazdani-2026-critical-study-automatic-evaluation-sign/)
- [Dicta-LM 3.0: Advancing The Frontier of Hebrew Sovereign LLMs](https://md.research.elamur.ai/free/en/papers/langstream-2026-shmidman-dicta-lm-3-0-advancing/)
- [Morphemes Without Borders: Evaluating Root-Pattern Morphology in Arabic Tokenizers and LLMs](https://md.research.elamur.ai/free/en/papers/alakeel-2026-morphemes-borders-evaluating-root-pattern/)
- [CLASE: A Hybrid Method for Chinese Legalese Stylistic Evaluation](https://md.research.elamur.ai/free/en/papers/ma-2026-clase-hybrid-method-chinese-legalese/)
- [Constructing a Japanese Claim Decomposition Dataset for Fact-Checking of LLM-Generated Texts](https://md.research.elamur.ai/free/en/papers/masano-2026-constructing-japanese-claim-decomposition-dataset/)
- [Multi-modal, Multi-task, Multi-criteria Automatic Evaluation with Vision Language Models](https://md.research.elamur.ai/free/en/papers/oi-2026-multi-modal-multi-task-multi/)
- [Modeling Clinical Uncertainty in Radiology Reports: From Explicit Uncertainty Markers to Implicit Reasoning Pathways](https://md.research.elamur.ai/free/en/papers/rabaey-2026-modeling-clinical-uncertainty-radiology-reports/)
- [Revisiting Metric Reliability for Fine-grained Evaluation of Machine Translation and Summarization in Indian Languages](https://md.research.elamur.ai/free/en/papers/yari-2026-revisiting-metric-reliability-fine-grained/)
- [SEFL: A Framework for Generating Synthetic Educational Assignment Feedback with LLM Agents](https://md.research.elamur.ai/free/en/papers/zhang-2026-sefl-framework-generating-synthetic-educational/)
- [LCHAIM - Investigating Long Context Reasoning in Hebrew](https://md.research.elamur.ai/free/en/papers/findings-2025-malul-lchaim-investigating-long-context-reasoning/)
- [Morphemes without Borders: Evaluating Root–Pattern Morphology in Arabic Tokenizers and LLMs](https://md.research.elamur.ai/free/en/papers/lingbulk-2026-alakeel-morphemes-without-borders-evaluating-root/)
- [DRInQ: Evaluating Conversational Implicature with Controlled Context Variation](https://md.research.elamur.ai/free/en/papers/arai-2026-drinq-evaluating-conversational-implicature-controlled/)
- [STAGEET: Stage-wise Typed Edit Tagging for Grammatical Error Correction with Arabic as a Case Study](https://md.research.elamur.ai/free/en/papers/lou-2026-stageet-stage-wise-typed-edit/)
- [Augmenting LLM Reasoning with Dynamic Notes Writing for Complex MultiHop QA](https://md.research.elamur.ai/free/en/papers/maheshwary-2026-augmenting-llm-reasoning-dynamic-notes/)
- [HistoryBankQA: Multilingual Temporal Question Answering on Historical Events](https://md.research.elamur.ai/free/en/papers/mandal-2026-historybankqa-multilingual-temporal-question-answering/)
- [Can LLMs Faithfully Explain Themselves in Low-Resource Languages? A Case Study on Emotion Detection in Persian](https://md.research.elamur.ai/free/en/papers/mehrazar-2026-can-llms-faithfully-explain-themselves/)
- [Uncovering Hidden Violent Tendencies in LLMs: A Demographic Analysis via Behavioral Vignettes](https://md.research.elamur.ai/free/en/papers/myers-2026-uncovering-hidden-violent-tendencies-llms/)
- [mSCoRe: A Multilingual and Scalable Benchmark for Skill-based Commonsense Reasoning](https://md.research.elamur.ai/free/en/papers/ngo-2026-mscore-multilingual-scalable-benchmark-skill/)
- [ViX-Ray: A Vietnamese Chest X-Ray Dataset for Vision-Language Models](https://md.research.elamur.ai/free/en/papers/nguyen-2026-vix-ray-vietnamese-chest-x/)
- [Data-Centric Approach at the LoResMT 2026 Turkic Translation Challenge: Russian-Kyrgyz](https://md.research.elamur.ai/free/en/papers/novokshanov-2026-data-centric-approach-loresmt-2026/)
- [A Single Model Ensemble Framework for Neural Machine Translation Using Pivot Translation](https://md.research.elamur.ai/free/en/papers/oh-2026-single-model-ensemble-framework-neural/)
- [Identifying Imaging Follow-Up in Radiology Reports: A Comparative Analysis of Traditional ML and LLM Approaches](https://md.research.elamur.ai/free/en/papers/park-2026-identifying-imaging-follow-up-radiology/)
- [Stands to Reason: Investigating the Effect of Reasoning on Idiomaticity Detection](https://md.research.elamur.ai/free/en/papers/phelps-2026-stands-reason-investigating-effect-reasoning/)
- [RelEx-PT: A Portuguese Sentence-Level Relation Extraction Dataset](https://md.research.elamur.ai/free/en/papers/pinto-2026-relex-pt-portuguese-sentence-level/)
- [Detecting Hallucinations in Authentic LLM–Human Interactions](https://md.research.elamur.ai/free/en/papers/ren-2026-detecting-hallucinations-authentic-llm-human/)
- [A Dataset for Oral Reading in Young English Readers](https://md.research.elamur.ai/free/en/papers/rose-2026-dataset-oral-reading-young-english/)
- [HotelCheckSpan: A Benchmark Dataset for LLM Faithfulness](https://md.research.elamur.ai/free/en/papers/schmidtova-2026-hotelcheckspan-benchmark-dataset-llm-faithfulness/)
- [A Scalable Pipeline for Novelty Detection in Skill Extraction Using Large Language Models](https://md.research.elamur.ai/free/en/papers/seifert-2026-scalable-pipeline-novelty-detection-skill/)
- [LLMs Out-of-the-Box Do Not Generate Context-Appropriate Word Order in Russian](https://md.research.elamur.ai/free/en/papers/shabaeva-2026-llms-out-box-do-not/)
- [Assessing Y-Axis Influence: Bias in Multimodal Language Models on Chart-to-Table Translation](https://md.research.elamur.ai/free/en/papers/song-2026-assessing-y-axis-influence-bias/)
- [Translation as Augmentation: Effect of Translated Data on Assessment of Difficulty](https://md.research.elamur.ai/free/en/papers/wu-2026-translation-as-augmentation-effect-translated/)
- [CRiT-QA: Evaluating Multi-hop Reasoning with Counterfactual Chains and Distractor Traps](https://md.research.elamur.ai/free/en/papers/yun-2026-crit-qa-evaluating-multi-hop/)
- [RadTimeline: Timeline Summarization for Longitudinal Radiological Lung Findings](https://md.research.elamur.ai/free/en/papers/zhou-2026-radtimeline-timeline-summarization-longitudinal-radiological/)
- [MuDRiC: Multi-Dialect Reasoning for Arabic Commonsense Validation](https://md.research.elamur.ai/free/en/papers/elozeiri-2025-mudric-multi-dialect-reasoning-arabic/)
- [Sarc7: Evaluating Sarcasm Detection and Generation with Seven Types and Emotion-Informed Techniques](https://md.research.elamur.ai/free/en/papers/xiong-2025-sarc7-evaluating-sarcasm-detection-generation/)
- [Leveraging Digitized Newspapers to Collect Summarization Data in Low-Resource Languages](https://md.research.elamur.ai/free/en/papers/aclbulk-2026-dahan-leveraging-digitized-newspapers-collect-summarization/)
- [HEBATRON: A Hebrew-Specialized Open-Weight Mixture-of-Experts Language Model](https://md.research.elamur.ai/free/en/papers/langstream-2026-kayzer-hebatron-hebrew-specialized-open-weight/)

## Cite

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@misc{openai2024gpt,
  title = {{GPT-4o System Card}},
  author = {OpenAI and Hurst, Aaron and Lerer, Adam and Goucher, Adam P. and Perelman, Adam and Ramesh, Aditya and Clark, Aidan and Ostrow, AJ and Welihinda, Akila and Hayes, Alan and Radford, Alec and Mądry, Aleksander and Baker-Whitcomb, Alex and Beutel, Alex and Borzunov, Alex and Carney, Alex and Chow, Alex and Kirillov, Alex and Nichol, Alex and Paino, Alex and Renzin, Alex and Passos, Alex Tachard and Kirillov, Alexander and Christakis, Alexi and Conneau, Alexis and Kamali, Ali and Jabri, Allan and Moyer, Allison and Tam, Allison and Crookes, Amadou and Tootoochian, Amin and Tootoonchian, Amin and Kumar, Ananya and Vallone, Andrea and Karpathy, Andrej and Braunstein, Andrew and Cann, Andrew and Codispoti, Andrew and Galu, Andrew and Kondrich, Andrew and Tulloch, Andrew and Mishchenko, Andrey and Baek, Angela and Jiang, Angela and Pelisse, Antoine and Woodford, Antonia and Gosalia, Anuj and Dhar, Arka and Pantuliano, Ashley and Nayak, Avi and Oliver, Avital and Zoph, Barret and Ghorbani, Behrooz and Leimberger, Ben and Rossen, Ben and Sokolowsky, Ben and Wang, Ben and Zweig, Benjamin and Hoover, Beth and Samic, Blake and McGrew, Bob and Spero, Bobby and Giertler, Bogo and Cheng, Bowen and Lightcap, Brad and Walkin, Brandon and Quinn, Brendan and Guarraci, Brian and Hsu, Brian and Kellogg, Bright and Eastman, Brydon and Lugaresi, Camillo and Wainwright, Carroll and Bassin, Cary and Hudson, Cary and Chu, Casey and Nelson, Chad and Li, Chak and Shern, Chan Jun and Conger, Channing and Barette, Charlotte and Voss, Chelsea and Ding, Chen and Lu, Cheng and Zhang, Chong and Beaumont, Chris and Hallacy, Chris and Koch, Chris and Gibson, Christian and Kim, Christina and Choi, Christine and McLeavey, Christine and Hesse, Christopher and Fischer, Claudia and Winter, Clemens and Czarnecki, Coley and Jarvis, Colin and Wei, Colin and Koumouzelis, Constantin and Sherburn, Dane and Kappler, Daniel and Levin, Daniel and Levy, Daniel and Carr, David and Farhi, David and Mely, David and Robinson, David and Sasaki, David and Jin, Denny and Valladares, Dev and Tsipras, Dimitris and Li, Doug and Nguyen, Duc Phong and Findlay, Duncan and Oiwoh, Edede and Wong, Edmund and Asdar, Ehsan and Proehl, Elizabeth and Yang, Elizabeth and Antonow, Eric and Kramer, Eric and Peterson, Eric and Sigler, Eric and Wallace, Eric and Brevdo, Eugene and Mays, Evan and Khorasani, Farzad and Such, Felipe Petroski and Raso, Filippo and Zhang, Francis and Lohmann, Fred von and Sulit, Freddie and Goh, Gabriel and Oden, Gene and Salmon, Geoff and Starace, Giulio and Brockman, Greg and Salman, Hadi and Bao, Haiming and Hu, Haitang and Wong, Hannah and Wang, Haoyu and Schmidt, Heather and Whitney, Heather and Jun, Heewoo and Kirchner, Hendrik and Pinto, Henrique Ponde de Oliveira and Ren, Hongyu and Chang, Huiwen and Chung, Hyung Won and Kivlichan, Ian and O'Connell, Ian and O'Connell, Ian and Osband, Ian and Silber, Ian and Sohl, Ian and Okuyucu, Ibrahim and Lan, Ikai and Kostrikov, Ilya and Sutskever, Ilya and Kanitscheider, Ingmar and Gulrajani, Ishaan and Coxon, Jacob and Menick, Jacob and Pachocki, Jakub and Aung, James and Betker, James and Crooks, James and Lennon, James and Kiros, Jamie and Leike, Jan and Park, Jane and Kwon, Jason and Phang, Jason and Teplitz, Jason and Wei, Jason and Wolfe, Jason and Chen, Jay and Harris, Jeff and Varavva, Jenia and Lee, Jessica Gan and Shieh, Jessica and Lin, Ji and Yu, Jiahui and Weng, Jiayi and Tang, Jie and Yu, Jieqi and Jang, Joanne and Candela, Joaquin Quinonero and Beutler, Joe and Landers, Joe and Parish, Joel and Heidecke, Johannes and Schulman, John and Lachman, Jonathan and McKay, Jonathan and Uesato, Jonathan and Ward, Jonathan and Kim, Jong Wook and Huizinga, Joost and Sitkin, Jordan and Kraaijeveld, Jos and Gross, Josh and Kaplan, Josh and Snyder, Josh and Achiam, Joshua and Jiao, Joy and Lee, Joyce and Zhuang, Juntang and Harriman, Justyn and Fricke, Kai and Hayashi, Kai and Singhal, Karan and Shi, Katy and Karthik, Kavin and Wood, Kayla and Rimbach, Kendra and Hsu, Kenny and Nguyen, Kenny and Gu-Lemberg, Keren and Button, Kevin and Liu, Kevin and Howe, Kiel and Muthukumar, Krithika and Luther, Kyle and Ahmad, Lama and Kai, Larry and Itow, Lauren and Workman, Lauren and Pathak, Leher and Chen, Leo and Jing, Li and Guy, Lia and Fedus, Liam and Zhou, Liang and Mamitsuka, Lien and Weng, Lilian and McCallum, Lindsay and Held, Lindsey and Ouyang, Long and Feuvrier, Louis and Zhang, Lu and Kondraciuk, Lukas and Kaiser, Lukasz and Hewitt, Luke and Metz, Luke and Doshi, Lyric and Aflak, Mada and Simens, Maddie and Boyd, Madelaine and Thompson, Madeleine and Dukhan, Marat and Chen, Mark and Gray, Mark and Hudnall, Mark and Zhang, Marvin and Aljubeh, Marwan and Litwin, Mateusz and Zeng, Matthew and Johnson, Max and Shetty, Maya and Gupta, Mayank and Shah, Meghan and Yatbaz, Mehmet and Yang, Meng Jia and Zhong, Mengchao and Glaese, Mia and Chen, Mianna and Janner, Michael and Lampe, Michael and Petrov, Michael and Wu, Michael and Wang, Michele and Fradin, Michelle and Pokrass, Michelle and Castro, Miguel and Castro, Miguel Oom Temudo de and Pavlov, Mikhail and Brundage, Miles and Wang, Miles and Khan, Minal and Murati, Mira and Bavarian, Mo and Lin, Molly and Yesildal, Murat and Soto, Nacho and Gimelshein, Natalia and Cone, Natalie and Staudacher, Natalie and Summers, Natalie and LaFontaine, Natan and Chowdhury, Neil and Ryder, Nick and Stathas, Nick and Turley, Nick and Tezak, Nik and Felix, Niko and Kudige, Nithanth and Keskar, Nitish and Deutsch, Noah and Bundick, Noel and Puckett, Nora and Nachum, Ofir and Okelola, Ola and Boiko, Oleg and Murk, Oleg and Jaffe, Oliver and Watkins, Olivia and Godement, Olivier and Campbell-Moore, Owen and Chao, Patrick and McMillan, Paul and Belov, Pavel and Su, Peng and Bak, Peter and Bakkum, Peter and Deng, Peter and Dolan, Peter and Hoeschele, Peter and Welinder, Peter and Tillet, Phil and Pronin, Philip and Tillet, Philippe and Dhariwal, Prafulla and Yuan, Qiming and Dias, Rachel and Lim, Rachel and Arora, Rahul and Troll, Rajan and Lin, Randall and Lopes, Rapha Gontijo and Puri, Raul and Miyara, Reah and Leike, Reimar and Gaubert, Renaud and Zamani, Reza and Wang, Ricky and Donnelly, Rob and Honsby, Rob and Smith, Rocky and Sahai, Rohan and Ramchandani, Rohit and Huet, Romain and Carmichael, Rory and Zellers, Rowan and Chen, Roy and Chen, Ruby and Nigmatullin, Ruslan and Cheu, Ryan and Jain, Saachi and Altman, Sam and Schoenholz, Sam and Toizer, Sam and Miserendino, Samuel and Agarwal, Sandhini and Culver, Sara and Ethersmith, Scott and Gray, Scott and Grove, Sean and Metzger, Sean and Hermani, Shamez and Jain, Shantanu and Zhao, Shengjia and Wu, Sherwin and Jomoto, Shino and Wu, Shirong and Shuaiqi and Xia and Phene, Sonia and Papay, Spencer and Narayanan, Srinivas and Coffey, Steve and Lee, Steve and Hall, Stewart and Balaji, Suchir and Broda, Tal and Stramer, Tal and Xu, Tao and Gogineni, Tarun and Christianson, Taya and Sanders, Ted and Patwardhan, Tejal and Cunninghman, Thomas and Degry, Thomas and Dimson, Thomas and Raoux, Thomas and Shadwell, Thomas and Zheng, Tianhao and Underwood, Todd and Markov, Todor and Sherbakov, Toki and Rubin, Tom and Stasi, Tom and Kaftan, Tomer and Heywood, Tristan and Peterson, Troy and Walters, Tyce and Eloundou, Tyna and Qi, Valerie and Moeller, Veit and Monaco, Vinnie and Kuo, Vishal and Fomenko, Vlad and Chang, Wayne and Zheng, Weiyi and Zhou, Wenda and Manassra, Wesam and Sheu, Will and Zaremba, Wojciech and Patil, Yash and Qian, Yilei and Kim, Yongjik and Cheng, Youlong and Zhang, Yu and He, Yuchen and Zhang, Yuchen and Jin, Yujia and Dai, Yunxing and Malkov, Yury},
  year = {2024},
  doi = {10.48550/arxiv.2410.21276},
  eprint = {2410.21276},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2410.21276}
}
```

- [next → DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models](https://md.research.elamur.ai/free/en/papers/shao-2024-deepseekmath-pushing-limits-mathematical-reasoning/)
- [↑ back to papers of 2024](https://md.research.elamur.ai/free/en/papers/2024/)

## References

59 references, 6 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. T. Stivers, N. J. Enfield, P. Brown, C. Englert, M. Hayashi, T. Heinemann, G. Hoymann, F. Rossano, J. P. de Ruiter, K. E. Yoon, and S. C. Levinson, “Universals and cultural variation in turn-taking in conversation,” Proceedings of the National Academy of Sciences, vol. 106, no. 26, pp. 10587–10592,… [find](https://scholar.google.com/scholar?q=T%2E+Stivers%2C+N%2E+J%2E+Enfield%2C+P%2E+Brown%2C+C%2E+Englert%2C+M%2E+Hayashi%2C+T%2E+Heinemann%2C+G%2E+Hoymann%2C+F%2E+Rossano%2C+J%2E+P%2E+de+Ruiter%2C+K%2E+E%2E+Yoon%2C+and+S%2E+C%2E+Levinson%2C+%E2%80%9CUniversals+and+cultural+variation+in+turn%2Dtaking+in)
2. The White House, “Fact sheet: Biden-harris administration secures voluntary commitments from leading artificial intelligence companies to manage the risks posed by ai,” 2023. [find](https://scholar.google.com/scholar?q=The+White+House%2C+%E2%80%9CFact+sheet%3A+Biden%2Dharris+administration+secures+voluntary+commitments+from+leading+artificial+intelligence+companies+to+manage+the+risks+posed+by+ai%2C%E2%80%9D+2023%2E)
3. OpenAI, “Openai preparedness framework openai-preparedness-framework-beta.pdf. beta,” 2023. [https://cdn.openai.com/](https://cdn.openai.com/) [↗](https://cdn.openai.com/)
4. Shutterstock, “Shutterstock press release,” 2023. [find](https://scholar.google.com/scholar?q=Shutterstock%2C+%E2%80%9CShutterstock+press+release%2C%E2%80%9D+2023%2E)
5. OpenAI, “Gpt-4 technical report,” 2023. [find](https://scholar.google.com/scholar?q=OpenAI%2C+%E2%80%9CGpt%2D4+technical+report%2C%E2%80%9D+2023%2E) [our card](https://md.research.elamur.ai/free/en/papers/openai-2023-gpt-4-technical-report/)
6. OpenAI, “Gpt-4v(ision) system card.” [https://openai.com/index/gpt-4v-system-card/](https://openai.com/index/gpt-4v-system-card/), 2023. Accessed: 2024-07-22. [↗](https://openai.com/index/gpt-4v-system-card/)
7. OpenAI, “Navigating the challenges and opportunities of synthetic voices.” [https://openai.com/index/](https://openai.com/index/) navigating-the-challenges-and-opportunities-of-synthetic-voices/, 2024. Accessed: 2024-07-22. [↗](https://openai.com/index/)
8. K. T. Mai, S. Bray, T. Davies, and L. D. Griffin, “Warning: Humans cannot reliably detect speech deepfakes,” PLoS One, vol. 18, p. e0285333, Aug. 2023. [find](https://scholar.google.com/scholar?q=K%2E+T%2E+Mai%2C+S%2E+Bray%2C+T%2E+Davies%2C+and+L%2E+D%2E+Griffin%2C+%E2%80%9CWarning%3A+Humans+cannot+reliably+detect+speech+deepfakes%2C%E2%80%9D+PLoS+One%2C+vol%2E+18%2C+p%2E+e0285333%2C+Aug%2E+2023%2E)
9. M. Mori, K. F. MacDorman, and N. Kageki, “The uncanny valley [from the field],” IEEE Robotics & automation magazine, vol. 19, no. 2, pp. 98–100, 2012. [find](https://scholar.google.com/scholar?q=M%2E+Mori%2C+K%2E+F%2E+MacDorman%2C+and+N%2E+Kageki%2C+%E2%80%9CThe+uncanny+valley+%5Bfrom+the+field%5D%2C%E2%80%9D+IEEE+Robotics+%26+automation+magazine%2C+vol%2E+19%2C+no%2E+2%2C+pp%2E+98%E2%80%93100%2C+2012%2E)
10. OpenAI, “How the voices for chatgpt were chosen,” 2024. [find](https://scholar.google.com/scholar?q=OpenAI%2C+%E2%80%9CHow+the+voices+for+chatgpt+were+chosen%2C%E2%80%9D+2024%2E)
11. I. Solaiman, Z. Talat, W. Agnew, L. Ahmad, D. Baker, S. L. Blodgett, C. Chen, H. D. I. au2, J. Dodge, I. Duan, E. Evans, F. Friedrich, A. Ghosh, U. Gohar, S. Hooker, Y. Jernite, R. Kalluri, A. Lusoli, A. Leidinger, M. Lin, X. Lin, S. Luccioni, J. Mickel, M. Mitchell, J. Newman, A. Ovalle, M.-T. Png,… [find](https://scholar.google.com/scholar?q=I%2E+Solaiman%2C+Z%2E+Talat%2C+W%2E+Agnew%2C+L%2E+Ahmad%2C+D%2E+Baker%2C+S%2E+L%2E+Blodgett%2C+C%2E+Chen%2C+H%2E+D%2E+I%2E+au2%2C+J%2E+Dodge%2C+I%2E+Duan%2C+E%2E+Evans%2C+F%2E+Friedrich%2C+A%2E+Ghosh%2C+U%2E+Gohar%2C+S%2E+Hooker%2C+Y%2E+Jernite%2C+R%2E+Kalluri%2C+A%2E+Lusoli%2C)
12. R. Shelby, S. Rismani, K. Henne, A. Moon, N. Rostamzadeh, P. Nicholas, N. Yilla, J. Gallegos, A. Smart, E. Garcia, and G. Virk, “Sociotechnical harms of algorithmic systems: Scoping a taxonomy for harm reduction,” 2023. [find](https://scholar.google.com/scholar?q=R%2E+Shelby%2C+S%2E+Rismani%2C+K%2E+Henne%2C+A%2E+Moon%2C+N%2E+Rostamzadeh%2C+P%2E+Nicholas%2C+N%2E+Yilla%2C+J%2E+Gallegos%2C+A%2E+Smart%2C+E%2E+Garcia%2C+and+G%2E+Virk%2C+%E2%80%9CSociotechnical+harms+of+algorithmic+systems%3A+Scoping+a+taxonomy+for+har)
13. S. L. Blodgett, Q. V. Liao, A. Olteanu, R. Mihalcea, M. Muller, M. K. Scheuerman, C. Tan, and Q. Yang, “Responsible language technologies: Foreseeing and mitigating harms,” in Extended Abstracts of the 2022 CHI Conference on Human Factors in Computing Systems, CHI EA ’22, (New York, NY, USA), Associ… [find](https://scholar.google.com/scholar?q=S%2E+L%2E+Blodgett%2C+Q%2E+V%2E+Liao%2C+A%2E+Olteanu%2C+R%2E+Mihalcea%2C+M%2E+Muller%2C+M%2E+K%2E+Scheuerman%2C+C%2E+Tan%2C+and+Q%2E+Yang%2C+%E2%80%9CResponsible+language+technologies%3A+Foreseeing+and+mitigating+harms%2C%E2%80%9D+in+Extended+Abstracts+of+th)
14. H. Suresh and J. Guttag, “A framework for understanding sources of harm throughout the machine learning life cycle,” in Equity and Access in Algorithms, Mechanisms, and Optimization, EAAMO ’21, ACM, Oct. 2021. [find](https://scholar.google.com/scholar?q=H%2E+Suresh+and+J%2E+Guttag%2C+%E2%80%9CA+framework+for+understanding+sources+of+harm+throughout+the+machine+learning+life+cycle%2C%E2%80%9D+in+Equity+and+Access+in+Algorithms%2C+Mechanisms%2C+and+Optimization%2C+EAAMO+%E2%80%9921%2C+ACM%2C+O)
15. S. Shahriar, S. Allana, S. M. Hazratifard, and R. Dara, “A survey of privacy risks and mitigation strategies in the artificial intelligence life cycle,” IEEE Access, vol. 11, pp. 61829–61854, 2023. [find](https://scholar.google.com/scholar?q=S%2E+Shahriar%2C+S%2E+Allana%2C+S%2E+M%2E+Hazratifard%2C+and+R%2E+Dara%2C+%E2%80%9CA+survey+of+privacy+risks+and+mitigation+strategies+in+the+artificial+intelligence+life+cycle%2C%E2%80%9D+IEEE+Access%2C+vol%2E+11%2C+pp%2E+61829%E2%80%9361854%2C+2023%2E)
16. OpenAI, “Moderation overview,” 2024. [find](https://scholar.google.com/scholar?q=OpenAI%2C+%E2%80%9CModeration+overview%2C%E2%80%9D+2024%2E)
17. A. Tamkin, M. Brundage, J. Clark, and D. Ganguli, “Understanding the capabilities, limitations, and societal impact of large language models,” 2021. [find](https://scholar.google.com/scholar?q=A%2E+Tamkin%2C+M%2E+Brundage%2C+J%2E+Clark%2C+and+D%2E+Ganguli%2C+%E2%80%9CUnderstanding+the+capabilities%2C+limitations%2C+and+societal+impact+of+large+language+models%2C%E2%80%9D+2021%2E)
18. B. Buchanan, A. Lohn, M. Musser, and K. Sedova, “Truth, lies, and automation: How language models could change disinformation,” May 2021. [find](https://scholar.google.com/scholar?q=B%2E+Buchanan%2C+A%2E+Lohn%2C+M%2E+Musser%2C+and+K%2E+Sedova%2C+%E2%80%9CTruth%2C+lies%2C+and+automation%3A+How+language+models+could+change+disinformation%2C%E2%80%9D+May+2021%2E)
19. OpenAI, “Openai usage policies,” 2023. [https://openai.com/policies/usage-policies/](https://openai.com/policies/usage-policies/). [↗](https://openai.com/policies/usage-policies/)
20. OpenAI, “Building an early warning system for llm-aided biological threat creation,” 2024. [https://openai.com/](https://openai.com/) index/building-an-early-warning-system-for-llm-aided-biological-threat-creation/. [↗](https://openai.com/)
21. Deloitte, “Deloitte acquires gryphon scientific business to expand security science and public health capabilities,” 2024. [https://www2.deloitte.com/us/en/pages/about-deloitte/articles/press-releases/](https://www2.deloitte.com/us/en/pages/about-deloitte/articles/press-releases/) deloitte-acquires-gryphon-scientific-business-to-expand-security-science-and-public-health-capabili… [↗](https://www2.deloitte.com/us/en/pages/about-deloitte/articles/press-releases/)
22. L. Weidinger, M. Rauh, N. Marchal, A. Manzini, L. A. Hendricks, J. Mateos-Garcia, S. Bergman, J. Kay, C. Griffin, B. Bariach, I. Gabriel, V. Rieser, and W. Isaac, “Sociotechnical safety evaluation of generative ai systems,” 2023. [find](https://scholar.google.com/scholar?q=L%2E+Weidinger%2C+M%2E+Rauh%2C+N%2E+Marchal%2C+A%2E+Manzini%2C+L%2E+A%2E+Hendricks%2C+J%2E+Mateos%2DGarcia%2C+S%2E+Bergman%2C+J%2E+Kay%2C+C%2E+Griffin%2C+B%2E+Bariach%2C+I%2E+Gabriel%2C+V%2E+Rieser%2C+and+W%2E+Isaac%2C+%E2%80%9CSociotechnical+safety+evaluation+of+)
23. A. Tamkin, A. Askell, L. Lovitt, E. Durmus, N. Joseph, S. Kravec, K. Nguyen, J. Kaplan, and D. Ganguli, “Evaluating and mitigating discrimination in language model decisions,” 2023. [find](https://scholar.google.com/scholar?q=A%2E+Tamkin%2C+A%2E+Askell%2C+L%2E+Lovitt%2C+E%2E+Durmus%2C+N%2E+Joseph%2C+S%2E+Kravec%2C+K%2E+Nguyen%2C+J%2E+Kaplan%2C+and+D%2E+Ganguli%2C+%E2%80%9CEvaluating+and+mitigating+discrimination+in+language+model+decisions%2C%E2%80%9D+2023%2E)
24. J. A. Goldstein, G. Sastry, M. Musser, R. DiResta, M. Gentzel, and K. Sedova, “Generative language models and automated influence operations: Emerging threats and potential mitigations,” 2023. [find](https://scholar.google.com/scholar?q=J%2E+A%2E+Goldstein%2C+G%2E+Sastry%2C+M%2E+Musser%2C+R%2E+DiResta%2C+M%2E+Gentzel%2C+and+K%2E+Sedova%2C+%E2%80%9CGenerative+language+models+and+automated+influence+operations%3A+Emerging+threats+and+potential+mitigations%2C%E2%80%9D+2023%2E)
25. I. Pentina, T. Hancock, and T. Xie, “Exploring relationship development with social chatbots: A mixed-method study of replika,” Computers in Human Behavior, vol. 140, p. 107600, 2023. [find](https://scholar.google.com/scholar?q=I%2E+Pentina%2C+T%2E+Hancock%2C+and+T%2E+Xie%2C+%E2%80%9CExploring+relationship+development+with+social+chatbots%3A+A+mixed%2Dmethod+study+of+replika%2C%E2%80%9D+Computers+in+Human+Behavior%2C+vol%2E+140%2C+p%2E+107600%2C+2023%2E)
26. Y. Bengio, G. Hinton, A. Yao, D. Song, P. Abbeel, T. Darrell, Y. N. Harari, Y.-Q. Zhang, L. Xue, S. Shalev-Shwartz, G. Hadfield, J. Clune, T. Maharaj, F. Hutter, A. G. Baydin, S. McIlraith, Q. Gao, A. Acharya, D. Krueger, A. Dragan, P. Torr, S. Russell, D. Kahneman, J. Brauner, and S. Mindermann, “M… [find](https://scholar.google.com/scholar?q=Y%2E+Bengio%2C+G%2E+Hinton%2C+A%2E+Yao%2C+D%2E+Song%2C+P%2E+Abbeel%2C+T%2E+Darrell%2C+Y%2E+N%2E+Harari%2C+Y%2E%2DQ%2E+Zhang%2C+L%2E+Xue%2C+S%2E+Shalev%2DShwartz%2C+G%2E+Hadfield%2C+J%2E+Clune%2C+T%2E+Maharaj%2C+F%2E+Hutter%2C+A%2E+G%2E+Baydin%2C+S%2E+McIlraith%2C+Q%2E+Gao%2C+A%2E)
27. S. B. Johnson, J. R. Clark, M. C. Luetke, N. M. Butala, A. T. Pearson, J. M. Shapiro, D. M. Aleman, J. M. Lee, M. M. Beil, C. V. Winkle, M. C. Boudreaux, R. C. D’Cunha, H. J. Krouse, and C. Li, “Chatgpt in medical education: a workshop-based large language model-powered intervention for evidence-bas… [find](https://scholar.google.com/scholar?q=S%2E+B%2E+Johnson%2C+J%2E+R%2E+Clark%2C+M%2E+C%2E+Luetke%2C+N%2E+M%2E+Butala%2C+A%2E+T%2E+Pearson%2C+J%2E+M%2E+Shapiro%2C+D%2E+M%2E+Aleman%2C+J%2E+M%2E+Lee%2C+M%2E+M%2E+Beil%2C+C%2E+V%2E+Winkle%2C+M%2E+C%2E+Boudreaux%2C+R%2E+C%2E+D%E2%80%99Cunha%2C+H%2E+J%2E+Krouse%2C+and+C%2E+Li%2C+%E2%80%9CChatg)
28. K. Kavukcuoglu, “Real-world challenges for agi,” Nov 2021. [find](https://scholar.google.com/scholar?q=K%2E+Kavukcuoglu%2C+%E2%80%9CReal%2Dworld+challenges+for+agi%2C%E2%80%9D+Nov+2021%2E)
29. S. Altman, “Planning for agi and beyond,” OpenAI, 2023. [find](https://scholar.google.com/scholar?q=S%2E+Altman%2C+%E2%80%9CPlanning+for+agi+and+beyond%2C%E2%80%9D+OpenAI%2C+2023%2E)
30. T. Eloundou, S. Manning, P. Mishkin, and D. Rock, “Gpts are gpts: An early look at the labor market impact potential of large language models,” arXiv preprint arXiv:2303.10130, 2023. — [arXiv:2303.10130](https://arxiv.org/abs/2303.10130) [find](https://scholar.google.com/scholar?q=T%2E+Eloundou%2C+S%2E+Manning%2C+P%2E+Mishkin%2C+and+D%2E+Rock%2C+%E2%80%9CGpts+are+gpts%3A+An+early+look+at+the+labor+market+impact+potential+of+large+language+models%2C%E2%80%9D+arXiv+preprint+arXiv%3A2303%2E10130%2C+2023%2E+%E2%80%94+%5BarXiv%3A2303%2E101)
31. L. Weidinger, M. Rauh, N. Marchal, A. Manzini, L. A. Hendricks, J. Mateos-Garcia, S. Bergman, J. Kay, C. Griffin, B. Bariach, et al., “Sociotechnical safety evaluation of generative ai systems,” arXiv preprint arXiv:2310.11986, 2023. — [arXiv:2310.11986](https://arxiv.org/abs/2310.11986) [find](https://scholar.google.com/scholar?q=L%2E+Weidinger%2C+M%2E+Rauh%2C+N%2E+Marchal%2C+A%2E+Manzini%2C+L%2E+A%2E+Hendricks%2C+J%2E+Mateos%2DGarcia%2C+S%2E+Bergman%2C+J%2E+Kay%2C+C%2E+Griffin%2C+B%2E+Bariach%2C+et+al%2E%2C+%E2%80%9CSociotechnical+safety+evaluation+of+generative+ai+systems%2C%E2%80%9D+arXiv)
32. S. Cox, M. Hammerling, J. Lála, J. Laurent, S. Rodriques, M. Rubashkin, and A. White, “Wikicrow: Automating synthesis of human scientific knowledge,” Future House, 2023. [find](https://scholar.google.com/scholar?q=S%2E+Cox%2C+M%2E+Hammerling%2C+J%2E+L%C3%A1la%2C+J%2E+Laurent%2C+S%2E+Rodriques%2C+M%2E+Rubashkin%2C+and+A%2E+White%2C+%E2%80%9CWikicrow%3A+Automating+synthesis+of+human+scientific+knowledge%2C%E2%80%9D+Future+House%2C+2023%2E)
33. S. A. Athaluri, S. V. Manthena, V. S. R. K. M. Kesapragada, V. Yarlagadda, T. Dave, and R. T. S. Duddumpudi, “Exploring the boundaries of reality: Investigating the phenomenon of artificial intelligence hallucination in scientific writing through chatgpt references,” Cureus, vol. 15, no. 4, p. e3743… [find](https://scholar.google.com/scholar?q=S%2E+A%2E+Athaluri%2C+S%2E+V%2E+Manthena%2C+V%2E+S%2E+R%2E+K%2E+M%2E+Kesapragada%2C+V%2E+Yarlagadda%2C+T%2E+Dave%2C+and+R%2E+T%2E+S%2E+Duddumpudi%2C+%E2%80%9CExploring+the+boundaries+of+reality%3A+Investigating+the+phenomenon+of+artificial+intelligen)
34. Z. Li, “The dark side of chatgpt: Legal and ethical challenges from stochastic parrots and hallucination,” 2023. [find](https://scholar.google.com/scholar?q=Z%2E+Li%2C+%E2%80%9CThe+dark+side+of+chatgpt%3A+Legal+and+ethical+challenges+from+stochastic+parrots+and+hallucination%2C%E2%80%9D+2023%2E)
35. M. Dubiel, A. Sergeeva, and L. A. Leiva, “Impact of voice fidelity on decision making: A potential dark pattern?,” 2024. [find](https://scholar.google.com/scholar?q=M%2E+Dubiel%2C+A%2E+Sergeeva%2C+and+L%2E+A%2E+Leiva%2C+%E2%80%9CImpact+of+voice+fidelity+on+decision+making%3A+A+potential+dark+pattern%3F%2C%E2%80%9D+2024%2E)
36. B. Waber, M. Williams, J. S. Carroll, and A. S. Pentland, “A voice is worth a thousand words: The implications of the micro-coding of social signals in speech for trust research,” in Handbook of Research Methods on Trust (G. M. Fergus Lyon and M. N. Saunders, eds.), ch. 23, p. 320, New York: Edward… [find](https://scholar.google.com/scholar?q=B%2E+Waber%2C+M%2E+Williams%2C+J%2E+S%2E+Carroll%2C+and+A%2E+S%2E+Pentland%2C+%E2%80%9CA+voice+is+worth+a+thousand+words%3A+The+implications+of+the+micro%2Dcoding+of+social+signals+in+speech+for+trust+research%2C%E2%80%9D+in+Handbook+of+Resea)
37. I. Pentina, B. Guo, and W. P. Fan, “Friend, mentor, lover: Does chatbot engagement lead to psychological dependence?,” Journal of Service Management, 2023. [find](https://scholar.google.com/scholar?q=I%2E+Pentina%2C+B%2E+Guo%2C+and+W%2E+P%2E+Fan%2C+%E2%80%9CFriend%2C+mentor%2C+lover%3A+Does+chatbot+engagement+lead+to+psychological+dependence%3F%2C%E2%80%9D+Journal+of+Service+Management%2C+2023%2E)
38. H. Nori, N. King, S. M. McKinney, D. Carignan, and E. Horvitz, “Capabilities of gpt-4 on medical challenge problems,” arXiv preprint arXiv:2303.13375, 2023. [find](https://scholar.google.com/scholar?q=H%2E+Nori%2C+N%2E+King%2C+S%2E+M%2E+McKinney%2C+D%2E+Carignan%2C+and+E%2E+Horvitz%2C+%E2%80%9CCapabilities+of+gpt%2D4+on+medical+challenge+problems%2C%E2%80%9D+arXiv+preprint+arXiv%3A2303%2E13375%2C+2023%2E) [our card](https://md.research.elamur.ai/free/en/papers/nori-2023-capabilities-gpt-4-medical-challenge/)
39. H. Nori, Y. T. Lee, S. Zhang, D. Carignan, R. Edgar, N. Fusi, N. King, J. Larson, Y. Li, W. Liu, et al., “Can generalist foundation models outcompete special-purpose tuning? case study in medicine,” arXiv preprint arXiv:2311.16452, 2023. [find](https://scholar.google.com/scholar?q=H%2E+Nori%2C+Y%2E+T%2E+Lee%2C+S%2E+Zhang%2C+D%2E+Carignan%2C+R%2E+Edgar%2C+N%2E+Fusi%2C+N%2E+King%2C+J%2E+Larson%2C+Y%2E+Li%2C+W%2E+Liu%2C+et+al%2E%2C+%E2%80%9CCan+generalist+foundation+models+outcompete+special%2Dpurpose+tuning%3F+case+study+in+medicine%2C%E2%80%9D+a)
40. K. Singhal, S. Azizi, T. Tu, S. S. Mahdavi, J. Wei, H. W. Chung, N. Scales, A. Tanwani, H. Cole-Lewis, S. Pfohl, P. Payne, M. Seneviratne, P. Gamble, C. Kelly, N. Scharli, A. Chowdhery, P. Mansfield, B. A. y Arcas, D. Webster, G. S. Corrado, Y. Matias, K. Chou, J. Gottweis, N. Tomasev, Y. Liu, A. Ra… [find](https://scholar.google.com/scholar?q=K%2E+Singhal%2C+S%2E+Azizi%2C+T%2E+Tu%2C+S%2E+S%2E+Mahdavi%2C+J%2E+Wei%2C+H%2E+W%2E+Chung%2C+N%2E+Scales%2C+A%2E+Tanwani%2C+H%2E+Cole%2DLewis%2C+S%2E+Pfohl%2C+P%2E+Payne%2C+M%2E+Seneviratne%2C+P%2E+Gamble%2C+C%2E+Kelly%2C+N%2E+Scharli%2C+A%2E+Chowdhery%2C+P%2E+Mansfield%2C+) [our card](https://md.research.elamur.ai/free/en/papers/singhal-2022-large-language-models-encode-clinical/)
41. K. Singhal, T. Tu, J. Gottweis, R. Sayres, E. Wulczyn, L. Hou, K. Clark, S. Pfohl, H. Cole-Lewis, D. Neal, M. Schaekermann, A. Wang, M. Amin, S. Lachgar, P. Mansfield, S. Prakash, B. Green, E. Dominowska, B. A. y Arcas, N. Tomasev, Y. Liu, R. Wong, C. Semturs, S. S. Mahdavi, J. Barral, D. Webster, G… [find](https://scholar.google.com/scholar?q=K%2E+Singhal%2C+T%2E+Tu%2C+J%2E+Gottweis%2C+R%2E+Sayres%2C+E%2E+Wulczyn%2C+L%2E+Hou%2C+K%2E+Clark%2C+S%2E+Pfohl%2C+H%2E+Cole%2DLewis%2C+D%2E+Neal%2C+M%2E+Schaekermann%2C+A%2E+Wang%2C+M%2E+Amin%2C+S%2E+Lachgar%2C+P%2E+Mansfield%2C+S%2E+Prakash%2C+B%2E+Green%2C+E%2E+Dominow) [our card](https://md.research.elamur.ai/free/en/papers/singhal-2023-towards-expert-level-medical-question/)
42. K. Saab, T. Tu, W.-H. Weng, R. Tanno, D. Stutz, E. Wulczyn, F. Zhang, T. Strother, C. Park, E. Vedadi, J. Z. Chaves, S.-Y. Hu, M. Schaekermann, A. Kamath, Y. Cheng, D. G. T. Barrett, C. Cheung, B. Mustafa, A. Palepu, D. McDuff, L. Hou, T. Golany, L. Liu, J. baptiste Alayrac, N. Houlsby, N. Tomasev,… [find](https://scholar.google.com/scholar?q=K%2E+Saab%2C+T%2E+Tu%2C+W%2E%2DH%2E+Weng%2C+R%2E+Tanno%2C+D%2E+Stutz%2C+E%2E+Wulczyn%2C+F%2E+Zhang%2C+T%2E+Strother%2C+C%2E+Park%2C+E%2E+Vedadi%2C+J%2E+Z%2E+Chaves%2C+S%2E%2DY%2E+Hu%2C+M%2E+Schaekermann%2C+A%2E+Kamath%2C+Y%2E+Cheng%2C+D%2E+G%2E+T%2E+Barrett%2C+C%2E+Cheung%2C+B%2E+Mus)
43. Epic Systems Corporation, “Epic and microsoft bring gpt-4 to ehrs,” Epic, 2023. [find](https://scholar.google.com/scholar?q=Epic+Systems+Corporation%2C+%E2%80%9CEpic+and+microsoft+bring+gpt%2D4+to+ehrs%2C%E2%80%9D+Epic%2C+2023%2E)
44. D. Van Veen, C. Van Uden, L. Blankemeier, J.-B. Delbrouck, A. Aali, C. Bluethgen, A. Pareek, M. Polacin, E. P. Reis, A. Seehofnerová, et al., “Adapted large language models can outperform medical experts in clinical text summarization,” Nature medicine, vol. 30, no. 4, pp. 1134–1142, 2024. [find](https://scholar.google.com/scholar?q=D%2E+Van+Veen%2C+C%2E+Van+Uden%2C+L%2E+Blankemeier%2C+J%2E%2DB%2E+Delbrouck%2C+A%2E+Aali%2C+C%2E+Bluethgen%2C+A%2E+Pareek%2C+M%2E+Polacin%2C+E%2E+P%2E+Reis%2C+A%2E+Seehofnerov%C3%A1%2C+et+al%2E%2C+%E2%80%9CAdapted+large+language+models+can+outperform+medical+expe) [our card](https://md.research.elamur.ai/free/en/papers/veen-2023-adapted-large-language-models-outperform/)
45. Epic, “Epic and microsoft bring gpt-4 to ehrs,” 2023. [find](https://scholar.google.com/scholar?q=Epic%2C+%E2%80%9CEpic+and+microsoft+bring+gpt%2D4+to+ehrs%2C%E2%80%9D+2023%2E)
46. P. Garcia, S. P. Ma, S. Shah, M. Smith, Y. Jeong, A. Devon-Sand, M. Tai-Seale, K. Takazawa, D. Clutter, K. Vogt, C. Lugtu, M. Rojo, S. Lin, T. Shanafelt, M. A. Pfeffer, and C. Sharp, “Artificial Intelligence–Generated Draft Replies to Patient Inbox Messages,” JAMA Network Open, vol. 7, pp. e243201–e… [find](https://scholar.google.com/scholar?q=P%2E+Garcia%2C+S%2E+P%2E+Ma%2C+S%2E+Shah%2C+M%2E+Smith%2C+Y%2E+Jeong%2C+A%2E+Devon%2DSand%2C+M%2E+Tai%2DSeale%2C+K%2E+Takazawa%2C+D%2E+Clutter%2C+K%2E+Vogt%2C+C%2E+Lugtu%2C+M%2E+Rojo%2C+S%2E+Lin%2C+T%2E+Shanafelt%2C+M%2E+A%2E+Pfeffer%2C+and+C%2E+Sharp%2C+%E2%80%9CArtificial+Intel)
47. OpenAI, “Paradigm: Improving patient access to clinical trials.” [https://openai.com/index/paradigm/](https://openai.com/index/paradigm/), 2024. Accessed: 2024-08-07. [↗](https://openai.com/index/paradigm/)
48. M. Hutson, “How ai is being used to accelerate clinical trials,” Nature, vol. 627, pp. S2–S5, 2024. [find](https://scholar.google.com/scholar?q=M%2E+Hutson%2C+%E2%80%9CHow+ai+is+being+used+to+accelerate+clinical+trials%2C%E2%80%9D+Nature%2C+vol%2E+627%2C+pp%2E+S2%E2%80%93S5%2C+2024%2E)
49. OpenAI, “Using gpt-4o reasoning to transform cancer care.” [https://openai.com/index/color-health/](https://openai.com/index/color-health/), 2024. Accessed: 2024-08-07. [↗](https://openai.com/index/color-health/)
50. J. Varghese and J.-L. Chapiro, “Systematic analysis of chatgpt, google search and llama 2 for clinical decision support tasks,” Nature Communications, vol. 15, no. 1, p. 46411, 2024. Accessed: 2024-08-07. [find](https://scholar.google.com/scholar?q=J%2E+Varghese+and+J%2E%2DL%2E+Chapiro%2C+%E2%80%9CSystematic+analysis+of+chatgpt%2C+google+search+and+llama+2+for+clinical+decision+support+tasks%2C%E2%80%9D+Nature+Communications%2C+vol%2E+15%2C+no%2E+1%2C+p%2E+46411%2C+2024%2E+Accessed%3A+2024%2D08)
51. E. Schmidt, “Ai will transform science.” [https://www.technologyreview.com/2023/07/05/1075865/](https://www.technologyreview.com/2023/07/05/1075865/) eric-schmidt-ai-will-transform-science/, 2023. Accessed: 2024-08-07. [↗](https://www.technologyreview.com/2023/07/05/1075865/)
52. N. Rosenberg, “Science, invention and economic growth,” The Economic Journal, vol. 84, no. 333, pp. 90–108, 1974. [find](https://scholar.google.com/scholar?q=N%2E+Rosenberg%2C+%E2%80%9CScience%2C+invention+and+economic+growth%2C%E2%80%9D+The+Economic+Journal%2C+vol%2E+84%2C+no%2E+333%2C+pp%2E+90%E2%80%93108%2C+1974%2E)
53. R. M. Atlas and M. Dando, “The dual-use dilemma for the life sciences: Perspectives, conundrums, and global solutions,” Biosecurity and Bioterrorism: Biodefense Strategy, Practice, and Science, vol. 4, no. 3, pp. 276–286, 2006. PMID: 16999588. [find](https://scholar.google.com/scholar?q=R%2E+M%2E+Atlas+and+M%2E+Dando%2C+%E2%80%9CThe+dual%2Duse+dilemma+for+the+life+sciences%3A+Perspectives%2C+conundrums%2C+and+global+solutions%2C%E2%80%9D+Biosecurity+and+Bioterrorism%3A+Biodefense+Strategy%2C+Practice%2C+and+Science%2C+vol%2E+4)
54. X. Gu and M. Krenn, “Generation and human-expert evaluation of interesting research ideas using knowledge graphs and large language models,” 2024. [find](https://scholar.google.com/scholar?q=X%2E+Gu+and+M%2E+Krenn%2C+%E2%80%9CGeneration+and+human%2Dexpert+evaluation+of+interesting+research+ideas+using+knowledge+graphs+and+large+language+models%2C%E2%80%9D+2024%2E)
55. A. Ghafarollahi and M. J. Buehler, “Atomagents: Alloy design and discovery through physics-aware multi-modal multi-agent artificial intelligence,” 2024. [find](https://scholar.google.com/scholar?q=A%2E+Ghafarollahi+and+M%2E+J%2E+Buehler%2C+%E2%80%9CAtomagents%3A+Alloy+design+and+discovery+through+physics%2Daware+multi%2Dmodal+multi%2Dagent+artificial+intelligence%2C%E2%80%9D+2024%2E)
56. J. M. Laurent, J. D. Janizek, M. Ruzo, M. M. Hinks, M. J. Hammerling, S. Narayanan, M. Ponnapati, A. D. White, and S. G. Rodriques, “Lab-bench: Measuring capabilities of language models for biology research,” 2024. [find](https://scholar.google.com/scholar?q=J%2E+M%2E+Laurent%2C+J%2E+D%2E+Janizek%2C+M%2E+Ruzo%2C+M%2E+M%2E+Hinks%2C+M%2E+J%2E+Hammerling%2C+S%2E+Narayanan%2C+M%2E+Ponnapati%2C+A%2E+D%2E+White%2C+and+S%2E+G%2E+Rodriques%2C+%E2%80%9CLab%2Dbench%3A+Measuring+capabilities+of+language+models+for+biology+re)
57. H. Cai, X. Cai, J. Chang, S. Li, L. Yao, C. Wang, Z. Gao, H. Wang, Y. Li, M. Lin, S. Yang, J. Wang, M. Xu, J. Huang, F. Xi, J. Zhuang, Y. Yin, Y. Li, C. Chen, Z. Cheng, Z. Zhao, L. Zhang, and G. Ke, “Sciassess: Benchmarking llm proficiency in scientific literature analysis,” 2024. [find](https://scholar.google.com/scholar?q=H%2E+Cai%2C+X%2E+Cai%2C+J%2E+Chang%2C+S%2E+Li%2C+L%2E+Yao%2C+C%2E+Wang%2C+Z%2E+Gao%2C+H%2E+Wang%2C+Y%2E+Li%2C+M%2E+Lin%2C+S%2E+Yang%2C+J%2E+Wang%2C+M%2E+Xu%2C+J%2E+Huang%2C+F%2E+Xi%2C+J%2E+Zhuang%2C+Y%2E+Yin%2C+Y%2E+Li%2C+C%2E+Chen%2C+Z%2E+Cheng%2C+Z%2E+Zhao%2C+L%2E+Zhang%2C+and+G%2E+Ke%2C+%E2%80%9C)
58. P. Clark, I. Cowhey, O. Etzioni, T. Khot, A. Sabharwal, C. Schoenick, and O. Tafjord, “Think you have solved question answering? try arc, the AI2 reasoning challenge,” CoRR, vol. abs/1803.05457, 2018. [find](https://scholar.google.com/scholar?q=P%2E+Clark%2C+I%2E+Cowhey%2C+O%2E+Etzioni%2C+T%2E+Khot%2C+A%2E+Sabharwal%2C+C%2E+Schoenick%2C+and+O%2E+Tafjord%2C+%E2%80%9CThink+you+have+solved+question+answering%3F+try+arc%2C+the+AI2+reasoning+challenge%2C%E2%80%9D+CoRR%2C+vol%2E+abs%2F1803%2E05457%2C+2018%2E)
59. S. Lin, J. Hilton, and O. Evans, “Truthfulqa: vol. abs/2109.07958, 2021. [find](https://scholar.google.com/scholar?q=S%2E+Lin%2C+J%2E+Hilton%2C+and+O%2E+Evans%2C+%E2%80%9CTruthfulqa%3A+vol%2E+abs%2F2109%2E07958%2C+2021%2E) [our card](https://md.research.elamur.ai/free/en/papers/lin-2022-truthfulqa-measuring-how-models-mimic/)