--- title: "A Framework for Automated Measurement of Responsible AI" description: "The work offers a reusable, scalable method for comparing models and mitigations. 4 references, cited by 7 papers here." authors: ["Ahmed Magooda", "Alec Helyar", "Kyle Jackson", "David Sullivan", "Chad Atalla", "Emily Sheng", "Dan Vann", "Richard Edgar", "Hamid Palangi", "Roman Lutz", "Hongliang Kong", "Vincent Yun", "Eslam Kamal", "Federico Zarfati", "Hanna Wallach", "Sarah Bird", "Mei Chen"] year: 2023 venue: "arXiv:2310.17750" source: "https://arxiv.org/abs/2310.17750" doi: "10.48550/arxiv.2310.17750" arxiv: "2310.17750" licence: "CC BY 4.0 (arXiv)" licence_url: "http://creativecommons.org/licenses/by/4.0/" summary_licence: "Elamur Cosmic License" canonical: "https://research.elamur.ai/free/en/papers/magooda-2023-framework-automated-measurement-responsible-ai/" --- [Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/magooda-2023-framework-automated-measurement-responsible-ai/) [home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2023](https://md.research.elamur.ai/free/en/papers/2023/page/14/) / A Framework for Automated Measurement of Responsible AI Harms in Generative AI Applications # A Framework for Automated Measurement of Responsible AI Harms in Generative AI Applications **Authors:** Ahmed Magooda, [Alec Helyar](https://md.research.elamur.ai/free/en/authors/alec-helyar/), Kyle Jackson, David Sullivan, Chad Atalla, [Emily Sheng](https://md.research.elamur.ai/free/en/authors/emily-sheng/), Dan Vann, Richard Edgar, [Hamid Palangi](https://md.research.elamur.ai/free/en/authors/hamid-palangi/), Roman Lutz, Hongliang Kong, Vincent Yun, Eslam Kamal, Federico Zarfati, [Hanna Wallach](https://md.research.elamur.ai/free/en/authors/hanna-wallach/), Sarah Bird and Mei Chen · **Year:** 2023 · **Venue:** arXiv:2310.17750 · **Licence:** [CC BY 4.0 (arXiv)](http://creativecommons.org/licenses/by/4.0/) **Links:** [Publisher's page](https://arxiv.org/abs/2310.17750) · [PDF](https://arxiv.org/pdf/2310.17750) · [DOI 10.48550/arxiv.2310.17750](https://doi.org/10.48550/arxiv.2310.17750) · [arXiv 2310.17750](https://arxiv.org/abs/2310.17750) · [OpenAlex](https://openalex.org/W4388068395) **Citations:** 2 citations (OpenAlex, 2026-10-14) · [7](https://md.research.elamur.ai/free/en/papers/magooda-2023-framework-automated-measurement-responsible-ai/cited-by/) in our library, of the 149,439 papers whose bibliographies we read ## Summary Microsoft researchers address the problem of measuring responsible AI (RAI) harms from LLM-based products quickly and at scale, since manual review cannot keep pace with how fast these systems are deployed. Their framework uses LLMs to test other LLMs and has two stages. In the first, a user-simulating model (LLM_user) follows expert-written persona templates to hold realistic interactions with the black-box system under test (LLM_test), such as chat, question answering or summarization. In the second, GPT-4 annotates the resulting samples against expert annotation guidelines, which define the harm, give examples and specify what counts as a defect. The scores are then aggregated into a defect rate. The authors apply the pipeline to a groundedness case study and compare three recent LLMs on IP leakage, potentially harmful content and jailbreaks. They stress that the results are relative, so a 0% defect rate does not prove a system is safe. They also note that the measurements are only as valid as the expert-built resources behind them. The work offers a reusable, scalable method for comparing models and mitigations. ## Key points - **Two-part architecture.** The first part generates data from Jinja-style templates filled with parameters such as topics or demographic groups. The second part is GPT-4-based annotation that produces both numeric severity scores and written reasoning. - **Defect rate metric.** The defect rate is the share of samples above a severity threshold. For example, on a 1–10 violence scale, any sample scored ≥ 7 could count as a defect. - **Execution environments.** The pipeline can run on a local machine or on Azure Machine Learning (AML) compute. - **Annotation stability.** Repeated LLM annotations of the same sample can disagree. To counter this, a stability factor reruns annotation several times (e.g., 5 or 7) and takes the majority value, at the cost of extra compute. - **Known limitations.** Simulated users may behave unrealistically, and the annotator may miss harms or hallucinate evidence. Human–LLM agreement checks help but are insufficient on their own, so measured harm should be treated as a lower bound. *Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.* ## Institutions - [Microsoft](https://md.research.elamur.ai/free/en/companies/microsoft/) — Company, Redmond, United States · 378 works · 161,630 citations - [Microsoft Research (United Kingdom)](https://md.research.elamur.ai/free/en/companies/microsoft-research-united-kingdom/) — Company, Cambridge, United Kingdom · 11 works · 3,215 citations 📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 18,614 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 17,965 works · [🤝 Social Sciences](https://md.research.elamur.ai/free/en/shelves/social-sciences/) · 1,954 works · [🤝 Safety Research](https://md.research.elamur.ai/free/en/shelves/social-sciences/safety-research/) · 626 works ## Cite Cite as: Ahmed Magooda, Alec Helyar, Kyle Jackson, David Sullivan, Chad Atalla, Emily Sheng, Dan Vann, Richard Edgar, Hamid Palangi, Roman Lutz, Hongliang Kong, Vincent Yun, Eslam Kamal, Federico Zarfati, Hanna Wallach, Sarah Bird and Mei Chen. 2023. A Framework for Automated Measurement of Responsible AI Harms in Generative AI Applications. arXiv:2310.17750. The paper itself, not this page. BibTeX from the library's record of the work. ```bibtex @misc{magooda2023framework, title = {{A Framework for Automated Measurement of Responsible AI Harms in Generative AI Applications}}, author = {Ahmed Magooda, Alec Helyar, Kyle Jackson, David Sullivan, Chad Atalla, Emily Sheng, Dan Vann, Richard Edgar, Hamid Palangi, Roman Lutz, Hongliang Kong, Vincent Yun, Eslam Kamal, Federico Zarfati, Hanna Wallach, Sarah Bird and Mei Chen}, year = {2023}, doi = {10.48550/arxiv.2310.17750}, eprint = {2310.17750}, archivePrefix = {arXiv}, url = {https://doi.org/10.48550/arxiv.2310.17750} } ``` - [← previous Yet Another Model for Arabic Dialect Identification](https://md.research.elamur.ai/free/en/papers/kulkarni-2023-yet-another-model-arabic-dialect/) - [next → Ashaar: Automatic Analysis and Generation of Arabic Poetry Using Deep Learning Approaches](https://md.research.elamur.ai/free/en/papers/alyafeai-2023-ashaar-automatic-analysis-generation-arabic/) - [↑ back to papers of 2023](https://md.research.elamur.ai/free/en/papers/2023/page/14/) ## References 4 references, 3 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet. 1. Sachin Kumar, Vidhisha Balachandran, Lucille Njoo, Antonios Anastasopoulos, and Yulia Tsvetkov. 2023. [Language Generation Models Can Cause Harm: So What Can We Do About It? An Actionable Survey](https://md.research.elamur.ai/free/en/papers/kumar-2023-language-generation-models-can-cause/). arXiv:2210.07700 [cs.CL] 2. OpenAI. 2023. [GPT-4 Technical Report](https://md.research.elamur.ai/free/en/papers/openai-2023-gpt-4-technical-report/). ArXiv abs/2303.08774 (2023). 3. Guilherme Penedo, Quentin Malartic, Daniel Hesslow, Ruxandra Cojocaru, Alessandro Cappelli, Hamza Alobeidli, Baptiste Pannier, Ebtesam Almazrouei, and Julien Launay. 2023. The RefinedWeb dataset for Falcon LLM: outperforming curated corpora with web data, and web data only. arXiv preprint arXiv:2306… [find](https://scholar.google.com/scholar?q=Guilherme+Penedo%2C+Quentin+Malartic%2C+Daniel+Hesslow%2C+Ruxandra+Cojocaru%2C+Alessandro+Cappelli%2C+Hamza+Alobeidli%2C+Baptiste+Pannier%2C+Ebtesam+Almazrouei%2C+and+Julien+Launay%2E+2023%2E+The+RefinedWeb+dataset+for+F) 4. Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al. 2023. [Llama 2: Open foundation and fine-tuned chat models](https://md.research.elamur.ai/free/en/papers/touvron-2023-llama-2-open-foundation-fine/). arXiv preprint arXiv:2307.09288 (2023). *Updated 30 October 2026*