--- title: "Open-Domain Safety Policy Construction — Wu 2026" description: "Content moderation systems depend on domain-specific safety policies, which are expensive to write and keep current. 0.01 F1. 11 references." authors: ["Di Wu", "Siyue Liu", "Zixiang Ji", "Ya-Liang Chang", "Zhe-Yu Liu", "Andrew Pleffer", "Kai-Wei Chang"] year: 2026 venue: "Findings of the Association for Computational Linguistics: EACL 2026" source: "https://aclanthology.org/2026.findings-eacl.262/" doi: "10.18653/v1/2026.findings-eacl.262" licence: "CC BY 4.0 (ACL Anthology)" licence_url: "https://creativecommons.org/licenses/by/4.0/" summary_licence: "Elamur Cosmic License" canonical: "https://research.elamur.ai/free/en/papers/wu-2026-open-domain-safety-policy-construction/" --- [Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/wu-2026-open-domain-safety-policy-construction/) [home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2026](https://md.research.elamur.ai/free/en/papers/2026/page/44/) / Open-Domain Safety Policy Construction # Open-Domain Safety Policy Construction **Authors:** [Di Wu](https://md.research.elamur.ai/free/en/authors/di-wu/), Siyue Liu, Zixiang Ji, Ya-Liang Chang, Zhe-Yu Liu, Andrew Pleffer and [Kai-Wei Chang](https://md.research.elamur.ai/free/en/authors/kai-wei-chang/) · **Year:** 2026 · **Venue:** [Findings of the Association for Computational Linguistics: EACL 2026](https://md.research.elamur.ai/free/en/venues/findings/) ([Findings of the Association for Computational Linguistics 2026](https://md.research.elamur.ai/free/en/venues/findings/2026/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/) **Links:** [Publisher's page](https://aclanthology.org/2026.findings-eacl.262/) · [PDF](https://aclanthology.org/2026.findings-eacl.262.pdf) · [DOI 10.18653/v1/2026.findings-eacl.262](https://doi.org/10.18653/v1/2026.findings-eacl.262) · [OpenAlex](https://openalex.org/W7140125003) **Citations:** 0 citations (OpenAlex, 2026-10-14) ## Summary Content moderation systems depend on domain-specific safety policies, which are expensive to write and keep current. Wu et al. frame "open-domain policy construction": starting from a brief seed description of a domain and a search engine, a system produces a structured policy. It is judged by how much the policy helps a fixed reader LLM perform safe/unsafe classification when the policy is placed in its context. Their agent, Deep Policy Research (DPR), uses web search as its only tool. Over several iterations it writes queries aimed at coverage gaps, turns the retrieved snippets into predicate-style rules, self-critiques and deduplicates those rules, and groups them into named sections by clustering keyphrases. On the OpenAI undesired content benchmark, DPR policies beat definition-only prompts, few-shot examples and OpenAI's Deep Research agent. On a proprietary multimodal ad benchmark, DPR approaches expert-written sections in some domains. The work suggests LLM agents can bootstrap draft policies cheaply, though human review is still needed. ## Key points - On five OpenAI benchmark domains, DPR raised average F1 from 0.752 to 0.792 on Llama 3.1 8B Instruct and from 0.810 to 0.831 on Qwen2.5 7B Instruct, compared with seed-only prompting. - DPR outperformed OpenAI Deep Research (run with GPT-5.1) under the same protocol: - Llama 3.1 8B: 0.776 → 0.792 - Qwen2.5 7B: 0.800 → 0.831 - The biggest improvements appeared in domains that call for more judgment, namely Violence, Harassment and Self-Harm. Results on Sexual stayed within 0.01 F1 of the baselines. - On the ad benchmark, DPR came close to the human policy on Misrepresentative and Exploitative (e.g., 0.908 vs 0.920). It lagged badly on Finance Claims (about 0.60 vs 0.83–0.88), which depends on proprietary compliance wording. - The code and the generated policies are released under MIT at github.com/xiaowu0162/deep-policy-research. *Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.* ## Institutions - [University of California, Los Angeles](https://md.research.elamur.ai/free/en/universities/university-of-california-los-angeles/) — University, Los Angeles, United States · 99 works · 56,333 citations 📚 Shelves (6): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 18,614 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 17,965 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 17,450 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 17,296 works · [🤝 Social Sciences](https://md.research.elamur.ai/free/en/shelves/social-sciences/) · 1,954 works · [🤝 Safety Research](https://md.research.elamur.ai/free/en/shelves/social-sciences/safety-research/) · 626 works 🧭 Topics (1): [🛡️ Bias, fairness & safety](https://md.research.elamur.ai/free/en/topics/bias-fairness-safety/) · 587 works ## Cite Cite as: Di Wu, Siyue Liu, Zixiang Ji, Ya-Liang Chang, Zhe-Yu Liu, Andrew Pleffer and Kai-Wei Chang. 2026. Open-Domain Safety Policy Construction. Findings of the Association for Computational Linguistics: EACL 2026. The paper itself, not this page. BibTeX from the library's record of the work. ```bibtex @inproceedings{wu2026open, title = {{Open-Domain Safety Policy Construction}}, author = {Di Wu, Siyue Liu, Zixiang Ji, Ya-Liang Chang, Zhe-Yu Liu, Andrew Pleffer and Kai-Wei Chang}, booktitle = {Findings of the Association for Computational Linguistics: EACL 2026}, year = {2026}, doi = {10.18653/v1/2026.findings-eacl.262}, url = {https://doi.org/10.18653/v1/2026.findings-eacl.262} } ``` - [← previous Open-access Dataset on Acceptability Ratings of Korean Clausal Constructions by Humans and GPT Models](https://md.research.elamur.ai/free/en/papers/shin-2026-open-access-dataset-acceptability-ratings/) - [next → OpenCor: Latin American and Iberian Languages Open Corpora Forum](https://md.research.elamur.ai/free/en/papers/real-2026-opencor-latin-american-iberian-languages/) - [↑ back to papers of 2026](https://md.research.elamur.ai/free/en/papers/2026/page/44/) ## References 11 references, 5 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet. 1. Yuntao Bai, Saurav Kadavath, Sandipan Kundu, Amanda Askell, Jackson Kernion, Andy Jones, Anna Chen, Anna Goldie, Azalia Mirhoseini, Cameron McKinnon, and et al. 2022. [Constitutional ai: Harmlessness from ai feedback](https://md.research.elamur.ai/free/en/papers/bai-2022-constitutional-ai-harmlessness-ai-feedback/). arXiv preprint arXiv:2212.08073. 2. Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, Anirudh Goyal, Anthony Hartshorn, Aobo Yang, Archi Mitra, Archie Sravankumar, Artem Korenev, Arthur Hinsvark, Arun Rao, Aston Zhang, Aurélien Rodriguez,… — [The Llama 3 Herd of Models](https://md.research.elamur.ai/free/en/papers/grattafiori-2024-llama-3-herd-models/) 3. Melody Y. Guan, Manas Joglekar, Eric Wallace, Saachi Jain, Boaz Barak, Alec Helyar, Rachel Dias, Andrea Vallone, Hongyu Ren, Jason Wei, Hyung Won Chung, Sam Toyer, Johannes Heidecke, Alex Beutel, and Amelia Glaese. 2024. [Deliberative alignment: Reasoning enables safer language models](https://md.research.elamur.ai/free/en/papers/guan-2024-deliberative-alignment-reasoning-enables-safer/). CoRR, abs/2412… 4. Todor Markov, Chong Zhang, Sandhini Agarwal, Florentine Eloundou Nekoul, Theodore Lee, Steven Adler, Angela Jiang, and Lilian Weng. 2023. [A holistic approach to undesired content detection in the real world](https://md.research.elamur.ai/free/en/papers/markov-2022-holistic-approach-undesired-content-detection/). In Proceedings of the 37th AAAI Conference on Artificial Intelligence (AAAI), pages 15009–15… 5. Yutao Mou, Yuxiao Luo, Shikun Zhang, and Wei Ye. 2025. [SaRO: Enhancing LLM safety through reasoning-based alignment](https://md.research.elamur.ai/free/en/papers/mou-2025-saro-enhancing-llm-safety-through/). arXiv preprint arXiv:2504.09420. 6. Tong Mu, Alec Helyar, Johannes Heidecke, Joshua Achiam, Andrea Vallone, Ian Kivlichan, Molly Lin, Alex Beutel, John Schulman, and Lilian Weng. 2024. Rule based rewards for language model safety. arXiv preprint arXiv:2411.01111. [find](https://scholar.google.com/scholar?q=Tong+Mu%2C+Alec+Helyar%2C+Johannes+Heidecke%2C+Joshua+Achiam%2C+Andrea+Vallone%2C+Ian+Kivlichan%2C+Molly+Lin%2C+Alex+Beutel%2C+John+Schulman%2C+and+Lilian+Weng%2E+2024%2E+Rule+based+rewards+for+language+model+safety%2E+arXiv) 7. OpenAI. 2023. Using GPT-4 for content moderation. [https://openai.com/index/using-gpt-4-for](https://openai.com/index/using-gpt-4-for) -content-moderation/. Accessed 2025-07-04. [↗](https://openai.com/index/using-gpt-4-for) 8. Bertie Vidgen and Leon Derczynski. 2020. Directions in abusive language training data, a systematic review: Garbage in, garbage out. Plos one, 15(12):e0243300. [find](https://scholar.google.com/scholar?q=Bertie+Vidgen+and+Leon+Derczynski%2E+2020%2E+Directions+in+abusive+language+training+data%2C+a+systematic+review%3A+Garbage+in%2C+garbage+out%2E+Plos+one%2C+15%2812%29%3Ae0243300%2E) 9. An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, et al. 2024. Qwen2. 5 technical report. ArXiv preprint, abs/2412.15115. [find](https://scholar.google.com/scholar?q=An+Yang%2C+Baosong+Yang%2C+Beichen+Zhang%2C+Binyuan+Hui%2C+Bo+Zheng%2C+Bowen+Yu%2C+Chengyuan+Li%2C+Dayiheng+Liu%2C+Fei+Huang%2C+Haoran+Wei%2C+et+al%2E+2024%2E+Qwen2%2E+5+technical+report%2E+ArXiv+preprint%2C+abs%2F2412%2E15115%2E) 10. Wenjie Yin and Arkaitz Zubiaga. 2021. Towards generalisable hate speech detection: a review on obstacles and solutions. PeerJ Computer Science, 7:e598. [find](https://scholar.google.com/scholar?q=Wenjie+Yin+and+Arkaitz+Zubiaga%2E+2021%2E+Towards+generalisable+hate+speech+detection%3A+a+review+on+obstacles+and+solutions%2E+PeerJ+Computer+Science%2C+7%3Ae598%2E) 11. Eric Zeng, Tadayoshi Kohno, and Franziska Roesner. 2020. Bad news: Clickbait and deceptive ads on news and misinformation websites. In Workshop on Technology and Consumer Protection (ConPro). [find](https://scholar.google.com/scholar?q=Eric+Zeng%2C+Tadayoshi+Kohno%2C+and+Franziska+Roesner%2E+2020%2E+Bad+news%3A+Clickbait+and+deceptive+ads+on+news+and+misinformation+websites%2E+In+Workshop+on+Technology+and+Consumer+Protection+%28ConPro%29%2E) *Updated 2 November 2026*