---
title: "Safety Assessment of Chinese Large Language Models (2023)"
description: "Chat models such as ChatGPT can produce toxic, biased or dangerous output and can be misused… 18 references, cited by 74 papers here."
authors: ["Hao Sun", "Zhexin Zhang", "Jiawen Deng", "Jiale Cheng", "Minlie Huang"]
year: 2023
venue: "arXiv:2304.10436"
source: "https://arxiv.org/abs/2304.10436"
doi: "10.48550/arxiv.2304.10436"
arxiv: "2304.10436"
licence: "arXiv — non-exclusive license to distribute"
licence_url: "http://arxiv.org/licenses/nonexclusive-distrib/1.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/sun-2023-safety-assessment-chinese-large-language/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/sun-2023-safety-assessment-chinese-large-language/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2023](https://md.research.elamur.ai/free/en/papers/2023/page/8/) / Safety Assessment of Chinese Large Language Models

# Safety Assessment of Chinese Large Language Models

**Authors:** [Hao Sun](https://md.research.elamur.ai/free/en/authors/hao-sun/), [Zhexin Zhang](https://md.research.elamur.ai/free/en/authors/zhexin-zhang/), [Jiawen Deng](https://md.research.elamur.ai/free/en/authors/jiawen-deng/), [Jiale Cheng](https://md.research.elamur.ai/free/en/authors/jiale-cheng/) and [Minlie Huang](https://md.research.elamur.ai/free/en/authors/minlie-huang/) · **Year:** 2023 · **Venue:** arXiv:2304.10436 · **Licence:** [arXiv — non-exclusive license to distribute](http://arxiv.org/licenses/nonexclusive-distrib/1.0/)

**Links:** [Publisher's page](https://arxiv.org/abs/2304.10436) · [PDF](https://arxiv.org/pdf/2304.10436) · [DOI 10.48550/arxiv.2304.10436](https://doi.org/10.48550/arxiv.2304.10436) · [arXiv 2304.10436](https://arxiv.org/abs/2304.10436) · [OpenAlex](https://openalex.org/W4366735819)

**Citations:** 15 citations (OpenAlex, 2026-10-14) · [74](https://md.research.elamur.ai/free/en/papers/sun-2023-safety-assessment-chinese-large-language/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
Chat models such as ChatGPT can produce toxic, biased or dangerous output and can be misused, yet Chinese-language LLMs had no systematic safety evaluation. Tsinghua's CoAI group built a Chinese safety benchmark around a 14-category taxonomy. It has eight ordinary risk scenarios, such as Insult, Mental Health and Privacy and Property, which extend earlier dialogue-safety work. It also has six natural-language "instruction attacks", including Goal Hijacking, Prompt Leaking and Role Play Instruction. Each model answers hand-written test prompts, and a prompted LLM serving as a safety judge scores the replies automatically. The results feed a public leaderboard. The authors tested 15 systems, including OpenAI GPT models and Chinese models such as ChatGLM and MiniChat. Every model scored lower under instruction attacks than in ordinary scenarios. Instruction-tuned models clearly outperformed older open-domain chatbots. The authors also released SafetyPrompts, a large augmented prompt set with model answers that can be used for red-teaming and safety tuning. The approach can serve as a template for safety benchmarks in other languages, including Ukrainian.

## Key points
- **Taxonomy:** The 8 typical scenarios cover Physical Harm, Insult, Sensitive Topics, Crimes and Illegal Activities, Unfairness and Discrimination, Mental Health, Ethics and Morality, and Privacy and Property. The 6 instruction attacks are Prompt Leaking, Goal Hijacking, Role Play Instruction, Reverse Exposure, Unsafe Instruction Topic, and Inquiry with Unsafe Opinion.
- **Leaderboard snapshot:** ChatGPT (turbo-0301) ranks first with an overall score of about 98.4. ChatGLM-130B follows at about 96.9. Older models such as davinci and BLOOM-7.1B score near 42–43.
- **Main gap:** ChatGPT's lead comes mostly from resisting instruction attacks. On Goal Hijacking it beats the runner-up by more than 20 points. In typical scenarios, leading Chinese LLMs are roughly on par with it.
- **SafetyPrompts:** 100k prompts covering 13 scenarios (Sensitive Topics was withheld). This comprises 10k per typical scenario and 5k per attack type. The prompts were generated by GPT-3.5-turbo from three seed examples per prompt.
- **Data details:** GPT-3.5-turbo's own responses are included as reference answers, although about 1.6% of them were judged unsafe. The public test set on the benchmark site contains only about 2k samples.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Tsinghua University](https://md.research.elamur.ai/free/en/universities/tsinghua-university/) — University, Beijing, China · 415 works · 94,028 citations
- [State Key Laboratory of Intelligent Technology and Systems](https://md.research.elamur.ai/free/en/labs/state-key-laboratory-of-intelligent-technology-and-systems/) — Research institute, Beijing, China · 24 works · 9,850 citations
- [Beijing National Research Center for Information Science and Technology](https://md.research.elamur.ai/free/en/labs/beijing-national-research-center-for-information-science-and-technology/) — Research institute, Beijing, China · 33 works · 6,706 citations
- [Viện Trí tuệ nhân tạo](https://md.research.elamur.ai/free/en/labs/vien-tri-tue-nhan-tao/) — Research institute, Hanoi, Vietnam · 4 works · 653 citations

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 18,614 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 17,965 works · [🤝 Social Sciences](https://md.research.elamur.ai/free/en/shelves/social-sciences/) · 1,954 works · [🤝 Safety Research](https://md.research.elamur.ai/free/en/shelves/social-sciences/safety-research/) · 626 works

🧭 Topics (2): [🧠 Language models & pretraining](https://md.research.elamur.ai/free/en/topics/language-models/) · 3,894 works · [🛡️ Bias, fairness & safety](https://md.research.elamur.ai/free/en/topics/bias-fairness-safety/) · 587 works

## Cite

Cite as: Hao Sun, Zhexin Zhang, Jiawen Deng, Jiale Cheng and Minlie Huang. 2023. Safety Assessment of Chinese Large Language Models. arXiv:2304.10436.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@misc{sun2023safety,
  title = {{Safety Assessment of Chinese Large Language Models}},
  author = {Hao Sun, Zhexin Zhang, Jiawen Deng, Jiale Cheng and Minlie Huang},
  year = {2023},
  doi = {10.48550/arxiv.2304.10436},
  eprint = {2304.10436},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2304.10436}
}
```

- [← previous RL4F: Generating Natural Language Feedback with Reinforcement Learning for Repairing Model Outputs](https://md.research.elamur.ai/free/en/papers/akyurek-2023-rl4f-generating-natural-language-feedback/)
- [next → Task Contamination: Language Models May Not Be Few-Shot Anymore](https://md.research.elamur.ai/free/en/papers/li-2023-task-contamination-language-models-may/)
- [↑ back to papers of 2023](https://md.research.elamur.ai/free/en/papers/2023/page/8/)

## References

18 references, 9 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. BigScience. 2023. Bloom: A 176b-parameter open-access multilingual language model. [find](https://scholar.google.com/scholar?q=BigScience%2E+2023%2E+Bloom%3A+A+176b%2Dparameter+open%2Daccess+multilingual+language+model%2E)
2. Clue.AI. 2023. Chatyuan: Large language model for dialogue in chinese and english. [https://github.com/clue-ai/ChatYuan](https://github.com/clue-ai/ChatYuan). [↗](https://github.com/clue-ai/ChatYuan)
3. Jiawen Deng, Hao Sun, Zhexin Zhang, Jiale Cheng, and Minlie Huang. 2023. Recent advances towards safe, responsible, and moral dialogue systems: A survey. [find](https://scholar.google.com/scholar?q=Jiawen+Deng%2C+Hao+Sun%2C+Zhexin+Zhang%2C+Jiale+Cheng%2C+and+Minlie+Huang%2E+2023%2E+Recent+advances+towards+safe%2C+responsible%2C+and+moral+dialogue+systems%3A+A+survey%2E)
4. Fabrizio Gilardi, Meysam Alizadeh, and Maël Kubli. 2023. [Chatgpt outperforms crowd-workers for text-annotation tasks](https://md.research.elamur.ai/free/en/papers/gilardi-2023-chatgpt-outperforms-crowd-workers-text/).
5. Yuxian Gu, Jiaxin Wen, Hao Sun, Yi Song, Pei Ke, Chujie Zheng, Zheng Zhang, Jianzhu Yao, Xiaoyan Zhu, Jie Tang, and Minlie Huang. 2022. Eva2.0: Investigating open-domain chinese dialogue systems with large-scale pre-training. [find](https://scholar.google.com/scholar?q=Yuxian+Gu%2C+Jiaxin+Wen%2C+Hao+Sun%2C+Yi+Song%2C+Pei+Ke%2C+Chujie+Zheng%2C+Zheng+Zhang%2C+Jianzhu+Yao%2C+Xiaoyan+Zhu%2C+Jie+Tang%2C+and+Minlie+Huang%2E+2022%2E+Eva2%2E0%3A+Investigating+open%2Ddomain+chinese+dialogue+systems+with+)
6. Yunjie Ji, Yong Deng, Yan Gong, Yiping Peng, Qiang Niu, Lei Zhang, Baochang Ma, and Xiangang Li. 2023. [Exploring the impact of instruction data scaling on large language models: An empirical study on real-world use cases](https://md.research.elamur.ai/free/en/papers/ji-2023-exploring-impact-instruction-data/).
7. Yang Liu, Dan Iter, Yichong Xu, Shuohang Wang, Ruochen Xu, and Chenguang Zhu. 2023. [Gpteval: Nlg evaluation using gpt-4 with better human alignment](https://md.research.elamur.ai/free/en/papers/liu-2023-g-eval-nlg-evaluation-gpt/).
8. OpenAI. 2022. Blog: Introducing chatgpt. [find](https://scholar.google.com/scholar?q=OpenAI%2E+2022%2E+Blog%3A+Introducing+chatgpt%2E)
9. OpenAI. 2023. [Gpt-4 technical report](https://md.research.elamur.ai/free/en/papers/openai-2023-gpt-4-technical-report/).
10. Long Ouyang, Jeff Wu, Xu Jiang, Diogo Almeida, Carroll L. Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, John Schulman, Jacob Hilton, Fraser Kelton, Luke Miller, Maddie Simens, Amanda Askell, Peter Welinder, Paul Christiano, Jan Leike, and Ryan Lowe. 2022. [Train…](https://md.research.elamur.ai/free/en/papers/ouyang-2022-training-language-models-follow-instructions/)
11. Fábio Perez and Ian Ribeiro. 2022. [Ignore previous prompt: Attack techniques for language models](https://md.research.elamur.ai/free/en/papers/perez-2022-ignore-previous-prompt-attack-techniques/).
12. David Rozado. 2023. The political biases of chatgpt. Social Sciences, 12(3). [find](https://scholar.google.com/scholar?q=David+Rozado%2E+2023%2E+The+political+biases+of+chatgpt%2E+Social+Sciences%2C+12%283%29%2E)
13. Hao Sun, Guangxuan Xu, Jiawen Deng, Jiale Cheng, Chujie Zheng, Hao Zhou, Nanyun Peng, Xiaoyan Zhu, and Minlie Huang. 2022. [On the safety of conversational models: Taxonomy, dataset, and benchmark](https://md.research.elamur.ai/free/en/papers/sun-2022-safety-conversational-models-taxonomy-dataset/).
14. Jiaan Wang, Yunlong Liang, Fandong Meng, Haoxiang Shi, Zhixu Li, Jinan Xu, Jianfeng Qu, and Jie Zhou. 2023. [Is chatgpt a good nlg evaluator? a preliminary study](https://md.research.elamur.ai/free/en/papers/wang-2023-chatgpt-good-nlg-evaluator-preliminary/).
15. Yizhong Wang, Yeganeh Kordi, Swaroop Mishra, Alisa Liu, Noah A. Smith, Daniel Khashabi, and Hannaneh Hajishirzi. 2022. Self-instruct: Aligning language model with self generated instructions. [find](https://scholar.google.com/scholar?q=Yizhong+Wang%2C+Yeganeh+Kordi%2C+Swaroop+Mishra%2C+Alisa+Liu%2C+Noah+A%2E+Smith%2C+Daniel+Khashabi%2C+and+Hannaneh+Hajishirzi%2E+2022%2E+Self%2Dinstruct%3A+Aligning+language+model+with+self+generated+instructions%2E)
16. Jing Xu, Da Ju, Margaret Li, Y-Lan Boureau, Jason Weston, and Emily Dinan. 2021. [Recipes for safety in open-domain chatbots](https://md.research.elamur.ai/free/en/papers/xu-2020-recipes-safety-open-domain-chatbots/).
17. Hao Zhou, Pei Ke, Zheng Zhang, Yuxian Gu, Yinhe Zheng, Chujie Zheng, Yida Wang, Chen Henry Wu, Hao Sun, Xiaocong Yang, Bosi Wen, Xiaoyan Zhu, Minlie Huang, and Jie Tang. 2021. Eva: An open-domain chinese dialogue system with large-scale generative pre-training. [find](https://scholar.google.com/scholar?q=Hao+Zhou%2C+Pei+Ke%2C+Zheng+Zhang%2C+Yuxian+Gu%2C+Yinhe+Zheng%2C+Chujie+Zheng%2C+Yida+Wang%2C+Chen+Henry+Wu%2C+Hao+Sun%2C+Xiaocong+Yang%2C+Bosi+Wen%2C+Xiaoyan+Zhu%2C+Minlie+Huang%2C+and+Jie+Tang%2E+2021%2E+Eva%3A+An+open%2Ddomain+chin)
18. Terry Yue Zhuo, Yujin Huang, Chunyang Chen, and Zhenchang Xing. 2023. Exploring ai ethics of chatgpt: A diagnostic analysis. [find](https://scholar.google.com/scholar?q=Terry+Yue+Zhuo%2C+Yujin+Huang%2C+Chunyang+Chen%2C+and+Zhenchang+Xing%2E+2023%2E+Exploring+ai+ethics+of+chatgpt%3A+A+diagnostic+analysis%2E)

*Updated 2 November 2026*