---
title: "大语言模型的中文多轮指令遵循能力评测研究 — 朱秦 2025"
description: "Benchmarks of instruction following for Chinese LLMs are often translated from English."
authors: ["朱秦", "何俊亮", "邱锡鹏", "黄萱菁"]
year: 2025
venue: "中文信息学报 (Journal of Chinese Information Processing)"
source: "http://jcip.cipsc.org.cn/article/id/zwxxxb_3941"
licence: "Free access (jcip.cipsc.org.cn), no open licence stated — readonly; © 中文信息学报"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/papers/zhu-2025-evaluation-chinese-multiturn/"
---
[Elamur Research](https://md.research.elamur.ai/) · [this page in HTML](https://research.elamur.ai/free/papers/zhu-2025-evaluation-chinese-multiturn/)

[home](https://md.research.elamur.ai/) / [papers](https://md.research.elamur.ai/free/papers/) / [2025](https://md.research.elamur.ai/free/papers/2025/page/3/) / 大语言模型的中文多轮指令遵循能力评测研究

# 大语言模型的中文多轮指令遵循能力评测研究

**Authors:** 朱秦, 何俊亮, 邱锡鹏, 黄萱菁 · **Year:** 2025 · **Venue:** 中文信息学报 (Journal of Chinese Information Processing) · **Licence:** Free access (jcip.cipsc.org.cn), no open licence stated — readonly; © 中文信息学报

**Links:** [Publisher's page](http://jcip.cipsc.org.cn/article/id/zwxxxb_3941) · [PDF](http://jcip.cipsc.org.cn/cn/article/pdf/preview/zwxxxb_3941.pdf)

## Summary
Benchmarks of instruction following for Chinese LLMs are often translated from English. They tend to test a single turn and carry over checks that suit only Latin script, such as letter case. This paper presents CMIF (Chinese Multiturn Instruction Following Benchmark), built for Chinese multi-turn evaluation. Each item pairs a real user question, drawn from AlignBench-style categories, with three atomic instructions that are given one per turn over a three-turn dialogue. The atomic instructions include Chinese-specific constraints, such as pinyin, tonal patterns, parallel sentences and classical-Chinese style. A two-stage filter removes contradictory instruction combinations: simple rules catch obvious clashes, and a strong LLM catches subtler ones. Scoring is rule-based rather than LLM-judged, which keeps evaluation cheap and objective. Fourteen open and closed models were tested. All of them handle single turns reasonably well, but accuracy falls steadily as turns accumulate. Chinese-specific instructions are much harder than generic formatting ones. The benchmark therefore exposes concrete weaknesses in phonological and grammatical control that general benchmarks miss.

## Key points
- **Instruction categories:** CMIF defines 25 instruction types grouped into five top-level classes: keywords, length limits, special symbols, extra generation, and Chinese-specific features.
- **Metrics:** Four accuracies are computed, each at instruction level and dialogue level, under strict and loose criteria. Their mean is reported as a composite score.
- **Overall results:** Claude-3.5-Sonnet ranks best among closed models with a composite of 0.538. Qwen2.5-72B-Instruct ranks best among open models with 0.490. Claude's instruction-level accuracy drops from 73.8% to 40.0% in multi-turn settings.
- **Chinese-specific tasks:** The best composite on these tasks is only 51.0%, achieved by GPT4o. The other four classes average about 79.0%. Tonal patterns (仄起平收), pinyin checks, keyword frequency and sentence-mood tasks all score below 30% on average.
- **Model size and context length:** Larger Qwen2.5 models lose less accuracy across turns. Doubao-pro-128k underperforms Doubao-pro-32k, suggesting a longer context window does not help here.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

Fields: [Linguistics & languages](https://md.research.elamur.ai/free/shelves/linguistics/) · [Natural language processing](https://md.research.elamur.ai/free/shelves/nlp/)

## Cite

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@article{2025,
  title = {{大语言模型的中文多轮指令遵循能力评测研究}},
  author = {朱秦 and 何俊亮 and 邱锡鹏 and 黄萱菁},
  journal = {中文信息学报 (Journal of Chinese Information Processing)},
  year = {2025},
  url = {http://jcip.cipsc.org.cn/article/id/zwxxxb_3941}
}
```

- [← previous 大语言模型推理中的存储优化技术综述](https://md.research.elamur.ai/free/papers/ge-2025-storage-optimization-techniques/)
- [next → 大语言模型的工具使用综述](https://md.research.elamur.ai/free/papers/zheng-2025-tool-use-large/)
- [↑ back to papers of 2025](https://md.research.elamur.ai/free/papers/2025/page/3/)