---
title: "CFBench — paper by Zhang et al. (2025) — summary and ideas"
description: "The authors mined real online instructions and combined them with synthesis and expert design. 33 references, cited by 30 papers here."
authors: ["Tao Zhang", "ChengLIn Zhu", "Yanjun Shen", "Wenjing Luo", "Yan Zhang", "Hao Liang", "Tao Zhang", "Fan Yang", "Mingan Lin", "Yujing Qiao", "Weipeng Chen", "Bin Cui", "Wentao Zhang", "Zenan Zhou"]
year: 2025
venue: "Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)"
source: "https://aclanthology.org/2025.acl-long.1581/"
doi: "10.18653/v1/2025.acl-long.1581"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/zhang-2025-cfbench-comprehensive-constraints-following-benchmark/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/zhang-2025-cfbench-comprehensive-constraints-following-benchmark/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2025](https://md.research.elamur.ai/free/en/papers/2025/page/3/) / CFBench: A Comprehensive Constraints-Following Benchmark for LLMs

# CFBench: A Comprehensive Constraints-Following Benchmark for LLMs

**Authors:** [Tao Zhang](https://md.research.elamur.ai/free/en/authors/tao-zhang/), ChengLIn Zhu, [Yanjun Shen](https://md.research.elamur.ai/free/en/authors/yanjun-shen/), Wenjing Luo, [Yan Zhang](https://md.research.elamur.ai/free/en/authors/yan-zhang/), [Hao Liang](https://md.research.elamur.ai/free/en/authors/hao-liang/), [Tao Zhang](https://md.research.elamur.ai/free/en/authors/tao-zhang/), [Fan Yang](https://md.research.elamur.ai/free/en/authors/fan-yang/), [Mingan Lin](https://md.research.elamur.ai/free/en/authors/mingan-lin/), Yujing Qiao, [Weipeng Chen](https://md.research.elamur.ai/free/en/authors/weipeng-chen/), [Bin Cui](https://md.research.elamur.ai/free/en/authors/bin-cui/), [Wentao Zhang](https://md.research.elamur.ai/free/en/authors/wentao-zhang/) and Zenan Zhou · **Year:** 2025 · **Venue:** [Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)](https://md.research.elamur.ai/free/en/venues/acl/) ([ACL 2025](https://md.research.elamur.ai/free/en/venues/acl/2025/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2025.acl-long.1581/) · [PDF](https://aclanthology.org/2025.acl-long.1581.pdf) · [DOI 10.18653/v1/2025.acl-long.1581](https://doi.org/10.18653/v1/2025.acl-long.1581) · [OpenAlex](https://openalex.org/W4412889603)

**Citations:** 0 citations (OpenAlex, 2026-10-14) · [30](https://md.research.elamur.ai/free/en/papers/zhang-2025-cfbench-comprehensive-constraints-following-benchmark/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
CFBench tackles a gap in how LLMs are tested on following instructions that carry many explicit constraints. Earlier benchmarks usually check isolated constraint types or narrow settings, and they rarely reflect how real users judge whether a response is good enough. The authors mined real online instructions and combined them with synthesis and expert design. This produced a taxonomy of constraints and a Chinese benchmark of 1,000 hand-curated samples covering many domains, scenarios and NLP tasks. Each sample comes with a checklist that splits the request into separate checkpoints. Every checkpoint is tagged with a constraint type and a priority (primary or secondary), and an LLM judge grades each one. Scores are reported at three levels: individual constraints (CSR), whole instructions (ISR), and priority-aware satisfaction (PSR). Even the strongest models leave substantial headroom, and model rankings on CFBench differ from those on knowledge and math benchmarks. This suggests that constraint following is a separate capability that deserves its own evaluation.

## Key points
- **Scale:** 1,000 samples spanning 200+ real-life scenarios, 20 domains and 50+ NLP tasks. The constraint taxonomy has 10 primary categories and 25+ subcategories.
- **Construction pipeline:** about 1.1M online log instructions were filtered down to about 100K, deduplicated to about 30K, narrowed to about 5K candidates, and finalized at 1K. Checklists were created and revised through human–machine iteration.
- **Model results:**
  - GPT-4o performs best overall.
  - DeepSeek-V3 and Qwen2-72B-Instruct lead the open-source models.
  - The top PSR on the Hard Set is only 0.582.
- **Weak spots:** models struggle with contradictory constraints and with lexical, word-count and sentence-count requirements. They do better on document-count and audience-style constraints.
- **What helps:**
  - SFT and larger models improve results; Qwen2-72B-Instruct gains about 40% relative PSR over Qwen2-7B-Instruct.
  - Training on complex-constraint data, as in Conifer, helps further.
  - The number of constraints affects ISR most, while the number of primary needs drives PSR.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 16,039 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 15,891 works

🧭 Topics (2): [🧠 Language models & pretraining](https://md.research.elamur.ai/free/en/topics/language-models/) · 3,320 works · [📏 Evaluation & benchmarks](https://md.research.elamur.ai/free/en/topics/evaluation-benchmarks/) · 1,220 works

## Cite

Cite as: Tao Zhang, ChengLIn Zhu, Yanjun Shen, Wenjing Luo, Yan Zhang, Hao Liang, Tao Zhang, Fan Yang, Mingan Lin, Yujing Qiao, Weipeng Chen, Bin Cui, Wentao Zhang and Zenan Zhou. 2025. CFBench: A Comprehensive Constraints-Following Benchmark for LLMs. Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers).

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{zhang2025cfbench,
  title = {{CFBench: A Comprehensive Constraints-Following Benchmark for LLMs}},
  author = {Tao Zhang, ChengLIn Zhu, Yanjun Shen, Wenjing Luo, Yan Zhang, Hao Liang, Tao Zhang, Fan Yang, Mingan Lin, Yujing Qiao, Weipeng Chen, Bin Cui, Wentao Zhang and Zenan Zhou},
  booktitle = {Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2025},
  doi = {10.18653/v1/2025.acl-long.1581},
  url = {https://doi.org/10.18653/v1/2025.acl-long.1581}
}
```

- [← previous Can Large Language Models Detect Errors in Long Chain-of-Thought Reasoning?](https://md.research.elamur.ai/free/en/papers/he-2025-can-large-language-models-detect/)
- [next → CRMArena: Understanding the Capacity of LLM Agents to Perform Professional CRM Tasks in Realistic Environments](https://md.research.elamur.ai/free/en/papers/huang-2025-crmarena-understanding-capacity-llm-agents/)
- [↑ back to papers of 2025](https://md.research.elamur.ai/free/en/papers/2025/page/3/)

## References

33 references, 15 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Yi Zheng, Yichao Zhang, Yifan Shi, Yiliang Xiong, Ying He, Ying Tang, Yishi Piao, Yisong Wang, Yixuan Tan, Yiyang Ma, Yiyuan Liu, Yongqiang Guo, Yu Wu, Yuan Ou, Yuchen Zhu, Yuduan Wang, Yue Gong, Yuheng Zou, Yujia He, Yukun Zha, Yunfan Xiong, Yunxian Ma, Yuting Yan, Yuxiang Luo, Yuxiang You, Yuxuan… [find](https://scholar.google.com/scholar?q=Yi+Zheng%2C+Yichao+Zhang%2C+Yifan+Shi%2C+Yiliang+Xiong%2C+Ying+He%2C+Ying+Tang%2C+Yishi+Piao%2C+Yisong+Wang%2C+Yixuan+Tan%2C+Yiyang+Ma%2C+Yiyuan+Liu%2C+Yongqiang+Guo%2C+Yu+Wu%2C+Yuan+Ou%2C+Yuchen+Zhu%2C+Yuduan+Wang%2C+Yue+Gong%2C+Yuhe)
2. Guanting Dong, Keming Lu, Chengpeng Li, Tingyu Xia, Bowen Yu, Chang Zhou, and Jingren Zhou. 2024. Self-play with execution feedback: Improving instruction-following capabilities of large language models. arXiv preprint arXiv:2406.13542. [find](https://scholar.google.com/scholar?q=Guanting+Dong%2C+Keming+Lu%2C+Chengpeng+Li%2C+Tingyu+Xia%2C+Bowen+Yu%2C+Chang+Zhou%2C+and+Jingren+Zhou%2E+2024%2E+Self%2Dplay+with+execution+feedback%3A+Improving+instruction%2Dfollowing+capabilities+of+large+language+mode)
3. Qianyu He, Jie Zeng, Qianxi He, Jiaqing Liang, and Yanghua Xiao. 2024a. From complex to simple: Enhancing multi-constraint complex instruction following ability of large language models. arXiv preprint arXiv:2404.15846. [find](https://scholar.google.com/scholar?q=Qianyu+He%2C+Jie+Zeng%2C+Qianxi+He%2C+Jiaqing+Liang%2C+and+Yanghua+Xiao%2E+2024a%2E+From+complex+to+simple%3A+Enhancing+multi%2Dconstraint+complex+instruction+following+ability+of+large+language+models%2E+arXiv+preprin)
4. Qianyu He, Jie Zeng, Wenhao Huang, Lina Chen, Jin Xiao, Qianxi He, Xunzhe Zhou, Jiaqing Liang, and Yanghua Xiao. 2024b. Can large language models understand real-world complex instructions? In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 18188–18196. [find](https://scholar.google.com/scholar?q=Qianyu+He%2C+Jie+Zeng%2C+Wenhao+Huang%2C+Lina+Chen%2C+Jin+Xiao%2C+Qianxi+He%2C+Xunzhe+Zhou%2C+Jiaqing+Liang%2C+and+Yanghua+Xiao%2E+2024b%2E+Can+large+language+models+understand+real%2Dworld+complex+instructions%3F+In+Proceed)
5. Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. 2020. [Measuring massive multitask language understanding](https://md.research.elamur.ai/free/en/papers/hendrycks-2020-measuring-massive-multitask-language-understanding/). arXiv preprint arXiv:2009.03300.
6. Yuxin Jiang, Yufei Wang, Xingshan Zeng, Wanjun Zhong, Liangyou Li, Fei Mi, Lifeng Shang, Xin Jiang, Qun Liu, and Wei Wang. 2023. [Followbench: A multi-level fine-grained constraints following benchmark for large language models](https://md.research.elamur.ai/free/en/papers/jiang-2024-followbench-multi-level-fine-grained/). arXiv preprint arXiv:2310.20410.
7. Md Tahmid Rahman Laskar, Sawsan Alqahtani, M Saiful Bari, Mizanur Rahman, Mohammad Abdullah Matin Khan, Haidar Khan, Israt Jahan, Amran Bhuiyan, Chee Wei Tan, Md Rizwan Parvez, et al. 2024. [A systematic survey and critical review on evaluating large language models: Challenges, limitations, and reco…](https://md.research.elamur.ai/free/en/papers/laskar-2024-systematic-survey-critical-review-evaluating/)
8. Xuechen Li, Tianyi Zhang, Yann Dubois, Rohan Taori, Ishaan Gulrajani, Carlos Guestrin, Percy Liang, and Tatsunori B Hashimoto. 2023. Alpacaeval: An automatic evaluator of instruction-following models. [find](https://scholar.google.com/scholar?q=Xuechen+Li%2C+Tianyi+Zhang%2C+Yann+Dubois%2C+Rohan+Taori%2C+Ishaan+Gulrajani%2C+Carlos+Guestrin%2C+Percy+Liang%2C+and+Tatsunori+B+Hashimoto%2E+2023%2E+Alpacaeval%3A+An+automatic+evaluator+of+instruction%2Dfollowing+models%2E)
9. Yizhi Li, Ge Zhang, Xingwei Qu, Jiali Li, Zhaoqun Li, Zekun Wang, Hao Li, Ruibin Yuan, Yinghao Ma, Kai Zhang, et al. 2024. Cif-bench: A chinese instruction-following benchmark for evaluating the generalizability of large language models. arXiv preprint arXiv:2402.13109. [find](https://scholar.google.com/scholar?q=Yizhi+Li%2C+Ge+Zhang%2C+Xingwei+Qu%2C+Jiali+Li%2C+Zhaoqun+Li%2C+Zekun+Wang%2C+Hao+Li%2C+Ruibin+Yuan%2C+Yinghao+Ma%2C+Kai+Zhang%2C+et+al%2E+2024%2E+Cif%2Dbench%3A+A+chinese+instruction%2Dfollowing+benchmark+for+evaluating+the+gener)
10. Xiao Liu, Hao Yu, Hanchen Zhang, Yifan Xu, Xuanyu Lei, Hanyu Lai, Yu Gu, Hangliang Ding, Kaiwen Men, Kejuan Yang, et al. 2023. [Agentbench: Evaluating llms as agents](https://md.research.elamur.ai/free/en/papers/liu-2023-agentbench-evaluating-llms-agents/). arXiv preprint arXiv:2308.03688.
11. Swaroop Mishra, Daniel Khashabi, Chitta Baral, and Hannaneh Hajishirzi. 2022. Cross-task generalization via natural language crowdsourcing instructions. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 3470–3487. [find](https://scholar.google.com/scholar?q=Swaroop+Mishra%2C+Daniel+Khashabi%2C+Chitta+Baral%2C+and+Hannaneh+Hajishirzi%2E+2022%2E+Cross%2Dtask+generalization+via+natural+language+crowdsourcing+instructions%2E+In+Proceedings+of+the+60th+Annual+Meeting+of+th)
12. Subhabrata Mukherjee, Arindam Mitra, Ganesh Jawahar, Sahaj Agarwal, Hamid Palangi, and Ahmed Awadallah. 2023. [Orca: Progressive learning from complex explanation traces of gpt-4](https://md.research.elamur.ai/free/en/papers/mukherjee-2023-orca-progressive-learning-complex-explanation/). arXiv preprint arXiv:2306.02707.
13. Yiwei Qin, Kaiqiang Song, Yebowen Hu, Wenlin Yao, Sangwoo Cho, Xiaoyang Wang, Xuansheng Wu, Fei Liu, Pengfei Liu, and Dong Yu. 2024. [Infobench: Evaluating instruction following ability in large language models](https://md.research.elamur.ai/free/en/papers/qin-2024-infobench-evaluating-instruction-following-ability/). arXiv preprint arXiv:2401.03601.
14. Victor Sanh, Albert Webson, Colin Raffel, Stephen H Bach, Lintang Sutawika, Zaid Alyafeai, Antoine Chaffin, Arnaud Stiegler, Teven Le Scao, Arun Raja, et al. 2022. [Multitask prompted training enables zero-shot task generalization](https://md.research.elamur.ai/free/en/papers/sanh-2021-multitask-prompted-training-enables-zero/). In ICLR 2022-Tenth International Conference on Learning Representatio…
15. Haoran Sun, Lixin Liu, Junjie Li, Fengyu Wang, Baohua Dong, Ran Lin, and Ruohui Huang. 2024a. [Conifer: Improving complex constrained instruction-following ability of large language models](https://md.research.elamur.ai/free/en/papers/sun-2024-conifer-improving-complex-constrained-instruction/). arXiv preprint arXiv:2404.02823.
16. Kun Sun, Rong Wang, Haitao Liu, and Anders Søgaard. 2024b. Comprehensive reassessment of large-scale evaluation outcomes in llms: A multifaceted statistical approach. arXiv preprint arXiv:2403.15250. — [arXiv:2403.15250](https://arxiv.org/abs/2403.15250) [find](https://scholar.google.com/scholar?q=Kun+Sun%2C+Rong+Wang%2C+Haitao+Liu%2C+and+Anders+S%C3%B8gaard%2E+2024b%2E+Comprehensive+reassessment+of+large%2Dscale+evaluation+outcomes+in+llms%3A+A+multifaceted+statistical+approach%2E+arXiv+preprint+arXiv%3A2403%2E15250%2E+)
17. Xiangru Tang, Yiming Zong, Yilun Zhao, Arman Cohan, and Mark Gerstein. 2023. Struc-bench: Are large language models really good at generating complex structured data? arXiv preprint arXiv:2309.08963. — [arXiv:2309.08963](https://arxiv.org/abs/2309.08963) [find](https://scholar.google.com/scholar?q=Xiangru+Tang%2C+Yiming+Zong%2C+Yilun+Zhao%2C+Arman+Cohan%2C+and+Mark+Gerstein%2E+2023%2E+Struc%2Dbench%3A+Are+large+language+models+really+good+at+generating+complex+structured+data%3F+arXiv+preprint+arXiv%3A2309%2E08963%2E+)
18. Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al. 2023. [Llama 2: Open foundation and fine-tuned chat models](https://md.research.elamur.ai/free/en/papers/touvron-2023-llama-2-open-foundation-fine/). arXiv preprint arXiv:2307.09288.
19. Alicia Tsai, Shereen Oraby, Vittorio Perera, Jiun-Yu Kao, Yuheng Du, Anjali Narayan-Chen, Tagyoung Chung, and Dilek Hakkani-Tur. 2021. Style control for schema-guided natural language generation. In Proceedings of the 3rd Workshop on Natural Language Processing for Conversational AI, pages 228– 242. [find](https://scholar.google.com/scholar?q=Alicia+Tsai%2C+Shereen+Oraby%2C+Vittorio+Perera%2C+Jiun%2DYu+Kao%2C+Yuheng+Du%2C+Anjali+Narayan%2DChen%2C+Tagyoung+Chung%2C+and+Dilek+Hakkani%2DTur%2E+2021%2E+Style+control+for+schema%2Dguided+natural+language+generation%2E+In+P)
20. Yizhong Wang, Yeganeh Kordi, Swaroop Mishra, Alisa Liu, Noah A Smith, Daniel Khashabi, and Hannaneh Hajishirzi. 2023. Self-instruct: Aligning language models with self-generated instructions. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long P… [find](https://scholar.google.com/scholar?q=Yizhong+Wang%2C+Yeganeh+Kordi%2C+Swaroop+Mishra%2C+Alisa+Liu%2C+Noah+A+Smith%2C+Daniel+Khashabi%2C+and+Hannaneh+Hajishirzi%2E+2023%2E+Self%2Dinstruct%3A+Aligning+language+models+with+self%2Dgenerated+instructions%2E+In+Proce)
21. Jason Wei, Maarten Bosma, Vincent Y Zhao, Kelvin Guu, Adams Wei Yu, Brian Lester, Nan Du, Andrew M Dai, and Quoc V Le. 2021. [Finetuned language models are zero-shot learners](https://md.research.elamur.ai/free/en/papers/wei-2021-finetuned-language-models-zero-shot/). arXiv preprint arXiv:2109.01652.
22. Orion Weller, Nicholas Lourie, Matt Gardner, and Matthew E Peters. 2020. [Learning from task descriptions](https://md.research.elamur.ai/free/en/papers/weller-2020-learning-task-descriptions/). In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 1361–1375.
23. Bosi Wen, Pei Ke, Xiaotao Gu, Lindong Wu, Hao Huang, Jinfeng Zhou, Wenchuang Li, Binxin Hu, Wendy Gao, Jiaxin Xu, et al. 2024. Benchmarking complex instruction-following with multiple constraints composition. arXiv preprint arXiv:2407.03978. — [arXiv:2407.03978](https://arxiv.org/abs/2407.03978) [find](https://scholar.google.com/scholar?q=Bosi+Wen%2C+Pei+Ke%2C+Xiaotao+Gu%2C+Lindong+Wu%2C+Hao+Huang%2C+Jinfeng+Zhou%2C+Wenchuang+Li%2C+Binxin+Hu%2C+Wendy+Gao%2C+Jiaxin+Xu%2C+et+al%2E+2024%2E+Benchmarking+complex+instruction%2Dfollowing+with+multiple+constraints+comp)
24. Congying Xia, Chen Xing, Jiangshu Du, Xinyi Yang, Yihao Feng, Ran Xu, Wenpeng Yin, and Caiming Xiong. 2024. Fofo: A benchmark to evaluate llms’ format-following capability. arXiv preprint arXiv:2402.18667. [find](https://scholar.google.com/scholar?q=Congying+Xia%2C+Chen+Xing%2C+Jiangshu+Du%2C+Xinyi+Yang%2C+Yihao+Feng%2C+Ran+Xu%2C+Wenpeng+Yin%2C+and+Caiming+Xiong%2E+2024%2E+Fofo%3A+A+benchmark+to+evaluate+llms%E2%80%99+format%2Dfollowing+capability%2E+arXiv+preprint+arXiv%3A2402%2E1)
25. Can Xu, Qingfeng Sun, Kai Zheng, Xiubo Geng, Pu Zhao, Jiazhan Feng, Chongyang Tao, and Daxin Jiang. 2023. [Wizardlm: Empowering large language models to follow complex instructions](https://md.research.elamur.ai/free/en/papers/xu-2023-wizardlm-empowering-large-pre-trained/). arXiv preprint arXiv:2304.12244.
26. An Yang, Baosong Yang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Zhou, Chengpeng Li, Chengyuan Li, Dayiheng Liu, Fei Huang, et al. 2024. Qwen2 technical report. arXiv preprint arXiv:2407.10671. [find](https://scholar.google.com/scholar?q=An+Yang%2C+Baosong+Yang%2C+Binyuan+Hui%2C+Bo+Zheng%2C+Bowen+Yu%2C+Chang+Zhou%2C+Chengpeng+Li%2C+Chengyuan+Li%2C+Dayiheng+Liu%2C+Fei+Huang%2C+et+al%2E+2024%2E+Qwen2+technical+report%2E+arXiv+preprint+arXiv%3A2407%2E10671%2E)
27. Sherry Yang, Ofir Nachum, Yilun Du, Jason Wei, Pieter Abbeel, and Dale Schuurmans. 2023. Foundation models for decision making: Problems, methods, and opportunities. arXiv preprint arXiv:2303.04129. [find](https://scholar.google.com/scholar?q=Sherry+Yang%2C+Ofir+Nachum%2C+Yilun+Du%2C+Jason+Wei%2C+Pieter+Abbeel%2C+and+Dale+Schuurmans%2E+2023%2E+Foundation+models+for+decision+making%3A+Problems%2C+methods%2C+and+opportunities%2E+arXiv+preprint+arXiv%3A2303%2E04129%2E)
28. Shunyu Yao, Howard Chen, Austin W Hanjie, Runzhe Yang, and Karthik Narasimhan. 2023. [Collie: Systematic construction of constrained text generation tasks](https://md.research.elamur.ai/free/en/papers/yao-2023-collie-systematic-construction-constrained-text/). arXiv preprint arXiv:2307.08689.
29. Hanqing Zhang, Haolin Song, Shaoyu Li, Ming Zhou, and Dawei Song. 2023. A survey of controllable text generation using transformer-based pre-trained language models. ACM Computing Surveys, 56(3):1– 37. [find](https://scholar.google.com/scholar?q=Hanqing+Zhang%2C+Haolin+Song%2C+Shaoyu+Li%2C+Ming+Zhou%2C+and+Dawei+Song%2E+2023%2E+A+survey+of+controllable+text+generation+using+transformer%2Dbased+pre%2Dtrained+language+models%2E+ACM+Computing+Surveys%2C+56%283%29%3A1%E2%80%93+37)
30. Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric Xing, et al. 2024. [Judging llm-as-a-judge with mt-bench and chatbot arena](https://md.research.elamur.ai/free/en/papers/zheng-2023-judging-llm-judge-mt-bench/). Advances in Neural Information Processing Systems, 36.
31. Ruiqi Zhong, Kristy Lee, Zheng Zhang, and Dan Klein. 2021. Adapting language models for zero-shot learning by meta-tuning on dataset and prompt collections. In Findings of the Association for Computational Linguistics: EMNLP 2021, pages 2856–2878. [find](https://scholar.google.com/scholar?q=Ruiqi+Zhong%2C+Kristy+Lee%2C+Zheng+Zhang%2C+and+Dan+Klein%2E+2021%2E+Adapting+language+models+for+zero%2Dshot+learning+by+meta%2Dtuning+on+dataset+and+prompt+collections%2E+In+Findings+of+the+Association+for+Computat)
32. Jeffrey Zhou, Tianjian Lu, Swaroop Mishra, Siddhartha Brahma, Sujoy Basu, Yi Luan, Denny Zhou, and Le Hou. 2023a. [Instruction-following evaluation for large language models](https://md.research.elamur.ai/free/en/papers/zhou-2023-instruction-following-evaluation-large-language/). arXiv preprint arXiv:2311.07911.
33. Wangchunshu Zhou, Yuchen Eleanor Jiang, Ethan Wilcox, Ryan Cotterell, and Mrinmaya Sachan. 2023b. Controlled text generation with natural language instructions. In International Conference on Machine Learning, pages 42602–42613. PMLR. [find](https://scholar.google.com/scholar?q=Wangchunshu+Zhou%2C+Yuchen+Eleanor+Jiang%2C+Ethan+Wilcox%2C+Ryan+Cotterell%2C+and+Mrinmaya+Sachan%2E+2023b%2E+Controlled+text+generation+with+natural+language+instructions%2E+In+International+Conference+on+Machine+)

*Updated 31 October 2026*