---
title: "The N+ Implementation Details of RLHF with PPO: A Case Study"
description: "The authors address this by revisiting the earlier TL;DR summarization study of Stiennon et al. (2020)… 34 references, cited by 12 papers here."
authors: ["Shengyi Huang", "Michael Noukhovitch", "Arian Hosseini", "Kashif Rasul", "Weixun Wang", "Lewis Tunstall"]
year: 2024
venue: "arXiv:2403.17031"
source: "https://arxiv.org/abs/2403.17031"
doi: "10.48550/arxiv.2403.17031"
arxiv: "2403.17031"
licence: "CC BY 4.0 (arXiv)"
licence_url: "http://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/huang-2024-n-implementation-details-rlhf-ppo/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/huang-2024-n-implementation-details-rlhf-ppo/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2024](https://md.research.elamur.ai/free/en/papers/2024/page/13/) / The N+ Implementation Details of RLHF with PPO: A Case Study on TL;DR Summarization

# The N+ Implementation Details of RLHF with PPO: A Case Study on TL;DR Summarization

**Authors:** [Shengyi Huang](https://md.research.elamur.ai/free/en/authors/shengyi-huang/), [Michael Noukhovitch](https://md.research.elamur.ai/free/en/authors/michael-noukhovitch/), [Arian Hosseini](https://md.research.elamur.ai/free/en/authors/arian-hosseini/), [Kashif Rasul](https://md.research.elamur.ai/free/en/authors/kashif-rasul/), [Weixun Wang](https://md.research.elamur.ai/free/en/authors/weixun-wang/) and [Lewis Tunstall](https://md.research.elamur.ai/free/en/authors/lewis-tunstall/) · **Year:** 2024 · **Venue:** arXiv:2403.17031 · **Licence:** [CC BY 4.0 (arXiv)](http://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://arxiv.org/abs/2403.17031) · [PDF](https://arxiv.org/pdf/2403.17031) · [DOI 10.48550/arxiv.2403.17031](https://doi.org/10.48550/arxiv.2403.17031) · [arXiv 2403.17031](https://arxiv.org/abs/2403.17031) · [OpenAlex](https://openalex.org/W4393247779)

**Citations:** 2 citations (OpenAlex, 2026-10-14) · [12](https://md.research.elamur.ai/free/en/papers/huang-2024-n-implementation-details-rlhf-ppo/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
RLHF pipelines like OpenAI's have been hard to replicate in open source. Subtle engineering choices affect stability, instruction-following outputs are difficult to judge, and training cycles are slow. The authors address this by revisiting the earlier TL;DR summarization study of Stiennon et al. (2020), where outputs are short and easier to evaluate. They built a full pipeline from scratch (SFT, reward model, then PPO) on Pythia models of 1B, 2.8B and 6.9B parameters, using the filtered Reddit TL;DR SFT data and OpenAI's summarize_from_feedback preference data. To keep the setup simple, one learning rate serves all three training stages, and every run is repeated with four seeds, including failed runs. The study catalogs more than twenty practical details, from query truncation and token handling to PPO tricks. The result is the first open reproduction of the original scaling trend: bigger models produce stronger summaries. The 2.8B and 6.9B policies beat OpenAI's released 1.3B checkpoint. All code and checkpoints are public, which gives the community a transparent RLHF baseline.

## Key points
- A single learning rate of 3e-6 is used for SFT, RM and PPO with no sweeps, and every configuration is run with four different seeds on each of the 1B, 2.8B and 6.9B Pythia models.
- Evaluation uses gpt-3.5-turbo-0125 to compare summaries against human references on the TL;DR validation split. The best 6.9B PPO model is preferred nearly 80% of the time, while two over-optimized 1B runs (KL roughly 50–85) fall below 20% and produce concatenated, space-free text.
- Key data handling details:
  - Queries are capped at 512 tokens by dropping the last paragraph of the post rather than cutting mid-sentence.
  - Completions get a leading space and an EOS token.
  - A dedicated [PAD] token is used instead of reusing EOS, so the model learns to stop generating.
- When summary length is controlled, PPO still outperforms SFT at every length. Reward whitening yields shorter outputs and a lower raw win rate, but performs similarly once length is accounted for. DPO is included as an RL-free comparison.
- Code is released at github.com/vwxyzjn/summarize_from_feedback_details, and the datasets are on Hugging Face (vwxyzjn/summarize_from_feedback_tldr_3_filtered, openai/summarize_from_feedback).

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Université de Montréal](https://md.research.elamur.ai/free/en/universities/universite-de-montreal/) — University, Montreal, Canada · 96 works · 92,262 citations

Also on the byline: [NetEase](https://ror.org/00fp6fj05).

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 19,503 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 18,805 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 17,959 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 17,789 works

🧭 Topics (1): [📝 Summarization](https://md.research.elamur.ai/free/en/topics/summarization/) · 246 works

## Cite

Cite as: Shengyi Huang, Michael Noukhovitch, Arian Hosseini, Kashif Rasul, Weixun Wang and Lewis Tunstall. 2024. The N+ Implementation Details of RLHF with PPO: A Case Study on TL;DR Summarization. arXiv:2403.17031.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@misc{huang2024n,
  title = {{The N+ Implementation Details of RLHF with PPO: A Case Study on TL;DR Summarization}},
  author = {Shengyi Huang, Michael Noukhovitch, Arian Hosseini, Kashif Rasul, Weixun Wang and Lewis Tunstall},
  year = {2024},
  doi = {10.48550/arxiv.2403.17031},
  eprint = {2403.17031},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2403.17031}
}
```

- [← previous Skywork-Math: Data Scaling Laws for Mathematical Reasoning in Large Language Models — The Story Goes On](https://md.research.elamur.ai/free/en/papers/zeng-2024-skywork-math-data-scaling-laws/)
- [next → Towards a dynamical model of English vowels. Evidence from diphthongisation](https://md.research.elamur.ai/free/en/papers/strycharczuk-2024-towards-dynamical-model-english-vowels/)
- [↑ back to papers of 2024](https://md.research.elamur.ai/free/en/papers/2024/page/13/)

## References

34 references, 13 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Nisan Stiennon, Long Ouyang, Jeffrey Wu, Daniel Ziegler, Ryan Lowe, Chelsea Voss, Alec Radford, Dario Amodei, and Paul F Christiano. Learning to summarize with human feedback. Advances in Neural Information Processing Systems, 33:3008–3021, 2020. [find](https://scholar.google.com/scholar?q=Nisan+Stiennon%2C+Long+Ouyang%2C+Jeffrey+Wu%2C+Daniel+Ziegler%2C+Ryan+Lowe%2C+Chelsea+Voss%2C+Alec+Radford%2C+Dario+Amodei%2C+and+Paul+F+Christiano%2E+Learning+to+summarize+with+human+feedback%2E+Advances+in+Neural+Infor)
2. Stella Biderman, USVSN Sai Prashanth, Lintang Sutawika, Hailey Schoelkopf, Quentin Anthony, Shivanshu Purohit, and Edward Raff. [Emergent and predictable memorization in large language models](https://md.research.elamur.ai/free/en/papers/biderman-2023-emergent-predictable-memorization-large-language/). 2023.
3. Alec Radford, Karthik Narasimhan, Tim Salimans, Ilya Sutskever, et al. Improving language understanding by generative pre-training. 2018. [find](https://scholar.google.com/scholar?q=Alec+Radford%2C+Karthik+Narasimhan%2C+Tim+Salimans%2C+Ilya+Sutskever%2C+et+al%2E+Improving+language+understanding+by+generative+pre%2Dtraining%2E+2018%2E)
4. Alec Radford, Jeff Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. Language models are unsupervised multitask learners. 2019. [find](https://scholar.google.com/scholar?q=Alec+Radford%2C+Jeff+Wu%2C+Rewon+Child%2C+David+Luan%2C+Dario+Amodei%2C+and+Ilya+Sutskever%2E+Language+models+are+unsupervised+multitask+learners%2E+2019%2E)
5. Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. [Language models are few-shot learners](https://md.research.elamur.ai/free/en/papers/brown-2020-language-models-few-shot-learners/). Advances in neural information processing systems, 33:1877–1901, 2020.
6. Jack W Rae, Sebastian Borgeaud, Trevor Cai, Katie Millican, Jordan Hoffmann, Francis Song, John Aslanides, Sarah Henderson, Roman Ring, Susannah Young, et al. Scaling language models: Methods, analysis & insights from training gopher. arXiv preprint arXiv:2112.11446, 2021. [find](https://scholar.google.com/scholar?q=Jack+W+Rae%2C+Sebastian+Borgeaud%2C+Trevor+Cai%2C+Katie+Millican%2C+Jordan+Hoffmann%2C+Francis+Song%2C+John+Aslanides%2C+Sarah+Henderson%2C+Roman+Ring%2C+Susannah+Young%2C+et+al%2E+Scaling+language+models%3A+Methods%2C+analysi)
7. Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. [Training language models to follow instructions with human feedback](https://md.research.elamur.ai/free/en/papers/ouyang-2022-training-language-models-follow-instructions/). Advances in neural information processing systems, 35:27730–27744, 2022.
8. Yuntao Bai, Andy Jones, Kamal Ndousse, Amanda Askell, Anna Chen, Nova DasSarma, Dawn Drain, Stanislav Fort, Deep Ganguli, Tom Henighan, et al. Training a helpful and harmless assistant with reinforcement learning from human feedback. arXiv preprint arXiv:2204.05862, 2022. [find](https://scholar.google.com/scholar?q=Yuntao+Bai%2C+Andy+Jones%2C+Kamal+Ndousse%2C+Amanda+Askell%2C+Anna+Chen%2C+Nova+DasSarma%2C+Dawn+Drain%2C+Stanislav+Fort%2C+Deep+Ganguli%2C+Tom+Henighan%2C+et+al%2E+Training+a+helpful+and+harmless+assistant+with+reinforcem)
9. OpenAI, Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, Red Avila, Igor Babuschkin, Suchir Balaji, Valerie Balcom, Paul Baltescu, Haiming Bao, Mohammad Bavarian, Jeff Belgum, Irwan Bello, Ja… — [GPT-4 Technical Report](https://md.research.elamur.ai/free/en/papers/openai-2023-gpt-4-technical-report/)
10. Logan Engstrom, Andrew Ilyas, Shibani Santurkar, Dimitris Tsipras, Firdaus Janoos, Larry Rudolph, and Aleksander Madry. Implementation matters in deep rl: A case study on ppo and trpo. In International Conference on Learning Representations, 2020. URL [https://openreview.net/forum?id=r1etN1rtPB](https://openreview.net/forum?id=r1etN1rtPB). [↗](https://openreview.net/forum?id=r1etN1rtPB)
11. Shengyi Huang, Rousslan Fernand Julien Dossa, Antonin Raffin, Anssi Kanervisto, and Weixun Wang. The 37 implementation details of proximal policy optimization. In ICLR Blog Track, 2022. URL [https://iclr-blog-track.github.io/2022/03/25/ppo-implementation-details/](https://iclr-blog-track.github.io/2022/03/25/ppo-implementation-details/). https://iclr-blog-track.github.io/20… [↗](https://iclr-blog-track.github.io/2022/03/25/ppo-implementation-details/)
12. Shengyi Costa Huang, Tianlin Liu, and Leandro von Werra. The n implementation details of rlhf with ppo. In ICLR Blogposts 2024, 2024. URL [https://d2jud02ci9yv69.cloudfront.net/2024-05-07-the-n-implementation-details-of-rlhf-with-ppo-130/blog/](https://d2jud02ci9yv69.cloudfront.net/2024-05-07-the-n-implementation-details-of-rlhf-with-ppo-130/blog/) the-n-implementation-details-of-rlhf-with-ppo/. https://… [↗](https://d2jud02ci9yv69.cloudfront.net/2024-05-07-the-n-implementation-details-of-rlhf-with-ppo-130/blog/)
13. Yann Dubois, Xuechen Li, Rohan Taori, Tianyi Zhang, Ishaan Gulrajani, Jimmy Ba, Carlos Guestrin, Percy Liang, and Tatsunori B. Hashimoto. [AlpacaFarm: A Simulation Framework for Methods that Learn from Human Feedback](https://md.research.elamur.ai/free/en/papers/dubois-2023-alpacafarm-simulation-framework-methods-that/). In Advances in Neural Information Processing Systems, 2023.
14. Paul F Christiano, Jan Leike, Tom Brown, Miljan Martic, Shane Legg, and Dario Amodei. Deep reinforcement learning from human preferences. Advances in neural information processing systems, 30, 2017. [find](https://scholar.google.com/scholar?q=Paul+F+Christiano%2C+Jan+Leike%2C+Tom+Brown%2C+Miljan+Martic%2C+Shane+Legg%2C+and+Dario+Amodei%2E+Deep+reinforcement+learning+from+human+preferences%2E+Advances+in+neural+information+processing+systems%2C+30%2C+2017%2E)
15. Daniel M Ziegler, Nisan Stiennon, Jeffrey Wu, Tom B Brown, Alec Radford, Dario Amodei, Paul Christiano, and Geoffrey Irving. Fine-tuning language models from human preferences. arXiv preprint arXiv:1909.08593, 2019. [find](https://scholar.google.com/scholar?q=Daniel+M+Ziegler%2C+Nisan+Stiennon%2C+Jeffrey+Wu%2C+Tom+B+Brown%2C+Alec+Radford%2C+Dario+Amodei%2C+Paul+Christiano%2C+and+Geoffrey+Irving%2E+Fine%2Dtuning+language+models+from+human+preferences%2E+arXiv+preprint+arXiv%3A19)
16. G̃emini Team, Rohan Anil, Sebastian Borgeaud, Yonghui Wu, Jean-Baptiste Alayrac, Jiahui Yu, Radu Soricut, Johan Schalkwyk, Andrew M Dai, Anja Hauth, et al. [Gemini: a family of highly capable multimodal models](https://md.research.elamur.ai/free/en/papers/gemini-team-2023-gemini-family-highly-capable-multimodal/). arXiv preprint arXiv:2312.11805, 2023.
17. Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D Manning, Stefano Ermon, and Chelsea Finn. [Direct preference optimization: Your language model is secretly a reward model](https://md.research.elamur.ai/free/en/papers/rafailov-2023-direct-preference-optimization-language-model/). In Thirty-seventh Conference on Neural Information Processing Systems, 2023. URL https://arxiv.org/abs/2305.18290. [↗](https://arxiv.org/abs/2305.18290)
18. Mohammad Gheshlaghi Azar, Mark Rowland, Bilal Piot, Daniel Guo, Daniele Calandriello, Michal Valko, and Rémi Munos. [A general theoretical paradigm to understand learning from human preferences](https://md.research.elamur.ai/free/en/papers/azar-2023-general-theoretical-paradigm-understand-learning/). arXiv preprint arXiv:2310.12036, 2023.
19. Jiwoo Hong, Noah Lee, and James Thorne. Reference-free monolithic preference optimization with odds ratio. arXiv preprint arXiv:2403.07691, 2024. [find](https://scholar.google.com/scholar?q=Jiwoo+Hong%2C+Noah+Lee%2C+and+James+Thorne%2E+Reference%2Dfree+monolithic+preference+optimization+with+odds+ratio%2E+arXiv+preprint+arXiv%3A2403%2E07691%2C+2024%2E)
20. Leandro von Werra, Younes Belkada, Lewis Tunstall, Edward Beeching, Tristan Thrush, Nathan Lambert, and Shengyi Huang. Trl: Transformer reinforcement learning. [https://github.com/huggingface/trl](https://github.com/huggingface/trl), 2020. [↗](https://github.com/huggingface/trl)
21. Lewis Tunstall, Edward Beeching, Nathan Lambert, Nazneen Rajani, Kashif Rasul, Younes Belkada, Shengyi Huang, Leandro von Werra, Clémentine Fourrier, Nathan Habib, Nathan Sarrazin, Omar Sanseviero, Alexander M. Rush, and Thomas Wolf. Zephyr: Direct distillation of lm alignment, 2023. [find](https://scholar.google.com/scholar?q=Lewis+Tunstall%2C+Edward+Beeching%2C+Nathan+Lambert%2C+Nazneen+Rajani%2C+Kashif+Rasul%2C+Younes+Belkada%2C+Shengyi+Huang%2C+Leandro+von+Werra%2C+Cl%C3%A9mentine+Fourrier%2C+Nathan+Habib%2C+Nathan+Sarrazin%2C+Omar+Sanseviero%2C+Al)
22. Hamish Ivison, Yizhong Wang, Valentina Pyatkin, Nathan Lambert, Matthew Peters, Pradeep Dasigi, Joel Jang, David Wadden, Noah A Smith, Iz Beltagy, et al. [Camels in a changing climate: Enhancing lm adaptation with tulu 2](https://md.research.elamur.ai/free/en/papers/ivison-2023-camels-changing-climate-enhancing-lm/). arXiv preprint arXiv:2311.10702, 2023.
23. Albert Q Jiang, Alexandre Sablayrolles, Antoine Roux, Arthur Mensch, Blanche Savary, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Emma Bou Hanna, Florian Bressand, et al. Mixtral of experts. arXiv preprint arXiv:2401.04088, 2024. [find](https://scholar.google.com/scholar?q=Albert+Q+Jiang%2C+Alexandre+Sablayrolles%2C+Antoine+Roux%2C+Arthur+Mensch%2C+Blanche+Savary%2C+Chris+Bamford%2C+Devendra+Singh+Chaplot%2C+Diego+de+las+Casas%2C+Emma+Bou+Hanna%2C+Florian+Bressand%2C+et+al%2E+Mixtral+of+expe)
24. Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Perric Cistac, Clara Ma, Yacine Jernite, Julien Plu, Canwen Xu, Teven Le Scao, Sylvain Gugger, Mariama Drame, Quentin Lhoest, and Alexander M. Rush. [Transformers: State-of-the-Art Natural Language Processing](https://md.research.elamur.ai/free/en/papers/wolf-2020-transformers-state-art-natural-language/). p…
25. Jeff Rasley, Samyam Rajbhandari, Olatunji Ruwase, and Yuxiong He. Deepspeed: System optimizations enable training deep learning models with over 100 billion parameters. In Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pages 3505–3506, 2020. [find](https://scholar.google.com/scholar?q=Jeff+Rasley%2C+Samyam+Rajbhandari%2C+Olatunji+Ruwase%2C+and+Yuxiong+He%2E+Deepspeed%3A+System+optimizations+enable+training+deep+learning+models+with+over+100+billion+parameters%2E+In+Proceedings+of+the+26th+ACM+)
26. Samyam Rajbhandari, Jeff Rasley, Olatunji Ruwase, and Yuxiong He. Zero: Memory optimizations toward training trillion parameter models. In SC20: International Conference for High Performance Computing, Networking, Storage and Analysis, pages 1–16. IEEE, 2020. [find](https://scholar.google.com/scholar?q=Samyam+Rajbhandari%2C+Jeff+Rasley%2C+Olatunji+Ruwase%2C+and+Yuxiong+He%2E+Zero%3A+Memory+optimizations+toward+training+trillion+parameter+models%2E+In+SC20%3A+International+Conference+for+High+Performance+Computing)
27. Sylvain Gugger, Lysandre Debut, Thomas Wolf, Philipp Schmid, Zachary Mueller, Sourab Mangrulkar, Marc Sun, and Benjamin Bossan. Accelerate: Training and inference at scale made simple, efficient and adaptable. [https://github.com/huggingface/accelerate](https://github.com/huggingface/accelerate), 2022. [↗](https://github.com/huggingface/accelerate)
28. Lukas Biewald. Experiment tracking with weights and biases, 2020. URL [https://www.wandb.com/](https://www.wandb.com/). Software available from wandb.com. [↗](https://www.wandb.com/)
29. Leo Gao, John Schulman, and Jacob Hilton. [Scaling laws for reward model overoptimization](https://md.research.elamur.ai/free/en/papers/gao-2022-scaling-laws-reward-model-overoptimization/). In International Conference on Machine Learning, pages 10835–10866. PMLR, 2023.
30. Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric Xing, et al. [Judging llm-as-a-judge with mt-bench and chatbot arena](https://md.research.elamur.ai/free/en/papers/zheng-2023-judging-llm-judge-mt-bench/). Advances in Neural Information Processing Systems, 36, 2023.
31. Michael Noukhovitch, Samuel Lavoie, Florian Strub, and Aaron Courville. Language Model Alignment with Elastic Reset. In Advances in Neural Information Processing Systems, 2023. [find](https://scholar.google.com/scholar?q=Michael+Noukhovitch%2C+Samuel+Lavoie%2C+Florian+Strub%2C+and+Aaron+Courville%2E+Language+Model+Alignment+with+Elastic+Reset%2E+In+Advances+in+Neural+Information+Processing+Systems%2C+2023%2E)
32. Marcin Andrychowicz, Anton Raichuk, Piotr Stańczyk, Manu Orsini, Sertan Girgin, Raphaël Marinier, Leonard Hussenot, Matthieu Geist, Olivier Pietquin, Marcin Michalski, Sylvain Gelly, and Olivier Bachem. What matters for on-policy deep actor-critic methods? a large-scale study. In International Conf… [find](https://scholar.google.com/scholar?q=Marcin+Andrychowicz%2C+Anton+Raichuk%2C+Piotr+Stan%CC%81czyk%2C+Manu+Orsini%2C+Sertan+Girgin%2C+Rapha%C3%ABl+Marinier%2C+Leonard+Hussenot%2C+Matthieu+Geist%2C+Olivier+Pietquin%2C+Marcin+Michalski%2C+Sylvain+Gelly%2C+and+Olivier+Bach)
33. Bill Yuchen Lin, Abhilasha Ravichander, Ximing Lu, Nouha Dziri, Melanie Sclar, Khyathi Chandu, Chandra Bhagavatula, and Yejin Choi. [The unlocking spell on base llms: Rethinking alignment via in-context learning](https://md.research.elamur.ai/free/en/papers/lin-2023-unlocking-spell-base-llms-rethinking/). In International Conference on Learning Representations, 2024. URL [https://arxiv.org/abs…](https://arxiv.org/abs…) [↗](https://arxiv.org/abs…)
34. Peiyi Wang, Lei Li, Liang Chen, Dawei Zhu, Binghuai Lin, Yunbo Cao, Qi Liu, Tianyu Liu, and Zhifang Sui. Large Language Models are not Fair Evaluators, May 2023. [find](https://scholar.google.com/scholar?q=Peiyi+Wang%2C+Lei+Li%2C+Liang+Chen%2C+Dawei+Zhu%2C+Binghuai+Lin%2C+Yunbo+Cao%2C+Qi+Liu%2C+Tianyu+Liu%2C+and+Zhifang+Sui%2E+Large+Language+Models+are+not+Fair+Evaluators%2C+May+2023%2E)

*Updated 3 November 2026*