---
title: "BiasGRPO — paper by Reddy et al. (2026) — summary and ideas"
description: "The authors propose BiasGRPO, which applies DeepSeek's Group Relative Policy Optimization to debiasing. 37 references."
authors: ["Saket Reddy", "Ke Yang", "ChengXiang Zhai"]
year: 2026
venue: "Findings of the Association for Computational Linguistics: ACL 2026"
source: "https://aclanthology.org/2026.findings-acl.2052/"
doi: "10.18653/v1/2026.findings-acl.2052"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/reddy-2026-biasgrpo-stabilizing-bias-mitigation-high/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/reddy-2026-biasgrpo-stabilizing-bias-mitigation-high/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2026](https://md.research.elamur.ai/free/en/papers/2026/page/24/) / BiasGRPO: Stabilizing Bias Mitigation in High-Variance Reward Landscapes via Group-Relative Policy Optimization

# BiasGRPO: Stabilizing Bias Mitigation in High-Variance Reward Landscapes via Group-Relative Policy Optimization

**Authors:** Saket Reddy, [Ke Yang](https://md.research.elamur.ai/free/en/authors/ke-yang/) and [ChengXiang Zhai](https://md.research.elamur.ai/free/en/authors/chengxiang-zhai/) · **Year:** 2026 · **Venue:** [Findings of the Association for Computational Linguistics: ACL 2026](https://md.research.elamur.ai/free/en/venues/findings/) ([Findings of the Association for Computational Linguistics 2026](https://md.research.elamur.ai/free/en/venues/findings/2026/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2026.findings-acl.2052/) · [PDF](https://aclanthology.org/2026.findings-acl.2052.pdf) · [DOI 10.18653/v1/2026.findings-acl.2052](https://doi.org/10.18653/v1/2026.findings-acl.2052) · [OpenAlex](https://openalex.org/W7166823759)

**Citations:** 0 citations (OpenAlex, 2026-10-14)

## Summary
Reducing social bias in LLMs is hard to optimise because no single "correct" unbiased answer exists, so reward signals are noisy and subjective. Existing preference-based approaches each have weaknesses. DPO (tested here in its IPO variant) learns offline from fixed pairs and generalises poorly. PPO explores online but depends on a critic whose value estimates become unreliable in such noisy settings.

The authors propose BiasGRPO, which applies DeepSeek's Group Relative Policy Optimization to debiasing. The pipeline has three parts:
- the base GRPO algorithm, which replaces the critic with rewards normalised within a group of sampled completions;
- a curated, synthetically extended dataset;
- a compact bias-specific reward model.

Because rewards are compared within a group, the policy still receives a learning signal when every completion is biased: the least biased one gets a positive advantage. On Microsoft's Phi-2, BiasGRPO beats DPO and PPO across bias benchmarks while keeping TruthfulQA performance, and it largely carries over to Llama 3.2 (3B). The released resources give RLHF pipelines a cheap, ready-made way to add bias mitigation.

## Key points
- **Released resources:** a dataset covering 11 domains and a 0.1B-parameter bias reward model, both on Hugging Face. The reward model is meant to plug into multi-objective RLHF without extra compute or loss of model knowledge.
- **Evaluation:** benchmarks were BOLD, RealToxicityPrompts, BBQ and TruthfulQA, with a default group size of G = 4.
- **PPO instability:** during PPO training, critic value loss fell only from 0.2507 to 0.2134. PPO reward standard deviation dropped only from 0.1703 to 0.1279.
- **Group-size ablation (Phi-2):** larger groups helped consistently across benchmarks.
  - BOLD (overall): .0293 (base) → .0140 (G = 4) → .0124 (G = 8)
  - BBQ (overall): .2750 → .3123 → .3781
  - TruthfulQA: .3843 → .3941 → .4137
  - G = 2 clearly lagged behind G = 4, as did Online DPO. The authors take this as evidence that group-relative normalisation, not online training alone, drives the gains.
- **Robustness and generalisation:**
  - Swapping in an alternative stereotype reward model still gave large improvements over the base model, though the custom reward model was better on most benchmarks.
  - On Llama 3.2 (3B), GRPO led on all benchmarks except BBQ, where every method hurt performance.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [University of Illinois Urbana-Champaign](https://md.research.elamur.ai/free/en/universities/university-of-illinois-urbana-champaign/) — University, Urbana, United States · 184 works · 69,372 citations

📚 Shelves (6): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 16,039 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 15,891 works · [🤝 Social Sciences](https://md.research.elamur.ai/free/en/shelves/social-sciences/) · 1,772 works · [🤝 Safety Research](https://md.research.elamur.ai/free/en/shelves/social-sciences/safety-research/) · 543 works

## Cite

Cite as: Saket Reddy, Ke Yang and ChengXiang Zhai. 2026. BiasGRPO: Stabilizing Bias Mitigation in High-Variance Reward Landscapes via Group-Relative Policy Optimization. Findings of the Association for Computational Linguistics: ACL 2026.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{reddy2026biasgrpo,
  title = {{BiasGRPO: Stabilizing Bias Mitigation in High-Variance Reward Landscapes via Group-Relative Policy Optimization}},
  author = {Saket Reddy, Ke Yang and ChengXiang Zhai},
  booktitle = {Findings of the Association for Computational Linguistics: ACL 2026},
  year = {2026},
  doi = {10.18653/v1/2026.findings-acl.2052},
  url = {https://doi.org/10.18653/v1/2026.findings-acl.2052}
}
```

- [← previous BIASEDTALES-ML: A Multilingual Dataset for Analyzing Narrative Attribute Distributions in LLM-Generated Stories](https://md.research.elamur.ai/free/en/papers/ouyang-2026-biasedtales-ml-multilingual-dataset-analyzing/)
- [next → Bibliography of Ethiopian Semitic, Cushitic, and Omotic Linguistics XXVIII: 2024](https://md.research.elamur.ai/free/en/papers/krzyzanowska-2026-bibliography-ethiopian-semitic-cushitic-omotic/)
- [↑ back to papers of 2026](https://md.research.elamur.ai/free/en/papers/2026/page/24/)

## References

37 references, 12 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Ahmed Allam. 2024. BiasDPO: Mitigating bias in language models through direct preference optimization. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 4: Student Research Workshop), pages 42–50. [find](https://scholar.google.com/scholar?q=Ahmed+Allam%2E+2024%2E+BiasDPO%3A+Mitigating+bias+in+language+models+through+direct+preference+optimization%2E+In+Proceedings+of+the+62nd+Annual+Meeting+of+the+Association+for+Computational+Linguistics+%28Volum)
2. Mohammad Gheshlaghi Azar, Zhaohan Daniel Guo, Bilal Piot, Remi Munos, Mark Rowland, Michal Valko, and Daniele Calandriello. 2024. [A general theoretical paradigm to understand learning from human preferences](https://md.research.elamur.ai/free/en/papers/azar-2023-general-theoretical-paradigm-understand-learning/). In Proceedings of The 27th International Conference on Artificial Intelligence and Statistic…
3. Daniel Borkan, Lucas Dixon, Jeffrey Sorensen, Nithum Thain, and Lucy Vasserman. 2019. Nuanced metrics for measuring unintended bias with real data for text classification. arXiv preprint arXiv:1903.04561. — [arXiv:1903.04561](https://arxiv.org/abs/1903.04561) [find](https://scholar.google.com/scholar?q=Daniel+Borkan%2C+Lucas+Dixon%2C+Jeffrey+Sorensen%2C+Nithum+Thain%2C+and+Lucy+Vasserman%2E+2019%2E+Nuanced+metrics+for+measuring+unintended+bias+with+real+data+for+text+classification%2E+arXiv+preprint+arXiv%3A1903%2E04)
4. Jwala Dhamala, Tony Sun, Varun Kumar, Satyapriya Krishna, Yada Pruksachatkun, Kai-Wei Chang, and Rahul Gupta. 2021. BOLD: Dataset and metrics for measuring biases in open-ended language generation. In Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency, pages 862–872… [find](https://scholar.google.com/scholar?q=Jwala+Dhamala%2C+Tony+Sun%2C+Varun+Kumar%2C+Satyapriya+Krishna%2C+Yada+Pruksachatkun%2C+Kai%2DWei+Chang%2C+and+Rahul+Gupta%2E+2021%2E+BOLD%3A+Dataset+and+metrics+for+measuring+biases+in+open%2Dended+language+generation%2E+In)
5. Farshid Faal, Ketra Schmitt, and Jia Yuan Yu. 2022. Reward modeling for mitigating toxicity in transformer-based language models. Applied Intelligence, pages 1–16. [find](https://scholar.google.com/scholar?q=Farshid+Faal%2C+Ketra+Schmitt%2C+and+Jia+Yuan+Yu%2E+2022%2E+Reward+modeling+for+mitigating+toxicity+in+transformer%2Dbased+language+models%2E+Applied+Intelligence%2C+pages+1%E2%80%9316%2E)
6. Dan Friedman and Adji Bousso Dieng. 2023. The Vendi score: A diversity evaluation metric for machine learning. Transactions on Machine Learning Research. [find](https://scholar.google.com/scholar?q=Dan+Friedman+and+Adji+Bousso+Dieng%2E+2023%2E+The+Vendi+score%3A+A+diversity+evaluation+metric+for+machine+learning%2E+Transactions+on+Machine+Learning+Research%2E)
7. Isabel O Gallegos, Ryan A Rossi, Joe Barrow, Md Mehrab Tanjim, Sungchul Kim, Franck Dernoncourt, Tong Yu, Ruiyi Zhang, and Nesreen K Ahmed. 2024. [Bias and fairness in large language models: A survey](https://md.research.elamur.ai/free/en/papers/gallegos-2023-bias-fairness-large-language-models/). Computational Linguistics, 50(3):1097– 1179.
8. Samuel Gehman, Suchin Gururangan, Maarten Sap, Yejin Choi, and Noah A Smith. 2020. [RealToxicityPrompts: Evaluating neural toxic degeneration in language models](https://md.research.elamur.ai/free/en/papers/gehman-2020-realtoxicityprompts-evaluating-neural-toxic-degeneration/). In Findings of the Association for Computational Linguistics: EMNLP 2020, pages 3356–3369.
9. Jan Hauke and Tomasz Kossowski. 2011. Comparison of values of Pearson’s and Spearman’s correlation coefficients on the same sets of data. Quaestiones Geographicae, 30(2):87–93. [find](https://scholar.google.com/scholar?q=Jan+Hauke+and+Tomasz+Kossowski%2E+2011%2E+Comparison+of+values+of+Pearson%E2%80%99s+and+Spearman%E2%80%99s+correlation+coefficients+on+the+same+sets+of+data%2E+Quaestiones+Geographicae%2C+30%282%29%3A87%E2%80%9393%2E)
10. Chenghua Huang, Lu Wang, Fangkai Yang, Pu Zhao, Zhixu Li, Qingwei Lin, Dongmei Zhang, Saravan Rajmohan, and Qi Zhang. 2025. Lean and mean: Decoupled value policy optimization with global value guidance. arXiv preprint arXiv:2502.16944. — [arXiv:2502.16944](https://arxiv.org/abs/2502.16944) [find](https://scholar.google.com/scholar?q=Chenghua+Huang%2C+Lu+Wang%2C+Fangkai+Yang%2C+Pu+Zhao%2C+Zhixu+Li%2C+Qingwei+Lin%2C+Dongmei+Zhang%2C+Saravan+Rajmohan%2C+and+Qi+Zhang%2E+2025%2E+Lean+and+mean%3A+Decoupled+value+policy+optimization+with+global+value+guidanc)
11. Ruipeng Jia, Yunyi Yang, Yongbo Gai, Kai Luo, Shihao Huang, Jianhe Lin, Xiaoxi Jiang, and Guanjun Jiang. 2025. Writing-Zero: Bridge the gap between non-verifiable tasks and verifiable rewards. arXiv preprint arXiv:2506.00103. — [arXiv:2506.00103](https://arxiv.org/abs/2506.00103) [find](https://scholar.google.com/scholar?q=Ruipeng+Jia%2C+Yunyi+Yang%2C+Yongbo+Gai%2C+Kai+Luo%2C+Shihao+Huang%2C+Jianhe+Lin%2C+Xiaoxi+Jiang%2C+and+Guanjun+Jiang%2E+2025%2E+Writing%2DZero%3A+Bridge+the+gap+between+non%2Dverifiable+tasks+and+verifiable+rewards%2E+arXiv+p)
12. Tao Li, Daniel Khashabi, Tushar Khot, Ashish Sabharwal, and Vivek Srikumar. 2020. [UNQOVERing stereotyping biases via underspecified questions](https://md.research.elamur.ai/free/en/papers/li-2020-unqovering-stereotyping-biases-underspecified-questions/). In Findings of the Association for Computational Linguistics: EMNLP 2020, pages 3475–3489.
13. Yuanzhi Li, Sébastien Bubeck, Ronen Eldan, Allie Del Giorno, Suriya Gunasekar, and Yin Tat Lee. 2023. [Textbooks are all you need II: Phi-1.5 technical report](https://md.research.elamur.ai/free/en/papers/li-2023-textbooks-all-need-ii-phi/). arXiv preprint arXiv:2309.05463.
14. Stephanie Lin, Jacob Hilton, and Owain Evans. 2022. [TruthfulQA: Measuring how models mimic human falsehoods](https://md.research.elamur.ai/free/en/papers/lin-2022-truthfulqa-measuring-how-models-mimic/). In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 3214–3252.
15. Yong Lin, Skyler Seto, Maartje Ter Hoeve, Katherine Metcalf, Barry-John Theobald, Xuan Wang, Yizhe Zhang, and Chen Huang. 2024. On the limited generalization capability of the implicit reward model induced by direct preference optimization. In Findings of the Association for Computational Linguistic… [find](https://scholar.google.com/scholar?q=Yong+Lin%2C+Skyler+Seto%2C+Maartje+Ter+Hoeve%2C+Katherine+Metcalf%2C+Barry%2DJohn+Theobald%2C+Xuan+Wang%2C+Yizhe+Zhang%2C+and+Chen+Huang%2E+2024%2E+On+the+limited+generalization+capability+of+the+implicit+reward+model+in)
16. Yang Liu. 2024. Quantifying stereotypes in language. In Proceedings of the 18th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1223– 1240. [find](https://scholar.google.com/scholar?q=Yang+Liu%2E+2024%2E+Quantifying+stereotypes+in+language%2E+In+Proceedings+of+the+18th+Conference+of+the+European+Chapter+of+the+Association+for+Computational+Linguistics+%28Volume+1%3A+Long+Papers%29%2C+pages+1223%E2%80%93)
17. Yiran Liu, Ke Yang, Zehan Qi, Xiao Liu, Yang Yu, and ChengXiang Zhai. 2025. Bias and volatility: A statistical framework for evaluating large language model’s stereotypes and the associated generation inconsistency. Preprint, arXiv:2402.15481. — [arXiv:2402.15481](https://arxiv.org/abs/2402.15481) [find](https://scholar.google.com/scholar?q=Yiran+Liu%2C+Ke+Yang%2C+Zehan+Qi%2C+Xiao+Liu%2C+Yang+Yu%2C+and+ChengXiang+Zhai%2E+2025%2E+Bias+and+volatility%3A+A+statistical+framework+for+evaluating+large+language+model%E2%80%99s+stereotypes+and+the+associated+generation)
18. Jordan J Louviere, Terry N Flynn, and Anthony AJ Marley. 2015. Best-Worst Scaling: Theory, Methods and Applications. Cambridge University Press. [find](https://scholar.google.com/scholar?q=Jordan+J+Louviere%2C+Terry+N+Flynn%2C+and+Anthony+AJ+Marley%2E+2015%2E+Best%2DWorst+Scaling%3A+Theory%2C+Methods+and+Applications%2E+Cambridge+University+Press%2E)
19. John Schulman, Philipp Moritz, Sergey Levine, Michael I. Jordan, and Pieter Abbeel. 2016. High-dimensional continuous control using generalized advantage estimation. In International Conference on Learning Representations. [find](https://scholar.google.com/scholar?q=John+Schulman%2C+Philipp+Moritz%2C+Sergey+Levine%2C+Michael+I%2E+Jordan%2C+and+Pieter+Abbeel%2E+2016%2E+High%2Ddimensional+continuous+control+using+generalized+advantage+estimation%2E+In+International+Conference+on+Lea)
20. Lucas Maystre and Matthias Grossglauser. 2015. Fast and accurate inference of Plackett-Luce models. In Advances in Neural Information Processing Systems, volume 28. [find](https://scholar.google.com/scholar?q=Lucas+Maystre+and+Matthias+Grossglauser%2E+2015%2E+Fast+and+accurate+inference+of+Plackett%2DLuce+models%2E+In+Advances+in+Neural+Information+Processing+Systems%2C+volume+28%2E)
21. Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Mingchuan Zhang, YK Li, Y Wu, and Daya Guo. 2024. [DeepSeekMath: Pushing the limits of mathematical reasoning in open language models](https://md.research.elamur.ai/free/en/papers/shao-2024-deepseekmath-pushing-limits-mathematical-reasoning/). arXiv preprint arXiv:2402.03300.
22. Quinn McNemar. 1947. Note on the sampling error of the difference between correlated proportions or percentages. Psychometrika, 12(2):153–157. [find](https://scholar.google.com/scholar?q=Quinn+McNemar%2E+1947%2E+Note+on+the+sampling+error+of+the+difference+between+correlated+proportions+or+percentages%2E+Psychometrika%2C+12%282%29%3A153%E2%80%93157%2E)
23. Charles Spearman. 1904. The proof and measurement of association between two things. The American Journal of Psychology, 15(1):72–101. [find](https://scholar.google.com/scholar?q=Charles+Spearman%2E+1904%2E+The+proof+and+measurement+of+association+between+two+things%2E+The+American+Journal+of+Psychology%2C+15%281%29%3A72%E2%80%93101%2E)
24. Meta. 2024. Llama 3.2 Model Card. [find](https://scholar.google.com/scholar?q=Meta%2E+2024%2E+Llama+3%2E2+Model+Card%2E)
25. Yixuan Tang and Yi Yang. 2025. SR-GRPO: Stable rank Https://github.com/meta-llama/llama-as an intrinsic geometric reward for large language models/blob/main/models/llama3_2/MODEL_CARD.md. model alignment. arXiv preprint arXiv:2512.02807. — [arXiv:2512.02807](https://arxiv.org/abs/2512.02807) [find](https://scholar.google.com/scholar?q=Yixuan+Tang+and+Yi+Yang%2E+2025%2E+SR%2DGRPO%3A+Stable+rank+Https%3A%2F%2Fgithub%2Ecom%2Fmeta%2Dllama%2Fllama%2Das+an+intrinsic+geometric+reward+for+large+language+models%2Fblob%2Fmain%2Fmodels%2Fllama3%5F2%2FMODEL%5FCARD%2Emd%2E+model+alignm)
26. Moin Nadeem, Anna Bethke, and Siva Reddy. 2021. [find](https://scholar.google.com/scholar?q=Moin+Nadeem%2C+Anna+Bethke%2C+and+Siva+Reddy%2E+2021%2E)
27. Valurank. 2022. [distilroberta-bias](https://md.research.elamur.ai/free/en/papers/nadeem-2021-stereoset-measuring-stereotypical-bias-pretrained/). StereoSet: Measuring stereotypical bias in pretrained [https://huggingface.co/valurank/](https://huggingface.co/valurank/) language models. In Proceedings of the 59th Annual distilroberta-bias. Meeting of the Association for Computational Linguistics and the 11th International Joint Conference Berti… [↗](https://huggingface.co/valurank/)
28. Roberto Navigli, Simone Conia, and Björn Ross. 2023. Biases in large language models: Origins, inventory and discussion. ACM Journal of Data and Information Quality. [find](https://scholar.google.com/scholar?q=Roberto+Navigli%2C+Simone+Conia%2C+and+Bj%C3%B6rn+Ross%2E+2023%2E+Biases+in+large+language+models%3A+Origins%2C+inventory+and+discussion%2E+ACM+Journal+of+Data+and+Information+Quality%2E)
29. Frank Wilcoxon. 1945. Individual comparisons by ranking methods. Biometrics Bulletin, 1(6):80–83. [find](https://scholar.google.com/scholar?q=Frank+Wilcoxon%2E+1945%2E+Individual+comparisons+by+ranking+methods%2E+Biometrics+Bulletin%2C+1%286%29%3A80%E2%80%9383%2E)
30. Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, and 1 others. 2022. [Training language models to follow instructions with human feedback](https://md.research.elamur.ai/free/en/papers/ouyang-2022-training-language-models-follow-instructions/). In Advances in Neural Information Processing Systems, volume 35, pag…
31. Alicia Parrish, Angelica Chen, Nikita Nangia, Vishakh Padmakumar, Jason Phang, Jana Thompson, Phu Mon Htut, and Samuel Bowman. 2022. [BBQ: A hand-built bias benchmark for question answering](https://md.research.elamur.ai/free/en/papers/parrish-2022-bbq-hand-built-bias-benchmark/). In Findings of the Association for Computational Linguistics: ACL 2022, pages 2086–2105.
32. Perspective API. 2017. Perspective API: Attributes and Languages. [https://developers.perspectiveapi.com/s/](https://developers.perspectiveapi.com/s/) about-the-api-attributes-and-languages? language=en_US. [↗](https://developers.perspectiveapi.com/s/)
33. Yihong Wu, Liheng Ma, Lei Ding, Muzhi Li, Xinyu Wang, Kejia Chen, Zhan Su, Zhanguang Zhang, Chenyang Huang, Yingxue Zhang, Mark Coates, and Jian-Yun Nie. 2025. It takes two: Your GRPO is secretly DPO. In Advances in Neural Information Processing Systems. [find](https://scholar.google.com/scholar?q=Yihong+Wu%2C+Liheng+Ma%2C+Lei+Ding%2C+Muzhi+Li%2C+Xinyu+Wang%2C+Kejia+Chen%2C+Zhan+Su%2C+Zhanguang+Zhang%2C+Chenyang+Huang%2C+Yingxue+Zhang%2C+Mark+Coates%2C+and+Jian%2DYun+Nie%2E+2025%2E+It+takes+two%3A+Your+GRPO+is+secretly+DPO%2E)
34. Xuan Xie, Xuan Wang, and Wenjie Wang. 2025. Da-GRPO: Rectifying gradient conflict in reasoning via distinctiveness-aware group relative policy optimization. arXiv preprint arXiv:2512.06337. — [arXiv:2512.06337](https://arxiv.org/abs/2512.06337) [find](https://scholar.google.com/scholar?q=Xuan+Xie%2C+Xuan+Wang%2C+and+Wenjie+Wang%2E+2025%2E+Da%2DGRPO%3A+Rectifying+gradient+conflict+in+reasoning+via+distinctiveness%2Daware+group+relative+policy+optimization%2E+arXiv+preprint+arXiv%3A2512%2E06337%2E+%E2%80%94+%5BarXiv%3A2)
35. Shaokun Zhang. 2024. bias-detection. [https://huggingface.co/D1V1DE/bias-detection](https://huggingface.co/D1V1DE/bias-detection). [↗](https://huggingface.co/D1V1DE/bias-detection)
36. Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric P. Xing, Hao Zhang, Joseph E. Gonzalez, and Ion Stoica. 2023. [Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena](https://md.research.elamur.ai/free/en/papers/zheng-2023-judging-llm-judge-mt-bench/). In Advances in Neural Information Processing Systems, volu…
37. Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D Manning, Stefano Ermon, and Chelsea Finn. 2023. [Direct preference optimization: Your language model is secretly a reward model](https://md.research.elamur.ai/free/en/papers/rafailov-2023-direct-preference-optimization-language-model/). In Advances in Neural Information Processing Systems, volume 36, pages 53728–53741.

*Updated 1 November 2026*