---
title: "V-RoLoRA — paper by Wang et al. (2026) — summary and ideas"
description: "The paper tackles steerable pluralistic alignment: making an LLM reliably express a chosen set of human values or moral… 39 references."
authors: ["Jing Wang", "Yaomin Wu", "Yinglin Wang", "Yitong Yang"]
year: 2026
venue: "Findings of the Association for Computational Linguistics: ACL 2026"
source: "https://aclanthology.org/2026.findings-acl.1377/"
doi: "10.18653/v1/2026.findings-acl.1377"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/wang-2026-v-rolora-rlvr-driven-moe/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/wang-2026-v-rolora-rlvr-driven-moe/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2026](https://md.research.elamur.ai/free/en/papers/2026/page/47/) / V-RoLoRA: RLVR-Driven MoE Routing for Steerable Pluralistic Alignment

# V-RoLoRA: RLVR-Driven MoE Routing for Steerable Pluralistic Alignment

**Authors:** [Jing Wang](https://md.research.elamur.ai/free/en/authors/jing-wang/), Yaomin Wu, Yinglin Wang and Yitong Yang · **Year:** 2026 · **Venue:** [Findings of the Association for Computational Linguistics: ACL 2026](https://md.research.elamur.ai/free/en/venues/findings/) ([Findings of the Association for Computational Linguistics 2026](https://md.research.elamur.ai/free/en/venues/findings/2026/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2026.findings-acl.1377/) · [PDF](https://aclanthology.org/2026.findings-acl.1377.pdf) · [DOI 10.18653/v1/2026.findings-acl.1377](https://doi.org/10.18653/v1/2026.findings-acl.1377) · [OpenAlex](https://openalex.org/W7166839968)

**Citations:** 0 citations (OpenAlex, 2026-10-14)

## Summary
The paper tackles steerable pluralistic alignment: making an LLM reliably express a chosen set of human values or moral foundations, including multi-value combinations, even when no steering prompt is given. Prompt-based steering tends to be brittle, so the authors move control into the model weights. V-RoLoRA encodes the target as a binary condition vector. A frozen sparse Gaussian projection followed by a small router converts this vector into mixing weights over a pool of LoRA experts, giving each condition its own adapter update.

Training has two stages. Supervised cold-start fine-tuning comes first. GRPO then optimizes a conditional consistency reward, which is supplied by an external value discriminator fine-tuned from Qwen3-0.6B and checks whether the response's values match the target.

On Touché23-ValueEval and MIC, with Llama-3.1-8B and Qwen3-8B backbones, the method beats prompt-based and multi-task PEFT baselines on controllability metrics, and human judges agree. The work shows that verifier-driven RL can bind compositional value control to parameter-efficient adapters, although discriminator bias remains a bottleneck.

## Key points
- **Data and models:** Evaluation uses Touché23-ValueEval (values) and MIC (morals) on Llama-3.1-8B and Qwen3-8B. Metrics are micro-F1, macro-F1 and Jaccard, all computed with the discriminator.
- **Main result on Touché23-ValueEval (micro-F1):** The full model reaches 0.5290 on Llama-3.1-8B and 0.5483 on Qwen3-8B. On Qwen3-8B, removing GRPO drops the score to about 0.503, and removing the gate drops it to about 0.481.
- **Human evaluation:** Pairwise comparisons on 100 sampled test items per dataset favour V-RoLoRA over every baseline.
- **Discriminator bias:** Confusion matrices over the top-20 value combinations show predictions collapsing onto a few frequent, low-density combinations. Touché samples mostly activate 2–3 values, while MIC samples mostly activate a single value. This imbalance skews the reward signal.
- **Limitations:** The small Qwen3-0.6B verifier may be biased. Only binary presence/absence vectors are tested, so control over value intensity (integer-valued vectors) is left for future work. Code is available at github.com/Bob-Annette/V-RoLoRA.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Shanghai University of Finance and Economics](https://md.research.elamur.ai/free/en/universities/shanghai-university-of-finance-and-economics/) — University, Shanghai, China · 13 works · 825 citations

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 16,039 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 15,891 works

## Cite

Cite as: Jing Wang, Yaomin Wu, Yinglin Wang and Yitong Yang. 2026. V-RoLoRA: RLVR-Driven MoE Routing for Steerable Pluralistic Alignment. Findings of the Association for Computational Linguistics: ACL 2026.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{wang2026v,
  title = {{V-RoLoRA: RLVR-Driven MoE Routing for Steerable Pluralistic Alignment}},
  author = {Jing Wang, Yaomin Wu, Yinglin Wang and Yitong Yang},
  booktitle = {Findings of the Association for Computational Linguistics: ACL 2026},
  year = {2026},
  doi = {10.18653/v1/2026.findings-acl.1377},
  url = {https://doi.org/10.18653/v1/2026.findings-acl.1377}
}
```

- [← previous v-HUB: A Benchmark for Video Humor Understanding from Vision and Sound](https://md.research.elamur.ai/free/en/papers/shi-2026-v-hub-benchmark-video-humor/)
- [next → VAIDYA: Validated Agents for Intelligent Diagnosis and Yielded Analysis](https://md.research.elamur.ai/free/en/papers/shah-2026-vaidya-validated-agents-intelligent-diagnosis/)
- [↑ back to papers of 2026](https://md.research.elamur.ai/free/en/papers/2026/page/47/)

## References

39 references, 15 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Jadie Adams, Brian Hu, Emily Veenhuis, David Joy, Bharadwaj Ravichandran, Aaron Bray, Anthony Hoogs, and Arslan Basharat. 2025. Steerable pluralism: Pluralistic alignment via few-shot comparative regression. Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society, 8(1):15–25. [find](https://scholar.google.com/scholar?q=Jadie+Adams%2C+Brian+Hu%2C+Emily+Veenhuis%2C+David+Joy%2C+Bharadwaj+Ravichandran%2C+Aaron+Bray%2C+Anthony+Hoogs%2C+and+Arslan+Basharat%2E+2025%2E+Steerable+pluralism%3A+Pluralistic+alignment+via+few%2Dshot+comparative+regr)
2. Shreyas Chaudhari, Pranjal Aggarwal, Vishvak Murahari, Tanmay Rajpurohit, Ashwin Kalyan, Karthik Narasimhan, Ameet Deshpande, and Bruno Castro da Silva. 2025. Rlhf deciphered: A critical analysis of reinforcement learning from human feedback for llms. ACM Comput. Surv., 58(2). [find](https://scholar.google.com/scholar?q=Shreyas+Chaudhari%2C+Pranjal+Aggarwal%2C+Vishvak+Murahari%2C+Tanmay+Rajpurohit%2C+Ashwin+Kalyan%2C+Karthik+Narasimhan%2C+Ameet+Deshpande%2C+and+Bruno+Castro+da+Silva%2E+2025%2E+Rlhf+deciphered%3A+A+critical+analysis+of+r)
3. Shangbin Feng, Taylor Sorensen, Yuhan Liu, Jillian Fisher, Chan Young Park, Yejin Choi, and Yulia Tsvetkov. 2024. [Modular pluralism: Pluralistic alignment via multi-LLM collaboration](https://md.research.elamur.ai/free/en/papers/feng-2024-modular-pluralism-pluralistic-alignment-multi/). In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 4151–4171, Miami, F…
4. Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, Amy Yang, Angela Fan, Anirudh Goyal, Anthony Hartshorn, Aobo Yang, Archi Mitra, Archie Sravankumar, and et al. 2024. [The llama 3 herd of mode…](https://md.research.elamur.ai/free/en/papers/grattafiori-2024-llama-3-herd-models/)
5. Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Peiyi Wang, Qihao Zhu, Runxin Xu, Ruoyu Zhang, Shirong Ma, Xiao Bi, and 1 others. 2025. Deepseekr1 incentivizes reasoning in llms through reinforcement learning. Nature, 645(8081):633–638. [find](https://scholar.google.com/scholar?q=Daya+Guo%2C+Dejian+Yang%2C+Haowei+Zhang%2C+Junxiao+Song%2C+Peiyi+Wang%2C+Qihao+Zhu%2C+Runxin+Xu%2C+Ruoyu+Zhang%2C+Shirong+Ma%2C+Xiao+Bi%2C+and+1+others%2E+2025%2E+Deepseekr1+incentivizes+reasoning+in+llms+through+reinforceme)
6. Dan Hendrycks, Collin Burns, Steven Basart, Andrew Critch, Jerry Li, Dawn Song, and Jacob Steinhardt. 2021. [Aligning {ai} with shared human values](https://md.research.elamur.ai/free/en/papers/hendrycks-2020-aligning-ai-shared-human-values/). In International Conference on Learning Representations.
7. Michael Luo, Sijun Tan, Justin Wong, Xiaoxiang Shi, William Tang, Manan Roongta, Colin Cai, Jeffrey Luo, Tianjun Zhang, Erran Li, Raluca Ada Popa, and Ion Stoica. 2025. Deepscaler: Surpassing o1-preview with a 1.5b model by scaling rl. Notion Blog. [find](https://scholar.google.com/scholar?q=Michael+Luo%2C+Sijun+Tan%2C+Justin+Wong%2C+Xiaoxiang+Shi%2C+William+Tang%2C+Manan+Roongta%2C+Colin+Cai%2C+Jeffrey+Luo%2C+Tianjun+Zhang%2C+Erran+Li%2C+Raluca+Ada+Popa%2C+and+Ion+Stoica%2E+2025%2E+Deepscaler%3A+Surpassing+o1%2Dprevi)
8. Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2021. [Lora: Low-rank adaptation of large language models](https://md.research.elamur.ai/free/en/papers/hu-2021-lora-low-rank-adaptation-large/). Preprint, arXiv:2106.09685.
9. Xueguang Ma, Qian Liu, Dongfu Jiang, Ge Zhang, Zejun Ma, and Wenhu Chen. 2025. [General-reasoner: Advancing llm reasoning across all domains](https://md.research.elamur.ai/free/en/papers/ma-2025-general-reasoner-advancing-llm-reasoning/). Preprint, arXiv:2505.14652.
10. Liwei Jiang, Taylor Sorensen, Sydney Levine, and Yejin Choi. 2025. [Can language models reason about individualistic human values and preferences?](https://md.research.elamur.ai/free/en/papers/jiang-2025-can-language-models-reason-about/) In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 6757–6794, Vienna, Austria. Asso…
11. Yufei Ma, Zihan Liang, Huangyu Dai, Ben Chen, Dehong Gao, Zhuoran Ran, Wang Zihan, Linbo Jin, Wen Jiang, Guannan Zhang, Xiaoyan Cai, and Libin Yang. 2024. MoDULA: Mixture of domain-specific and universal LoRA for multi-task learning. In Proceedings of the 2024 Conference on Empirical Methods in Natu… [find](https://scholar.google.com/scholar?q=Yufei+Ma%2C+Zihan+Liang%2C+Huangyu+Dai%2C+Ben+Chen%2C+Dehong+Gao%2C+Zhuoran+Ran%2C+Wang+Zihan%2C+Linbo+Jin%2C+Wen+Jiang%2C+Guannan+Zhang%2C+Xiaoyan+Cai%2C+and+Libin+Yang%2E+2024%2E+MoDULA%3A+Mixture+of+domain%2Dspecific+and+univer)
12. Dongjun Kang, Joonsuk Park, Yohan Jo, and JinYeong Bak. 2023. From values to opinions: Predicting human behaviors and stances using value-injected large language models. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 15539–15559, Singapore. Associati… [find](https://scholar.google.com/scholar?q=Dongjun+Kang%2C+Joonsuk+Park%2C+Yohan+Jo%2C+and+JinYeong+Bak%2E+2023%2E+From+values+to+opinions%3A+Predicting+human+behaviors+and+stances+using+value%2Dinjected+large+language+models%2E+In+Proceedings+of+the+2023+Con)
13. Johannes Kiesel, Milad Alshomary, Nicolas Handke, Xiaoni Cai, Henning Wachsmuth, and Benno Stein. 2022. Identifying the human values behind arguments. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 4459–4471, Dublin, Ireland.… [find](https://scholar.google.com/scholar?q=Johannes+Kiesel%2C+Milad+Alshomary%2C+Nicolas+Handke%2C+Xiaoni+Cai%2C+Henning+Wachsmuth%2C+and+Benno+Stein%2E+2022%2E+Identifying+the+human+values+behind+arguments%2E+In+Proceedings+of+the+60th+Annual+Meeting+of+the+)
14. Nathan Lambert, Jacob Morrison, Valentina Pyatkin, Shengyi Huang, Hamish Ivison, Faeze Brahman, Lester James V. Miranda, Alisa Liu, Nouha Dziri, Shane Lyu, Yuling Gu, Saumya Malik, Victoria Graf, Jena D. Hwang, Jiangjiang Yang, Ronan Le Bras, Oyvind Tafjord, Chris Wilhelm, Luca Soldaini, and 4 other… — [Tulu 3: Pushing Frontiers in Open Language Model Post-Training](https://md.research.elamur.ai/free/en/papers/lambert-2024-tulu-3-pushing-frontiers-open/)
15. Mengqi Liao, Wei Chen, Junfeng Shen, Shengnan Guo, and Huaiyu Wan. 2025. Hmora: Making llms more effective with hierarchical mixture of lora experts. In International Conference on Representation Learning, volume 2025, pages 91309–91330. [find](https://scholar.google.com/scholar?q=Mengqi+Liao%2C+Wei+Chen%2C+Junfeng+Shen%2C+Shengnan+Guo%2C+and+Huaiyu+Wan%2E+2025%2E+Hmora%3A+Making+llms+more+effective+with+hierarchical+mixture+of+lora+experts%2E+In+International+Conference+on+Representation+Lear)
16. Chi Liu, Derek Li, Yan Shu, Robin Chen, Derek Duan, Teng Fang, and Bryan Dai. 2025. Fleming-r1: Toward expert-level medical reasoning via reinforcement learning. Preprint, arXiv:2509.15279. — [arXiv:2509.15279](https://arxiv.org/abs/2509.15279) [find](https://scholar.google.com/scholar?q=Chi+Liu%2C+Derek+Li%2C+Yan+Shu%2C+Robin+Chen%2C+Derek+Duan%2C+Teng+Fang%2C+and+Bryan+Dai%2E+2025%2E+Fleming%2Dr1%3A+Toward+expert%2Dlevel+medical+reasoning+via+reinforcement+learning%2E+Preprint%2C+arXiv%3A2509%2E15279%2E+%E2%80%94+%5BarXiv%3A2)
17. Qidong Liu, Xian Wu, Xiangyu Zhao, Yuanshao Zhu, Derong Xu, Feng Tian, and Yefeng Zheng. 2024. When moe meets llms: Parameter efficient fine-tuning for multi-task medical applications. In Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval,… [find](https://scholar.google.com/scholar?q=Qidong+Liu%2C+Xian+Wu%2C+Xiangyu+Zhao%2C+Yuanshao+Zhu%2C+Derong+Xu%2C+Feng+Tian%2C+and+Yefeng+Zheng%2E+2024%2E+When+moe+meets+llms%3A+Parameter+efficient+fine%2Dtuning+for+multi%2Dtask+medical+applications%2E+In+Proceedings+)
18. Shalom Schwartz. 2012. An overview of the schwartz theory of basic values. Online Readings in Psychology and Culture, 2. [find](https://scholar.google.com/scholar?q=Shalom+Schwartz%2E+2012%2E+An+overview+of+the+schwartz+theory+of+basic+values%2E+Online+Readings+in+Psychology+and+Culture%2C+2%2E)
19. ByteDance Seed, :, Jiaze Chen, Tiantian Fan, Xin Liu, Lingjun Liu, Zhiqi Lin, Mingxuan Wang, Chengyi Wang, Xiangpeng Wei, Wenyuan Xu, Yufeng Yuan, Yu Yue, Lin Yan, Qiying Yu, Xiaochen Zuo, Chi Zhang, Ruofei Zhu, Zhecheng An, and 255 others. 2025. [Seed1.5-thinking: Advancing superb reasoning models w…](https://md.research.elamur.ai/free/en/papers/bytedance-seed-2025-seed1-5-thinking-advancing/)
20. Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, Y. K. Li, Y. Wu, and Daya Guo. 2024. [Deepseekmath: Pushing the limits of mathematical reasoning in open language models](https://md.research.elamur.ai/free/en/papers/shao-2024-deepseekmath-pushing-limits-mathematical-reasoning/). Preprint, arXiv:2402.03300.
21. Noam Shazeer, Azalia Mirhoseini, Krzysztof Maziarz, Andy Davis, Quoc Le, Geoffrey Hinton, and Jeff Dean. 2017. Outrageously large neural networks: The sparsely-gated mixture-of-experts layer. Preprint, arXiv:1701.06538. [find](https://scholar.google.com/scholar?q=Noam+Shazeer%2C+Azalia+Mirhoseini%2C+Krzysztof+Maziarz%2C+Andy+Davis%2C+Quoc+Le%2C+Geoffrey+Hinton%2C+and+Jeff+Dean%2E+2017%2E+Outrageously+large+neural+networks%3A+The+sparsely%2Dgated+mixture%2Dof%2Dexperts+layer%2E+Preprint)
22. Taylor Sorensen, Liwei Jiang, Jena D. Hwang, Sydney Levine, Valentina Pyatkin, Peter West, Nouha Dziri, Ximing Lu, Kavel Rao, Chandra Bhagavatula, Maarten Sap, John Tasioulas, and Yejin Choi. 2024a. [Value kaleidoscope: Engaging ai with pluralistic human values, rights, and duties](https://md.research.elamur.ai/free/en/papers/sorensen-2023-value-kaleidoscope-engaging-ai-pluralistic/). Proceedings of the…
23. Taylor Sorensen, Jared Moore, Jillian Fisher, Mitchell L Gordon, Niloofar Mireshghallah, Christopher Michael Rytting, Andre Ye, Liwei Jiang, Ximing Lu, Nouha Dziri, Tim Althoff, and Yejin Choi. 2024b. Position: [A roadmap to pluralistic alignment](https://md.research.elamur.ai/free/en/papers/sorensen-2024-roadmap-pluralistic-alignment/). In Proceedings of the 41st International Conference o…
24. Chunlin Tian, Zhan Shi, Zhijiang Guo, Li Li, and Chengzhong Xu. 2024. Hydralora: An asymmetric lora architecture for efficient fine-tuning. In Advances in Neural Information Processing Systems (NeurIPS). [find](https://scholar.google.com/scholar?q=Chunlin+Tian%2C+Zhan+Shi%2C+Zhijiang+Guo%2C+Li+Li%2C+and+Chengzhong+Xu%2E+2024%2E+Hydralora%3A+An+asymmetric+lora+architecture+for+efficient+fine%2Dtuning%2E+In+Advances+in+Neural+Information+Processing+Systems+%28NeurIP)
25. Zhiqi Lin, Bole Ma, Guangming Sheng, Yuxuan Tong, Chi Zhang, Mofan Zhang, Wang Zhang, and 16 others. 2025. [Dapo: An open-source llm reinforcement learning system at scale](https://md.research.elamur.ai/free/en/papers/yu-2025-dapo-open-source-llm-reinforcement/). Preprint, arXiv:2503.14476.
26. Chen Wang, Lai Wei, Yanzhi Zhang, Chenyang Shao, Zedong Dan, Weiran Huang, Yuzhi Zhang, and Yue Wang. 2025a. Eframe: Deeper reasoning via exploration-filter-replay reinforcement learning framework. Preprint, arXiv:2506.22200. — [arXiv:2506.22200](https://arxiv.org/abs/2506.22200) [find](https://scholar.google.com/scholar?q=Chen+Wang%2C+Lai+Wei%2C+Yanzhi+Zhang%2C+Chenyang+Shao%2C+Zedong+Dan%2C+Weiran+Huang%2C+Yuzhi+Zhang%2C+and+Yue+Wang%2E+2025a%2E+Eframe%3A+Deeper+reasoning+via+exploration%2Dfilter%2Dreplay+reinforcement+learning+framework%2E+Pr)
27. Yunfan Zhang, Kathleen McKeown, and Smaranda Muresan. 2025. Exploring chain-of-thought reasoning for steerable pluralistic alignment. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pages 25647–25660, Suzhou, China. Association for Computational Linguistics… [find](https://scholar.google.com/scholar?q=Yunfan+Zhang%2C+Kathleen+McKeown%2C+and+Smaranda+Muresan%2E+2025%2E+Exploring+chain%2Dof%2Dthought+reasoning+for+steerable+pluralistic+alignment%2E+In+Proceedings+of+the+2025+Conference+on+Empirical+Methods+in+Natu)
28. Xujia Wang, Haiyan Zhao, Shuo Wang, Hanqing Wang, and Zhiyuan Liu. 2025b. MALoRA: Mixture of asymmetric low-rank adaptation for enhanced multi-task learning. In Findings of the Association for Computational Linguistics: NAACL 2025, pages 5609– 5626, Albuquerque, New Mexico. Association for Computati… [find](https://scholar.google.com/scholar?q=Xujia+Wang%2C+Haiyan+Zhao%2C+Shuo+Wang%2C+Hanqing+Wang%2C+and+Zhiyuan+Liu%2E+2025b%2E+MALoRA%3A+Mixture+of+asymmetric+low%2Drank+adaptation+for+enhanced+multi%2Dtask+learning%2E+In+Findings+of+the+Association+for+Computa)
29. Chujie Zheng, Shixuan Liu, Mingze Li, Xiong-Hui Chen, Bowen Yu, Chang Gao, Kai Dang, Yuqiong Liu, Rui Men, An Yang, Jingren Zhou, and Junyang Lin. 2025. [Group sequence policy optimization](https://md.research.elamur.ai/free/en/papers/zheng-2025-group-sequence-policy-optimization/). Preprint, arXiv:2507.18071.
30. Zhangchen Xu, Yuetai Li, Fengqing Jiang, Bhaskar Ramasubramanian, Luyao Niu, Bill Yuchen Lin, and Radha Poovendran. 2025. Tinyv: Reducing false negatives in verification improves rl for llm reasoning. Preprint, arXiv:2505.14625. [find](https://scholar.google.com/scholar?q=Zhangchen+Xu%2C+Yuetai+Li%2C+Fengqing+Jiang%2C+Bhaskar+Ramasubramanian%2C+Luyao+Niu%2C+Bill+Yuchen+Lin%2C+and+Radha+Poovendran%2E+2025%2E+Tinyv%3A+Reducing+false+negatives+in+verification+improves+rl+for+llm+reasoning%2E)
31. Yiyun Zhou, Chang Yao, and Jingyuan Chen. 2025. CoLA: Collaborative low-rank adaptation. In Findings of the Association for Computational Linguistics: ACL 2025, pages 14115–14130, Vienna, Austria. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Yiyun+Zhou%2C+Chang+Yao%2C+and+Jingyuan+Chen%2E+2025%2E+CoLA%3A+Collaborative+low%2Drank+adaptation%2E+In+Findings+of+the+Association+for+Computational+Linguistics%3A+ACL+2025%2C+pages+14115%E2%80%9314130%2C+Vienna%2C+Austria%2E+Ass)
32. An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, Chujie Zheng, Dayiheng Liu, Fan Zhou, Fei Huang, Feng Hu, Hao Ge, Haoran Wei, Huan Lin, Jialong Tang, and 41 others. 2025a. [Qwen3 technical report](https://md.research.elamur.ai/free/en/papers/yang-2025-qwen3-technical-report/). Preprint, arXiv:2505.09388.
33. Zhanhui Zhou, Jie Liu, Jing Shao, Xiangyu Yue, Chao Yang, Wanli Ouyang, and Yu Qiao. 2024. Beyond one-preference-fits-all alignment: Multi-objective direct preference optimization. In Findings of the Association for Computational Linguistics ACL 2024, pages 10586–10613. [find](https://scholar.google.com/scholar?q=Zhanhui+Zhou%2C+Jie+Liu%2C+Jing+Shao%2C+Xiangyu+Yue%2C+Chao+Yang%2C+Wanli+Ouyang%2C+and+Yu+Qiao%2E+2024%2E+Beyond+one%2Dpreference%2Dfits%2Dall+alignment%3A+Multi%2Dobjective+direct+preference+optimization%2E+In+Findings+of+the+)
34. Yaming Yang, Dilxat Muhtar, Yelong Shen, Yuefeng Zhan, Jianfeng Liu, Yujing Wang, Hao Sun, Weiwei Deng, Feng Sun, Qi Zhang, Weizhu Chen, and Yunhai Tong. 2025b. Mtl-lora: Low-rank adaptation for multi-task learning. Proceedings of the AAAI Conference on Artificial Intelligence, 39(20):22010–22018. [find](https://scholar.google.com/scholar?q=Yaming+Yang%2C+Dilxat+Muhtar%2C+Yelong+Shen%2C+Yuefeng+Zhan%2C+Jianfeng+Liu%2C+Yujing+Wang%2C+Hao+Sun%2C+Weiwei+Deng%2C+Feng+Sun%2C+Qi+Zhang%2C+Weizhu+Chen%2C+and+Yunhai+Tong%2E+2025b%2E+Mtl%2Dlora%3A+Low%2Drank+adaptation+for+multi)
35. Caleb Ziems, Jane Yu, Yi-Chia Wang, Alon Halevy, and Diyi Yang. 2022. [The moral integrity corpus: A benchmark for ethical dialogue systems](https://md.research.elamur.ai/free/en/papers/ziems-2022-moral-integrity-corpus-benchmark-ethical/). In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 3755–3773, Dublin, Ireland. Associatio…
36. Jing Yao, Xiaoyuan Yi, Shitong Duan, Jindong Wang, Yuzhuo Bai, Muhua Huang, Yang Ou, Scarlett Li, Peng Zhang, Tun Lu, Zhicheng Dou, Maosong Sun, James Evans, and Xing Xie. 2025. Value compass benchmarks: A comprehensive, generative and self-evolving platform for LLMs’ value evaluation. In Proceeding… [find](https://scholar.google.com/scholar?q=Jing+Yao%2C+Xiaoyuan+Yi%2C+Shitong+Duan%2C+Jindong+Wang%2C+Yuzhuo+Bai%2C+Muhua+Huang%2C+Yang+Ou%2C+Scarlett+Li%2C+Peng+Zhang%2C+Tun+Lu%2C+Zhicheng+Dou%2C+Maosong+Sun%2C+James+Evans%2C+and+Xing+Xie%2E+2025%2E+Value+compass+benchmar)
37. Heming Zou, Yunliang Zang, Wutong Xu, Yao Zhu, and Xiangyang Ji. 2025. FlyloRA: Boosting task decoupling and parameter efficiency via implicit rank-wise mixture-of-experts. In The Thirty-ninth Annual Conference on Neural Information Processing Systems. [find](https://scholar.google.com/scholar?q=Heming+Zou%2C+Yunliang+Zang%2C+Wutong+Xu%2C+Yao+Zhu%2C+and+Xiangyang+Ji%2E+2025%2E+FlyloRA%3A+Boosting+task+decoupling+and+parameter+efficiency+via+implicit+rank%2Dwise+mixture%2Dof%2Dexperts%2E+In+The+Thirty%2Dninth+Annual+)
38. A Jing Yao, Xiaoyuan Yi, Yifan Gong, Xiting Wang, and Xing Xie. 2024. Value FULCRA: Mapping large language models to the multidimensional spectrum of basic human value. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Languag… [find](https://scholar.google.com/scholar?q=A+Jing+Yao%2C+Xiaoyuan+Yi%2C+Yifan+Gong%2C+Xiting+Wang%2C+and+Xing+Xie%2E+2024%2E+Value+FULCRA%3A+Mapping+large+language+models+to+the+multidimensional+spectrum+of+basic+human+value%2E+In+Proceedings+of+the+2024+Conf)
39. Qiying Yu, Zheng Zhang, Ruofei Zhu, Yufeng Yuan, Xiaochen Zuo, Yu Yue, Weinan Dai, Tiantian Fan, Gaohong Liu, Lingjun Liu, Xin Liu, Haibin Lin, Notation Summary • x: input prompt; y: generated response; π θ (y \| x, v): conditional generation policy. • v ∈ {0, 1} K : discrete condition vector with K… [find](https://scholar.google.com/scholar?q=Qiying+Yu%2C+Zheng+Zhang%2C+Ruofei+Zhu%2C+Yufeng+Yuan%2C+Xiaochen+Zuo%2C+Yu+Yue%2C+Weinan+Dai%2C+Tiantian+Fan%2C+Gaohong+Liu%2C+Lingjun+Liu%2C+Xin+Liu%2C+Haibin+Lin%2C+Notation+Summary+%E2%80%A2+x%3A+input+prompt%3B+y%3A+generated+respons)

*Updated 1 November 2026*