---
title: "H3Fusion: Helpful, Harmless, Honest Fusion of Aligned LLMs"
description: "Aligning an LLM to be helpful, harmless and honest (HHH) at once is difficult. 60 references, cited by 1 paper here."
authors: ["Selim Furkan Tekin", "Fatih Ilhan", "Sihao Hu", "Tiansheng Huang", "Yichang Xu", "Zachary Yahn", "Ling Liu"]
year: 2026
venue: "Proceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers)"
source: "https://aclanthology.org/2026.eacl-long.329/"
doi: "10.18653/v1/2026.eacl-long.329"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/tekin-2026-h3fusion-helpful-harmless-honest-fusion/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/tekin-2026-h3fusion-helpful-harmless-honest-fusion/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2026](https://md.research.elamur.ai/free/en/papers/2026/page/13/) / H3Fusion: Helpful, Harmless, Honest Fusion of Aligned LLMs

# H3Fusion: Helpful, Harmless, Honest Fusion of Aligned LLMs

**Authors:** Selim Furkan Tekin, Fatih Ilhan, Sihao Hu, [Tiansheng Huang](https://md.research.elamur.ai/free/en/authors/tiansheng-huang/), Yichang Xu, Zachary Yahn and [Ling Liu](https://md.research.elamur.ai/free/en/authors/ling-liu/) · **Year:** 2026 · **Venue:** [Proceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers)](https://md.research.elamur.ai/free/en/venues/eacl/) ([EACL 2026](https://md.research.elamur.ai/free/en/venues/eacl/2026/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2026.eacl-long.329/) · [PDF](https://aclanthology.org/2026.eacl-long.329.pdf) · [DOI 10.18653/v1/2026.eacl-long.329](https://doi.org/10.18653/v1/2026.eacl-long.329) · [OpenAlex](https://openalex.org/W7140073040)

**Citations:** 0 citations (OpenAlex, 2026-10-14) · 1 in our library, of the 149,439 papers whose bibliographies we read

## Summary
Aligning an LLM to be helpful, harmless and honest (HHH) at once is difficult. Tuning for one property often hurts another: safety tuning can make a model overly cautious, and pushing helpfulness can increase hallucinations. H3Fusion addresses this by merging three separately aligned copies of one base model into a single mixture-of-experts (MoE) model. Each copy is fine-tuned on helpfulness, safety or truthfulness data.

The FFN layers are replaced with sparsely-gated MoE layers whose experts are initialized from the three aligned models. Self-attention is kept, so only the router weights need a short fine-tuning on mixed data. Two extra losses guide this training:
- a drift-regularization term with per-expert constants (γ) limits how far each expert's weights move;
- a gating loss (λ) penalizes the router when it picks the wrong expert.

The method was tested on Alpaca-Eval, BeaverTails and TruthfulQA. It beats the individually aligned models, LLM ensembling methods and weight-merging methods. This matters because it reuses existing aligned models and lets practitioners adjust the balance between alignment dimensions.

## Key points
- Base architectures used: LLaMA-2 7B, LLaMA-3 8B and Qwen-2 7B. The approach should transfer to any self-attention model with FFN layers.
- Reported gains: +11.37% over the individually aligned models, +13.77% over state-of-the-art LLM ensembles and +6.18% over model-merging methods.
- Two simpler fusion variants serve as comparisons: H3Fusion-Instruct (instruction-tuned output fusion) and H3Fusion-Summary (in the style of LLM-TOPLA). Both require generating outputs from every base model first.
- Evaluation relies on LLM judges (GPT-4o, GPT-Judge, a locally trained BeaverTails model). The gating and regularization losses add roughly 35 minutes of training time.
- Code: github.com/git-disl/h3fusion (Georgia Tech, EACL 2026).

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Georgia Institute of Technology](https://md.research.elamur.ai/free/en/universities/georgia-institute-of-technology/) — University, Atlanta, United States · 133 works · 24,497 citations

📚 Shelves (5): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 16,039 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 15,891 works · [👁️ Computer Vision and Pattern Recognition](https://md.research.elamur.ai/free/en/shelves/computer-science/computer-vision-and-pattern-recognition/) · 2,869 works

🧭 Topics (2): [🧠 Language models & pretraining](https://md.research.elamur.ai/free/en/topics/language-models/) · 3,320 works · [🛡️ Bias, fairness & safety](https://md.research.elamur.ai/free/en/topics/bias-fairness-safety/) · 517 works

## Cite

Cite as: Selim Furkan Tekin, Fatih Ilhan, Sihao Hu, Tiansheng Huang, Yichang Xu, Zachary Yahn and Ling Liu. 2026. H3Fusion: Helpful, Harmless, Honest Fusion of Aligned LLMs. Proceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers).

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{tekin2026h3fusion,
  title = {{H3Fusion: Helpful, Harmless, Honest Fusion of Aligned LLMs}},
  author = {Selim Furkan Tekin, Fatih Ilhan, Sihao Hu, Tiansheng Huang, Yichang Xu, Zachary Yahn and Ling Liu},
  booktitle = {Proceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2026},
  doi = {10.18653/v1/2026.eacl-long.329},
  url = {https://doi.org/10.18653/v1/2026.eacl-long.329}
}
```

- [← previous H-Mem: Hybrid Multi-Dimensional Memory Management for Long-Context Conversational Agents](https://md.research.elamur.ai/free/en/papers/ye-2026-h-mem-hybrid-multi-dimensional/)
- [next → HABIB_TAZ at SemEval-2026 Task 11: Disentangling Formal Logic from Content via Synthetic Training and Multi-Objective Optimization](https://md.research.elamur.ai/free/en/papers/shaikh-2026-habib-taz-semeval-2026-task/)
- [↑ back to papers of 2026](https://md.research.elamur.ai/free/en/papers/2026/page/13/)

## References

60 references, 33 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, and 1 others. 2023. [Gpt-4 technical report](https://md.research.elamur.ai/free/en/papers/openai-2023-gpt-4-technical-report/). arXiv preprint arXiv:2303.08774.
2. Amanda Askell, Yuntao Bai, Anna Chen, Dawn Drain, Deep Ganguli, Tom Henighan, Andy Jones, Nicholas Joseph, Ben Mann, Nova DasSarma, and 1 others. 2021. A general language assistant as a laboratory for alignment. arXiv preprint arXiv:2112.00861. [find](https://scholar.google.com/scholar?q=Amanda+Askell%2C+Yuntao+Bai%2C+Anna+Chen%2C+Dawn+Drain%2C+Deep+Ganguli%2C+Tom+Henighan%2C+Andy+Jones%2C+Nicholas+Joseph%2C+Ben+Mann%2C+Nova+DasSarma%2C+and+1+others%2E+2021%2E+A+general+language+assistant+as+a+laboratory+for)
3. Yuntao Bai, Andy Jones, Kamal Ndousse, Amanda Askell, Anna Chen, Nova DasSarma, Dawn Drain, Stanislav Fort, Deep Ganguli, Tom Henighan, and 1 others. 2022. Training a helpful and harmless assistant with reinforcement learning from human feedback. arXiv preprint arXiv:2204.05862. [find](https://scholar.google.com/scholar?q=Yuntao+Bai%2C+Andy+Jones%2C+Kamal+Ndousse%2C+Amanda+Askell%2C+Anna+Chen%2C+Nova+DasSarma%2C+Dawn+Drain%2C+Stanislav+Fort%2C+Deep+Ganguli%2C+Tom+Henighan%2C+and+1+others%2E+2022%2E+Training+a+helpful+and+harmless+assistant+wi)
4. Iz Beltagy, Matthew E Peters, and Arman Cohan. 2020. [Longformer: The long-document transformer](https://md.research.elamur.ai/free/en/papers/beltagy-2020-longformer-long-document-transformer/). arXiv preprint arXiv:2004.05150.
5. Federico Bianchi, Mirac Suzgun, Giuseppe Attanasio, Paul Röttger, Dan Jurafsky, Tatsunori Hashimoto, and James Zou. 2023. [Safety-tuned llamas: Lessons from improving the safety of large language models that follow instructions](https://md.research.elamur.ai/free/en/papers/bianchi-2023-safety-tuned-llamas-lessons-improving/). arXiv preprint arXiv:2309.07875.
6. Tom B Brown. 2020. [Language models are few-shot learners](https://md.research.elamur.ai/free/en/papers/brown-2020-language-models-few-shot-learners/). arXiv preprint arXiv:2005.14165.
7. Chi-Min Chan, Weize Chen, Yusheng Su, Jianxuan Yu, Wei Xue, Shanghang Zhang, Jie Fu, and Zhiyuan Liu. 2023. [Chateval: Towards better llm-based evaluators through multi-agent debate](https://md.research.elamur.ai/free/en/papers/chan-2023-chateval-better-llm-based/). arXiv preprint arXiv:2308.07201.
8. Josef Dai, Xuehai Pan, Ruiyang Sun, Jiaming Ji, Xinbo Xu, Mickel Liu, Yizhou Wang, and Yaodong Yang. 2023. [Safe rlhf: Safe reinforcement learning from human feedback](https://md.research.elamur.ai/free/en/papers/dai-2023-safe-rlhf-safe-reinforcement-learning/). arXiv preprint arXiv:2310.12773.
9. Hanze Dong, Wei Xiong, Deepanshu Goyal, Yihan Zhang, Winnie Chow, Rui Pan, Shizhe Diao, Jipeng Zhang, Kashun Shum, and Tong Zhang. 2023. [Raft: Reward ranked finetuning for generative foundation model alignment](https://md.research.elamur.ai/free/en/papers/dong-2023-raft-reward-ranked-finetuning-generative/). arXiv preprint arXiv:2304.06767.
10. Yilun Du, Shuang Li, Antonio Torralba, Joshua B Tenenbaum, and Igor Mordatch. 2023. [Improving factuality and reasoning in language models through multiagent debate](https://md.research.elamur.ai/free/en/papers/du-2023-improving-factuality-reasoning-language-models/). arXiv preprint arXiv:2305.14325.
11. Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, and 1 others. 2024. [The llama 3 herd of models](https://md.research.elamur.ai/free/en/papers/grattafiori-2024-llama-3-herd-models/). arXiv e-prints, pages arXiv–2407.
12. Wang et. al. 2024. [Interpretable preferences via multi-objective reward modeling and mixture-of-experts](https://md.research.elamur.ai/free/en/papers/wang-2024-interpretable-preferences-multi-objective-reward/). arXiv preprint arXiv:2406.12845.
13. Yao Fu, Hao Peng, Ashish Sabharwal, Peter Clark, and Tushar Khot. 2022. [Complexity-based prompting for multi-step reasoning](https://md.research.elamur.ai/free/en/papers/fu-2022-complexity-based-prompting-multi-step/). In The Eleventh International Conference on Learning Representations.
14. Yifei He, Yuzheng Hu, Yong Lin, Tong Zhang, and Han Zhao. 2024. Localize-and-stitch: Efficient model merging via sparse task arithmetic. arXiv preprint arXiv:2408.13656. — [arXiv:2408.13656](https://arxiv.org/abs/2408.13656) [find](https://scholar.google.com/scholar?q=Yifei+He%2C+Yuzheng+Hu%2C+Yong+Lin%2C+Tong+Zhang%2C+and+Han+Zhao%2E+2024%2E+Localize%2Dand%2Dstitch%3A+Efficient+model+merging+via+sparse+task+arithmetic%2E+arXiv+preprint+arXiv%3A2408%2E13656%2E+%E2%80%94+%5BarXiv%3A2408%2E13656%5D%28https%3A%2F%2Fa)
15. Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2021. [Lora: Low-rank adaptation of large language models](https://md.research.elamur.ai/free/en/papers/hu-2021-lora-low-rank-adaptation-large/). arXiv preprint arXiv:2106.09685.
16. Tiansheng Huang, Sihao Hu, Fatih Ilhan, Selim Furkan Tekin, and Ling Liu. 2024a. Booster: Tackling harmful fine-tuing for large language models via attenuating harmful perturbation. arXiv preprint arXiv:2409.01586. — [arXiv:2409.01586](https://arxiv.org/abs/2409.01586) [find](https://scholar.google.com/scholar?q=Tiansheng+Huang%2C+Sihao+Hu%2C+Fatih+Ilhan%2C+Selim+Furkan+Tekin%2C+and+Ling+Liu%2E+2024a%2E+Booster%3A+Tackling+harmful+fine%2Dtuing+for+large+language+models+via+attenuating+harmful+perturbation%2E+arXiv+preprint+arX)
17. Tiansheng Huang, Sihao Hu, Fatih Ilhan, Selim Furkan Tekin, and Ling Liu. 2024b. Lazy safety alignment for large language models against harmful fine-tuning. arXiv preprint arXiv:2405.18641. — [arXiv:2405.18641](https://arxiv.org/abs/2405.18641) [find](https://scholar.google.com/scholar?q=Tiansheng+Huang%2C+Sihao+Hu%2C+Fatih+Ilhan%2C+Selim+Furkan+Tekin%2C+and+Ling+Liu%2E+2024b%2E+Lazy+safety+alignment+for+large+language+models+against+harmful+fine%2Dtuning%2E+arXiv+preprint+arXiv%3A2405%2E18641%2E+%E2%80%94+%5BarXiv%3A)
18. Jiaming Ji, Mickel Liu, Josef Dai, Xuehai Pan, Chi Zhang, Ce Bian, Boyuan Chen, Ruiyang Sun, Yizhou Wang, and Yaodong Yang. 2024. [Beavertails: Towards improved safety alignment of llm via a human-preference dataset](https://md.research.elamur.ai/free/en/papers/ji-2023-beavertails-towards-improved-safety-alignment/). Advances in Neural Information Processing Systems, 36.
19. Albert Q Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, and 1 others. 2023a. Mistral 7b. arXiv preprint arXiv:2310.06825. [find](https://scholar.google.com/scholar?q=Albert+Q+Jiang%2C+Alexandre+Sablayrolles%2C+Arthur+Mensch%2C+Chris+Bamford%2C+Devendra+Singh+Chaplot%2C+Diego+de+las+Casas%2C+Florian+Bressand%2C+Gianna+Lengyel%2C+Guillaume+Lample%2C+Lucile+Saulnier%2C+and+1+others%2E+202)
20. Albert Q Jiang, Alexandre Sablayrolles, Antoine Roux, Arthur Mensch, Blanche Savary, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Emma Bou Hanna, Florian Bressand, and 1 others. 2024. Mixtral of experts. arXiv preprint arXiv:2401.04088. [find](https://scholar.google.com/scholar?q=Albert+Q+Jiang%2C+Alexandre+Sablayrolles%2C+Antoine+Roux%2C+Arthur+Mensch%2C+Blanche+Savary%2C+Chris+Bamford%2C+Devendra+Singh+Chaplot%2C+Diego+de+las+Casas%2C+Emma+Bou+Hanna%2C+Florian+Bressand%2C+and+1+others%2E+2024%2E+Mi)
21. Dongfu Jiang, Xiang Ren, and Bill Yuchen Lin. 2023b. [Llm-blender: Ensembling large language models with pairwise ranking and generative fusion](https://md.research.elamur.ai/free/en/papers/jiang-2023-llm-blender-ensembling-large-language/). arXiv preprint arXiv:2306.02561.
22. Xisen Jin, Xiang Ren, Daniel Preotiuc-Pietro, and Pengxiang Cheng. 2022. Dataless knowledge fusion by merging weights of language models. arXiv preprint arXiv:2212.09849. [find](https://scholar.google.com/scholar?q=Xisen+Jin%2C+Xiang+Ren%2C+Daniel+Preotiuc%2DPietro%2C+and+Pengxiang+Cheng%2E+2022%2E+Dataless+knowledge+fusion+by+merging+weights+of+language+models%2E+arXiv+preprint+arXiv%3A2212%2E09849%2E)
23. Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen-tau Yih, Tim Rocktäschel, and 1 others. 2020. [Retrieval-augmented generation for knowledge-intensive nlp tasks](https://md.research.elamur.ai/free/en/papers/lewis-2020-retrieval-augmented-generation-knowledge-intensive/). Advances in Neural Information Processing Systems, 33:9459–…
24. Junyou Li, Qin Zhang, Yangbin Yu, Qiang Fu, and Deheng Ye. 2024a. More agents is all you need. arXiv preprint arXiv:2402.05120. [find](https://scholar.google.com/scholar?q=Junyou+Li%2C+Qin+Zhang%2C+Yangbin+Yu%2C+Qiang+Fu%2C+and+Deheng+Ye%2E+2024a%2E+More+agents+is+all+you+need%2E+arXiv+preprint+arXiv%3A2402%2E05120%2E)
25. Kenneth Li, Oam Patel, Fernanda Viégas, Hanspeter Pfister, and Martin Wattenberg. 2024b. [Inference-time intervention: Eliciting truthful answers from a language model](https://md.research.elamur.ai/free/en/papers/li-2023-inference-time-intervention-eliciting-truthful/). Advances in Neural Information Processing Systems, 36.
26. Xuechen Li, Tianyi Zhang, Yann Dubois, Rohan Taori, Ishaan Gulrajani, Carlos Guestrin, Percy Liang, and Tatsunori B. Hashimoto. 2023. Alpacaeval: An automatic evaluator of instruction-following models. [https://github.com/tatsu-lab/alpaca_eval](https://github.com/tatsu-lab/alpaca_eval). [↗](https://github.com/tatsu-lab/alpaca_eval)
27. Tiansheng Huang, Sihao Hu, and Ling Liu. 2024c. Vaccine: Perturbation-aware alignment for large language model. arXiv preprint arXiv:2402.01109. — [arXiv:2402.01109](https://arxiv.org/abs/2402.01109) [find](https://scholar.google.com/scholar?q=Tiansheng+Huang%2C+Sihao+Hu%2C+and+Ling+Liu%2E+2024c%2E+Vaccine%3A+Perturbation%2Daware+alignment+for+large+language+model%2E+arXiv+preprint+arXiv%3A2402%2E01109%2E+%E2%80%94+%5BarXiv%3A2402%2E01109%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2402%2E01109%29)
28. Yifei Li, Zeqi Lin, Shizhuo Zhang, Qiang Fu, Bei Chen, Jian-Guang Lou, and Weizhu Chen. 2022. On the advance of making language models better reasoners. arXiv preprint arXiv:2206.02336. — [arXiv:2206.02336](https://arxiv.org/abs/2206.02336) [find](https://scholar.google.com/scholar?q=Yifei+Li%2C+Zeqi+Lin%2C+Shizhuo+Zhang%2C+Qiang+Fu%2C+Bei+Chen%2C+Jian%2DGuang+Lou%2C+and+Weizhu+Chen%2E+2022%2E+On+the+advance+of+making+language+models+better+reasoners%2E+arXiv+preprint+arXiv%3A2206%2E02336%2E+%E2%80%94+%5BarXiv%3A2206%2E)
29. Gabriel Ilharco, Marco Tulio Ribeiro, Mitchell Wortsman, Suchin Gururangan, Ludwig Schmidt, Hannaneh Hajishirzi, and Ali Farhadi. 2022. [Editing models with task arithmetic](https://md.research.elamur.ai/free/en/papers/ilharco-2022-editing-models-task-arithmetic/). arXiv preprint arXiv:2212.04089.
30. Tian Liang, Zhiwei He, Wenxiang Jiao, Xing Wang, Yan Wang, Rui Wang, Yujiu Yang, Shuming Shi, and Zhaopeng Tu. 2023. [Encouraging divergent thinking in large language models through multi-agent debate](https://md.research.elamur.ai/free/en/papers/liang-2023-encouraging-divergent-thinking-large-language/). arXiv preprint arXiv:2305.19118.
31. Stephanie Lin, Jacob Hilton, and Owain Evans. 2021. [Truthfulqa: Measuring how models mimic human falsehoods](https://md.research.elamur.ai/free/en/papers/lin-2022-truthfulqa-measuring-how-models-mimic/). arXiv preprint arXiv:2109.07958.
32. I Loshchilov. 2017. Decoupled weight decay regularization. arXiv preprint arXiv:1711.05101. [find](https://scholar.google.com/scholar?q=I+Loshchilov%2E+2017%2E+Decoupled+weight+decay+regularization%2E+arXiv+preprint+arXiv%3A1711%2E05101%2E)
33. Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, and 1 others. 2023. [Llama 2: Open foundation and fine-tuned chat models](https://md.research.elamur.ai/free/en/papers/touvron-2023-llama-2-open-foundation-fine/). arXiv preprint arXiv:2307.09288.
34. Michael S Matena and Colin A Raffel. 2022. [Merging models with fisher-weighted averaging](https://md.research.elamur.ai/free/en/papers/matena-2021-merging-models-fisher-weighted-averaging/). Advances in Neural Information Processing Systems, 35:17703– 17716.
35. Laurens Van der Maaten and Geoffrey Hinton. 2008. Visualizing data using t-sne. Journal of machine learning research, 9(11). [find](https://scholar.google.com/scholar?q=Laurens+Van+der+Maaten+and+Geoffrey+Hinton%2E+2008%2E+Visualizing+data+using+t%2Dsne%2E+Journal+of+machine+learning+research%2C+9%2811%29%2E)
36. Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, and 1 others. 2022. [Training language models to follow instructions with human feedback](https://md.research.elamur.ai/free/en/papers/ouyang-2022-training-language-models-follow-instructions/). Advances in neural information processing systems, 35:27730–27744.
37. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. [Attention is all you need](https://md.research.elamur.ai/free/en/papers/vaswani-2017-attention-all-need/). Advances in neural information processing systems, 30.
38. Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D Manning, Stefano Ermon, and Chelsea Finn. 2023. [Direct preference optimization: Your language model is secretly a reward model](https://md.research.elamur.ai/free/en/papers/rafailov-2023-direct-preference-optimization-language-model/). Advances in neural information processing systems, 36:53728–53741.
39. Mathieu Ravaut, Shafiq Joty, and Nancy F Chen. 2022. Towards summary candidates fusion. arXiv preprint arXiv:2210.08779. [find](https://scholar.google.com/scholar?q=Mathieu+Ravaut%2C+Shafiq+Joty%2C+and+Nancy+F+Chen%2E+2022%2E+Towards+summary+candidates+fusion%2E+arXiv+preprint+arXiv%3A2210%2E08779%2E)
40. Noam Shazeer, Azalia Mirhoseini, Krzysztof Maziarz, Andy Davis, Quoc Le, Geoffrey Hinton, and Jeff Dean. 2017. Outrageously large neural networks: The sparsely-gated mixture-of-experts layer. arXiv preprint arXiv:1701.06538. [find](https://scholar.google.com/scholar?q=Noam+Shazeer%2C+Azalia+Mirhoseini%2C+Krzysztof+Maziarz%2C+Andy+Davis%2C+Quoc+Le%2C+Geoffrey+Hinton%2C+and+Jeff+Dean%2E+2017%2E+Outrageously+large+neural+networks%3A+The+sparsely%2Dgated+mixture%2Dof%2Dexperts+layer%2E+arXiv+pr)
41. Sheng Shen, Le Hou, Yanqi Zhou, Nan Du, Shayne Longpre, Jason Wei, Hyung Won Chung, Barret Zoph, William Fedus, Xinyun Chen, and 1 others. 2023a. [Mixture-of-experts meets instruction tuning: A winning combination for large language models](https://md.research.elamur.ai/free/en/papers/shen-2023-mixture-experts-meets-instruction-tuning/). arXiv preprint arXiv:2305.14705.
42. Tianhao Shen, Renren Jin, Yufei Huang, Chuang Liu, Weilong Dong, Zishan Guo, Xinwei Wu, Yan Liu, and Deyi Xiong. 2023b. [Large language model alignment: A survey](https://md.research.elamur.ai/free/en/papers/shen-2023-large-language-model-alignment-survey/). arXiv preprint arXiv:2309.15025.
43. Rohan Taori, Ishaan Gulrajani, Tianyi Zhang, Yann Dubois, Xuechen Li, Carlos Guestrin, Percy Liang, and Tatsunori B. Hashimoto. 2023. Stanford alpaca: An instruction-following llama model. [https://github.com/tatsu-lab/stanford_alpaca](https://github.com/tatsu-lab/stanford_alpaca). [↗](https://github.com/tatsu-lab/stanford_alpaca)
44. Gemma Team, Thomas Mesnard, Cassidy Hardin, Robert Dadashi, Surya Bhupatiraju, Shreya Pathak, Laurent Sifre, Morgane Rivière, Mihir Sanjay Kale, Juliette Love, and 1 others. 2024a. Gemma: Open models based on gemini research and technology. arXiv preprint arXiv:2403.08295. [find](https://scholar.google.com/scholar?q=Gemma+Team%2C+Thomas+Mesnard%2C+Cassidy+Hardin%2C+Robert+Dadashi%2C+Surya+Bhupatiraju%2C+Shreya+Pathak%2C+Laurent+Sifre%2C+Morgane+Rivi%C3%A8re%2C+Mihir+Sanjay+Kale%2C+Juliette+Love%2C+and+1+others%2E+2024a%2E+Gemma%3A+Open+models+)
45. Qwen Team and 1 others. 2024b. Qwen2 technical report. arXiv preprint arXiv:2407.10671, 2:3. [find](https://scholar.google.com/scholar?q=Qwen+Team+and+1+others%2E+2024b%2E+Qwen2+technical+report%2E+arXiv+preprint+arXiv%3A2407%2E10671%2C+2%3A3%2E)
46. Selim Furkan Tekin, Fatih Ilhan, Tiansheng Huang, Sihao Hu, and Ling Liu. 2024. Llm-topla: Efficient llm ensemble by maximising diversity. arXiv preprint arXiv:2410.03953. [find](https://scholar.google.com/scholar?q=Selim+Furkan+Tekin%2C+Fatih+Ilhan%2C+Tiansheng+Huang%2C+Sihao+Hu%2C+and+Ling+Liu%2E+2024%2E+Llm%2Dtopla%3A+Efficient+llm+ensemble+by+maximising+diversity%2E+arXiv+preprint+arXiv%3A2410%2E03953%2E)
47. Fanqi Wan, Xinting Huang, Deng Cai, Xiaojun Quan, Wei Bi, and Shuming Shi. 2024. Knowledge fusion of large language models. arXiv preprint arXiv:2401.10491. [find](https://scholar.google.com/scholar?q=Fanqi+Wan%2C+Xinting+Huang%2C+Deng+Cai%2C+Xiaojun+Quan%2C+Wei+Bi%2C+and+Shuming+Shi%2E+2024%2E+Knowledge+fusion+of+large+language+models%2E+arXiv+preprint+arXiv%3A2401%2E10491%2E)
48. Hongyi Wang, Felipe Maia Polo, Yuekai Sun, Souvik Kundu, Eric Xing, and Mikhail Yurochkin. 2023. Fusing models with complementary expertise. arXiv preprint arXiv:2310.01542. — [arXiv:2310.01542](https://arxiv.org/abs/2310.01542) [find](https://scholar.google.com/scholar?q=Hongyi+Wang%2C+Felipe+Maia+Polo%2C+Yuekai+Sun%2C+Souvik+Kundu%2C+Eric+Xing%2C+and+Mikhail+Yurochkin%2E+2023%2E+Fusing+models+with+complementary+expertise%2E+arXiv+preprint+arXiv%3A2310%2E01542%2E+%E2%80%94+%5BarXiv%3A2310%2E01542%5D%28https)
49. Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc Le, Ed Chi, and Denny Zhou. 2022a. Rationale-augmented ensembles in language models. arXiv preprint arXiv:2207.00747. [find](https://scholar.google.com/scholar?q=Xuezhi+Wang%2C+Jason+Wei%2C+Dale+Schuurmans%2C+Quoc+Le%2C+Ed+Chi%2C+and+Denny+Zhou%2E+2022a%2E+Rationale%2Daugmented+ensembles+in+language+models%2E+arXiv+preprint+arXiv%3A2207%2E00747%2E)
50. Yizhong Wang, Yeganeh Kordi, Swaroop Mishra, Alisa Liu, Noah A Smith, Daniel Khashabi, and Hannaneh Hajishirzi. 2022b. Self-instruct: Aligning language models with self-generated instructions. arXiv preprint arXiv:2212.10560. [find](https://scholar.google.com/scholar?q=Yizhong+Wang%2C+Yeganeh+Kordi%2C+Swaroop+Mishra%2C+Alisa+Liu%2C+Noah+A+Smith%2C+Daniel+Khashabi%2C+and+Hannaneh+Hajishirzi%2E+2022b%2E+Self%2Dinstruct%3A+Aligning+language+models+with+self%2Dgenerated+instructions%2E+arXiv+p)
51. Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, and 1 others. 2022. [Chain-of-thought prompting elicits reasoning in large language models](https://md.research.elamur.ai/free/en/papers/wei-2022-chain-thought-prompting-elicits-reasoning/). Advances in neural information processing systems, 35:24824– 24837.
52. Mitchell Wortsman, Gabriel Ilharco, Samir Ya Gadre, Rebecca Roelofs, Raphael Gontijo-Lopes, Ari S Morcos, Hongseok Namkoong, Ali Farhadi, Yair Carmon, Simon Kornblith, and 1 others. 2022. [Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference time](https://md.research.elamur.ai/free/en/papers/wortsman-2022-model-soups-averaging-weights-multiple/)…
53. Tianhao Wu, Banghua Zhu, Ruoyu Zhang, Zhaojin Wen, Kannan Ramchandran, and Jiantao Jiao. 2023. Pairwise proximal policy optimization: Harnessing relative feedback for llm alignment. arXiv preprint arXiv:2310.00212. — [arXiv:2310.00212](https://arxiv.org/abs/2310.00212) [find](https://scholar.google.com/scholar?q=Tianhao+Wu%2C+Banghua+Zhu%2C+Ruoyu+Zhang%2C+Zhaojin+Wen%2C+Kannan+Ramchandran%2C+and+Jiantao+Jiao%2E+2023%2E+Pairwise+proximal+policy+optimization%3A+Harnessing+relative+feedback+for+llm+alignment%2E+arXiv+preprint+arX)
54. Prateek Yadav, Derek Tam, Leshem Choshen, Colin A Raffel, and Mohit Bansal. 2023. [Ties-merging: Resolving interference when merging models](https://md.research.elamur.ai/free/en/papers/yadav-2023-ties-merging-resolving-interference-merging/). Advances in Neural Information Processing Systems, 36:7093–7115.
55. Shunyu Yao, Dian Yu, Jeffrey Zhao, Izhak Shafran, Tom Griffiths, Yuan Cao, and Karthik Narasimhan. 2024. Tree of thoughts: Deliberate problem solving with large language models. Advances in Neural Information Processing Systems, 36. [find](https://scholar.google.com/scholar?q=Shunyu+Yao%2C+Dian+Yu%2C+Jeffrey+Zhao%2C+Izhak+Shafran%2C+Tom+Griffiths%2C+Yuan+Cao%2C+and+Karthik+Narasimhan%2E+2024%2E+Tree+of+thoughts%3A+Deliberate+problem+solving+with+large+language+models%2E+Advances+in+Neural+Inf)
56. Le Yu, Bowen Yu, Haiyang Yu, Fei Huang, and Yongbin Li. 2024. [Language models are super mario: Absorbing abilities from homologous models as a free lunch](https://md.research.elamur.ai/free/en/papers/yu-2023-language-models-super-mario-absorbing/). In Forty-first International Conference on Machine Learning.
57. Shaolei Zhang, Tian Yu, and Yang Feng. 2024. [Truthx: Alleviating hallucinations by editing large language models in truthful space](https://md.research.elamur.ai/free/en/papers/zhang-2024-truthx-alleviating-hallucinations-editing-large/). arXiv preprint arXiv:2402.17811.
58. Wayne Xin Zhao, Kun Zhou, Junyi Li, Tianyi Tang, Xiaolei Wang, Yupeng Hou, Yingqian Min, Beichen Zhang, Junjie Zhang, Zican Dong, and 1 others. 2023. [A survey of large language models](https://md.research.elamur.ai/free/en/papers/zhao-2023-survey-large-language-models/). arXiv preprint arXiv:2303.18223.
59. Tong Zhu, Xiaoye Qu, Daize Dong, Jiacheng Ruan, Jingqi Tong, Conghui He, and Yu Cheng. 2024. [Llama-moe: Building mixture-of-experts from llama with continual pre-training](https://md.research.elamur.ai/free/en/papers/zhu-2024-llama-moe-building-mixture-experts/). arXiv preprint arXiv:2406.16554.
60. Barret Zoph, Irwan Bello, Sameer Kumar, Nan Du, Yanping Huang, Jeff Dean, Noam Shazeer, and William Fedus. 2022. St-moe: Designing stable and transferable sparse expert models. arXiv preprint arXiv:2202.08906. [find](https://scholar.google.com/scholar?q=Barret+Zoph%2C+Irwan+Bello%2C+Sameer+Kumar%2C+Nan+Du%2C+Yanping+Huang%2C+Jeff+Dean%2C+Noam+Shazeer%2C+and+William+Fedus%2E+2022%2E+St%2Dmoe%3A+Designing+stable+and+transferable+sparse+expert+models%2E+arXiv+preprint+arXiv%3A22)

*Updated 31 October 2026*