---
title: "Efficient Low-Resource Language Adaptation via Multi-Source"
description: "The authors propose TriMix, a logit fusion method applied at inference time. 68 references."
authors: ["Chen Zhang", "Jiuheng Lin", "Zhiyuan Liao", "Yansong Feng"]
year: 2026
venue: "Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)"
source: "https://aclanthology.org/2026.acl-long.207/"
doi: "10.18653/v1/2026.acl-long.207"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/zhang-2026-efficient-low-resource-language-adaptation/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/zhang-2026-efficient-low-resource-language-adaptation/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2026](https://md.research.elamur.ai/free/en/papers/2026/page/31/) / Efficient Low-Resource Language Adaptation via Multi-Source Dynamic Logit Fusion

# Efficient Low-Resource Language Adaptation via Multi-Source Dynamic Logit Fusion

**Authors:** [Chen Zhang](https://md.research.elamur.ai/free/en/authors/chen-zhang/), Jiuheng Lin, Zhiyuan Liao and [Yansong Feng](https://md.research.elamur.ai/free/en/authors/yansong-feng/) · **Year:** 2026 · **Venue:** [Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)](https://md.research.elamur.ai/free/en/venues/acl/) ([ACL 2026](https://md.research.elamur.ai/free/en/venues/acl/2026/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2026.acl-long.207/) · [PDF](https://aclanthology.org/2026.acl-long.207.pdf) · [DOI 10.18653/v1/2026.acl-long.207](https://doi.org/10.18653/v1/2026.acl-long.207) · [OpenAlex](https://openalex.org/W7166864766)

**Citations:** 0 citations (OpenAlex, 2026-10-14)

## Summary
Adapting LLMs to low-resource languages (LRLs) is difficult because labeled task data in these languages is scarce and continual pretraining (CPT) of large models is expensive. Proxy Tuning combines model logits, but it tends to let the large model dominate, and that model's weak LRL ability can override what a specialized small model has learned. The authors propose TriMix, a logit fusion method applied at inference time. It combines three logit sources: a small model continually pretrained on raw LRL text (language competence), a large off-the-shelf instruction-tuned model (task skills learned in high-resource languages, plus the benefits of scale), and the small base model as a reference. The fusion weights are chosen per input by minimizing prompt perplexity or first-token entropy, so no LRL annotations are required. Across several model families and eight LRLs, TriMix beats single-model baselines and Proxy Tuning. The analysis shows that the best results come from giving the small CPT model more weight than the large model. This matters for teams with limited compute that work on underserved languages.

## Key points
- **Fusion formula:** α·L(large-ins) + β·L(small-cpt) + (1−α−β)·L(small-base). The scaling coefficient is tied to α (γ = α), so the large base model never has to be loaded.
- **Models and languages:** Qwen2.5 (1.5B–7B CPT paired with 3B–14B instruct), Llama2 7B/13B (CPT checkpoints from Tao et al., 2024), Llama3.2 1B/3B and Gemma3 4B/12B. Languages are bod, uig, kaz, mvf, Tamil, Telugu, Odia and Bengali.
- **Main result:** Fusing a CPT Qwen2.5-1.5B with the 14B instruct model gives about a 5% average relative gain over the 14B model alone. The perplexity-guided weighting comes close to the empirical upper bound.
- **Efficiency:** With INT8 quantization, the performance loss is under 0.5%. The method still requires access to logits and a compatible tokenizer across models, and it adds inference overhead.
- **Code:** github.com/luciusssss/TriMix

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Peking University](https://md.research.elamur.ai/free/en/universities/peking-university/) — University, Beijing, China · 315 works · 43,267 citations

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 18,614 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 17,965 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 17,450 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 17,296 works

🧭 Topics (1): [🌱 Low-resource languages](https://md.research.elamur.ai/free/en/topics/low-resource-languages/) · 287 works

## Cite

Cite as: Chen Zhang, Jiuheng Lin, Zhiyuan Liao and Yansong Feng. 2026. Efficient Low-Resource Language Adaptation via Multi-Source Dynamic Logit Fusion. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers).

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{zhang2026efficient,
  title = {{Efficient Low-Resource Language Adaptation via Multi-Source Dynamic Logit Fusion}},
  author = {Chen Zhang, Jiuheng Lin, Zhiyuan Liao and Yansong Feng},
  booktitle = {Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2026},
  doi = {10.18653/v1/2026.acl-long.207},
  url = {https://doi.org/10.18653/v1/2026.acl-long.207}
}
```

- [← previous Efficient Learned Data Compression via Dual-Stream Feature Decoupling](https://md.research.elamur.ai/free/en/papers/ma-2026-efficient-learned-data-compression-dual/)
- [next → Efficient Prior-Guided Reasoning for Robust Retrieval-Augmented Generation under Conflicts](https://md.research.elamur.ai/free/en/papers/yuan-2026-efficient-prior-guided-reasoning-robust/)
- [↑ back to papers of 2026](https://md.research.elamur.ai/free/en/papers/2026/page/31/)

## References

68 references, 23 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. David Ifeoluwa Adelani, Hannah Liu, Xiaoyu Shen, Nikita Vassilyev, Jesujoba O. Alabi, Yanke Mao, Haonan Gao, and En-Shiun Annie Lee. 2024. [SIB-200: A simple, inclusive, and big evaluation dataset for topic classification in 200+ languages and dialects](https://md.research.elamur.ai/free/en/papers/adelani-2023-sib-200-simple-inclusive-big/). In Proceedings of the 18th Conference of the Eu…
2. Divyanshu Aggarwal, Ashutosh Sathe, and Sunayana Sitaram. 2025. Improving cross lingual transfer by pretraining with active forgetting. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pages 2367–2378, Suzhou, China. Association for Computational Linguistics… [find](https://scholar.google.com/scholar?q=Divyanshu+Aggarwal%2C+Ashutosh+Sathe%2C+and+Sunayana+Sitaram%2E+2025%2E+Improving+cross+lingual+transfer+by+pretraining+with+active+forgetting%2E+In+Proceedings+of+the+2025+Conference+on+Empirical+Methods+in+Na)
3. Takuya Akiba, Makoto Shing, Yujin Tang, Qi Sun, and David Ha. 2025. Evolutionary optimization of model merging recipes. Nature Machine Intelligence, 7(2):195–204. [find](https://scholar.google.com/scholar?q=Takuya+Akiba%2C+Makoto+Shing%2C+Yujin+Tang%2C+Qi+Sun%2C+and+David+Ha%2E+2025%2E+Evolutionary+optimization+of+model+merging+recipes%2E+Nature+Machine+Intelligence%2C+7%282%29%3A195%E2%80%93204%2E)
4. Lucas Bandarkar, Davis Liang, Benjamin Muller, Mikel Artetxe, Satya Narayan Shukla, Donald Husa, Naman Goyal, Abhinandan Krishnan, Luke Zettlemoyer, and Madian Khabsa. 2024. [The belebele benchmark: a parallel reading comprehension dataset in 122 language variants](https://md.research.elamur.ai/free/en/papers/bandarkar-2023-belebele-benchmark-parallel-reading-comprehension/). In Proceedings of the 62nd Annual M…
5. Lucas Bandarkar, Benjamin Muller, Pritish Yuvraj, Rui Hou, Nayan Singhal, Hongjiang Lv, and Bing Liu. 2025. Layer swapping for zero-shot cross-lingual transfer in large language models. In The Thirteenth International Conference on Learning Representations. [find](https://scholar.google.com/scholar?q=Lucas+Bandarkar%2C+Benjamin+Muller%2C+Pritish+Yuvraj%2C+Rui+Hou%2C+Nayan+Singhal%2C+Hongjiang+Lv%2C+and+Bing+Liu%2E+2025%2E+Layer+swapping+for+zero%2Dshot+cross%2Dlingual+transfer+in+large+language+models%2E+In+The+Thirtee)
6. Sheng Cao, Mingrui Wu, Karthik Prasad, Yuandong Tian, and Zechun Liu. 2025. Param$\\delta$ for direct mixing: Post-train large language model at zero cost. In The Thirteenth International Conference on Learning Representations. [find](https://scholar.google.com/scholar?q=Sheng+Cao%2C+Mingrui+Wu%2C+Karthik+Prasad%2C+Yuandong+Tian%2C+and+Zechun+Liu%2E+2025%2E+Param%24%5C%5Cdelta%24+for+direct+mixing%3A+Post%2Dtrain+large+language+model+at+zero+cost%2E+In+The+Thirteenth+International+Conference+o)
7. Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde de Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, Alex Ray, Raul Puri, Gretchen Krueger, Michael Petrov, Heidy Khlaaf, Girish Sastry, Pamela Mishkin, Brooke Chan, Scott Gray, and 39 others. 202… — [Evaluating Large Language Models Trained on Code](https://md.research.elamur.ai/free/en/papers/chen-2021-evaluating-large-language-models-trained/)
8. Xin Cheng, Xun Wang, Tao Ge, Si-Qing Chen, Furu Wei, Dongyan Zhao, and Rui Yan. 2024. SCALE: Synergized collaboration of asymmetric language translation engines. In Findings of the Association for Computational Linguistics: ACL 2024, pages 15903–15918, Bangkok, Thailand. Association for Computationa… [find](https://scholar.google.com/scholar?q=Xin+Cheng%2C+Xun+Wang%2C+Tao+Ge%2C+Si%2DQing+Chen%2C+Furu+Wei%2C+Dongyan+Zhao%2C+and+Rui+Yan%2E+2024%2E+SCALE%3A+Synergized+collaboration+of+asymmetric+language+translation+engines%2E+In+Findings+of+the+Association+for+Com)
9. Yung-Sung Chuang, Yujia Xie, Hongyin Luo, Yoon Kim, James R. Glass, and Pengcheng He. 2024. [Dola: Decoding by contrasting layers improves factuality in large language models](https://md.research.elamur.ai/free/en/papers/chuang-2023-dola-decoding-contrasting-layers-improves/). In The Twelfth International Conference on Learning Representations.
10. Sourabh Deoghare, Diptesh Kanojia, and Pushpak Bhattacharyya. 2024. Together we can: Multilingual automatic post-editing for low-resource languages. In Findings of the Association for Computational Linguistics: EMNLP 2024, pages 10800–10812, Miami, Florida, USA. Association for Computational Linguis… [find](https://scholar.google.com/scholar?q=Sourabh+Deoghare%2C+Diptesh+Kanojia%2C+and+Pushpak+Bhattacharyya%2E+2024%2E+Together+we+can%3A+Multilingual+automatic+post%2Dediting+for+low%2Dresource+languages%2E+In+Findings+of+the+Association+for+Computational+Li)
11. Jesse Dodge, Maarten Sap, Ana Marasović, William Agnew, Gabriel Ilharco, Dirk Groeneveld, Margaret Mitchell, and Matt Gardner. 2021. Documenting large webtext corpora: A case study on the colossal clean crawled corpus. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language P… [find](https://scholar.google.com/scholar?q=Jesse+Dodge%2C+Maarten+Sap%2C+Ana+Marasovic%CC%81%2C+William+Agnew%2C+Gabriel+Ilharco%2C+Dirk+Groeneveld%2C+Margaret+Mitchell%2C+and+Matt+Gardner%2E+2021%2E+Documenting+large+webtext+corpora%3A+A+case+study+on+the+colossal+cl)
12. Chenghao Fan, Zhenyi Lu, Wei Wei, Jie Tian, Xiaoye Qu, Dangyang Chen, and Yu Cheng. 2024. On giant’s shoulders: Effortless weak to strong by dynamic logits fusion. In The Thirty-eighth Annual Conference on Neural Information Processing Systems. [find](https://scholar.google.com/scholar?q=Chenghao+Fan%2C+Zhenyi+Lu%2C+Wei+Wei%2C+Jie+Tian%2C+Xiaoye+Qu%2C+Dangyang+Chen%2C+and+Yu+Cheng%2E+2024%2E+On+giant%E2%80%99s+shoulders%3A+Effortless+weak+to+strong+by+dynamic+logits+fusion%2E+In+The+Thirty%2Deighth+Annual+Conferen)
13. Tingchen Fu, Yupeng Hou, Julian McAuley, and Rui Yan. 2025. Unlocking decoding-time controllability: Gradient-free multi-objective alignment with contrastive prompts. In Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human… [find](https://scholar.google.com/scholar?q=Tingchen+Fu%2C+Yupeng+Hou%2C+Julian+McAuley%2C+and+Rui+Yan%2E+2025%2E+Unlocking+decoding%2Dtime+controllability%3A+Gradient%2Dfree+multi%2Dobjective+alignment+with+contrastive+prompts%2E+In+Proceedings+of+the+2025+Confer)
14. Kazuki Fujii, Taishi Nakamura, Mengsay Loem, Hiroki Iida, Masanari Ohi, Kakeru Hattori, Hirai Shota, Sakae Mizuki, Rio Yokota, and Naoaki Okazaki. 2024. [Continual pre-training for cross-lingual LLM adaptation: Enhancing japanese language capabilities](https://md.research.elamur.ai/free/en/papers/fujii-2024-continual-pre-training-cross-lingual/). In First Conference on Language Modeling.
15. Esteban Garces Arias, Julian Rodemann, Meimingwei Li, Christian Heumann, and Matthias Aßenmacher. 2024. Adaptive contrastive search: Uncertainty-guided decoding for open-ended text generation. In Findings of the Association for Computational Linguistics: EMNLP 2024, pages 15060–15080, Miami, Florida… [find](https://scholar.google.com/scholar?q=Esteban+Garces+Arias%2C+Julian+Rodemann%2C+Meimingwei+Li%2C+Christian+Heumann%2C+and+Matthias+A%C3%9Fenmacher%2E+2024%2E+Adaptive+contrastive+search%3A+Uncertainty%2Dguided+decoding+for+open%2Dended+text+generation%2E+In+Find)
16. Charles Goddard, Shamane Siriwardhana, Malikeh Ehghaghi, Luke Meyers, Vladimir Karpukhin, Brian Benedict, Mark McQuade, and Jacob Solawetz. 2024. [Arcee’s MergeKit: A toolkit for merging large language models](https://md.research.elamur.ai/free/en/papers/goddard-2024-arcee-s-mergekit-toolkit-merging/). In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing:…
17. Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, and 1 others. 2024. [The llama 3 herd of models](https://md.research.elamur.ai/free/en/papers/grattafiori-2024-llama-3-herd-models/). arXiv preprint arXiv:2407.21783.
18. Andrés Domínguez Hernández, Diana Mosquera, and Francisco Gallegos. 2025. Lessons from the margins: Contextualizing, reimagining, and hacking generative ai in the global south. Harvard Data Science Review, 7(4). [find](https://scholar.google.com/scholar?q=Andr%C3%A9s+Dom%C3%ADnguez+Hern%C3%A1ndez%2C+Diana+Mosquera%2C+and+Francisco+Gallegos%2E+2025%2E+Lessons+from+the+margins%3A+Contextualizing%2C+reimagining%2C+and+hacking+generative+ai+in+the+global+south%2E+Harvard+Data+Science+Re)
19. Shih-Cheng Huang, Pin-Zu Li, Yu-chi Hsu, Kuang-Ming Chen, Yu Tung Lin, Shih-Kai Hsiao, Richard Tsai, and Hung-yi Lee. 2024a. Chat vector: A simple approach to equip LLMs with instruction following and model alignment in new languages. In Proceedings of the 62nd Annual Meeting of the Association for… [find](https://scholar.google.com/scholar?q=Shih%2DCheng+Huang%2C+Pin%2DZu+Li%2C+Yu%2Dchi+Hsu%2C+Kuang%2DMing+Chen%2C+Yu+Tung+Lin%2C+Shih%2DKai+Hsiao%2C+Richard+Tsai%2C+and+Hung%2Dyi+Lee%2E+2024a%2E+Chat+vector%3A+A+simple+approach+to+equip+LLMs+with+instruction+following+and)
20. Zixian Huang, Wenhao Zhu, Gong Cheng, Lei Li, and Fei Yuan. 2024b. Mindmerger: Efficiently boosting LLM reasoning in non-english languages. In The Thirty-eighth Annual Conference on Neural Information Processing Systems. [find](https://scholar.google.com/scholar?q=Zixian+Huang%2C+Wenhao+Zhu%2C+Gong+Cheng%2C+Lei+Li%2C+and+Fei+Yuan%2E+2024b%2E+Mindmerger%3A+Efficiently+boosting+LLM+reasoning+in+non%2Denglish+languages%2E+In+The+Thirty%2Deighth+Annual+Conference+on+Neural+Information)
21. Arthur Jacot, Franck Gabriel, and Clément Hongler. 2018. Neural tangent kernel: Convergence and generalization in neural networks. Advances in neural information processing systems, 31. [find](https://scholar.google.com/scholar?q=Arthur+Jacot%2C+Franck+Gabriel%2C+and+Cl%C3%A9ment+Hongler%2E+2018%2E+Neural+tangent+kernel%3A+Convergence+and+generalization+in+neural+networks%2E+Advances+in+neural+information+processing+systems%2C+31%2E)
22. Jing Jin, Houfeng Wang, Hao Zhang, Xiaoguang Li, and Zhijiang Guo. 2024. DVD: Dynamic contrastive decoding for knowledge amplification in multi-document question answering. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 4624–4637, Miami, Florida, USA… [find](https://scholar.google.com/scholar?q=Jing+Jin%2C+Houfeng+Wang%2C+Hao+Zhang%2C+Xiaoguang+Li%2C+and+Zhijiang+Guo%2E+2024%2E+DVD%3A+Dynamic+contrastive+decoding+for+knowledge+amplification+in+multi%2Ddocument+question+answering%2E+In+Proceedings+of+the+2024+)
23. Pratik Joshi, Sebastin Santy, Amar Budhiraja, Kalika Bali, and Monojit Choudhury. 2020. [The state and fate of linguistic diversity and inclusion in the NLP world](https://md.research.elamur.ai/free/en/papers/joshi-2020-state-fate-linguistic-diversity-inclusion/). In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 6282–6293, Online. Association for Comp…
24. Jared Kaplan, Sam McCandlish, Tom Henighan, Tom B Brown, Benjamin Chess, Rewon Child, Scott Gray, Alec Radford, Jeffrey Wu, and Dario Amodei. 2020. [Scaling laws for neural language models](https://md.research.elamur.ai/free/en/papers/kaplan-2020-scaling-laws-neural-language-models/). arXiv preprint arXiv:2001.08361.
25. Zixuan Ke, Yifei Ming, and Shafiq Joty. 2025. Adaptation of large language models. In Proceedings of the 2025 Annual Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 5: Tutorial Abstracts), pages 30–37, Albuquerqu… [find](https://scholar.google.com/scholar?q=Zixuan+Ke%2C+Yifei+Ming%2C+and+Shafiq+Joty%2E+2025%2E+Adaptation+of+large+language+models%2E+In+Proceedings+of+the+2025+Annual+Conference+of+the+Nations+of+the+Americas+Chapter+of+the+Association+for+Computatio)
26. Youna Kim, Hyuhng Joon Kim, Cheonbok Park, Choonghyun Park, Hyunsoo Cho, Junyeob Kim, Kang Min Yoo, Sang-goo Lee, and Taeuk Kim. 2024. Adaptive contrastive decoding in retrieval-augmented generation for handling noisy contexts. In Findings of the Association for Computational Linguistics: EMNLP 2024… [find](https://scholar.google.com/scholar?q=Youna+Kim%2C+Hyuhng+Joon+Kim%2C+Cheonbok+Park%2C+Choonghyun+Park%2C+Hyunsoo+Cho%2C+Junyeob+Kim%2C+Kang+Min+Yoo%2C+Sang%2Dgoo+Lee%2C+and+Taeuk+Kim%2E+2024%2E+Adaptive+contrastive+decoding+in+retrieval%2Daugmented+generation+f)
27. Tomasz Korbak, Hady Elsahar, Germán Kruszewski, and Marc Dymetman. 2022. On reinforcement learning and distribution matching for fine-tuning language models with no catastrophic forgetting. Advances in Neural Information Processing Systems, 35:16203–16220. [find](https://scholar.google.com/scholar?q=Tomasz+Korbak%2C+Hady+Elsahar%2C+Germ%C3%A1n+Kruszewski%2C+and+Marc+Dymetman%2E+2022%2E+On+reinforcement+learning+and+distribution+matching+for+fine%2Dtuning+language+models+with+no+catastrophic+forgetting%2E+Advances+i)
28. Binyuan Hui, Jian Yang, Zeyu Cui, Jiaxi Yang, Dayiheng Liu, Lei Zhang, Tianyu Liu, Jiajun Zhang, Bowen Yu, Keming Lu, and 1 others. 2024. Qwen2. 5-coder technical report. arXiv preprint arXiv:2409.12186. — [Qwen2.5-Coder Technical Report](https://md.research.elamur.ai/free/en/papers/hui-2024-qwen2-5-coder-technical-report/)
29. Hakyung Lee, Subeen Park, Joowang Kim, Sungjun Lim, and Kyungwoo Song. 2025. Uncertainty-aware contrastive decoding. In Findings of the Association for Computational Linguistics: ACL 2025, pages 26376–26391, Vienna, Austria. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Hakyung+Lee%2C+Subeen+Park%2C+Joowang+Kim%2C+Sungjun+Lim%2C+and+Kyungwoo+Song%2E+2025%2E+Uncertainty%2Daware+contrastive+decoding%2E+In+Findings+of+the+Association+for+Computational+Linguistics%3A+ACL+2025%2C+pages+26376)
30. Jonathan Hus and Antonios Anastasopoulos. 2024. Back to school: Translation using grammar books. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 20207–20219, Miami, Florida, USA. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Jonathan+Hus+and+Antonios+Anastasopoulos%2E+2024%2E+Back+to+school%3A+Translation+using+grammar+books%2E+In+Proceedings+of+the+2024+Conference+on+Empirical+Methods+in+Natural+Language+Processing%2C+pages+20207%E2%80%93)
31. Chong Li, Yingzhuo Deng, Jiajun Zhang, and Chengqing Zong. 2025a. Group then scale: Dynamic mixture-of-experts multilingual language model. In Findings of the Association for Computational Linguistics: ACL 2025, pages 1730–1754, Vienna, Austria. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Chong+Li%2C+Yingzhuo+Deng%2C+Jiajun+Zhang%2C+and+Chengqing+Zong%2E+2025a%2E+Group+then+scale%3A+Dynamic+mixture%2Dof%2Dexperts+multilingual+language+model%2E+In+Findings+of+the+Association+for+Computational+Linguistics)
32. Xiang Lisa Li, Ari Holtzman, Daniel Fried, Percy Liang, Jason Eisner, Tatsunori Hashimoto, Luke Zettlemoyer, and Mike Lewis. 2023. [Contrastive decoding: Open-ended text generation as optimization](https://md.research.elamur.ai/free/en/papers/li-2022-contrastive-decoding-open-ended-text/). In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: L…
33. Yue Li, Zhixue Zhao, and Carolina Scarton. 2025b. It’s all about in-context learning! teaching extremely low-resource languages to LLMs. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pages 29532–29547, Suzhou, China. Association for Computational Linguist… [find](https://scholar.google.com/scholar?q=Yue+Li%2C+Zhixue+Zhao%2C+and+Carolina+Scarton%2E+2025b%2E+It%E2%80%99s+all+about+in%2Dcontext+learning%21+teaching+extremely+low%2Dresource+languages+to+LLMs%2E+In+Proceedings+of+the+2025+Conference+on+Empirical+Methods+in+N)
34. Zhuang Li, Levon Haroutunian, Raj Tumuluri, Philip Cohen, and Reza Haf. 2024. Improving cross-domain low-resource text generation through LLM post-editing: A programmer-interpreter approach. In Findings of the Association for Computational Linguistics: EACL 2024, pages 347–354, St. Julian’s, Malta.… [find](https://scholar.google.com/scholar?q=Zhuang+Li%2C+Levon+Haroutunian%2C+Raj+Tumuluri%2C+Philip+Cohen%2C+and+Reza+Haf%2E+2024%2E+Improving+cross%2Ddomain+low%2Dresource+text+generation+through+LLM+post%2Dediting%3A+A+programmer%2Dinterpreter+approach%2E+In+Findin)
35. Alisa Liu, Xiaochuang Han, Yizhong Wang, Yulia Tsvetkov, Yejin Choi, and Noah A. Smith. 2024. [Tuning language models by proxy](https://md.research.elamur.ai/free/en/papers/liu-2024-tuning-language-models-proxy/). In First Conference on Language Modeling.
36. Costas Mavromatis, Petros Karypis, and George Karypis. 2024. Pack of LLMs: Model fusion at test-time via perplexity optimization. In First Conference on Language Modeling. [find](https://scholar.google.com/scholar?q=Costas+Mavromatis%2C+Petros+Karypis%2C+and+George+Karypis%2E+2024%2E+Pack+of+LLMs%3A+Model+fusion+at+test%2Dtime+via+perplexity+optimization%2E+In+First+Conference+on+Language+Modeling%2E)
37. Eric Mitchell, Rafael Rafailov, Archit Sharma, Chelsea Finn, and Christopher D Manning. 2024. [An emulator for fine-tuning large language models using small language models](https://md.research.elamur.ai/free/en/papers/mitchell-2023-emulator-fine-tuning-large-language/). In The Twelfth International Conference on Learning Representations.
38. Sean O’Brien and Mike Lewis. 2023. [Contrastive decoding improves reasoning in large language models](https://md.research.elamur.ai/free/en/papers/o-brien-2023-contrastive-decoding-improves-reasoning-large/). arXiv preprint arXiv:2309.09117.
39. Renhao Pei, Yihong Liu, Peiqin Lin, François Yvon, and Hinrich Schuetze. 2025. Understanding in-context machine translation for low-resource languages: A case study on Manchu. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 87… [find](https://scholar.google.com/scholar?q=Renhao+Pei%2C+Yihong+Liu%2C+Peiqin+Lin%2C+Fran%C3%A7ois+Yvon%2C+and+Hinrich+Schuetze%2E+2025%2E+Understanding+in%2Dcontext+machine+translation+for+low%2Dresource+languages%3A+A+case+study+on+Manchu%2E+In+Proceedings+of+the+63)
40. Fabian David Schmidt, Philipp Borchert, Ivan Vulić, and Goran Glavaš. 2024. Self-distillation for model stacking unlocks cross-lingual NLU in 200+ languages. In Findings of the Association for Computational Linguistics: EMNLP 2024, pages 6724–6743, Miami, Florida, USA. Association for Computational… [find](https://scholar.google.com/scholar?q=Fabian+David+Schmidt%2C+Philipp+Borchert%2C+Ivan+Vulic%CC%81%2C+and+Goran+Glava%C5%A1%2E+2024%2E+Self%2Ddistillation+for+model+stacking+unlocks+cross%2Dlingual+NLU+in+200%2B+languages%2E+In+Findings+of+the+Association+for+Comput)
41. Rico Sennrich, Jannis Vamvas, and Alireza Mohammadshahi. 2024. [Mitigating hallucinations and off-target machine translation with source-contrastive and language-contrastive decoding](https://md.research.elamur.ai/free/en/papers/sennrich-2024-mitigating-hallucinations-off-target-machine/). In Proceedings of the 18th Conference of the European Chapter of the Association for Computational Linguistics (Volum…
42. Uri Shaham, Jonathan Herzig, Roee Aharoni, Idan Szpektor, Reut Tsarfaty, and Matan Eyal. 2024. [Multilingual instruction tuning with just a pinch of multilinguality](https://md.research.elamur.ai/free/en/papers/shaham-2024-multilingual-instruction-tuning-just-pinch/). In Findings of the Association for Computational Linguistics: ACL 2024, pages 2304–2317, Bangkok, Thailand. Association for Computation…
43. Zejiang Shen, Hunter Lang, Bailin Wang, Yoon Kim, and David Sontag. 2024. Learning to decode collaboratively with multiple language models. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 12974–12990, Bangkok, Thailand. Associ… [find](https://scholar.google.com/scholar?q=Zejiang+Shen%2C+Hunter+Lang%2C+Bailin+Wang%2C+Yoon+Kim%2C+and+David+Sontag%2E+2024%2E+Learning+to+decode+collaboratively+with+multiple+language+models%2E+In+Proceedings+of+the+62nd+Annual+Meeting+of+the+Association)
44. Weijia Shi, Xiaochuang Han, Mike Lewis, Yulia Tsvetkov, Luke Zettlemoyer, and Wen-tau Yih. 2024. [Trusting your evidence: Hallucinate less with context-aware decoding](https://md.research.elamur.ai/free/en/papers/shi-2024-trusting-your-evidence-hallucinate-less/). In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language…
45. Mohammad Shoeybi, Mostofa Patwary, Raul Puri, Patrick LeGresley, Jared Casper, and Bryan Catanzaro. 2019. Megatron-lm: Training multi-billion parameter language models using model parallelism. arXiv preprint arXiv:1909.08053. [find](https://scholar.google.com/scholar?q=Mohammad+Shoeybi%2C+Mostofa+Patwary%2C+Raul+Puri%2C+Patrick+LeGresley%2C+Jared+Casper%2C+and+Bryan+Catanzaro%2E+2019%2E+Megatron%2Dlm%3A+Training+multi%2Dbillion+parameter+language+models+using+model+parallelism%2E+arXiv+p)
46. Shivalika Singh, Angelika Romanou, Clémentine Fourrier, David Ifeoluwa Adelani, Jian Gang Ngui, Daniel Vila-Suero, Peerat Limkonchotiwat, Kelly Marchisio, Wei Qi Leong, Yosephine Susanto, Raymond Ng, Shayne Longpre, Sebastian Ruder, Wei-Yin Ko, Antoine Bosselut, Alice Oh, Andre Martins, Leshem Chosh… — [Global MMLU: Understanding and Addressing Cultural and Linguistic Biases in Multilingual Evaluation](https://md.research.elamur.ai/free/en/papers/singh-2025-global-mmlu-understanding-addressing-cultural/)
47. Shivalika Singh, Freddie Vargus, Daniel D’souza, Börje F. Karlsson, Abinaya Mahendiran, Wei-Yin Ko, Herumb Shandilya, Jay Patel, Deividas Mataciunas, Laura O’Mahony, Mike Zhang, Ramith Hettiarachchi, Joseph Wilson, Marina Machado, Luisa Moura, Dominik Krzemiński, Hakimeh Fadaei, Irem Ergun, Ifeoma… — [Aya Dataset: An Open-Access Collection for Multilingual Instruction Tuning](https://md.research.elamur.ai/free/en/papers/singh-2024-aya-dataset-open-access-collection/)
48. Tong Su, Xin Peng, Sarubi Thillainathan, David Guzmán, Surangika Ranathunga, and En-Shiun Lee. 2024. Unlocking parameter-efficient fine-tuning for low-resource language translation. In Findings of the Association for Computational Linguistics: NAACL 2024, pages 4217–4225, Mexico City, Mexico. Associ… [find](https://scholar.google.com/scholar?q=Tong+Su%2C+Xin+Peng%2C+Sarubi+Thillainathan%2C+David+Guzm%C3%A1n%2C+Surangika+Ranathunga%2C+and+En%2DShiun+Lee%2E+2024%2E+Unlocking+parameter%2Defficient+fine%2Dtuning+for+low%2Dresource+language+translation%2E+In+Findings+of+the)
49. Zeli Su, Ziyin Zhang, Guixian Xu, Jianing Liu, Xu Han, Ting Zhang, and Yushuang Dong. 2025. Multilingual encoder knows more than you realize: Shared weights pretraining for extremely low-resource languages. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (V… [find](https://scholar.google.com/scholar?q=Zeli+Su%2C+Ziyin+Zhang%2C+Guixian+Xu%2C+Jianing+Liu%2C+Xu+Han%2C+Ting+Zhang%2C+and+Yushuang+Dong%2E+2025%2E+Multilingual+encoder+knows+more+than+you+realize%3A+Shared+weights+pretraining+for+extremely+low%2Dresource+lang)
50. Garrett Tanzer, Mirac Suzgun, Eline Visser, Dan Jurafsky, and Luke Melas-Kyriazi. 2024. A benchmark for learning to translate a new language from one grammar book. In The Twelfth International Conference on Learning Representations. [find](https://scholar.google.com/scholar?q=Garrett+Tanzer%2C+Mirac+Suzgun%2C+Eline+Visser%2C+Dan+Jurafsky%2C+and+Luke+Melas%2DKyriazi%2E+2024%2E+A+benchmark+for+learning+to+translate+a+new+language+from+one+grammar+book%2E+In+The+Twelfth+International+Confere)
51. Mingxu Tao, Jie Hu, Mingchuan Yang, Yunhuai Liu, Dongyan Zhao, and Yansong Feng. 2025. EpiCoDe: Boosting model performance beyond training with extrapolation and contrastive decoding. In Findings of the Association for Computational Linguistics: ACL 2025, pages 14874–14885, Vienna, Austria. Associat… [find](https://scholar.google.com/scholar?q=Mingxu+Tao%2C+Jie+Hu%2C+Mingchuan+Yang%2C+Yunhuai+Liu%2C+Dongyan+Zhao%2C+and+Yansong+Feng%2E+2025%2E+EpiCoDe%3A+Boosting+model+performance+beyond+training+with+extrapolation+and+contrastive+decoding%2E+In+Findings+of+t)
52. Mingxu Tao, Chen Zhang, Quzhe Huang, Tianyao Ma, Songfang Huang, Dongyan Zhao, and Yansong Feng. 2024. Unlocking the potential of model merging for low-resource languages. In Findings of the Association for Computational Linguistics: EMNLP 2024, pages 8705–8720, Miami, Florida, USA. Association for… [find](https://scholar.google.com/scholar?q=Mingxu+Tao%2C+Chen+Zhang%2C+Quzhe+Huang%2C+Tianyao+Ma%2C+Songfang+Huang%2C+Dongyan+Zhao%2C+and+Yansong+Feng%2E+2024%2E+Unlocking+the+potential+of+model+merging+for+low%2Dresource+languages%2E+In+Findings+of+the+Associati)
53. Gemma Team, Aishwarya Kamath, Johan Ferret, Shreya Pathak, Nino Vieillard, Ramona Merhej, Sarah Perrin, Tatiana Matejovicova, Alexandre Ramé, Morgane Rivière, and 1 others. 2025. Gemma 3 technical report. arXiv preprint arXiv:2503.19786. [find](https://scholar.google.com/scholar?q=Gemma+Team%2C+Aishwarya+Kamath%2C+Johan+Ferret%2C+Shreya+Pathak%2C+Nino+Vieillard%2C+Ramona+Merhej%2C+Sarah+Perrin%2C+Tatiana+Matejovicova%2C+Alexandre+Ram%C3%A9%2C+Morgane+Rivi%C3%A8re%2C+and+1+others%2E+2025%2E+Gemma+3+technical+rep)
54. Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, and 1 others. 2023. [Llama 2: Open foundation and fine-tuned chat models](https://md.research.elamur.ai/free/en/papers/touvron-2023-llama-2-open-foundation-fine/). arXiv preprint arXiv:2307.09288.
55. Gorka Urbizu, Ander Corral, Xabier Saralegi, and Iñaki San Vicente. 2025. Sub-1B language models for low-resource languages: Training strategies and insights for Basque. In Proceedings of the 5th Workshop on Multilingual Representation Learning (MRL 2025), pages 519–530, Suzhuo, China. Association f… [find](https://scholar.google.com/scholar?q=Gorka+Urbizu%2C+Ander+Corral%2C+Xabier+Saralegi%2C+and+I%C3%B1aki+San+Vicente%2E+2025%2E+Sub%2D1B+language+models+for+low%2Dresource+languages%3A+Training+strategies+and+insights+for+Basque%2E+In+Proceedings+of+the+5th+Work)
56. Jonas Waldendorf, Barry Haddow, and Alexandra Birch. 2024. Contrastive decoding reduces hallucinations in large multilingual machine translation models. In Proceedings of the 18th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers), pages 2526–… [find](https://scholar.google.com/scholar?q=Jonas+Waldendorf%2C+Barry+Haddow%2C+and+Alexandra+Birch%2E+2024%2E+Contrastive+decoding+reduces+hallucinations+in+large+multilingual+machine+translation+models%2E+In+Proceedings+of+the+18th+Conference+of+the+Eu)
57. Yangyifan Xu, Jianghao Chen, Junhong Wu, and Jiajun Zhang. 2025. [Hit the sweet spot! span-level ensemble for large language models](https://md.research.elamur.ai/free/en/papers/xu-2025-hit-sweet-spot-span-level/). In Proceedings of the 31st International Conference on Computational Linguistics, pages 8314–8325, Abu Dhabi, UAE. Association for Computational Linguistics.
58. Prateek Yadav, Derek Tam, Leshem Choshen, Colin Raffel, and Mohit Bansal. 2023. [TIES-merging: Resolving interference when merging models](https://md.research.elamur.ai/free/en/papers/yadav-2023-ties-merging-resolving-interference-merging/). In Thirty-seventh Conference on Neural Information Processing Systems.
59. Atsuki Yamaguchi, Terufumi Morishita, Aline Villavicencio, and Nikolaos Aletras. 2025. Adapting chat language models using only target unlabeled language data. Transactions on Machine Learning Research. [find](https://scholar.google.com/scholar?q=Atsuki+Yamaguchi%2C+Terufumi+Morishita%2C+Aline+Villavicencio%2C+and+Nikolaos+Aletras%2E+2025%2E+Adapting+chat+language+models+using+only+target+unlabeled+language+data%2E+Transactions+on+Machine+Learning+Researc)
60. An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, and 1 others. 2024. Qwen2. 5 technical report. arXiv preprint arXiv:2412.15115. [find](https://scholar.google.com/scholar?q=An+Yang%2C+Baosong+Yang%2C+Beichen+Zhang%2C+Binyuan+Hui%2C+Bo+Zheng%2C+Bowen+Yu%2C+Chengyuan+Li%2C+Dayiheng+Liu%2C+Fei+Huang%2C+Haoran+Wei%2C+and+1+others%2E+2024%2E+Qwen2%2E+5+technical+report%2E+arXiv+preprint+arXiv%3A2412%2E15115)
61. Zheng Xin Yong, Hailey Schoelkopf, Niklas Muennighoff, Alham Fikri Aji, David Ifeoluwa Adelani, Khalid Almubarak, M Saiful Bari, Lintang Sutawika, Jungo Kasai, Ahmed Baruwa, Genta Winata, Stella Biderman, Edward Raff, Dragomir Radev, and Vassilina Nikoulina. 2023. [BLOOM+1: Adding language support to…](https://md.research.elamur.ai/free/en/papers/yong-2023-bloom-1-adding-language-support/)
62. Byeongho Yu, Changhun Lee, Jun-gyu Jin, and Eunhyeok Park. 2025. PruneCD: Contrasting pruned self model to improve decoding factuality. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pages 32450–32461, Suzhou, China. Association for Computational Linguisti… [find](https://scholar.google.com/scholar?q=Byeongho+Yu%2C+Changhun+Lee%2C+Jun%2Dgyu+Jin%2C+and+Eunhyeok+Park%2E+2025%2E+PruneCD%3A+Contrasting+pruned+self+model+to+improve+decoding+factuality%2E+In+Proceedings+of+the+2025+Conference+on+Empirical+Methods+in+Na)
63. Chen Zhang, Jiuheng Lin, Xiao Liu, Zekai Zhang, and Yansong Feng. 2025a. Read it in two steps: Translating extremely low-resource languages with code-augmented grammar books. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 397… [find](https://scholar.google.com/scholar?q=Chen+Zhang%2C+Jiuheng+Lin%2C+Xiao+Liu%2C+Zekai+Zhang%2C+and+Yansong+Feng%2E+2025a%2E+Read+it+in+two+steps%3A+Translating+extremely+low%2Dresource+languages+with+code%2Daugmented+grammar+books%2E+In+Proceedings+of+the+63r)
64. Chen Zhang, Xiao Liu, Jiuheng Lin, and Yansong Feng. 2024a. Teaching large language models an unseen language on the fly. In Findings of the Association for Computational Linguistics: ACL 2024, pages 8783–8800, Bangkok, Thailand. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Chen+Zhang%2C+Xiao+Liu%2C+Jiuheng+Lin%2C+and+Yansong+Feng%2E+2024a%2E+Teaching+large+language+models+an+unseen+language+on+the+fly%2E+In+Findings+of+the+Association+for+Computational+Linguistics%3A+ACL+2024%2C+pages+)
65. Chen Zhang, Mingxu Tao, Quzhe Huang, Jiuheng Lin, Zhibin Chen, and Yansong Feng. 2024b. MC 2 : Towards transparent and culturally-aware NLP for minority languages in China. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 8832–… [find](https://scholar.google.com/scholar?q=Chen+Zhang%2C+Mingxu+Tao%2C+Quzhe+Huang%2C+Jiuheng+Lin%2C+Zhibin+Chen%2C+and+Yansong+Feng%2E+2024b%2E+MC+2+%3A+Towards+transparent+and+culturally%2Daware+NLP+for+minority+languages+in+China%2E+In+Proceedings+of+the+62nd+)
66. Name Xuandong Zhao, Xianjun Yang, Tianyu Pang, Chao Du, Lei Li, Yu-Xiang Wang, and William Yang Wang. 2024. Weak-to-strong jailbreaking on large language models. In ICML 2024 Next Generation of AI Safety Workshop. [find](https://scholar.google.com/scholar?q=Name+Xuandong+Zhao%2C+Xianjun+Yang%2C+Tianyu+Pang%2C+Chao+Du%2C+Lei+Li%2C+Yu%2DXiang+Wang%2C+and+William+Yang+Wang%2E+2024%2E+Weak%2Dto%2Dstrong+jailbreaking+on+large+language+models%2E+In+ICML+2024+Next+Generation+of+AI+Saf)
67. Qihuang Zhong, Liang Ding, Juhua Liu, Bo Du, and Dacheng Tao. 2024. ROSE doesn’t do that: Boosting the safety of instruction-tuned large language models with reverse prompt contrastive decoding. In Findings of the Association for Computational Linguistics: ACL 2024, pages 13721–13736, Bangkok, Thail… [find](https://scholar.google.com/scholar?q=Qihuang+Zhong%2C+Liang+Ding%2C+Juhua+Liu%2C+Bo+Du%2C+and+Dacheng+Tao%2E+2024%2E+ROSE+doesn%E2%80%99t+do+that%3A+Boosting+the+safety+of+instruction%2Dtuned+large+language+models+with+reverse+prompt+contrastive+decoding%2E+In+Fi)
68. Wenhao Zhu, Sizhe Liu, Shujian Huang, Shuaijie She, Chris Wendler, and Jiajun Chen. 2024. Multilingual contrastive decoding via language-agnostic layers skipping. In Findings of the Association for Computational Linguistics: EMNLP 2024, pages 8775– 8782, Miami, Florida, USA. Association for Computat… [find](https://scholar.google.com/scholar?q=Wenhao+Zhu%2C+Sizhe+Liu%2C+Shujian+Huang%2C+Shuaijie+She%2C+Chris+Wendler%2C+and+Jiajun+Chen%2E+2024%2E+Multilingual+contrastive+decoding+via+language%2Dagnostic+layers+skipping%2E+In+Findings+of+the+Association+for+Co)

*Updated 2 November 2026*