--- title: "Are Small Language Models the Silver Bullet to — Song 2026" description: "The paper examines whether compact language models can translate low-resource languages (LRLs) reliably." authors: ["Yewei Song", "Lujun Li", "Cedric Lothritz", "Saad Ezzini", "Lama Sleem", "Niccolo' Gentile", "Radu State", "Tegawendé F. Bissyandé", "Jacques Klein"] year: 2026 venue: "Proceedings for the Ninth Workshop on Technologies for Machine Translation of Low Resource Languages (LoResMT 2026)" source: "https://aclanthology.org/2026.loresmt-1.1/" doi: "10.18653/v1/2026.loresmt-1.1" licence: "CC BY 4.0 (ACL Anthology)" licence_url: "https://creativecommons.org/licenses/by/4.0/" summary_licence: "Elamur Cosmic License" canonical: "https://research.elamur.ai/free/papers/song-2026-are-small-language-models-silver/" --- [Elamur Research](https://md.research.elamur.ai/) · [this page in HTML](https://research.elamur.ai/free/papers/song-2026-are-small-language-models-silver/) [home](https://md.research.elamur.ai/) / [papers](https://md.research.elamur.ai/free/papers/) / [2026](https://md.research.elamur.ai/free/papers/2026/) / Are Small Language Models the Silver Bullet to Low-Resource Languages Machine Translation? # Are Small Language Models the Silver Bullet to Low-Resource Languages Machine Translation? **Authors:** Yewei Song, Lujun Li, Cedric Lothritz, Saad Ezzini, Lama Sleem, Niccolo' Gentile, Radu State, Tegawendé F. Bissyandé, Jacques Klein · **Year:** 2026 · **Venue:** Proceedings for the Ninth Workshop on Technologies for Machine Translation of Low Resource Languages (LoResMT 2026) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/) **Links:** [Publisher's page](https://aclanthology.org/2026.loresmt-1.1/) · [PDF](https://aclanthology.org/2026.loresmt-1.1.pdf) · [DOI 10.18653/v1/2026.loresmt-1.1](https://doi.org/10.18653/v1/2026.loresmt-1.1) · [OpenAlex](https://openalex.org/W7140087451) **Citations:** 1 citations (openalex-snapshot · 2026-10-14) · [2](#cited-by) in our library, of the 140673 papers whose bibliographies we read ## Summary The paper examines whether compact language models can translate low-resource languages (LRLs) reliably. Such models are attractive where cost, latency, privacy or offline use matter, but they often hallucinate on underrepresented languages. The authors first benchmark several small decoder-only models (Llama-3.x, Gemma2, Qwen2.5, Phi-3/3.5, Ministral-8B) against GPT-4o-mini on all of FLORES-200. They translate into English and score outputs mainly with an LLM-as-a-Judge (LLMaaJ) metric, plus BLEU. Small models lose far more quality on LRLs than larger ones do, and African, Austronesian and Indigenous American languages score lowest. To close this gap, the authors build synthetic training data by having strong teacher models translate mostly monolingual LRL text, then fine-tune the small models on it. Luxembourgish is the main case study. With this distillation, 2B–3B models can reach or beat systems with up to 70B parameters. General abilities are preserved. This offers a cheap and practical route to deployable LRL translation. ## Key points - **Benchmark:** All 200 FLORES-200 languages were evaluated in the source→English direction. LLMaaJ scores range from 0 to 1, and a score of at least 0.8 is treated as a good translation. - **Training method:** Full-parameter fine-tuning beat LoRA, and decoder-only teachers (Llama 3.3-70B, GPT-4o-mini) beat the encoder–decoder NLLB-200-3.3B teacher. - **Effect of data size:** Increasing training data from 25% to 100% roughly doubled spBLEU for EN→LB with Llama-3.2-3B-Instruct and Gemma-2-2b-it. Gemma learned faster with little data but gained less beyond 50%. - **Generalisation to other languages:** The pipeline was repeated with Llama-3.2-3B-Instruct on Khasi, Assamese and Ukrainian, using 10,000 monolingual sentences and 1,000 validation pairs per language. - Gains were largest where the base model started weak, such as En–Kh and En–As. - NLLB could not serve as a teacher for Khasi because it does not support that language. - **No catastrophic forgetting:** SuperGLUE scores changed only slightly after fine-tuning. Training ran on 4×H100 GPUs with TRL, and the code and models are released openly. *Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.* Fields: [Linguistics & languages](https://md.research.elamur.ai/free/shelves/linguistics/) · [Machine translation](https://md.research.elamur.ai/free/shelves/machine-translation/) Topics: [Machine translation](https://md.research.elamur.ai/free/topics/machine-translation/) ## Cited by Cited by 2 papers of our library cite it (of the 140673 whose bibliographies we read); 1 of them have a page here, 1 not yet. - [IndigiEval: Evaluating LLMs in North American Indigenous Languages](https://md.research.elamur.ai/free/papers/mainzinger-2026-indigieval-evaluating-llms-north-american/) ## Cite The paper itself, not this page. BibTeX from the library's record of the work. ```bibtex @inproceedings{song2026are, title = {{Are Small Language Models the Silver Bullet to Low-Resource Languages Machine Translation?}}, author = {Song, Yewei and Li, Lujun and Lothritz, Cedric and Ezzini, Saad and Sleem, Lama and Gentile, Niccolo' and State, Radu and Bissyandé, Tegawendé F. and Klein, Jacques}, booktitle = {Proceedings for the Ninth Workshop on Technologies for Machine Translation of Low Resource Languages (LoResMT 2026)}, year = {2026}, doi = {10.18653/v1/2026.loresmt-1.1}, url = {https://doi.org/10.18653/v1/2026.loresmt-1.1} } ``` - [← previous Are LLMs Good Text Diacritizers? An Arabic and Yoruba Case Study](https://md.research.elamur.ai/free/papers/lingbulk-2026-toyin-are-llms-good-text-diacritizers/) - [next → Are the LLMs Capable of Maintaining at Least the Language Genus?](https://md.research.elamur.ai/free/papers/mitrovic-2026-are-llms-capable-maintaining-least/) - [↑ back to papers of 2026](https://md.research.elamur.ai/free/papers/2026/) ## References 2 references, 0 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet. 1. SFT Methods Llama-3.2-3B-Instruct Gemma-2-2b-it (Llama, 2024) (Google, 2024) [find](https://scholar.google.com/scholar?q=SFT+Methods+Llama%2D3%2E2%2D3B%2DInstruct+Gemma%2D2%2D2b%2Dit+%28Llama%2C+2024%29+%28Google%2C+2024%29) 2. FS/ LoRA SFT FS/ LoRA SFT *Table 5: Various models and their SFT methods. "FS/ Lora SFT" refers to full-size and "Lora SFT" denotes Low-Rank Adaptation SFT only.* [find](https://scholar.google.com/scholar?q=FS%2F+LoRA+SFT+FS%2F+LoRA+SFT+%2ATable+5%3A+Various+models+and+their+SFT+methods%2E+%22FS%2F+Lora+SFT%22+refers+to+full%2Dsize+and+%22Lora+SFT%22+denotes+Low%2DRank+Adaptation+SFT+only%2E%2A)