--- title: "DevLake at LoResMT 2026 — Tyurin 2026" description: "This paper reports Team DevLake's entry in the LoResMT 2026 shared task on Russian-to-Bashkir machine translation." authors: ["Vyacheslav Tyurin"] year: 2026 venue: "Proceedings for the Ninth Workshop on Technologies for Machine Translation of Low Resource Languages (LoResMT 2026)" source: "https://aclanthology.org/2026.loresmt-1.18/" doi: "10.18653/v1/2026.loresmt-1.18" licence: "CC BY 4.0 (ACL Anthology)" licence_url: "https://creativecommons.org/licenses/by/4.0/" summary_licence: "Elamur Cosmic License" canonical: "https://research.elamur.ai/free/en/papers/tyurin-2026-devlake-loresmt-2026-impact-pre/" --- [Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/tyurin-2026-devlake-loresmt-2026-impact-pre/) [home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2026](https://md.research.elamur.ai/free/en/papers/2026/page/5/) / DevLake at LoResMT 2026: The Impact of Pre-training and Model Scale on Russian-Bashkir Low-Resource Translation # DevLake at LoResMT 2026: The Impact of Pre-training and Model Scale on Russian-Bashkir Low-Resource Translation **Authors:** Vyacheslav Tyurin · **Year:** 2026 · **Venue:** Proceedings for the Ninth Workshop on Technologies for Machine Translation of Low Resource Languages (LoResMT 2026) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/) **Links:** [Publisher's page](https://aclanthology.org/2026.loresmt-1.18/) · [PDF](https://aclanthology.org/2026.loresmt-1.18.pdf) · [DOI 10.18653/v1/2026.loresmt-1.18](https://doi.org/10.18653/v1/2026.loresmt-1.18) · [OpenAlex](https://openalex.org/W7140091404) **Citations:** 0 citations (OpenAlex, 2026-10-14) ## Summary This paper reports Team DevLake's entry in the LoResMT 2026 shared task on Russian-to-Bashkir machine translation. It asks whether cheaply adapting a large multilingual model beats fully training a compact one when only a single consumer GPU (RTX 3080, 10GB) is available. The organizers supplied about 1.2M sentence pairs, but the data was noisy. The author scored each pair with the slone/bert-base-multilingual-cased-bak-rus-similarity classifier and kept two subsets: a strict one (score ≥ 0.80, ~486k pairs) and a permissive one (≥ 0.10, ~923k pairs). Three systems were compared: - NLLB-200-1.3B-Distilled, tuned with 4-bit QLoRA on the strict subset. - M2M-100 (418M). - MarianMT (opus-mt-en-trk, 77M), fully fine-tuned after its vocabulary was extended with missing Bashkir Cyrillic letters. NLLB already covers Bashkir in pre-training, and it achieved the best CHRF++ despite seeing roughly half as much data. MarianMT, by contrast, produced fluent output whose meaning often drifted. The broader lesson for low-resource MT is that native pre-training coverage and scale outweigh raw data volume, and that such models can be adapted on modest hardware. ## Key points - **Official CHRF++ scores:** NLLB-200 (1.3B) 52.67, M2M-100 48.80, and MarianMT (77M) 43.15, despite MarianMT being trained on the larger ~923k-pair set. - **Training setups:** - NLLB: nf4 QLoRA with r=64, α=64, dropout 0.1, adapters on all linear projections, 1 epoch at lr 2e-4. - MarianMT: full FP32 fine-tuning for 3 epochs at lr 5e-5, with a Bashkir language token prepended to source sentences. - **Inference tricks:** - Seven test sentences matched the training corpus exactly, so their reference translations were inserted directly. - Decoding used beam size 5, no_repeat_ngram_size=3, and repetition penalty 1.2 to suppress repetition loops. - **Qualitative errors:** MarianMT's output shifted with punctuation, turning "apple cider" into words about sitting or planting, or into "leafy chair." NLLB consistently produced Алма сидары. - **Released resources and limitations:** - Code is on GitHub and checkpoints are on the Hugging Face Hub (e.g., Voldis/nllb-1.3b-rus-bak). - Inference with the 1.3B model needs about 2GB VRAM and has higher latency. - The BERT-based filter may discard rare dialectal Bashkir. *Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.* Fields: [Linguistics & languages](https://md.research.elamur.ai/free/en/shelves/linguistics/) · [Machine translation](https://md.research.elamur.ai/free/en/shelves/machine-translation/) Topics: [Machine translation](https://md.research.elamur.ai/free/en/topics/machine-translation/) · [Low-resource languages](https://md.research.elamur.ai/free/en/topics/low-resource-languages/) ## Cite The paper itself, not this page. BibTeX from the library's record of the work. ```bibtex @inproceedings{tyurin2026devlake, title = {{DevLake at LoResMT 2026: The Impact of Pre-training and Model Scale on Russian-Bashkir Low-Resource Translation}}, author = {Tyurin, Vyacheslav}, booktitle = {Proceedings for the Ninth Workshop on Technologies for Machine Translation of Low Resource Languages (LoResMT 2026)}, year = {2026}, doi = {10.18653/v1/2026.loresmt-1.18}, url = {https://doi.org/10.18653/v1/2026.loresmt-1.18} } ``` - [← previous Development of an Arabic Grammar Instructional Medium Based on Illustrated Stories in Elementary School](https://md.research.elamur.ai/free/en/papers/solihah-2026-development-arabic-grammar-instructional-medium/) - [next → Diagnose, Then Repair: A Two-Stage MQM-Guided Post-Editing Framework for Domain-Specific Machine Translation](https://md.research.elamur.ai/free/en/papers/wang-2026-diagnose-then-repair-two-stage/) - [↑ back to papers of 2026](https://md.research.elamur.ai/free/en/papers/2026/page/5/) ## References 6 references, 1 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet. 1. Marta R. Costa-jussà and 1 others. 2022. No language left behind: Scaling human-centered machine translation. arXiv preprint arXiv:2207.04672. [find](https://scholar.google.com/scholar?q=Marta+R%2E+Costa%2Djuss%C3%A0+and+1+others%2E+2022%2E+No+language+left+behind%3A+Scaling+human%2Dcentered+machine+translation%2E+arXiv+preprint+arXiv%3A2207%2E04672%2E) 2. Tim Dettmers and 1 others. 2023. Qlora: Efficient finetuning of quantized llms. arXiv preprint arXiv:2305.14314. [find](https://scholar.google.com/scholar?q=Tim+Dettmers+and+1+others%2E+2023%2E+Qlora%3A+Efficient+finetuning+of+quantized+llms%2E+arXiv+preprint+arXiv%3A2305%2E14314%2E) 3. Angela Fan and 1 others. 2021. Beyond english-centric multilingual machine translation. Journal of Machine Learning Research, 22(107):1–48. [find](https://scholar.google.com/scholar?q=Angela+Fan+and+1+others%2E+2021%2E+Beyond+english%2Dcentric+multilingual+machine+translation%2E+Journal+of+Machine+Learning+Research%2C+22%28107%29%3A1%E2%80%9348%2E) 4. Edward J Hu and 1 others. 2021. Lora: Low-rank adaptation of large language models. arXiv preprint arXiv:2106.09685. [find](https://scholar.google.com/scholar?q=Edward+J+Hu+and+1+others%2E+2021%2E+Lora%3A+Low%2Drank+adaptation+of+large+language+models%2E+arXiv+preprint+arXiv%3A2106%2E09685%2E) 5. Marcin Junczys-Dowmunt and 1 others. 2018. Marian: Fast neural machine translation in C++. In Proceedings of ACL 2018, System Demonstrations. [find](https://scholar.google.com/scholar?q=Marcin+Junczys%2DDowmunt+and+1+others%2E+2018%2E+Marian%3A+Fast+neural+machine+translation+in+C%2B%2B%2E+In+Proceedings+of+ACL+2018%2C+System+Demonstrations%2E) [our card](https://md.research.elamur.ai/free/en/papers/junczys-2018-dowmunt-marian-fast-neural-machine-translation/) 6. Slone Team. 2023. Bert-base-multilingual-cased-bak-rus-similarity. slone/bert-base-multilingual-cased-bak-rus-similarity. [find](https://scholar.google.com/scholar?q=Slone+Team%2E+2023%2E+Bert%2Dbase%2Dmultilingual%2Dcased%2Dbak%2Drus%2Dsimilarity%2E+slone%2Fbert%2Dbase%2Dmultilingual%2Dcased%2Dbak%2Drus%2Dsimilarity%2E)