--- title: "Dense Retrieval for Low Resource Languages — the Case of Amharic Language — Yeshambel 2025" description: "They create a small training set, Train Amharic, and evaluate on two collections: 2AIRTC and the Amharic part of… 15 references, cited by 1 paper here." authors: ["Tilahun Yeshambel", "Moncef Garouani", "Serge Molina", "Josiane Mothe"] year: 2025 venue: "arXiv:2503.18570" source: "https://arxiv.org/abs/2503.18570" doi: "10.48550/arxiv.2503.18570" arxiv: "2503.18570" licence: "CC BY 4.0 (arXiv)" licence_url: "http://creativecommons.org/licenses/by/4.0/" summary_licence: "Elamur Cosmic License" canonical: "https://research.elamur.ai/free/en/papers/yeshambel-2025-dense-retrieval-low-resource-languages/" --- [Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/yeshambel-2025-dense-retrieval-low-resource-languages/) [home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2025](https://md.research.elamur.ai/free/en/papers/2025/page/3/) / Dense Retrieval for Low Resource Languages — the Case of Amharic Language # Dense Retrieval for Low Resource Languages — the Case of Amharic Language **Authors:** Tilahun Yeshambel, Moncef Garouani, Serge Molina and Josiane Mothe · **Year:** 2025 · **Venue:** arXiv:2503.18570 · **Licence:** [CC BY 4.0 (arXiv)](http://creativecommons.org/licenses/by/4.0/) **Links:** [Publisher's page](https://arxiv.org/abs/2503.18570) · [PDF](https://arxiv.org/pdf/2503.18570) · [DOI 10.48550/arxiv.2503.18570](https://doi.org/10.48550/arxiv.2503.18570) · [arXiv 2503.18570](https://arxiv.org/abs/2503.18570) · [OpenAlex](https://openalex.org/W6892012517) **Citations:** 0 citations (OpenAlex, 2026-10-14) · 1 in our library, of the 140,884 papers whose bibliographies we read ## Summary The paper asks whether modern neural retrievers can work for Amharic, a Semitic language with about 120 million speakers. Its rich inflection and derivation produce many surface forms per word, which hurts exact-match ranking, and annotated data and tools are scarce. The authors compare BM25 (Lemur Indri) against ColBERTv2 built on several Amharic masked language models. They also try SPLADE. They create a small training set, Train Amharic, and evaluate on two collections: 2AIRTC and the Amharic part of AfriCLIRMatrix, whose English test queries they machine-translated into Amharic. Untuned encoders give near-zero scores, whether English or Amharic. After fine-tuning on only about 150 examples, ColBERT becomes usable, and the best model reaches NDCG@10 of 0.704 on 2AIRTC. SPLADE yields nothing. The authors also point to three practical problems: indexing is much more expensive, the pretraining text may overlap with the test documents, and local compute is lacking. The work matters because it suggests a tiny, manually built training set may be enough to adapt dense retrieval to under-served languages. ## Key points - Data: - 2AIRTC has more than 10,000 documents and 240 curated queries with human relevance judgments. - Train Amharic has 152 queries and 152 documents. - AfriCLIRMatrix (Amharic side) has 15,458 documents and 1,500 test queries, translated with NLLB-200. - Encoders come from rasyosef on Hugging Face: bert-medium-amharic, roberta-base-amharic and roberta-amharic-text-embedding-base. They were pretrained on about 290M tokens from Oscar, C4 and the Amharic Sentences Corpus. - Scores without fine-tuning were near zero: - English BERT models gave NDCG@10 of 0.000–0.061. - Amharic-pretrained models gave 0.000–0.005. - The best fine-tuned model, roberta-amharic-text-embedding-base, scored NDCG@10 of 0.704 on 2AIRTC but only 0.177 on AfriCLIRMatrix. DistilSPLADE v2 produced only null results. - Timing compared with Indri: - ColBERTv2 indexing took about 29× (2AIRTC) and 46× (AfriCLIRMatrix) longer. - Query time was roughly the same (0.78× and 1.17×). ## Interesting ideas - About 150 hand-made query–document pairs were enough to turn a useless encoder into a working retriever, well below the roughly 1000 examples previously reported. Small curated seed sets can therefore be worth more than waiting for big data. - The paper separates the effect of architecture from the effect of extra fine-tuning. The best model wins because of its additional tuning, not because of its design. - The authors warn that in low-resource settings the pretraining corpus may contain the very documents used for evaluation, a form of collection contamination. - A test set in the target language can be built by machine-translating queries from an existing English benchmark (here with NLLB-200). - The paper reports one-time indexing cost separately from per-query cost. That split shows a heavy upfront investment can still give fast answers. ## For Mova How these ideas could serve Mova, our small open language model — its files, stages and tests: - **I1** (learn): Before scaling silver data, try adapting a weak component with a small hand-checked seed set and measure how far that alone carries it. - **I2** (learn): When a parser or teacher tier improves, run ablations that isolate the training data or tuning step from the model change, so gains are credited correctly. - **I3** (learn): Check that held-out test texts (fables, school tests) never appear in silver or teacher data, and treat such overlap as a negative control that must fail. - **I4** (answer): Translated variants of QA or word-problem sets could probe whether answering depends on the meaning graph rather than on English surface wording. - **I5** (world): Measure the compiled knowledge layer (ontology, verb classes) the same way, keeping build cost and lookup cost as separate numbers so expensive precompilation stays justified. *Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.* Fields: [Linguistics & languages](https://md.research.elamur.ai/free/en/shelves/linguistics/) Topics: [🌱 Low-resource languages](https://md.research.elamur.ai/free/en/topics/low-resource-languages/) · [🐪 Semitic languages](https://md.research.elamur.ai/free/en/topics/semitic-languages/) · [🔍 Information retrieval & search](https://md.research.elamur.ai/free/en/topics/information-retrieval/) ## Cite Cite as: Tilahun Yeshambel, Moncef Garouani, Serge Molina and Josiane Mothe. 2025. Dense Retrieval for Low Resource Languages — the Case of Amharic Language. arXiv:2503.18570. The paper itself, not this page. BibTeX from the library's record of the work. ```bibtex @misc{yeshambel2025dense, title = {{Dense Retrieval for Low Resource Languages — the Case of Amharic Language}}, author = {Tilahun Yeshambel, Moncef Garouani, Serge Molina and Josiane Mothe}, year = {2025}, doi = {10.48550/arxiv.2503.18570}, eprint = {2503.18570}, archivePrefix = {arXiv}, url = {https://doi.org/10.48550/arxiv.2503.18570} } ``` - [← previous Cross-Lingual SynthDocs: A Large-Scale Synthetic Corpus for Any to Arabic OCR and Document Understanding](https://md.research.elamur.ai/free/en/papers/al-homoud-2025-cross-lingual-synthdocs-large-scale/) - [next → Development of Android Game-Based Media to Enhance Students' Arabic Grammar Skills](https://md.research.elamur.ai/free/en/papers/utomo-2025-development-android-game-based-media/) - [↑ back to papers of 2025](https://md.research.elamur.ai/free/en/papers/2025/page/3/) ## References 15 references, 6 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet. 1. S. Robertson et al. Okapi at trec-3. Nist Special Publication, 1995. [our card](https://md.research.elamur.ai/free/en/papers/robertson-1994-okapi-trec-3/) 2. K. Santhanam et al. Colbertv2: Effective and efficient retrieval via lightweight late interaction. arXiv preprint arXiv:2112.01488, 2021. [↗](https://arxiv.org/abs/2112.01488) [our card](https://md.research.elamur.ai/free/en/papers/santhanam-2022-colbertv2-effective-efficient-retrieval-lightweight/) 3. T. Yeshambel et al. 2airtc: the amharic adhoc information retrieval test collection. In Experimental IR Meets Multilinguality, Multimodality, and Interaction: 11th International Conf. of the CLEF Association, pages 55–66, 2020. [find](https://scholar.google.com/scholar?q=T%2E+Yeshambel+et+al%2E+2airtc%3A+the+amharic+adhoc+information+retrieval+test+collection%2E+In+Experimental+IR+Meets+Multilinguality%2C+Multimodality%2C+and+Interaction%3A+11th+International+Conf%2E+of+the+CLEF+Asso) 4. O. Ogundepo et al. AfriCLIRMatrix: Enabling Cross-Lingual Information Retrieval for African Languages. In Proc. of the Conf. on Empirical Methods in Natural Language Processing, pages 8721–8728. ACL, 2022. [find](https://scholar.google.com/scholar?q=O%2E+Ogundepo+et+al%2E+AfriCLIRMatrix%3A+Enabling+Cross%2DLingual+Information+Retrieval+for+African+Languages%2E+In+Proc%2E+of+the+Conf%2E+on+Empirical+Methods+in+Natural+Language+Processing%2C+pages+8721%E2%80%938728%2E+ACL%2C+) 5. NLLB Team. No Language Left Behind: Scaling Human-Centered Machine Translation, 2022. [find](https://scholar.google.com/scholar?q=NLLB+Team%2E+No+Language+Left+Behind%3A+Scaling+Human%2DCentered+Machine+Translation%2C+2022%2E) 6. T. Strohman et al. Indri: A language model-based search engine for complex queries. In Proc. of the international conf. on intelligent analysis, volume 2, pages 2–6, 2005. [find](https://scholar.google.com/scholar?q=T%2E+Strohman+et+al%2E+Indri%3A+A+language+model%2Dbased+search+engine+for+complex+queries%2E+In+Proc%2E+of+the+international+conf%2E+on+intelligent+analysis%2C+volume+2%2C+pages+2%E2%80%936%2C+2005%2E) 7. T. Yeshambel et al. Construction of morphem-based amharic stopword list for information retrieval. In the 8th EAI international conference on advancement of science and technology. Springer, 2020. [find](https://scholar.google.com/scholar?q=T%2E+Yeshambel+et+al%2E+Construction+of+morphem%2Dbased+amharic+stopword+list+for+information+retrieval%2E+In+the+8th+EAI+international+conference+on+advancement+of+science+and+technology%2E+Springer%2C+2020%2E) 8. PJO. Suárez et al. A monolingual approach to contextualized word embeddings for mid-resource languages. In Proc. ACL, pages 1703–1714, 2020. [our card](https://md.research.elamur.ai/free/en/papers/suarez-2020-monolingual-approach-contextualized-word-embeddings/) 9. PJO. Suárez et al. Asynchronous pipelines for processing huge corpora on medium to low resource infrastructures. Proc. of the Workshop on Challenges in the Management of Large Corpora, pages 9 – 16, 2019. [find](https://scholar.google.com/scholar?q=PJO%2E+Su%C3%A1rez+et+al%2E+Asynchronous+pipelines+for+processing+huge+corpora+on+medium+to+low+resource+infrastructures%2E+Proc%2E+of+the+Workshop+on+Challenges+in+the+Management+of+Large+Corpora%2C+pages+9+%E2%80%93+16%2C+2) 10. C. Raffel et al. Exploring the limits of transfer learning with a unified text-to-text transformer. J. of machine learning research, 21(140):1–67, 2020. [our card](https://md.research.elamur.ai/free/en/papers/raffel-2019-exploring-limits-transfer-learning-unified/) 11. S. M. Yimam et al. Introducing various semantic models for amharic: Experimentation and evaluation with multiple tasks and datasets. Future Internet, 13(11):275, 2021. [our card](https://md.research.elamur.ai/free/en/papers/yimam-2020-introducing-various-semantic-models-amharic/) 12. A. Conneau et al. Unsupervised cross-lingual representation learning at scale. arXiv preprint arXiv:1911.02116, 2019. [↗](https://arxiv.org/abs/1911.02116) 13. C. Lassance et al. Splade-v3: New baselines for splade. arXiv preprint arXiv:2403.06789, 2024. [↗](https://arxiv.org/abs/2403.06789) 14. V. Karpukhin et al. Dense passage retrieval for open-domain question answering. In EMNLP (1), pages 6769–6781, 2020. [our card](https://md.research.elamur.ai/free/en/papers/karpukhin-2020-dense-passage-retrieval-open-domain/) 15. Vishakha Suresh Kalal, Andrew Parry, and Sean MacAvaney. Training on the test model: Contamination in ranking distillation. arXiv preprint arXiv:2411.02284, 2024. [↗](https://arxiv.org/abs/2411.02284)