---
title: "AfriMTEB and AfriE5 — paper by Uemura et al. (2026)"
description: "The authors build AfriMTEB, an Africa-focused extension of MMTEB with 59 languages, 14 tasks and… 65 references, cited by 3 papers here."
authors: ["Kosei Uemura", "Miaoran Zhang", "David Ifeoluwa Adelani"]
year: 2026
venue: "Proceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers)"
source: "https://aclanthology.org/2026.eacl-long.171/"
doi: "10.18653/v1/2026.eacl-long.171"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/uemura-2026-afrimteb-afrie5-benchmarking-adapting-text/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/uemura-2026-afrimteb-afrie5-benchmarking-adapting-text/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2026](https://md.research.elamur.ai/free/en/papers/2026/page/4/) / AfriMTEB and AfriE5: Benchmarking and Adapting Text Embedding Models for African Languages

# AfriMTEB and AfriE5: Benchmarking and Adapting Text Embedding Models for African Languages

**Authors:** [Kosei Uemura](https://md.research.elamur.ai/free/en/authors/kosei-uemura/), [Miaoran Zhang](https://md.research.elamur.ai/free/en/authors/miaoran-zhang/) and [David Ifeoluwa Adelani](https://md.research.elamur.ai/free/en/authors/david-ifeoluwa-adelani/) · **Year:** 2026 · **Venue:** [Proceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers)](https://md.research.elamur.ai/free/en/venues/eacl/) ([EACL 2026](https://md.research.elamur.ai/free/en/venues/eacl/2026/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2026.eacl-long.171/) · [PDF](https://aclanthology.org/2026.eacl-long.171.pdf) · [DOI 10.18653/v1/2026.eacl-long.171](https://doi.org/10.18653/v1/2026.eacl-long.171) · [OpenAlex](https://openalex.org/W7140100985)

**Citations:** 1 citations (OpenAlex, 2026-10-14) · [3](https://md.research.elamur.ai/free/en/papers/uemura-2026-afrimteb-afrie5-benchmarking-adapting-text/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
Existing text-embedding benchmarks such as MMTEB cover African languages poorly. Some task types have no African data at all, and others include only one to three languages, so it is unclear how well embedding models serve the region.

The authors build AfriMTEB, an Africa-focused extension of MMTEB with 59 languages, 14 tasks and 38 datasets. These are grouped into eight families following the MTEB taxonomy, and six datasets are new to the suite. Because language coverage differs sharply between tasks, they also define AfriMTEB-Lite, a balanced subset where nearly every task includes the same nine languages.

They then adapt mE5-large-instruct into AfriE5 using cross-lingual contrastive learning combined with knowledge distillation. The training data are MNLI and SNLI, machine-translated with NLLB-200, filtered with SSA-COMET, and scored by a BGE Reranker v2 m3 teacher.

AfriE5 was tuned on only nine languages, yet it achieves the best overall average among open-weight models on both suites and is competitive with Gemini Embedding-001. This suggests that targeted adaptation on a few well-chosen languages can transfer to many more. The benchmark gives the community a standardized way to compare embedding models for African languages.

## Key points
- **Six added datasets:** AfriXNLI (15 languages), EmotionAnalysisPlus (14 African languages; the first multi-label task for African languages), AfriHate (14), InjongoIntent (16 languages, 40 intents), KinNews (Kinyarwanda news topics) and SIB200_14Classes (a harder 14-class SIB-200 variant).
- **AfriMTEB-Lite languages:** its 13 datasets share a common set of nine languages, namely Hausa, Igbo, Yoruba, Swahili, Kinyarwanda, Amharic, Oromo, Zulu and Xhosa.
- **Scoring:** results are macro-averaged first over languages within each task, then over tasks and families, so that no single task or language dominates.
- **Lite results:** AfriE5-Large-Instruct scores 63.7, ahead of Gemini embedding-001 (63.1) and mE5-large-instruct (62.0).
- **Full results:** the text reports 62.4 for AfriE5 versus 61.3 for mE5-large-instruct and 60.6 for Gemini. The Figure 1 caption instead gives a mean of 64.6, likely due to different averaging.
- **Ablation (Lite):** an SSA-COMET threshold of 0.75 (about 60K training pairs) gave the best average, 63.2. A threshold of 0.80 kept only about 7,500 samples and dropped the average to 58.3. Removing cross-lingual expansion lowered the average to 62.3.
- **Code:** released at github.com/LLMforLRL/FlagEmbedding-AfriE5.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 16,039 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 15,891 works

## Cite

Cite as: Kosei Uemura, Miaoran Zhang and David Ifeoluwa Adelani. 2026. AfriMTEB and AfriE5: Benchmarking and Adapting Text Embedding Models for African Languages. Proceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers).

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{uemura2026afrimteb,
  title = {{AfriMTEB and AfriE5: Benchmarking and Adapting Text Embedding Models for African Languages}},
  author = {Kosei Uemura, Miaoran Zhang and David Ifeoluwa Adelani},
  booktitle = {Proceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2026},
  doi = {10.18653/v1/2026.eacl-long.171},
  url = {https://doi.org/10.18653/v1/2026.eacl-long.171}
}
```

- [← previous A Goal Without a Plan Is Just a Wish: Efficient and Effective Global Planner Training for Long-Horizon Agent Tasks](https://md.research.elamur.ai/free/en/papers/si-2026-goal-without-plan-is-just/)
- [next → AgencyBench: Benchmarking the Frontiers of Autonomous Agents in 1M-Token Real-World Contexts](https://md.research.elamur.ai/free/en/papers/li-2026-agencybench-benchmarking-frontiers-autonomous-agents/)
- [↑ back to papers of 2026](https://md.research.elamur.ai/free/en/papers/2026/page/4/)

## References

65 references, 38 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. ExploreAI Academy and Joanne M. 2022. South african language identification. [https://kaggle.com/competitions/](https://kaggle.com/competitions/) south-african-language-identification. Kaggle. [↗](https://kaggle.com/competitions/)
2. Sibanda, Godson Koffi Kalipe, Jonathan Mukiibi, Salomon Kabongo Kabenamualu, Foutse Yuehgoh, Mmasibidi Setaka, Lolwethu Ndolela, and 8 others. 2025. [IrokoBench: A new benchmark for African languages in the age of large language models](https://md.research.elamur.ai/free/en/papers/adelani-2025-irokobench-new-benchmark-african-languages/). In Proceedings of the 2025 Conference of the Nations of the Amer…
3. Jesujoba O Alabi, Michael A Hedderich, David Ifeoluwa Adelani, and Dietrich Klakow. 2025. Charting the landscape of african nlp: Mapping progress and shaping the road ahead. arXiv preprint arXiv:2505.21315. [find](https://scholar.google.com/scholar?q=Jesujoba+O+Alabi%2C+Michael+A+Hedderich%2C+David+Ifeoluwa+Adelani%2C+and+Dietrich+Klakow%2E+2025%2E+Charting+the+landscape+of+african+nlp%3A+Mapping+progress+and+shaping+the+road+ahead%2E+arXiv+preprint+arXiv%3A2505%2E)
4. Mikel Artetxe, Sebastian Ruder, and Dani Yogatama. 2020. [On the cross-lingual transferability of monolingual representations](https://md.research.elamur.ai/free/en/papers/artetxe-2020-cross-lingual-transferability-monolingual-representations/). In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, page 4623–4637. Association for Computational Linguistics.
5. Lucas Bandarkar, Davis Liang, Benjamin Muller, Mikel Artetxe, Satya Narayan Shukla, Donald Husa, Naman Goyal, Abhinandan Krishnan, Luke Zettlemoyer, and Madian Khabsa. 2024a. [The belebele benchmark: a parallel reading comprehension dataset in 122 language variants](https://md.research.elamur.ai/free/en/papers/bandarkar-2023-belebele-benchmark-parallel-reading-comprehension/). In Proceedings of the 62nd Annual…
6. Lucas Bandarkar, Davis Liang, Benjamin Muller, Mikel Artetxe, Satya Narayan Shukla, Donald Husa, Naman Goyal, Abhinandan Krishnan, Luke Zettlemoyer, and Madian Khabsa. 2024b. [The belebele benchmark: a parallel reading comprehension dataset in 122 language variants](https://md.research.elamur.ai/free/en/papers/bandarkar-2023-belebele-benchmark-parallel-reading-comprehension/). In Proceedings of the 62nd Annual…
7. David Ifeoluwa Adelani, Hannah Liu, Xiaoyu Shen, Nikita Vassilyev, Jesujoba O. Alabi, Yanke Mao, Haonan Gao, and Annie En-Shiun Lee. 2024. [Sib-200: A simple, inclusive, and big evaluation dataset for topic classification in 200+ languages and dialects](https://md.research.elamur.ai/free/en/papers/adelani-2023-sib-200-simple-inclusive-big/). Preprint, arXiv:2309.07445.
8. Tadesse Destaw Belay, Israel Abebe Azime, Abinew Ali Ayele, Grigori Sidorov, Dietrich Klakow, Philip Slusallek, Olga Kolesnikova, and Seid Muhie Yimam. 2025. [Evaluating the capabilities of large language models for multi-label emotion understanding](https://md.research.elamur.ai/free/en/papers/belay-2025-evaluating-capabilities-large-language-models/). In Proceedings of the 31st International Conferenc…
9. David Ifeoluwa Adelani, Marek Masiak, Israel Abebe Azime, Jesujoba Alabi, Atnafu Lambebo Tonja, and et. al. 2023. [Masakhanews: News topic classification for african languages](https://md.research.elamur.ai/free/en/papers/adelani-2023-masakhanews-news-topic-classification-african/). Preprint, arXiv:2304.09972.
10. Samuel R. Bowman, Gabor Angeli, Christopher Potts, and Christopher D. Manning. 2015. [A large annotated corpus for learning natural language inference](https://md.research.elamur.ai/free/en/papers/bowman-2015-large-annotated-corpus-learning-natural/). Preprint, arXiv:1508.05326.
11. David Ifeoluwa Adelani, Jessica Ojo, Israel Abebe Azime, Jian Yun Zhuang, Jesujoba Oluwadara Alabi, Xuanli He, Millicent Ochieng, Sara Hooker, Andiswa Bukula, En-Shiun Annie Lee, Chiamaka Ijeoma Chukwuneke, Happy Buzaaba, Blessing Kudzaishe Jianlv Chen, Shitao Xiao, Peitian Zhang, Kun Luo, Defu Lian… — [M3-Embedding: Multi-Linguality, Multi-Functionality, Multi-Granularity Text Embeddings Through Self-Knowledge Distillation](https://md.research.elamur.ai/free/en/papers/chen-2024-m3-embedding-multi-linguality/)
12. Mathieu Ciancone, Imene Kerboua, Marion Schaeffer, and Wissam Siblini. 2024. Mteb-french: Resources for french sentence embedding evaluation and analysis. Preprint, arXiv:2405.20468. [find](https://scholar.google.com/scholar?q=Mathieu+Ciancone%2C+Imene+Kerboua%2C+Marion+Schaeffer%2C+and+Wissam+Siblini%2E+2024%2E+Mteb%2Dfrench%3A+Resources+for+french+sentence+embedding+evaluation+and+analysis%2E+Preprint%2C+arXiv%3A2405%2E20468%2E)
13. Alexis Conneau, Kartikay Khandelwal, Naman Goyal, Vishrav Chaudhary, Guillaume Wenzek, Francisco Guzmán, Edouard Grave, Myle Ott, Luke Zettlemoyer, and Veselin Stoyanov. 2020. Unsupervised cross-lingual representation learning at scale. Preprint, arXiv:1911.02116. [find](https://scholar.google.com/scholar?q=Alexis+Conneau%2C+Kartikay+Khandelwal%2C+Naman+Goyal%2C+Vishrav+Chaudhary%2C+Guillaume+Wenzek%2C+Francisco+Guzm%C3%A1n%2C+Edouard+Grave%2C+Myle+Ott%2C+Luke+Zettlemoyer%2C+and+Veselin+Stoyanov%2E+2020%2E+Unsupervised+cross%2Dlingu)
14. Alexis Conneau, Guillaume Lample, Ruty Rinott, Adina Williams, Samuel R. Bowman, Holger Schwenk, and Veselin Stoyanov. 2018. Xnli: Evaluating cross-lingual sentence representations. Preprint, arXiv:1809.05053. [find](https://scholar.google.com/scholar?q=Alexis+Conneau%2C+Guillaume+Lample%2C+Ruty+Rinott%2C+Adina+Williams%2C+Samuel+R%2E+Bowman%2C+Holger+Schwenk%2C+and+Veselin+Stoyanov%2E+2018%2E+Xnli%3A+Evaluating+cross%2Dlingual+sentence+representations%2E+Preprint%2C+arXiv%3A18)
15. Kenneth Enevoldsen, Isaac Chung, Imene Kerboua, Márton Kardos, Ashwin Mathur, David Stap, Jay Gala, Wissam Siblini, Dominik Krzemiński, Genta Indra Winata, Saba Sturua, Saiteja Utpala, Mathieu Ciancone, Marion Schaeffer, Diganta Misra, Shreeya Dhakal, Jonathan Rystrøm, Roman Solomatin, Ömer Veysel… — [MMTEB: Massive Multilingual Text Embedding Benchmark](https://md.research.elamur.ai/free/en/papers/enevoldsen-2025-mmteb-massive-multilingual-text-embedding/)
16. Xinshuo Hu, Zifei Shan, Xinping Zhao, Zetian Sun, Zhenyu Liu, Dongfang Li, Shaolin Ye, Xinyuan Wei, Qian Chen, Baotian Hu, and 1 others. 2025. Kalm-embedding: Superior training data brings a stronger embedding model. arXiv preprint arXiv:2501.01028. — [arXiv:2501.01028](https://arxiv.org/abs/2501.01028) [find](https://scholar.google.com/scholar?q=Xinshuo+Hu%2C+Zifei+Shan%2C+Xinping+Zhao%2C+Zetian+Sun%2C+Zhenyu+Liu%2C+Dongfang+Li%2C+Shaolin+Ye%2C+Xinyuan+Wei%2C+Qian+Chen%2C+Baotian+Hu%2C+and+1+others%2E+2025%2E+Kalm%2Dembedding%3A+Superior+training+data+brings+a+stronger+)
17. Amanda Kann. 2024. Massively multilingual token-based typology using the parallel Bible corpus. In Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024), pages 11070–11079, Torino, Italia. ELRA and ICCL. [find](https://scholar.google.com/scholar?q=Amanda+Kann%2E+2024%2E+Massively+multilingual+token%2Dbased+typology+using+the+parallel+Bible+corpus%2E+In+Proceedings+of+the+2024+Joint+International+Conference+on+Computational+Linguistics%2C+Language+Resourc)
18. Junseong Kim, Seolhwa Lee, Jihoon Kwon, Sangmo Gu, Yejin Kim, Minkyung Cho, Jy yong Sohn, and Chanyeol Choi. 2024. Linq-embed-mistral:elevating text retrieval with improved gpt data through task-specific control and quality refinement. Linq AI Research Blog. [find](https://scholar.google.com/scholar?q=Junseong+Kim%2C+Seolhwa+Lee%2C+Jihoon+Kwon%2C+Sangmo+Gu%2C+Yejin+Kim%2C+Minkyung+Cho%2C+Jy+yong+Sohn%2C+and+Chanyeol+Choi%2E+2024%2E+Linq%2Dembed%2Dmistral%3Aelevating+text+retrieval+with+improved+gpt+data+through+task%2Dspeci)
19. Jinhyuk Lee, Feiyang Chen, Sahil Dua, Daniel Cer, Madhuri Shanbhogue, Iftekhar Naim, Gustavo Hernández Ábrego, Zhe Li, Kaifeng Chen, Henrique Schechter Vera, Xiaoqi Ren, Shanfeng Zhang, Daniel Salz, Michael Boratko, Jay Han, Blair Chen, Shuo Huang, Vikram Rao, Paul Suganthan, and 28 others. 2025. Ge… — [Gemini Embedding: Generalizable Embeddings from Gemini](https://md.research.elamur.ai/free/en/papers/lee-2025-gemini-embedding-generalizable-embeddings-gemini/)
20. Christian Federmann, Tom Kocmi, and Ying Xin. 2022. [NTREX-128 – news test references for MT evaluation of 128 languages](https://md.research.elamur.ai/free/en/papers/federmann-2022-ntrex-128-news-test-references/). In Proceedings of the First Workshop on Scaling Up Multilingual Evaluation, pages 21–24, Online. Association for Computational Linguistics.
21. Senyu Li, Jiayi Wang, Felermino D. M. A. Ali, Colin Cherry, Daniel Deutsch, Eleftheria Briakou, Rui Sousa-Silva, Henrique Lopes Cardoso, Pontus Stenetorp, and David Ifeoluwa Adelani. 2025. [Ssa-comet: Do llms outperform learned metrics in evaluating mt for under-resourced african languages?](https://md.research.elamur.ai/free/en/papers/li-2025-ssa-comet-do-llms-outperform/) Preprint,…
22. Fangxiaoyu Feng, Yinfei Yang, Daniel Cer, Naveen Arivazhagan, and Wei Wang. 2022. [Language-agnostic bert sentence embedding](https://md.research.elamur.ai/free/en/papers/feng-2020-language-agnostic-bert-sentence-embedding/). Preprint, arXiv:2007.01852.
23. Shengzhe Li, Masaya Ohagi, and Ryokan Ri. 2024. JMTEB: Japanese Massive Text Embedding Benchmark. [https://huggingface.co/](https://huggingface.co/) datasets/sbintuitions/JMTEB. [↗](https://huggingface.co/)
24. Jack FitzGerald, Christopher Hench, Charith Peris, Scott Mackie, Kay Rottmann, Ana Sanchez, Aaron Nash, Liam Urbach, Vishesh Kakarala, Richa Singh, Swetha Ranganath, Laurie Crist, Misha Britan, Wouter Leeuwis, Gokhan Tur, and Prem Natarajan. 2023. MASSIVE: A 1M-example multilingual natural language… [find](https://scholar.google.com/scholar?q=Jack+FitzGerald%2C+Christopher+Hench%2C+Charith+Peris%2C+Scott+Mackie%2C+Kay+Rottmann%2C+Ana+Sanchez%2C+Aaron+Nash%2C+Liam+Urbach%2C+Vishesh+Kakarala%2C+Richa+Singh%2C+Swetha+Ranganath%2C+Laurie+Crist%2C+Misha+Britan%2C+Wouter)
25. Zehan Li, Xin Zhang, Yanzhao Zhang, Dingkun Long, Pengjun Xie, and Meishan Zhang. 2023a. [Towards general text embeddings with multi-stage contrastive learning](https://md.research.elamur.ai/free/en/papers/li-2023-towards-general-text-embeddings-multi/). Preprint, arXiv:2308.03281.
26. Tianyu Gao, Xingcheng Yao, and Danqi Chen. 2022. [Simcse: Simple contrastive learning of sentence embeddings](https://md.research.elamur.ai/free/en/papers/gao-2021-simcse-simple-contrastive-learning-sentence/). Preprint, arXiv:2104.08821.
27. Naman Goyal, Cynthia Gao, Vishrav Chaudhary, Peng-Jen Chen, Guillaume Wenzek, Da Ju, Sanjana Krishnan, Marc’Aurelio Ranzato, Francisco Guzmán, and Angela Fan. 2022. The Flores-101 evaluation benchmark for low-resource and multilingual machine translation. Transactions of the Association for Computat… [find](https://scholar.google.com/scholar?q=Naman+Goyal%2C+Cynthia+Gao%2C+Vishrav+Chaudhary%2C+Peng%2DJen+Chen%2C+Guillaume+Wenzek%2C+Da+Ju%2C+Sanjana+Krishnan%2C+Marc%E2%80%99Aurelio+Ranzato%2C+Francisco+Guzm%C3%A1n%2C+and+Angela+Fan%2E+2022%2E+The+Flores%2D101+evaluation+benchmark)
28. Zehan Li, Xin Zhang, Yanzhao Zhang, Dingkun Long, Pengjun Xie, and Meishan Zhang. 2023b. [Towards general text embeddings with multi-stage contrastive learning](https://md.research.elamur.ai/free/en/papers/li-2023-towards-general-text-embeddings-multi/). arXiv preprint arXiv:2308.03281.
29. Andani Madodonga, Vukosi Marivate, and Matthew Adendorff. 2023. Izindaba-tindzaba: Machine learning news categorisation for long and short text for isizulu and siswati. Preprint, arXiv:2306.07426. — [arXiv:2306.07426](https://arxiv.org/abs/2306.07426) [find](https://scholar.google.com/scholar?q=Andani+Madodonga%2C+Vukosi+Marivate%2C+and+Matthew+Adendorff%2E+2023%2E+Izindaba%2Dtindzaba%3A+Machine+learning+news+categorisation+for+long+and+short+text+for+isizulu+and+siswati%2E+Preprint%2C+arXiv%3A2306%2E07426%2E+%E2%80%94+%5B)
30. Marc Marone, Orion Weller, William Fleshman, Eugene Yang, Dawn Lawrie, and Benjamin Van Durme. 2025. [mmbert: A modern multilingual encoder with annealed language learning](https://md.research.elamur.ai/free/en/papers/marone-2025-mmbert-modern-multilingual-encoder-annealed/). Preprint, arXiv:2509.06888.
31. Rui Meng, Ye Liu, Shafiq Rayhan Joty, Caiming Xiong, Yingbo Zhou, and Semih Yavuz. 2024. Sfr-embedding-mistral:enhance text retrieval with transfer learning. Salesforce AI Research Blog. [find](https://scholar.google.com/scholar?q=Rui+Meng%2C+Ye+Liu%2C+Shafiq+Rayhan+Joty%2C+Caiming+Xiong%2C+Yingbo+Zhou%2C+and+Semih+Yavuz%2E+2024%2E+Sfr%2Dembedding%2Dmistral%3Aenhance+text+retrieval+with+transfer+learning%2E+Salesforce+AI+Research+Blog%2E)
32. Niklas Muennighoff, Hongjin Su, Liang Wang, Nan Yang, Furu Wei, Tao Yu, Amanpreet Singh, and Douwe Kiela. 2024. [Generative representational instruction tuning](https://md.research.elamur.ai/free/en/papers/muennighoff-2024-generative-representational-instruction-tuning/). Preprint, arXiv:2402.09906.
33. Niklas Muennighoff, Nouamane Tazi, Loic Magne, and Nils Reimers. 2023. [MTEB: Massive text embedding benchmark](https://md.research.elamur.ai/free/en/papers/muennighoff-2022-mteb-massive-text-embedding-benchmark/). In Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics, pages 2014–2037, Dubrovnik, Croatia. Association for Computational Linguistic…
34. Shamsuddeen Hassan Muhammad, Idris Abdulmumin, Abinew Ali Ayele, David Ifeoluwa Adelani, Ibrahim Said Ahmad, Saminu Mohammad Aliyu, Nelson Odhiambo Onyango, Lilian D. A. Wanzare, Samuel Rutunda, Lukman Jibril Aliyu, Esubalew Alemneh, Oumaima Hourrane, Hagos Tesfahun Gebremichael, Elyas Abdi Ismail,… [find](https://scholar.google.com/scholar?q=Shamsuddeen+Hassan+Muhammad%2C+Idris+Abdulmumin%2C+Abinew+Ali+Ayele%2C+David+Ifeoluwa+Adelani%2C+Ibrahim+Said+Ahmad%2C+Saminu+Mohammad+Aliyu%2C+Nelson+Odhiambo+Onyango%2C+Lilian+D%2E+A%2E+Wanzare%2C+Samuel+Rutunda%2C+Lukma)
35. Shamsuddeen Hassan Muhammad, Idris Abdulmumin, Abinew Ali Ayele, Nedjma Ousidhoum, David Ifeoluwa Adelani, Seid Muhie Yimam, Ibrahim Sa’id Ahmad, Meriem Beloucif, Saif M. Mohammad, Sebastian Ruder, Oumaima Hourrane, Pavel Brazdil, Felermino Dário Mário António Ali, Davis David, Salomey Osei, Bello S… — [AfriSenti: A Twitter Sentiment Analysis Benchmark for African Languages](https://md.research.elamur.ai/free/en/papers/muhammad-2023-afrisenti-twitter-sentiment-analysis-benchmark/)
36. Shamsuddeen Hassan Muhammad, David Ifeoluwa Adelani, Sebastian Ruder, Ibrahim Said Ahmad, Idris Abdulmumin, Bello Shehu Bello, Monojit Choudhury, Chris Chinenye Emezue, Saheed Salahudeen Abdullahi, Anuoluwapo Aremu, Alipio Jeorge, and Pavel Brazdil. 2022. [Naijasenti: A nigerian twitter sentiment cor…](https://md.research.elamur.ai/free/en/papers/muhammad-2022-naijasenti-nigerian-twitter-sentiment-corpus/)
37. Alexander Panchenko, and 2 others. 2025c. Semeval-2025 task 11: Bridging the gap in text-based emotion detection. Preprint, arXiv:2503.07269. [find](https://scholar.google.com/scholar?q=Alexander+Panchenko%2C+and+2+others%2E+2025c%2E+Semeval%2D2025+task+11%3A+Bridging+the+gap+in+text%2Dbased+emotion+detection%2E+Preprint%2C+arXiv%3A2503%2E07269%2E)
38. Arvind Neelakantan, Tao Xu, Raul Puri, Alec Radford, Jesse Michael Han, Jerry Tworek, Qiming Yuan, Nikolas Tezak, Jong Wook Kim, Chris Hallacy, Johannes Heidecke, Pranav Shyam, Boris Power, Tyna Eloundou Nekoul, Girish Sastry, Gretchen Krueger, David Schnurr, Felipe Petroski Such, Kenny Hsu, and 6 o… — [Text and Code Embeddings by Contrastive Pre-Training](https://md.research.elamur.ai/free/en/papers/neelakantan-2022-text-code-embeddings-contrastive-pre/)
39. Rubungo Andre Niyongabo, Hong Qu, Julia Kreutzer, and Li Huang. 2020. Kinnews and kirnews: Benchmarking cross-lingual text classification for kinyarwanda and kirundi. Preprint, arXiv:2010.12174. [find](https://scholar.google.com/scholar?q=Rubungo+Andre+Niyongabo%2C+Hong+Qu%2C+Julia+Kreutzer%2C+and+Li+Huang%2E+2020%2E+Kinnews+and+kirnews%3A+Benchmarking+cross%2Dlingual+text+classification+for+kinyarwanda+and+kirundi%2E+Preprint%2C+arXiv%3A2010%2E12174%2E)
40. NLLB-Team, Marta R. Costa-jussà, James Cross, Onur Çelebi, Maha Elbayad, Kenneth Heafield, Kevin Heffernan, Elahe Kalbassi, Janice Lam, Daniel Licht, Jean Maillard, Anna Sun, Skyler Wang, Guillaume Wenzek, Al Youngblood, Bapi Akula, Loic Barrault, Gabriel Mejia Gonzalez, Prangthip Hansanti, and 20 o… [find](https://scholar.google.com/scholar?q=NLLB%2DTeam%2C+Marta+R%2E+Costa%2Djuss%C3%A0%2C+James+Cross%2C+Onur+%C3%87elebi%2C+Maha+Elbayad%2C+Kenneth+Heafield%2C+Kevin+Heffernan%2C+Elahe+Kalbassi%2C+Janice+Lam%2C+Daniel+Licht%2C+Jean+Maillard%2C+Anna+Sun%2C+Skyler+Wang%2C+Guillaume+We)
41. Jessica Ojo, Odunayo Ogundepo, Akintunde Oladipo, Kelechi Ogueji, Jimmy Lin, Pontus Stenetorp, and David Ifeoluwa Adelani. 2025. Afrobench: How good are large language models on african languages? Preprint, arXiv:2311.07978. [find](https://scholar.google.com/scholar?q=Jessica+Ojo%2C+Odunayo+Ogundepo%2C+Akintunde+Oladipo%2C+Kelechi+Ogueji%2C+Jimmy+Lin%2C+Pontus+Stenetorp%2C+and+David+Ifeoluwa+Adelani%2E+2025%2E+Afrobench%3A+How+good+are+large+language+models+on+african+languages%3F+Pre)
42. Nedjma Ousidhoum, Shamsuddeen Hassan Muhammad, Mohamed Abdalla, Idris Abdulmumin, Ibrahim Said Ahmad, Sanchit Ahuja, Alham Fikri Aji, Vladimir Araujo, Abinew Ali Ayele, Pavan Baswani, Meriem Beloucif, Chris Biemann, Sofia Bourhim, Christine De Kock, Genet Shanko Dekebo, Oumaima Hourrane, Gopichand K… [find](https://scholar.google.com/scholar?q=Nedjma+Ousidhoum%2C+Shamsuddeen+Hassan+Muhammad%2C+Mohamed+Abdalla%2C+Idris+Abdulmumin%2C+Ibrahim+Said+Ahmad%2C+Sanchit+Ahuja%2C+Alham+Fikri+Aji%2C+Vladimir+Araujo%2C+Abinew+Ali+Ayele%2C+Pavan+Baswani%2C+Meriem+Beloucif%2C)
43. Shamsuddeen Hassan Muhammad, Nedjma Ousidhoum, Idris Abdulmumin, Jan Philip Wahle, Terry Ruas, Meriem Beloucif, Christine de Kock, Nirmal Surange, Daniela Teodorescu, Ibrahim Said Ahmad, David Ifeoluwa Adelani, Alham Fikri Aji, Felermino D. M. A. Ali, Ilseyar Alimova, Vladimir Araujo, Nikolay Babako… [find](https://scholar.google.com/scholar?q=Shamsuddeen+Hassan+Muhammad%2C+Nedjma+Ousidhoum%2C+Idris+Abdulmumin%2C+Jan+Philip+Wahle%2C+Terry+Ruas%2C+Meriem+Beloucif%2C+Christine+de+Kock%2C+Nirmal+Surange%2C+Daniela+Teodorescu%2C+Ibrahim+Said+Ahmad%2C+David+Ifeoluw)
44. Loc Pham, Tung Luu, Thu Vo, Minh Nguyen, and Viet Hoang. 2025. [Vn-mteb: Vietnamese massive text embedding benchmark](https://md.research.elamur.ai/free/en/papers/pham-2026-vn-mteb-vietnamese-massive-text/). Preprint, arXiv:2507.21500.
45. Shamsuddeen Hassan Muhammad, Nedjma Ousidhoum, Idris Abdulmumin, Seid Muhie Yimam, Jan Philip Wahle, Terry Ruas, Meriem Beloucif, Christine De Kock, Tadesse Destaw Belay, Ibrahim Said Ahmad, Nirmal Surange, Daniela Teodorescu, David Ifeoluwa Adelani, Alham Fikri Aji, Felermino Ali, Vladimir Araujo,… — [PL-MTEB: Polish Massive Text Embedding Benchmark](https://md.research.elamur.ai/free/en/papers/poswiata-2026-pl-mteb-polish-massive-text/)
46. Wuttikorn Ponwitayarat, Raymond Ng, Jann Railey Montalan, Thura Aung, Jian Gang Ngui, Yosephine Susanto, William Tjhi, Panuthep Tasawong, Erik Cambria, Ekapol Chuangsuwanich, Sarana Nutanong, and Peerat Limkonchotiwat. 2025. Sea-bed: Southeast asia embedding benchmark. Preprint, arXiv:2508.12243. — [arXiv:2508.12243](https://arxiv.org/abs/2508.12243) [find](https://scholar.google.com/scholar?q=Wuttikorn+Ponwitayarat%2C+Raymond+Ng%2C+Jann+Railey+Montalan%2C+Thura+Aung%2C+Jian+Gang+Ngui%2C+Yosephine+Susanto%2C+William+Tjhi%2C+Panuthep+Tasawong%2C+Erik+Cambria%2C+Ekapol+Chuangsuwanich%2C+Sarana+Nutanong%2C+and+Peer)
47. Ricardo Rei, Craig Stewart, Ana C Farinha, and Alon Lavie. 2020. [COMET: A neural framework for MT evaluation](https://md.research.elamur.ai/free/en/papers/rei-2020-comet-neural-framework-mt-evaluation/). In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 2685–2702, Online. Association for Computational Linguistics.
48. Nils Reimers and Iryna Gurevych. 2019. [Sentence-bert: Sentence embeddings using siamese bert-networks](https://md.research.elamur.ai/free/en/papers/reimers-2019-sentence-bert-sentence-embeddings-siamese/). Preprint, arXiv:1908.10084.
49. Iyanuoluwa Shode, David Ifeoluwa Adelani, Jing Peng, and Anna Feldman. 2023. Nollysenti: Leveraging transfer learning and machine translation for nigerian movie sentiment classification. Preprint, arXiv:2305.10971. [find](https://scholar.google.com/scholar?q=Iyanuoluwa+Shode%2C+David+Ifeoluwa+Adelani%2C+Jing+Peng%2C+and+Anna+Feldman%2E+2023%2E+Nollysenti%3A+Leveraging+transfer+learning+and+machine+translation+for+nigerian+movie+sentiment+classification%2E+Preprint%2C+arX)
50. Artem Snegirev, Maria Tikhonova, Anna Maksimova, Alena Fenogenova, and Alexander Abramov. 2025. [The russian-focused embedders’ exploration: rumteb benchmark and russian embedding model design](https://md.research.elamur.ai/free/en/papers/snegirev-2025-russian-focused-embedders-exploration-rumteb/). Preprint, arXiv:2408.12503.
51. Ashish V. Thapliyal, Jordi Pont-Tuset, Xi Chen, and Radu Soricut. 2022. [Crossmodal-3600: A massively multilingual multimodal evaluation dataset](https://md.research.elamur.ai/free/en/papers/thapliyal-2022-crossmodal-3600-massively-multilingual-multimodal/). Preprint, arXiv:2205.12522.
52. Jörg Tiedemann. 2020. [The tatoeba translation challenge – realistic data sets for low resource and multilingual mt](https://md.research.elamur.ai/free/en/papers/tiedemann-2020-tatoeba-translation-challenge-realistic-data/). Preprint, arXiv:2010.06354.
53. Henrique Schechter Vera, Sahil Dua, Biao Zhang, Daniel Salz, and et. al. 2025. Embeddinggemma: Powerful and lightweight text representations. Preprint, arXiv:2509.20354. — [arXiv:2509.20354](https://arxiv.org/abs/2509.20354) [find](https://scholar.google.com/scholar?q=Henrique+Schechter+Vera%2C+Sahil+Dua%2C+Biao+Zhang%2C+Daniel+Salz%2C+and+et%2E+al%2E+2025%2E+Embeddinggemma%3A+Powerful+and+lightweight+text+representations%2E+Preprint%2C+arXiv%3A2509%2E20354%2E+%E2%80%94+%5BarXiv%3A2509%2E20354%5D%28https%3A%2F%2Fa)
54. Adina Williams, Nikita Nangia, and Samuel R. Bowman. 2018. [A broad-coverage challenge corpus for sentence understanding through inference](https://md.research.elamur.ai/free/en/papers/williams-2018-broad-coverage-challenge-corpus-sentence/). Preprint, arXiv:1704.05426.
55. Shitao Xiao, Zheng Liu, Peitian Zhang, Niklas Muennighoff, Defu Lian, and Jian-Yun Nie. 2024. [C-pack: Packed resources for general chinese embeddings](https://md.research.elamur.ai/free/en/papers/xiao-2023-c-pack-packed-resources/). Preprint, arXiv:2309.07597.
56. Hao Yu, Jesujoba O. Alabi, Andiswa Bukula, Jian Yun Zhuang, En-Shiun Annie Lee, Tadesse Kebede Guge, Israel Abebe Azime, Happy Buzaaba, Blessing Kudzaishe Sibanda, Godson K. Kalipe, Jonathan Mukiibi, Salomon Kabongo Kabenamualu, Mmasibidi Setaka, Lolwethu Ndolela, Nkiruka Odu, Rooweither Mabuya, Sha… [find](https://scholar.google.com/scholar?q=Hao+Yu%2C+Jesujoba+O%2E+Alabi%2C+Andiswa+Bukula%2C+Jian+Yun+Zhuang%2C+En%2DShiun+Annie+Lee%2C+Tadesse+Kebede+Guge%2C+Israel+Abebe+Azime%2C+Happy+Buzaaba%2C+Blessing+Kudzaishe+Sibanda%2C+Godson+K%2E+Kalipe%2C+Jonathan+Mukiibi%2C+)
57. Xinyu Zhang, Xueguang Ma, Peng Shi, and Jimmy Lin. 2021. Mr. tydi: A multi-lingual benchmark for dense retrieval. Preprint, arXiv:2108.08787. [find](https://scholar.google.com/scholar?q=Xinyu+Zhang%2C+Xueguang+Ma%2C+Peng+Shi%2C+and+Jimmy+Lin%2E+2021%2E+Mr%2E+tydi%3A+A+multi%2Dlingual+benchmark+for+dense+retrieval%2E+Preprint%2C+arXiv%3A2108%2E08787%2E)
58. Xinyu Zhang, Nandan Thakur, Odunayo Ogundepo, Ehsan Kamalloo, David Alfonso-Hermelo, Xiaoguang Li, Qun Liu, Mehdi Rezagholizadeh, and Jimmy Lin. 2023. [Miracl: A multilingual retrieval dataset covering 18 diverse languages](https://md.research.elamur.ai/free/en/papers/zhang-2023-miracl-multilingual-retrieval-dataset-covering/). Transactions of the Association for Computational Linguistics, 11:1114–1131.
59. Liang Wang, Nan Yang, Xiaolong Huang, Binxing Jiao, Linjun Yang, Daxin Jiang, Rangan Majumder, and Furu Wei. 2024a. [Text embeddings by weakly-supervised contrastive pre-training](https://md.research.elamur.ai/free/en/papers/wang-2022-text-embeddings-weakly-supervised-contrastive/). Preprint, arXiv:2212.03533.
60. Yanzhao Zhang, Mingxin Li, Dingkun Long, Xin Zhang, Huan Lin, Baosong Yang, Pengjun Xie, An Yang, Dayiheng Liu, Junyang Lin, Fei Huang, and Jingren Zhou. 2025. [Qwen3 embedding: Advancing text embedding and reranking through foundation models](https://md.research.elamur.ai/free/en/papers/zhang-2025-qwen3-embedding-advancing-text/). Preprint, arXiv:2506.05176.
61. Liang Wang, Nan Yang, Xiaolong Huang, Linjun Yang, Rangan Majumder, and Furu Wei. 2024b. [Improving text embeddings with large language models](https://md.research.elamur.ai/free/en/papers/wang-2024-improving-text-embeddings-large-language/). Preprint, arXiv:2401.00368.
62. Erfan Zinvandi, Morteza Alikhani, Mehran Sarmadi, Zahra Pourbahman, Sepehr Arvin, Reza Kazemi, and Arash Amini. 2025. Famteb: Massive text embedding benchmark in persian language. Preprint, arXiv:2502.11571. [find](https://scholar.google.com/scholar?q=Erfan+Zinvandi%2C+Morteza+Alikhani%2C+Mehran+Sarmadi%2C+Zahra+Pourbahman%2C+Sepehr+Arvin%2C+Reza+Kazemi%2C+and+Arash+Amini%2E+2025%2E+Famteb%3A+Massive+text+embedding+benchmark+in+persian+language%2E+Preprint%2C+arXiv%3A2502)
63. Liang Wang, Nan Yang, Xiaolong Huang, Linjun Yang, Rangan Majumder, and Furu Wei. 2024c. [Multilingual e5 text embeddings: A technical report](https://md.research.elamur.ai/free/en/papers/wang-2024-multilingual-e5-text-embeddings-technical/). Preprint, arXiv:2402.05672.
64. Benjamin Warner, Antoine Chaffin, Benjamin Clavié, Orion Weller, Oskar Hallström, Said Taghadouini, Alexis Gallagher, Raja Biswas, Faisal Ladhak, Tom Aarsen, Griffin Thomas Adams, Jeremy Howard, and Iacopo Poli. 2025. [Smarter, better, faster, longer: A modern bidirectional encoder for fast, memory e…](https://md.research.elamur.ai/free/en/papers/warner-2024-smarter-better-faster-longer-modern/)
65. Silvan Wehrli, Bert Arnrich, and Christopher Irrgang. 2024. German text embedding clustering benchmark. Preprint, arXiv:2401.02709. [find](https://scholar.google.com/scholar?q=Silvan+Wehrli%2C+Bert+Arnrich%2C+and+Christopher+Irrgang%2E+2024%2E+German+text+embedding+clustering+benchmark%2E+Preprint%2C+arXiv%3A2401%2E02709%2E)

*Updated 31 October 2026*