---
title: "Enhanced Arabic Text Retrieval with Attentive Relevance Scoring — Bekhouche 2025"
description: "They propose Adaptive Passage Retrieval (APR), a dual-encoder system initialised from the compact Arabic MiniBERT. 27 references, cited by 2 papers here."
authors: ["Salah Eddine Bekhouche", "Azeddine Benlamoudi", "Yazid Bounab", "Fadi Dornaika", "Abdenour Hadid"]
year: 2025
venue: "arXiv:2507.23404"
source: "https://arxiv.org/abs/2507.23404"
doi: "10.48550/arxiv.2507.23404"
arxiv: "2507.23404"
licence: "CC BY 4.0 (arXiv)"
licence_url: "http://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/bekhouche-2025-enhanced-arabic-text-retrieval-attentive/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/bekhouche-2025-enhanced-arabic-text-retrieval-attentive/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2025](https://md.research.elamur.ai/free/en/papers/2025/page/2/) / Enhanced Arabic Text Retrieval with Attentive Relevance Scoring

# Enhanced Arabic Text Retrieval with Attentive Relevance Scoring

**Authors:** Salah Eddine Bekhouche, Azeddine Benlamoudi, Yazid Bounab, Fadi Dornaika and [Abdenour Hadid](https://md.research.elamur.ai/free/en/authors/abdenour-hadid/) · **Year:** 2025 · **Venue:** arXiv:2507.23404 · **Licence:** [CC BY 4.0 (arXiv)](http://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://arxiv.org/abs/2507.23404) · [PDF](https://arxiv.org/pdf/2507.23404) · [DOI 10.48550/arxiv.2507.23404](https://doi.org/10.48550/arxiv.2507.23404) · [arXiv 2507.23404](https://arxiv.org/abs/2507.23404) · [OpenAlex](https://openalex.org/W4415524526)

**Citations:** 0 citations (OpenAlex, 2026-10-14) · 2 in our library, of the 140,884 papers whose bibliographies we read

## Summary
Arabic is hard for retrieval systems. Its morphology is rich, diacritics are optional, and Modern Standard Arabic sits alongside many dialects. Dedicated dense retrievers for the language are also scarce. Prior work such as AraDPR showed that contrastively trained Arabic retrievers outperform both sparse methods and multilingual models. However, AraDPR and similar systems still rank passages by a plain dot product or cosine between two vectors. The authors argue that this final step is too crude for Arabic. They propose Adaptive Passage Retrieval (APR), a dual-encoder system initialised from the compact Arabic MiniBERT. APR replaces the fixed similarity with a small trainable module called Attentive Relevance Scoring (ARS). ARS projects the question and passage embeddings into a shared space, multiplies them element by element, applies tanh, and turns the result into a single sigmoid score through a learned attention vector. Passage vectors can still be computed ahead of time. The authors report clearly better ranking accuracy on Arabic question answering, although the result tables were not in the excerpt we received. They release their code publicly.

## Key points
- APR is a DPR-style dual encoder. The separate question encoder and passage encoder are both initialised from MiniBERT, an Arabic-pretrained model from Safaya et al.
- The global representation of each text is its final-layer [CLS] vector, ℓ2-normalised onto the unit hypersphere to stabilise contrastive similarity.
- ARS computes a score in four steps:
  - It projects the query and passage with matrices W_q and W_p (h×d).
  - It takes their element-wise product.
  - It applies tanh.
  - It combines the result with a learned vector w_a and passes it through a sigmoid, giving a score between 0 and 1.
- At inference time, embeddings for all passages are precomputed. Only the query is encoded live, and ARS then scores the query against each stored passage vector.
- The main reference point is AraDPR, which uses an AraBERT backbone and was trained on translated plus native Arabic QA data such as ARCD. Exact result numbers were not available in the provided text.

## Interesting ideas
- Keep both sides encoded independently, and swap the fixed dot product for a tiny learned interaction head. Retrieval stays cacheable, but relevance is no longer limited to raw geometric closeness.
- Bound the relevance score to (0,1) with a sigmoid instead of leaving it as an unbounded similarity. This gives an absolute value that can be thresholded, not just a ranking.
- Put the modelling capacity into a cheap scoring layer on top of a lightweight encoder, rather than scaling up the backbone. This suits low-resource or CPU settings.
- Normalise vectors to the unit sphere before the learned interaction. This keeps contrastive training stable, because magnitude cannot dominate the score.

## For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

- **I1** (answer): A QA reader could rank candidate story sentences for a question with such a head over frame-feature vectors. Because the features are named slots rather than opaque dimensions, each component of the element-wise product can be shown as an explanation.
- **I2** (answer): The fable and school-test readers could abstain or say "not stated in the text" when the best supporting fact falls below a tuned threshold, instead of always returning the top candidate.
- **I3** (frames): Coreference could store one vector per entity and score each new mention with a small projected-product-sigmoid head. Antecedent search would stay fast, and the per-feature contributions could be logged.
- **I4** (learn): When training any pairwise matcher, such as aligning silver frames between witness parsers, normalising inputs first could make agreement scores comparable across sentences and parsers.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

Fields: [Linguistics & languages](https://md.research.elamur.ai/free/en/shelves/linguistics/)

Topics: [🐪 Semitic languages](https://md.research.elamur.ai/free/en/topics/semitic-languages/) · [🔍 Information retrieval & search](https://md.research.elamur.ai/free/en/topics/information-retrieval/)

## Institutions

- [University of the Basque Country](https://md.research.elamur.ai/free/en/universities/university-of-the-basque-country/) — University, Leioa, Spain · 34 works · 14,069 citations
- [University of Helsinki](https://md.research.elamur.ai/free/en/universities/university-of-helsinki/) — University, Helsinki, Finland · 24 works · 3,902 citations

Also on the byline: [University of Ouargla](https://ror.org/05amrd548).

## Cite

Cite as: Salah Eddine Bekhouche, Azeddine Benlamoudi, Yazid Bounab, Fadi Dornaika and Abdenour Hadid. 2025. Enhanced Arabic Text Retrieval with Attentive Relevance Scoring. arXiv:2507.23404.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@misc{bekhouche2025enhanced,
  title = {{Enhanced Arabic Text Retrieval with Attentive Relevance Scoring}},
  author = {Salah Eddine Bekhouche, Azeddine Benlamoudi, Yazid Bounab, Fadi Dornaika and Abdenour Hadid},
  year = {2025},
  doi = {10.48550/arxiv.2507.23404},
  eprint = {2507.23404},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2507.23404}
}
```

- [← previous Detecting Hope, Hate, and Emotion in Arabic Textual Speech and Multi-modal Memes Using Large Language Models](https://md.research.elamur.ai/free/en/papers/aldahoul-2025-detecting-hope-hate-emotion-arabic/)
- [next → EvaCun 2025 Shared Task: Lemmatization and Token Prediction in Akkadian and Sumerian using LLMs](https://md.research.elamur.ai/free/en/papers/gordin-2025-evacun-2025-shared-task-lemmatization/)
- [↑ back to papers of 2025](https://md.research.elamur.ai/free/en/papers/2025/page/2/)

## References

27 references, 15 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Vladimir Karpukhin, Barlas Oğuz, Sewon Min, Patrick Lewis, Ledell Wu, Sergey Edunov, Danqi Chen, and Wen-tau Yih, “Dense passage retrieval for open-domain question answering,” in Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP). 2020, pp. 6769–6781, Association for Computational Linguistics. [our card](https://md.research.elamur.ai/free/en/papers/karpukhin-2020-dense-passage-retrieval-open-domain/)
2. Abdelrahman Abdallah, Mahmoud Kasem, Mahmoud Abdalla, Mohamed Mahmoud, Mohamed Elkasaby, Yasser Elbendary, and Adam Jatowt, “Arabicaqa: A comprehensive dataset for arabic question answering,” in Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval, 2024, pp. 2049–2059. [our card](https://md.research.elamur.ai/free/en/papers/abdallah-2024-arabicaqa-comprehensive-dataset-arabic-question/)
3. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova, “Bert: Pre-training of deep bidirectional transformers for language understanding,” in Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers). 2019, pp. 4171–4186, Association for Computational Linguistics. [our card](https://md.research.elamur.ai/free/en/papers/devlin-2019-bert-pre-training-deep-bidirectional/)
4. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin, “Attention is all you need,” Advances in neural information processing systems, vol. 30, 2017. [our card](https://md.research.elamur.ai/free/en/papers/vaswani-2017-attention-all-need/)
5. Jeff Johnson, Matthijs Douze, and Hervé Jégou, “Billion-scale similarity search with gpus,” IEEE Transactions on Big Data, vol. 7, no. 3, pp. 535–547, 2019. [our card](https://md.research.elamur.ai/free/en/papers/johnson-2017-billion-scale-similarity-search-gpus/)
6. Stephen Robertson, Hugo Zaragoza, et al., “The probabilistic relevance framework: Bm25 and beyond,” Foundations and Trends® in Information Retrieval, vol. 3, no. 4, pp. 333–389, 2009. [find](https://scholar.google.com/scholar?q=Stephen+Robertson%2C+Hugo+Zaragoza%2C+et+al%2E%2C+%E2%80%9CThe+probabilistic+relevance+framework%3A+Bm25+and+beyond%2C%E2%80%9D+Foundations+and+Trends%C2%AE+in+Information+Retrieval%2C+vol%2E+3%2C+no%2E+4%2C+pp%2E+333%E2%80%93389%2C+2009%2E)
7. Lee Xiong, Chenyan Xiong, Ye Li, Kwok-Fung Lau, Jimmy Lin, and Paul Bennett, “Approximate nearest neighbor negative contrastive learning for dense text retrieval,” 2020. [our card](https://md.research.elamur.ai/free/en/papers/xiong-2020-approximate-nearest-neighbor-negative-contrastive/)
8. Yingqi Qu, Yujing Ding, Jing Liu, Kai Liu, Ruiyang Ren, Wayne Xin Zhao, Daxiang Dong, Hua Wu, and Haifeng Wang, “RocketQA: An optimized training approach to dense passage retrieval for open-domain question answering,” in Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. 2021, pp. 5835–5847, Association for Computational Linguistics. [our card](https://md.research.elamur.ai/free/en/papers/qu-2021-rocketqa-optimized-training-approach-dense/)
9. Omar Khattab and Matei Zaharia, “Colbert: Efficient and effective passage search via contextualized late interaction over bert,” in Proceedings of the 43rd International ACM SIGIR conference on research and development in Information Retrieval, 2020, pp. 39–48. [our card](https://md.research.elamur.ai/free/en/papers/khattab-2020-colbert-efficient-effective-passage/)
10. Keshav Santhanam, Omar Khattab, Jon Saad-Falcon, Christopher Potts, and Matei Zaharia, “Colbertv2: Effective and efficient retrieval via lightweight late interaction,” arXiv preprint arXiv:2112.01488, 2021. [↗](https://arxiv.org/abs/2112.01488) [our card](https://md.research.elamur.ai/free/en/papers/santhanam-2022-colbertv2-effective-efficient-retrieval-lightweight/)
11. Thibault Formal, Benjamin Piwowarski, and Stéphane Clinchant, “Splade: Sparse lexical and expansion model for first stage ranking,” in Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval, 2021. [our card](https://md.research.elamur.ai/free/en/papers/formal-2021-splade-sparse-lexical-expansion/)
12. Victor Sanh, Lysandre Debut, Julien Chaumond, and Thomas Wolf, “Distilbert, a distilled version of bert: smaller, faster, cheaper and lighter,” arXiv preprint arXiv:1910.01108, 2019. [↗](https://arxiv.org/abs/1910.01108)
13. Xiaoqi Jiao, Yichun Yin, Lifeng Shang, Xin Jiang, Xiao Chen, Linlin Li, Fang Wang, and Qun Liu, “Tinybert: Distilling bert for natural language understanding,” arXiv preprint arXiv:1909.10351, 2019. [↗](https://arxiv.org/abs/1909.10351)
14. Ahmed Magdy Ezzeldin and Mohamed Shaheen, “A survey of arabic question answering: challenges, tasks, approaches, tools, and future trends,” in Proceedings of The 13th international Arab conference on information technology (ACIT 2012), 2012, pp. 1–8. [find](https://scholar.google.com/scholar?q=Ahmed+Magdy+Ezzeldin+and+Mohamed+Shaheen%2C+%E2%80%9CA+survey+of+arabic+question+answering%3A+challenges%2C+tasks%2C+approaches%2C+tools%2C+and+future+trends%2C%E2%80%9D+in+Proceedings+of+The+13th+international+Arab+conference+on+)
15. Wissam Antoun, Fady Baly, and Hazem Hajj, “Arabert: Transformer-based model for arabic language understanding,” arXiv preprint arXiv:2003.00104, 2020. [↗](https://arxiv.org/abs/2003.00104) [our card](https://md.research.elamur.ai/free/en/papers/antoun-2020-arabert-transformer-based-model-arabic/)
16. Muhammad Abdul-Mageed, AbdelRahim Elmadany, and El Moatez Billah Nagoudi, “Arbert & marbert: Deep bidirectional transformers for arabic,” arXiv preprint arXiv:2101.01785, 2020. [↗](https://arxiv.org/abs/2101.01785) [our card](https://md.research.elamur.ai/free/en/papers/abdul-mageed-2021-arbert-marbert-deep-bidirectional-transformers/)
17. Wissam Antoun, Fady Baly, and Hazem Hajj, “Araelectra: Pre-training text discriminators for arabic language understanding,” arXiv preprint arXiv:2012.15516, 2020. [↗](https://arxiv.org/abs/2012.15516)
18. Go Inoue, Bashar Alhafni, Nurpeiis Baimukan, Houda Bouamor, and Nizar Habash, “The interplay of variant, size, and task type in arabic pre-trained language models,” arXiv preprint arXiv:2103.06678, 2021. [↗](https://arxiv.org/abs/2103.06678) [our card](https://md.research.elamur.ai/free/en/papers/inoue-2021-interplay-variant-size-task-type/)
19. Kareem Darwish, Walid Magdy, et al., “Arabic information retrieval,” Foundations and Trends® in Information Retrieval, vol. 7, no. 4, pp. 239–342, 2014. [find](https://scholar.google.com/scholar?q=Kareem+Darwish%2C+Walid+Magdy%2C+et+al%2E%2C+%E2%80%9CArabic+information+retrieval%2C%E2%80%9D+Foundations+and+Trends%C2%AE+in+Information+Retrieval%2C+vol%2E+7%2C+no%2E+4%2C+pp%2E+239%E2%80%93342%2C+2014%2E)
20. Rasha Obeidat, Marwa Al-Harbi, Mahmoud Al-Ayyoub, and Luay Alawneh, “Arquad: An expert-annotated arabic machine reading comprehension dataset,” Cognitive Computation, vol. 16, no. 3, pp. 984–1003, 2024. [find](https://scholar.google.com/scholar?q=Rasha+Obeidat%2C+Marwa+Al%2DHarbi%2C+Mahmoud+Al%2DAyyoub%2C+and+Luay+Alawneh%2C+%E2%80%9CArquad%3A+An+expert%2Dannotated+arabic+machine+reading+comprehension+dataset%2C%E2%80%9D+Cognitive+Computation%2C+vol%2E+16%2C+no%2E+3%2C+pp%2E+984%E2%80%931003%2C+202)
21. Jonathan H Clark, Eunsol Choi, Michael Collins, Dan Garrette, Tom Kwiatkowski, Vitaly Nikolaev, and Jennimaria Palomaki, “Tydi qa: A benchmark for information-seeking question answering in ty pologically di verse languages,” Transactions of the Association for Computational Linguistics, vol. 8, pp. 454– 470, 2020. [find](https://scholar.google.com/scholar?q=Jonathan+H+Clark%2C+Eunsol+Choi%2C+Michael+Collins%2C+Dan+Garrette%2C+Tom+Kwiatkowski%2C+Vitaly+Nikolaev%2C+and+Jennimaria+Palomaki%2C+%E2%80%9CTydi+qa%3A+A+benchmark+for+information%2Dseeking+question+answering+in+ty+pologica)
22. Po-Sen Huang, Xiaodong He, Jianfeng Gao, Li Deng, Alex Acero, and Larry Heck, “Learning deep structured semantic models for web search using clickthrough data,” in Proceedings of the 22nd ACM international conference on Information & Knowledge Management, 2013, pp. 2333–2338. [find](https://scholar.google.com/scholar?q=Po%2DSen+Huang%2C+Xiaodong+He%2C+Jianfeng+Gao%2C+Li+Deng%2C+Alex+Acero%2C+and+Larry+Heck%2C+%E2%80%9CLearning+deep+structured+semantic+models+for+web+search+using+clickthrough+data%2C%E2%80%9D+in+Proceedings+of+the+22nd+ACM+internat)
23. Sparsh Gupta and Vitor R Carvalho, “Faq retrieval using attentive matching,” in Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval, 2019, pp. 929–932. [find](https://scholar.google.com/scholar?q=Sparsh+Gupta+and+Vitor+R+Carvalho%2C+%E2%80%9CFaq+retrieval+using+attentive+matching%2C%E2%80%9D+in+Proceedings+of+the+42nd+International+ACM+SIGIR+Conference+on+Research+and+Development+in+Information+Retrieval%2C+2019%2C+p)
24. Vladimir Karpukhin, Barlas Oguz, Sewon Min, Patrick SH Lewis, Ledell Wu, Sergey Edunov, Danqi Chen, and Wen-tau Yih, “Dense passage retrieval for open-domain question answering.,” in EMNLP (1), 2020, pp. 6769–6781. [our card](https://md.research.elamur.ai/free/en/papers/karpukhin-2020-dense-passage-retrieval-open-domain/)
25. Ali Safaya, Moutasem Abdullatif, and Deniz Yuret, “Kuisail at semeval-2020 task 12: Bert-cnn for offensive speech identification in social media,” arXiv preprint arXiv:2007.13184, 2020. [↗](https://arxiv.org/abs/2007.13184) [our card](https://md.research.elamur.ai/free/en/papers/safaya-2020-kuisail-semeval-2020-task-12/)
26. Ilya Loshchilov and Frank Hutter, “Decoupled weight decay regularization,” in International Conference on Learning Representations (ICLR), 2019. [find](https://scholar.google.com/scholar?q=Ilya+Loshchilov+and+Frank+Hutter%2C+%E2%80%9CDecoupled+weight+decay+regularization%2C%E2%80%9D+in+International+Conference+on+Learning+Representations+%28ICLR%29%2C+2019%2E)
27. Hussein Mozannar, Karl El Hajal, Elie Maamary, and Hazem Hajj, “Neural arabic question answering,” arXiv preprint arXiv:1906.05394, 2019. [↗](https://arxiv.org/abs/1906.05394)