---
title: "Qur'an QA 2022 — paper by Malhas et al. (2022)"
description: "Qur'an QA 2022 was the first shared task on extractive question answering over the Holy Qur'an. 31 references, cited by 32 papers here."
authors: ["Rana Malhas", "Watheq Mansour", "Tamer Elsayed"]
year: 2022
venue: "Proceedinsg of the 5th Workshop on Open-Source Arabic Corpora and Processing Tools with Shared Tasks on Qur'an QA and Fine-Grained Hate Speech Detection"
source: "https://aclanthology.org/2022.osact-1.9/"
licence: "CC BY-NC 4.0 (ELRA, LREC proceedings)"
licence_url: "https://creativecommons.org/licenses/by-nc/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/malhas-2022-qur-qa-2022-overview-first/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/malhas-2022-qur-qa-2022-overview-first/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2022](https://md.research.elamur.ai/free/en/papers/2022/page/5/) / Qur'an QA 2022: Overview of The First Shared Task on Question Answering over the Holy Qur'an

# Qur'an QA 2022: Overview of The First Shared Task on Question Answering over the Holy Qur'an

**Authors:** [Rana Malhas](https://md.research.elamur.ai/free/en/authors/rana-malhas/), [Watheq Mansour](https://md.research.elamur.ai/free/en/authors/watheq-mansour/) and [Tamer Elsayed](https://md.research.elamur.ai/free/en/authors/tamer-elsayed/) · **Year:** 2022 · **Venue:** [Proceedinsg of the 5th Workshop on Open-Source Arabic Corpora and Processing Tools with Shared Tasks on Qur'an QA and Fine-Grained Hate Speech Detection](https://md.research.elamur.ai/free/en/venues/osact/) ([Workshop on Open-Source Arabic Corpora and Processing Tools with Shared Tasks 2022](https://md.research.elamur.ai/free/en/venues/osact/2022/)) · **Publisher:** [European Language Resources Association (ELRA)](https://md.research.elamur.ai/free/en/publishers/european-language-resources-association-elra/) · **Licence:** [CC BY-NC 4.0 (ELRA, LREC proceedings)](https://creativecommons.org/licenses/by-nc/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2022.osact-1.9/) · [PDF](https://aclanthology.org/2022.osact-1.9.pdf) · [OpenAlex](https://openalex.org/W7042765526)

**Citations:** 0 citations (OpenAlex, 2026-10-14) · [32](https://md.research.elamur.ai/free/en/papers/malhas-2022-qur-qa-2022-overview-first/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
Qur'an QA 2022 was the first shared task on extractive question answering over the Holy Qur'an. It was organized to push Arabic machine reading comprehension (MRC) forward on a hard domain. Systems receive a passage of consecutive verses in Classical Arabic and a question in Modern Standard Arabic. They must return up to five ranked answer spans taken from that passage, and questions can be factoid or non-factoid.

The organizers released QRCD, the Qur'anic Reading Comprehension Dataset, which extends AyaTEC. They scored runs mainly with partial Reciprocal Rank (pRR), which gives credit for answers that overlap a gold span or appear below the top rank.

Thirteen teams took part in the final phase and submitted 30 runs. Every team built on pre-trained transformers, most often AraBERT and AraELECTRA. The top three teams fine-tuned AraELECTRA on large MSA MRC datasets before fine-tuning on QRCD. The task gives the field a common benchmark and shows how hard it is to transfer from MSA to Classical Arabic.

## Key points
- QRCD contains 1,093 question-passage pairs, which expand to 1,337 question-passage-answer triplets. The splits are 710 train, 109 dev and 274 test pairs (65/10/25%), distributed as JSONL. The text comes from Tanzil (simple-clean v1.0.2).
- The measures are pRR (primary), plus Exact Match and F1@1, which look only at the top answer. The naive baseline returns the whole passage as its answer.
- Thirty teams signed up and 13 teams from 21 institutes reached the final phase. They submitted 30 runs, and 10 teams wrote system papers.
- The second-place team, TCE, scored a pRR of 0.567 with an ensemble of AraBERTv0.2-Base, AraBERTv0.2-Large and ARBERT. Third place reached 0.559 by paraphrasing questions to augment the data, and GOF came fourth with 0.546.
- The Stars and eRock teams augmented training data with AQQAC, which yielded 0.528 and 0.308 respectively. Runs built on CAMeLBERT-CA or CAMeLBERT-Mix scored roughly 0.29–0.41.

## Interesting ideas
- pRR gives partial credit for answers that only partly match a gold span and discounts correct answers that appear lower in the list, so a near-miss scores better than a total miss.
- Returning the entire passage gives a trivial reference point on the leaderboard, so any system must beat "answer with everything" to show it adds value.
- External QA pairs were converted into the target format only when the answer could be extracted from the attached evidence, and a matching context passage was built for each one.
- Self-ensembling trains the same model with several random seeds and merges their predictions; it buys stable outputs at a small possible cost in accuracy.
- Answers written in older or literary language often share no words with the modern question, so a system has to locate evidence for the answer instead of matching the question's words.

## For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

- **I1** (learn): Mova's QA evaluation could report a graded, rank-aware score alongside exact match, which would show when the reader finds roughly the right evidence but cuts the span badly.
- **I2** (answer): Fable and tale QA could include a whole-passage baseline as a negative control, which would show that Mova's chosen answers are more precise than simply quoting the context.
- **I3** (learn): When reusing outside QA corpora as silver data, Mova could keep only the items whose answer it can locate in the text, which filters out noisy free-form answers.
- **I4** (tree): gparse could be run under several seeds and treated as extra witness parsers, so that disagreement among them flags unstable attachments.
- **I5** (world): Mova's world model could link question and passage through compiled knowledge (WordNet synonyms, FrameNet frames) instead of word overlap, which helps with archaic phrasing in tales.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Qatar University](https://md.research.elamur.ai/free/en/universities/qatar-university/) — University, Doha, Qatar · 53 works · 1,027 citations

📚 Shelves (5): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 18,614 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 17,965 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 17,450 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 17,296 works · [🗄️ Information Systems](https://md.research.elamur.ai/free/en/shelves/computer-science/information-systems/) · 1,730 works

🧭 Topics (3): [Semitic languages](https://md.research.elamur.ai/free/en/topics/semitic-languages/) · 3,066 works · [📏 Evaluation & benchmarks](https://md.research.elamur.ai/free/en/topics/evaluation-benchmarks/) · 1,341 works · [❓ Question answering](https://md.research.elamur.ai/free/en/topics/question-answering/) · 487 works

## Cite

Cite as: Rana Malhas, Watheq Mansour and Tamer Elsayed. 2022. Qur'an QA 2022: Overview of The First Shared Task on Question Answering over the Holy Qur'an. Proceedinsg of the 5th Workshop on Open-Source Arabic Corpora and Processing Tools with Shared Tasks on Qur'an QA and Fine-Grained Hate Speech Detection.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{malhas2022qur,
  title = {{Qur'an QA 2022: Overview of The First Shared Task on Question Answering over the Holy Qur'an}},
  author = {Rana Malhas, Watheq Mansour and Tamer Elsayed},
  booktitle = {Proceedinsg of the 5th Workshop on Open-Source Arabic Corpora and Processing Tools with Shared Tasks on Qur'an QA and Fine-Grained Hate Speech Detection},
  year = {2022},
  url = {https://aclanthology.org/2022.osact-1.9/}
}
```

- [← previous OpenStance: Real-world Zero-shot Stance Detection](https://md.research.elamur.ai/free/en/papers/xu-2022-openstance-real-world-zero-shot/)
- [next → RST Discourse Parsing with Second-Stage EDU-Level Pre-training](https://md.research.elamur.ai/free/en/papers/yu-2022-rst-discourse-parsing-second-stage/)
- [↑ back to papers of 2022](https://md.research.elamur.ai/free/en/papers/2022/page/5/)

## References

31 references, 23 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Abdul-Mageed, M., Elmadany, A., et al. (2021). [Arbert & marbert: Deep bidirectional transformers for arabic](https://md.research.elamur.ai/free/en/papers/abdul-mageed-2021-arbert-marbert-deep-bidirectional-transformers/). In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 7088–7105. [↗](https://doi.org/10.48550/arxiv.2101.01785)
2. Aftab, E. and Malik, M. K. (2022). [eRock at Qur’an QA 2022: Contemporary Deep Neural Networks for Qur’an based Reading Comprehension Question Answers](https://md.research.elamur.ai/free/en/papers/aftab-2022-erock-qur-qa-2022-contemporary/). In Proceedings of the 5th Workshop on Open-Source Arabic Corpora and Processing Tools (OS-ACT5) at the 13th Language Resources and Evaluation Conference (LREC 2022).
3. Ahmed, B. H., Saad, M. K., and Refaee, E. A. (2022). [QQATeam at Quran QA 2022: Fine-Tunning Arabic QA Models for Quran QA Task](https://md.research.elamur.ai/free/en/papers/ahmed-2022-qqateam-qur-qa-2022-fine/). In Proceedings of the 5th Workshop on Open-Source Arabic Corpora and Processing Tools (OSACT5) at the 13th Language Resources and Evaluation Conference (LREC 2022).
4. Al-Azami, M. (2020). The History of the Qur’anic Text from Revelation to Compilation: A Comparative Study with the Old and New Testaments, 2nd ed. Turath Publishing, UK. [find](https://scholar.google.com/scholar?q=Al%2DAzami%2C+M%2E+%282020%29%2E+The+History+of+the+Qur%E2%80%99anic+Text+from+Revelation+to+Compilation%3A+A+Comparative+Study+with+the+Old+and+New+Testaments%2C+2nd+ed%2E+Turath+Publishing%2C+UK%2E)
5. Alqahtani, M. and Atwell, E. (2018). Annotated corpus of Arabic al-quran question and answer. [find](https://scholar.google.com/scholar?q=Alqahtani%2C+M%2E+and+Atwell%2C+E%2E+%282018%29%2E+Annotated+corpus+of+Arabic+al%2Dquran+question+and+answer%2E)
6. Alsaleh, A., Althabiti, S., Alshammari, I., Alnefaie, S., Alowaidi, S., Alsaqer, A., Atwell, E., Altahhan, A., and Alsalka, M. A. (2022). [LK2022 at Qur’an QA 2022: Simple Transformers Model for Finding Answers to Questions from Qur’an](https://md.research.elamur.ai/free/en/papers/alsaleh-2022-lk2022-qur-qa-2022-simple/). In Proceedings of the 5th Workshop on Open-Source Arabic Corpora and Processing Tools (OSACT5) at the 13th Language Resources and Evaluation Conference (LREC 2022).
7. Antoun, W., Baly, F., and Hajj, H. (2020). [Arabert: Transformer-based model for arabic language understanding](https://md.research.elamur.ai/free/en/papers/antoun-2020-arabert-transformer-based-model-arabic/). In LREC 2020 Workshop Language Resources and Evaluation Conference 11–16 May 2020, page 9. [↗](https://doi.org/10.48550/arxiv.2003.00104)
8. Antoun, W., Baly, F., and Hajj, H. (2021). [Araelectra: Pre-training text discriminators for arabic language understanding](https://md.research.elamur.ai/free/en/papers/antoun-2021-araelectra-pre-training-text-discriminators/). In Proceedings of the Sixth Arabic Natural Language Processing Workshop, pages 191–195. Association for Computational Linguistics. [↗](https://doi.org/10.48550/arxiv.2012.15516)
9. Artetxe, M., Ruder, S., and Yogatama, D. (2020). [On the cross-lingual transferability of monolingual representations](https://md.research.elamur.ai/free/en/papers/artetxe-2020-cross-lingual-transferability-monolingual-representations/). In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 4623–4637. Association for Computational Linguistics, July.
10. Atwell, E., Brierley, C., Dukes, K., Sawalha, M., and Sharaf, A.-B. (2011). An artificial intelligence approach to arabic and islamic content on the internet. In Proceedings of NITS 3rd National Information Technology Symposium, pages 1–8. Leeds. [find](https://scholar.google.com/scholar?q=Atwell%2C+E%2E%2C+Brierley%2C+C%2E%2C+Dukes%2C+K%2E%2C+Sawalha%2C+M%2E%2C+and+Sharaf%2C+A%2E%2DB%2E+%282011%29%2E+An+artificial+intelligence+approach+to+arabic+and+islamic+content+on+the+internet%2E+In+Proceedings+of+NITS+3rd+National+Infor)
11. Bashir, M. H., M. Azmi, A., Nawaz, H., Zaghouani, W., Diab, M., Al-Fuqaha, A., and Qadir, J. (2021). Arabic natural language processing for qur’anic research: A systematic review. April. [↗](https://doi.org/10.1007/s10462-022-10313-2)
12. Chen, D., Fisch, A., Weston, J., and Bordes, A. (2017). [Reading wikipedia to answer open-domain questions](https://md.research.elamur.ai/free/en/papers/chen-2017-reading-wikipedia-answer-open-domain/). In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), page 1870–1879. Association for Computational Linguistics.
13. Chen, D. (2018). Neural Reading Comprehension and Beyond. Ph.D. thesis, Stanford University. [find](https://scholar.google.com/scholar?q=Chen%2C+D%2E+%282018%29%2E+Neural+Reading+Comprehension+and+Beyond%2E+Ph%2ED%2E+thesis%2C+Stanford+University%2E)
14. Clark, J. H., Choi, E., Collins, M., Garrette, D., Kwiatkowski, T., Nikolaev, V., and Palomaki, J. (2020). [Tydi qa: A benchmark for information-seeking question answering in typologically diverse languages](https://md.research.elamur.ai/free/en/papers/clark-2020-tydi-qa-benchmark-information-seeking/). Transactions of the Association for Computational Linguistics, 8:454–470. [↗](https://doi.org/10.48550/arxiv.2003.05002)
15. Devlin, J., Chang, M.-W., Lee, K., and Toutanova, K. (2019). [Bert: Pre-training of deep bidirectional transformers for language understanding](https://md.research.elamur.ai/free/en/papers/devlin-2019-bert-pre-training-deep-bidirectional/). In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), page 4171–4186. Association for Computational Linguistics, June.
16. ElKomy, M. and Sarhan, A. M. (2022). [TCE at Qur’an QA 2022: Arabic Language Question Answering Over Holy Qur’an Using a Post-Processed Ensemble of BERT-based Models](https://md.research.elamur.ai/free/en/papers/elkomy-2022-tce-qur-qa-2022-arabic/). In Proceedings of the 5th Workshop on Open-Source Arabic Corpora and Processing Tools (OSACT5) at the 13th Language Resources and Evaluation Conference (LREC 2022). [↗](https://doi.org/10.48550/arxiv.2206.01550)
17. Joshi, M., Choi, E., Weld, D., and Zettlemoyer, L. (2017). [Triviaqa: A large scale distantly supervised challenge dataset for reading comprehension](https://md.research.elamur.ai/free/en/papers/joshi-2017-triviaqa-large-scale-distantly-supervised/). In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), page 1601–1611. Association for Computational Linguistics.
18. Keleg, A. and Magdy, W. (2022). [Smash at qur’an qa 2022: Creating better faithful data splits for low-resourced question answering scenarios](https://md.research.elamur.ai/free/en/papers/keleg-2022-smash-qur-qa-2022-creating/). In Proceedings of the 5th Workshop on Open-Source Arabic Corpora and Processing Tools (OSACT5) at the 13th Language Resources and Evaluation Conference (LREC 2022).
19. Lewis, P., Oguz, B., Rinott, R., Riedel, S., and Schwenk, H. (2020). [MLQA: Evaluating cross-lingual extractive question answering](https://md.research.elamur.ai/free/en/papers/lewis-2019-mlqa-evaluating-cross-lingual-extractive/). In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 7315–7330. Association for Computational Linguistics, July.
20. Malhas, R. and Elsayed, T. (2020). AyaTEC: Building a Reusable Verse-based Test Collection for Arabic Question Answering on the Holy Qur’an. ACM Transactions on Asian and Low-Resource Language Information Processing, 19(6):1–21, Nov. [find](https://scholar.google.com/scholar?q=Malhas%2C+R%2E+and+Elsayed%2C+T%2E+%282020%29%2E+AyaTEC%3A+Building+a+Reusable+Verse%2Dbased+Test+Collection+for+Arabic+Question+Answering+on+the+Holy+Qur%E2%80%99an%2E+ACM+Transactions+on+Asian+and+Low%2DResource+Language+Informa)
21. Malhas, R. and Elsayed, T. (2022). Official Repository of Qur’an QA Shared Task. [https://gitlab.com/bigirqu/quranqa](https://gitlab.com/bigirqu/quranqa). [↗](https://gitlab.com/bigirqu/quranqa)
22. Mellah, Y., Touahri, I., Kaddari, Z., Haja, Z., Berrich, J., and Bouchentouf, T. (2022). [LARSA22 at Qur’an QA 2022: Text-to-Text Transformer for Finding Answers to Questions from Qur’an](https://md.research.elamur.ai/free/en/papers/mellah-2022-larsa22-qur-qa-2022-text/). In Proceedings of the 5th Workshop on Open-Source Arabic Corpora and Processing Tools (OSACT5) at the 13th Language Resources and Evaluation Conference (LREC 2022).
23. Mostafa, A. and Mohamed, O. (2022). [GOF at Qur’an QA 2022: Towards an Efficient Question Answering For The Holy Qu’ran In The Arabic Language Using Deep Learning-Based Approach](https://md.research.elamur.ai/free/en/papers/mostafa-2022-gof-qur-qa-2022-efficient/). In Proceedings of the 5th Workshop on Open-Source Arabic Corpora and Processing Tools (OSACT5) at the 13th Language Resources and Evaluation Conference (LREC 2022).
24. Mozannar, H., Maamary, E., El Hajal, K., and Hajj, H. (2019). [Neural arabic question answering](https://md.research.elamur.ai/free/en/papers/mozannar-2019-neural-arabic-question-answering/). In Proceedings of the Fourth Arabic Natural Language Processing Workshop, page 108–118. Association for Computational Linguistics, Aug.
25. Premasiri, D., Ranasinghe, T., Zaghouani, W., Mitkov, R., Berrich, J., and Bouchentouf, T. (2022). [DTW at Qur’an QA 2022: Utilising Transfer Learning with Transformers for Question Answering in a Low-resource Domain](https://md.research.elamur.ai/free/en/papers/premasiri-2022-dtw-qur-qa-2022-utilising/). In Proceedings of the 5th Workshop on Open-Source Arabic Corpora and Processing Tools (OSACT5) at the 13th Language Resources and Evaluation Conference (LREC 2022).
26. Rajpurkar, P., Zhang, J., Lopyrev, K., and Liang, P. (2016). [Squad: 100, 000+ questions for machine comprehension of text](https://md.research.elamur.ai/free/en/papers/rajpurkar-2016-squad-100-000-questions-machine/). In EMNLP. [↗](https://doi.org/10.48550/arxiv.1606.05250)
27. Safaya, A., Abdullatif, M., and Yuret, D. (2020). [Kuisail at semeval-2020 task 12: Bert-cnn for offensive speech identification in social media](https://md.research.elamur.ai/free/en/papers/safaya-2020-kuisail-semeval-2020-task-12/). In Proceedings of the Fourteenth Workshop on Semantic Evaluation, pages 2054–2059.
28. Sagi, O. and Rokach, L. (2018). Ensemble learning: A survey. Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery, 8(4):e1249. [↗](https://doi.org/10.1002/widm.1249)
29. Singh, N. (2022). [niksss at Qur’an QA 2022: A Heavily Optimized BERT Based Model for Answering Questions from the Holy Qu’ran](https://md.research.elamur.ai/free/en/papers/singh-2022-niksss-qur-qa-2022-heavily/). In Proceedings of the 5th Workshop on Open-Source Arabic Corpora and Processing Tools (OSACT5) at the 13th Language Resources and Evaluation Conference (LREC 2022).
30. Wasfey, A., Elrefai, E., Marwa, M., and Haq, N. (2022). [Stars at Qur’an QA 2022: Building Automatic Extractive Question Answering Systems for the Holy Qur’an with Transformer Models and Releasing a New Dataset](https://md.research.elamur.ai/free/en/papers/sleem-2022-stars-qur-qa-2022-building/). In Proceedings of the 5th Workshop on Open-Source Arabic Corpora and Processing Tools (OSACT5) at the 13th Language Resources and Evaluation Conference (LREC 2022).
31. Xue, L., Constant, N., Roberts, A., Kale, M., Al-Rfou, R., Siddhant, A., Barua, A., and Raffel, C. (2021). [mt5: A massively multilingual pre-trained text-to-text transformer](https://md.research.elamur.ai/free/en/papers/xue-2021-mt5-massively-multilingual-pre-trained/). In NAACL-HLT. [↗](https://doi.org/10.48550/arxiv.2010.11934)

*Updated 1 November 2026*