--- title: "Few-Shot Prompting for Extractive Quranic QA with Instruction-Tuned LLMs — Basem 2025" description: "This work addresses extractive question answering over the Qur'an, using Task B of the Qur'an QA 2023 Shared Task. 20 references, cited by 2 papers here." authors: ["Mohamed Basem", "Islam Oshallah", "Ali Hamdi", "Ammar Mohammed"] year: 2025 venue: "arXiv:2508.06103" source: "https://arxiv.org/abs/2508.06103" doi: "10.48550/arxiv.2508.06103" arxiv: "2508.06103" licence: "CC BY 4.0 (arXiv)" licence_url: "http://creativecommons.org/licenses/by/4.0/" summary_licence: "Elamur Cosmic License" canonical: "https://research.elamur.ai/free/en/papers/basem-2025-few-shot-prompting-extractive-quranic/" --- [Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/basem-2025-few-shot-prompting-extractive-quranic/) [home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2025](https://md.research.elamur.ai/free/en/papers/2025/page/2/) / Few-Shot Prompting for Extractive Quranic QA with Instruction-Tuned LLMs # Few-Shot Prompting for Extractive Quranic QA with Instruction-Tuned LLMs **Authors:** Mohamed Basem, Islam Oshallah, [Ali Hamdi](https://md.research.elamur.ai/free/en/authors/ali-hamdi/) and [Ammar Mohammed](https://md.research.elamur.ai/free/en/authors/ammar-mohammed/) · **Year:** 2025 · **Venue:** arXiv:2508.06103 · **Licence:** [CC BY 4.0 (arXiv)](http://creativecommons.org/licenses/by/4.0/) **Links:** [Publisher's page](https://arxiv.org/abs/2508.06103) · [PDF](https://arxiv.org/pdf/2508.06103) · [DOI 10.48550/arxiv.2508.06103](https://doi.org/10.48550/arxiv.2508.06103) · [arXiv 2508.06103](https://arxiv.org/abs/2508.06103) · [OpenAlex](https://openalex.org/W4414463779) **Citations:** 1 citations (OpenAlex, 2026-10-14) · 2 in our library, of the 140,884 papers whose bibliographies we read ## Summary This work addresses extractive question answering over the Qur'an, using Task B of the Qur'an QA 2023 Shared Task. Given a Quranic passage and a question in Modern Standard Arabic, a system must return one or more exact sub-spans of the passage, or nothing when no answer exists. Classical Arabic makes this hard: rich morphology, free word order, metaphor, missing diacritics, and themes that span several chapters. The authors compare two routes. The first fine-tunes Arabic encoders (AraBERT, AraELECTRA, CAMeLBERT), starting from TyDi QA checkpoints, on a merged corpus of QUQA, ARCD and QRCDv1.2. The second gives Gemini and DeepSeek three-shot prompts written in Arabic. Both routes feed a post-processing stage that aligns subwords, suppresses overlapping spans, and filters vague or redundant answers. Prompted Gemini scores best, at pAP@10 0.637, which is above the shared-task winner. This suggests that careful prompting with span-faithful cleanup can beat fine-tuning in a low-resource domain with dense meaning. ## Key points - The benchmark is QRCDv1.2 from the Qur'an QA 2023 Shared Task (Task B). Roughly 81% of its question–passage pairs have a single answer, about 14% have several, and about 5% have none. - Fine-tuned models were built with SimpleTransformers from TyDi QA checkpoints and trained on QUQA, ARCD and QRCDv1.2 (lr 2e-5, 10 epochs, batch size 16). The best was "bert-base-arabertv02-tydiqa" at pAP@10 0.503, which would place seventh in the standings. - Gemini with three-shot Arabic prompts reached pAP@10 0.637. Prior Task B entries scored lower: Tce 0.571 (ranked first), Al-Jawaab 0.539 (GPT-4 span extraction after dense retrieval), LKAU23 0.49, and GYM 0.461. - The few-shot demonstrations were taken from QRCDv1.2 and deliberately included single-answer, multi-answer and zero-answer cases. - Post-processing has four parts: subword alignment, non-maximum suppression of overlapping spans, semantic filtering, and re-ranking. The authors credit it with improving precision and reducing hallucinated answers. ## Interesting ideas - An LLM's free-text output is accepted only once it can be mapped back to an exact stretch of the source passage, which turns a generative model into a strictly extractive one and cuts hallucination. - Candidate answers that overlap are pruned with non-maximum suppression, so near-duplicate spans with slightly different boundaries do not crowd the top of the ranked list. - The few-shot demonstrations are chosen to cover the full range of answer types, including questions with no answer, so the model learns that answering "nothing" is a valid option. - Several heterogeneous QA datasets are first rewritten into one common schema and then pooled, so that general-domain Arabic data (ARCD) can support a niche domain. ## For Mova How these ideas could serve Mova, our small open language model — its files, stages and tests: - **I1** (answer): every answer a reader produces could be required to resolve to concrete token offsets in the text, rejecting outputs with no source span and keeping each answer explainable. - **I2** (answer): when several frames or nodes support overlapping answer spans in a tale QA item, NMS-style pruning could keep the single best-bounded span and drop the redundant variants. - **I3** (learn): prompts for the LLM teacher tiers could always include unanswerable examples, giving built-in negative controls and discouraging the teacher from inventing labels. - **I4** (tree): the same approach fits UD dialect conversion, where treebanks with different conventions are normalized into one target format before being pooled for gparse or the perceptron parser. *Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.* Fields: [Linguistics & languages](https://md.research.elamur.ai/free/en/shelves/linguistics/) Topics: [🧠 Language models & pretraining](https://md.research.elamur.ai/free/en/topics/language-models/) · [❓ Question answering](https://md.research.elamur.ai/free/en/topics/question-answering/) · [🐪 Semitic languages](https://md.research.elamur.ai/free/en/topics/semitic-languages/) ## Institutions - [October University of Modern Sciences and Arts](https://md.research.elamur.ai/free/en/universities/october-university-of-modern-sciences-and-arts/) — University, Cairo, Egypt · 7 works · 25 citations ## Cite Cite as: Mohamed Basem, Islam Oshallah, Ali Hamdi and Ammar Mohammed. 2025. Few-Shot Prompting for Extractive Quranic QA with Instruction-Tuned LLMs. arXiv:2508.06103. The paper itself, not this page. BibTeX from the library's record of the work. ```bibtex @misc{basem2025few, title = {{Few-Shot Prompting for Extractive Quranic QA with Instruction-Tuned LLMs}}, author = {Mohamed Basem, Islam Oshallah, Ali Hamdi and Ammar Mohammed}, year = {2025}, doi = {10.48550/arxiv.2508.06103}, eprint = {2508.06103}, archivePrefix = {arXiv}, url = {https://doi.org/10.48550/arxiv.2508.06103} } ``` - [← previous Enhanced Arabic-language cyberbullying detection: deep embedding and transformer (BERT) approaches](https://md.research.elamur.ai/free/en/papers/aljohani-2025-enhanced-arabic-language-cyberbullying-detection/) - [next → Modeling the Evolution of English Noun Compounds with Feature-Rich Diachronic Compositionality Prediction](https://md.research.elamur.ai/free/en/papers/miletic-2025-modeling-evolution-english-noun-compounds/) - [↑ back to papers of 2025](https://md.research.elamur.ai/free/en/papers/2025/page/2/) ## References 20 references, 1 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet. 1. A. Alrayzah, F. Alsolami, and M. Saleh, “Challenges and opportunities for Arabic question-answering systems: current techniques and future directions,” PeerJ Computer Science, vol. 9, p. e1633, 2023. [find](https://scholar.google.com/scholar?q=A%2E+Alrayzah%2C+F%2E+Alsolami%2C+and+M%2E+Saleh%2C+%E2%80%9CChallenges+and+opportunities+for+Arabic+question%2Danswering+systems%3A+current+techniques+and+future+directions%2C%E2%80%9D+PeerJ+Computer+Science%2C+vol%2E+9%2C+p%2E+e1633%2C+2023%2E) 2. R. Alnefaie et al., “Lkau23 team: Transformer-based ensemble for Qur’anic QA,” in Proc. 1st Workshop on Question Answering on the Quran, pp. 141–147, 2023. [find](https://scholar.google.com/scholar?q=R%2E+Alnefaie+et+al%2E%2C+%E2%80%9CLkau23+team%3A+Transformer%2Dbased+ensemble+for+Qur%E2%80%99anic+QA%2C%E2%80%9D+in+Proc%2E+1st+Workshop+on+Question+Answering+on+the+Quran%2C+pp%2E+141%E2%80%93147%2C+2023%2E) 3. TheDevastator, “ARCD Dataset: Unlocking Arabic Language Comprehension with the Arabic Reading Comprehension Dataset,” 2021. [Online]. Available: [https://www.kaggle.com/datasets/thedevastator/](https://www.kaggle.com/datasets/thedevastator/) unlocking-arabic-language-comprehension-with-the/data [↗](https://www.kaggle.com/datasets/thedevastator/) 4. M. Basem, I. Oshallah, B. Hikal, A. Hamdi, and A. Mohamed, “Optimized Quran Passage Retrieval Using an Expanded QA Dataset and Fine-Tuned Language Models,” arXiv preprint arXiv:2412.11431, 2024. [↗](https://arxiv.org/abs/2412.11431) [our card](https://md.research.elamur.ai/free/en/papers/basem-2024-optimized-quran-passage-retrieval-using/) 5. A. Elkomy, M. Elsayed, and M. Hefny, “TCE team: Transformer-based ensemble for extractive Qur’anic QA,” in Proc. 1st Workshop on Question Answering on the Quran, pp. 110–117, 2023. [find](https://scholar.google.com/scholar?q=A%2E+Elkomy%2C+M%2E+Elsayed%2C+and+M%2E+Hefny%2C+%E2%80%9CTCE+team%3A+Transformer%2Dbased+ensemble+for+extractive+Qur%E2%80%99anic+QA%2C%E2%80%9D+in+Proc%2E+1st+Workshop+on+Question+Answering+on+the+Quran%2C+pp%2E+110%E2%80%93117%2C+2023%2E) 6. M. Essam et al., “Decoding Queries: An In-Depth Survey of Quality Techniques for Question Analysis in Arabic Question Answering Systems,” IEEE Access, 2024. [find](https://scholar.google.com/scholar?q=M%2E+Essam+et+al%2E%2C+%E2%80%9CDecoding+Queries%3A+An+In%2DDepth+Survey+of+Quality+Techniques+for+Question+Analysis+in+Arabic+Question+Answering+Systems%2C%E2%80%9D+IEEE+Access%2C+2024%2E) 7. M. U. Hadi et al., “Large language models: a comprehensive survey of its applications, challenges, limitations, and future prospects,” Authorea Preprints, vol. 1, pp. 1–26, 2023. [find](https://scholar.google.com/scholar?q=M%2E+U%2E+Hadi+et+al%2E%2C+%E2%80%9CLarge+language+models%3A+a+comprehensive+survey+of+its+applications%2C+challenges%2C+limitations%2C+and+future+prospects%2C%E2%80%9D+Authorea+Preprints%2C+vol%2E+1%2C+pp%2E+1%E2%80%9326%2C+2023%2E) 8. S. Kazi, S. Khoja, and A. Daud, “A survey of deep learning techniques for machine reading comprehension,” Artificial Intelligence Review, vol. 56, no. Suppl 2, pp. 2509–2569, 2023. [find](https://scholar.google.com/scholar?q=S%2E+Kazi%2C+S%2E+Khoja%2C+and+A%2E+Daud%2C+%E2%80%9CA+survey+of+deep+learning+techniques+for+machine+reading+comprehension%2C%E2%80%9D+Artificial+Intelligence+Review%2C+vol%2E+56%2C+no%2E+Suppl+2%2C+pp%2E+2509%E2%80%932569%2C+2023%2E) 9. A. Liu, B. Feng, et al., “DeepSeek-V3 Technical Report,” arXiv preprint arXiv:2412.19437, 2025. [Online]. Available: https://arxiv.org/abs/2412. 19437 [↗](https://arxiv.org/abs/2412.19437) 10. M. Mahmoudi, M. Elaraby, and A. Mourad, “Gym team: Multi-task transfer learning for semantic span detection in Qur’anic question answering,” in Proc. 1st Workshop on Question Answering on the Quran, pp. 134–140, 2023. [find](https://scholar.google.com/scholar?q=M%2E+Mahmoudi%2C+M%2E+Elaraby%2C+and+A%2E+Mourad%2C+%E2%80%9CGym+team%3A+Multi%2Dtask+transfer+learning+for+semantic+span+detection+in+Qur%E2%80%99anic+question+answering%2C%E2%80%9D+in+Proc%2E+1st+Workshop+on+Question+Answering+on+the+Quran%2C+p) 11. M. Malhas, H. Mubarak, and K. Darwish, “Qur’an QA 2023: Overview of the First Workshop on Question Answering on the Quran,” in Proc. 1st Workshop on Question Answering on the Quran, pp. 1–13, 2023. [Online]. Available: https://aclanthology.org/2023.quran-1.1 [↗](https://aclanthology.org/2023.quran-1.1/) 12. I. Oshallah, M. Basem, A. Hamdi, and A. Mohammed, “Cross-Language Approach for Quranic QA,” arXiv preprint arXiv:2501.17449, 2025. [↗](https://arxiv.org/abs/2501.17449) 13. B. I. OsmanHamoud, “A Question Answering System Design about the Holy Quran,” Ph.D. dissertation, Sudan Univ. of Science and Technology, 2017. [find](https://scholar.google.com/scholar?q=B%2E+I%2E+OsmanHamoud%2C+%E2%80%9CA+Question+Answering+System+Design+about+the+Holy+Quran%2C%E2%80%9D+Ph%2ED%2E+dissertation%2C+Sudan+Univ%2E+of+Science+and+Technology%2C+2017%2E) 14. H. S. Al-Khalifa and N. Alnazzawi, “QUQA: Question Answering Dataset for the Holy Qur’an,” 2023. [Online]. Available: [https://github.com/scsaln/HAQA-and-QUQA/blob/main/QUQA.xlsx](https://github.com/scsaln/HAQA-and-QUQA/blob/main/QUQA.xlsx) [↗](https://github.com/scsaln/HAQA-and-QUQA/blob/main/QUQA.xlsx) 15. bigIR Research Group, “Qur’an QA 2023 Shared Task - Task B Dataset,” 2023. [Online]. Available: [https://gitlab.com/bigirqu/quran-qa-2023/-/](https://gitlab.com/bigirqu/quran-qa-2023/-/) tree/main/Task-B/data [↗](https://gitlab.com/bigirqu/quran-qa-2023/-/) 16. A. Ramachandran, “Unveiling Google’s Gemini 2.0: A Comprehensive Study of its Multimodal AI Design, Advanced Architecture, and Real-World Applications,” ResearchGate, 2024. [Online]. Available: [https://www.researchgate.net/publication/387089907](https://www.researchgate.net/publication/387089907) Unveiling Google% 27s Gemini 20 A Comprehensive Study of its Multimodal AI Design Advanced Architecture and Real-World Applications [↗](https://www.researchgate.net/publication/387089907) 17. M. Salama, The Qur’an and Modern Arabic Literary Criticism: From Taha to Nasr, Bloomsbury Publishing, 2018. [find](https://scholar.google.com/scholar?q=M%2E+Salama%2C+The+Qur%E2%80%99an+and+Modern+Arabic+Literary+Criticism%3A+From+Taha+to+Nasr%2C+Bloomsbury+Publishing%2C+2018%2E) 18. M. Zekiye, M. Bakouri, M. Oudah, and A. Mourad, “Al-Jawaab team: Generative constraint modeling with dense passage retrieval for Qur’anic question answering,” in Proc. 1st Workshop on Question Answering on the Quran, pp. 126–133, 2023. [find](https://scholar.google.com/scholar?q=M%2E+Zekiye%2C+M%2E+Bakouri%2C+M%2E+Oudah%2C+and+A%2E+Mourad%2C+%E2%80%9CAl%2DJawaab+team%3A+Generative+constraint+modeling+with+dense+passage+retrieval+for+Qur%E2%80%99anic+question+answering%2C%E2%80%9D+in+Proc%2E+1st+Workshop+on+Question+Answeri) 19. Z. Zhang, E. Strubell, and E. Hovy, “Comparing span extraction methods for semantic role labeling,” in Proc. 5th Workshop on Structured Prediction for NLP (SPNLP), pp. 67–77, 2021. [find](https://scholar.google.com/scholar?q=Z%2E+Zhang%2C+E%2E+Strubell%2C+and+E%2E+Hovy%2C+%E2%80%9CComparing+span+extraction+methods+for+semantic+role+labeling%2C%E2%80%9D+in+Proc%2E+5th+Workshop+on+Structured+Prediction+for+NLP+%28SPNLP%29%2C+pp%2E+67%E2%80%9377%2C+2021%2E) 20. X. Zhang, R. Mao, and E. Cambria, “A survey on syntactic processing techniques,” Artificial Intelligence Review, vol. 56, no. 6, pp. 5645–5728, 2023. [find](https://scholar.google.com/scholar?q=X%2E+Zhang%2C+R%2E+Mao%2C+and+E%2E+Cambria%2C+%E2%80%9CA+survey+on+syntactic+processing+techniques%2C%E2%80%9D+Artificial+Intelligence+Review%2C+vol%2E+56%2C+no%2E+6%2C+pp%2E+5645%E2%80%935728%2C+2023%2E)