---
title: "TCE at Qur'an QA 2023 Shared Task: Low Resource Enhanced"
description: "The paper describes the TCE team's systems for the two Qur'an QA 2023 shared tasks. 34.83% MRR. 17 references, cited by 6 papers here."
authors: ["Mohammed Elkomy", "Amany Sarhan"]
year: 2023
venue: "Proceedings of ArabicNLP 2023"
source: "https://aclanthology.org/2023.arabicnlp-1.81/"
doi: "10.18653/v1/2023.arabicnlp-1.81"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/elkomy-2023-tce-qur-qa-2023-shared/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/elkomy-2023-tce-qur-qa-2023-shared/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2023](https://md.research.elamur.ai/free/en/papers/2023/page/13/) / TCE at Qur'an QA 2023 Shared Task: Low Resource Enhanced Transformer-based Ensemble Approach for Qur'anic QA

# TCE at Qur'an QA 2023 Shared Task: Low Resource Enhanced Transformer-based Ensemble Approach for Qur'anic QA

**Authors:** Mohammed Elkomy and [Amany Sarhan](https://md.research.elamur.ai/free/en/authors/amany-sarhan/) · **Year:** 2023 · **Venue:** [Proceedings of ArabicNLP 2023](https://md.research.elamur.ai/free/en/venues/arabicnlp/) ([ArabicNLP 2023](https://md.research.elamur.ai/free/en/venues/arabicnlp/2023/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2023.arabicnlp-1.81/) · [PDF](https://aclanthology.org/2023.arabicnlp-1.81.pdf) · [DOI 10.18653/v1/2023.arabicnlp-1.81](https://doi.org/10.18653/v1/2023.arabicnlp-1.81) · [OpenAlex](https://openalex.org/W4389518327)

**Citations:** 5 citations (OpenAlex, 2026-10-14) · [6](https://md.research.elamur.ai/free/en/papers/elkomy-2023-tce-qur-qa-2023-shared/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
The paper describes the TCE team's systems for the two Qur'an QA 2023 shared tasks. Task A is ad hoc passage retrieval: questions are written in MSA, and the answers are passages of the Thematic Qur'an Passage Collection (QPC), which are in Classical Arabic. Task B is ranked extractive MRC over QRCDv1.2, in the style of SQuADv2.0. Training data is very small, so the authors combine several techniques:

- **Pipelined fine-tuning** of Arabic encoders (AraBERTv0.2-base, CAMeLBERT-CA, AraELECTRA), passing through external resources such as TyDi QA AR and Tafseer before the target task.
- **Retrieval architectures** for Task A: both dual-encoders and cross-encoders.
- **Two span losses** for Task B: first-answer and multi-answer.
- **Self-ensembles** that vote across runs to reduce run-to-run variance.
- **Thresholds** that decide when a question has no answer.

For Task B they also build leakage-aware "faithful" splits and apply overlap-removing post-processing. On the hidden test set, the best systems reach a MAP of 25.05% on Task A and a pAP of 57.11% on Task B, well above the baselines (9.03 and 32.68). The work matters because it shows how standard transformer pipelines can be stabilised for an extremely small, imbalanced QA benchmark.

## Key points
- **Task A design:** dual-encoders were trained with STAR, using 16 queries per batch. Cross-encoders feed the question and passage to the model as one joint input. On the dev split, the best single model is a pipelined AraBERT cross-encoder (MAP 34.83%, MRR 47.09%), and its self-ensemble reaches a MAP of 36.70%.
- **Hard negatives:** mining hard negatives from a fine-tuned checkpoint improves the AraBERT dual-encoder by roughly 4.5 MAP points compared with random negatives.
- **Multi-answer loss (MAL):** on multi-answer samples, MAL clearly beats the first-answer loss (FAL). For example, AraELECTRA scores 51.55 vs 43.69 pAP on that subset. On the standard split overall, however, MAL does not always win.
- **Post-processing:** removing overlapping spans always improves on the raw span rankings, by about 3 pAP for AraBERT-MAL.
- **Leakage:** the authors extended the leakage analysis to QRCDv1.2. About 90% of its no-answer samples fall into the trivial leakage group D(1), which makes the dev scores on unanswerable cases unreliable.
- **Released resources:** code and trained models are on GitHub (mohammed-elkomy/quran-qa).

## Interesting ideas
- **Leakage-aware splits:** the authors re-split the data so that shared questions, passages or answers cannot sit on both sides of train and dev. This exposes how much of a score comes from memorisation rather than generalisation.
- **Multi-answer training loss:** when a question has several correct spans, the loss spreads probability across all of them instead of rewarding only the first gold span. This keeps the model from becoming overconfident in a single answer.
- **Prior-calibrated abstention:** the no-answer threshold is set so that the share of rejected items matches the expected rate of unanswerable questions (about 15% for Task A, about 5% for Task B). This is used instead of tuning the threshold on a tiny, unrepresentative dev set.
- **Self-ensemble voting:** the same model is trained several times and the runs vote, which smooths out the variance that small data causes.
- **Hard negatives from a trained checkpoint:** negatives for retrieval are mined using an already fine-tuned model, so the training signal sharpens around confusable passages.

## For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

- **I1** (learn): Mova's QA and parser evaluations could group items by shared text before splitting, as a negative control against inflated scores from near-duplicate fables or sentences.
- **I2** (answer): Mova's readers could keep and score every grounded answer candidate in the world model, rather than forcing one winner when the text supports several.
- **I3** (answer): Mova's "cannot answer from text" decision could be calibrated against a known base rate of unanswerable test questions, which makes abstention explicit and auditable.
- **I4** (learn): this mirrors Mova's agreement among witness parsers; repeated gparse runs with different seeds could serve as extra witnesses when filtering silver trees.
- **I5** (world): when Mova links mentions to WordNet or Wikidata entries, near-miss candidates proposed by an earlier linker could serve as hard negatives for training the disambiguation step.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Tanta University](https://md.research.elamur.ai/free/en/universities/tanta-university/) — University, Tanta, Egypt · 7 works · 278 citations

📚 Shelves (5): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 18,614 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 17,965 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 17,450 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 17,296 works · [🗄️ Information Systems](https://md.research.elamur.ai/free/en/shelves/computer-science/information-systems/) · 1,730 works

🧭 Topics (3): [Semitic languages](https://md.research.elamur.ai/free/en/topics/semitic-languages/) · 3,066 works · [❓ Question answering](https://md.research.elamur.ai/free/en/topics/question-answering/) · 487 works · [🌱 Low-resource languages](https://md.research.elamur.ai/free/en/topics/low-resource-languages/) · 287 works

## Cite

Cite as: Mohammed Elkomy and Amany Sarhan. 2023. TCE at Qur'an QA 2023 Shared Task: Low Resource Enhanced Transformer-based Ensemble Approach for Qur'anic QA. Proceedings of ArabicNLP 2023.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{elkomy2023tce,
  title = {{TCE at Qur'an QA 2023 Shared Task: Low Resource Enhanced Transformer-based Ensemble Approach for Qur'anic QA}},
  author = {Mohammed Elkomy and Amany Sarhan},
  booktitle = {Proceedings of ArabicNLP 2023},
  year = {2023},
  doi = {10.18653/v1/2023.arabicnlp-1.81},
  url = {https://doi.org/10.18653/v1/2023.arabicnlp-1.81}
}
```

- [← previous Optimizing the Size of Subword Vocabularies in Dialect Classification](https://md.research.elamur.ai/free/en/papers/kanjirangat-2023-optimizing-size-subword-vocabularies-dialect/)
- [next → The Trickle-down Impact of Reward (In-)consistency on RLHF](https://md.research.elamur.ai/free/en/papers/shen-2023-trickle-down-impact-reward-consistency/)
- [↑ back to papers of 2023](https://md.research.elamur.ai/free/en/papers/2023/page/13/)

## References

17 references, 10 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Wissam Antoun, Fady Baly, and Hazem Hajj. 2020. [Arabert: Transformer-based model for arabic language understanding](https://md.research.elamur.ai/free/en/papers/antoun-2020-arabert-transformer-based-model-arabic/). pages 9–15. European Language Resource Association.
2. Wissam Antoun, Fady Baly, and Hazem Hajj. 2021. [Araelectra: Pre-training text discriminators for arabic language understanding](https://md.research.elamur.ai/free/en/papers/antoun-2021-araelectra-pre-training-text-discriminators/). pages 191–195. Association for Computational Linguistics.
3. Muhammad Huzaifa Bashir, Aqil M Azmi, Haq Nawaz, Wajdi Zaghouani, Mona Diab, Ala Al-Fuqaha, and Junaid Qadir. 2021. Arabic natural language processing for qur’anic research: A systematic review. [↗](https://doi.org/10.1007/s10462-022-10313-2)
4. Jonathan H Clark, Eunsol Choi, Michael Collins, Dan Garrette, Tom Kwiatkowski, Vitaly Nikolaev, and Jennimaria Palomaki. 2020. [Tydi qa: A benchmark for information-seeking question answering in typologically diverse languages](https://md.research.elamur.ai/free/en/papers/clark-2020-tydi-qa-benchmark-information-seeking/). Transactions of the Association for Computational Linguistics. [↗](https://doi.org/10.48550/arxiv.2003.05002)
5. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. [Bert: Pre-training of deep bidirectional transformers for language understanding](https://md.research.elamur.ai/free/en/papers/devlin-2019-bert-pre-training-deep-bidirectional/). pages 4171–4186. Association for Computational Linguistics.
6. Mohamemd Elkomy and Amany M Sarhan. 2022. [Tce at qur’an qa 2022: Arabic language question answering over holy qur’an using a post-processed ensemble of bert-based models](https://md.research.elamur.ai/free/en/papers/elkomy-2022-tce-qur-qa-2022-arabic/). pages 154–161. European Language Resources Association.
7. Siddhant Garg, Thuy Vu, and Alessandro Moschitti. 2020. Tanda: Transfer and adapt pre-trained transformer models for answer sentence selection. volume 34, pages 7780–7788. [find](https://scholar.google.com/scholar?q=Siddhant+Garg%2C+Thuy+Vu%2C+and+Alessandro+Moschitti%2E+2020%2E+Tanda%3A+Transfer+and+adapt+pre%2Dtrained+transformer+models+for+answer+sentence+selection%2E+volume+34%2C+pages+7780%E2%80%937788%2E)
8. Go Inoue, Bashar Alhafni, Nurpeiis Baimukan, Houda Bouamor, and Nizar Habash. 2021. [The interplay of variant, size, and task type in arabic pre-trained language models](https://md.research.elamur.ai/free/en/papers/inoue-2021-interplay-variant-size-task-type/). Association for Computational Linguistics.
9. Amr Keleg and Walid Magdy. 2022. [Smash at qur’an qa 2022: Creating better faithful data splits for low-resourced question answering scenarios](https://md.research.elamur.ai/free/en/papers/keleg-2022-smash-qur-qa-2022-creating/). pages 136– 145. European Language Resources Association.
10. Rana Malhas and Tamer Elsayed. 2020. AyaTEC: Building a Reusable Verse-based Test Collection for Arabic Question Answering on the Holy Qur’an. ACM Transactions on Asian and Low-Resource Language Information Processing, 19(6):1–21. [find](https://scholar.google.com/scholar?q=Rana+Malhas+and+Tamer+Elsayed%2E+2020%2E+AyaTEC%3A+Building+a+Reusable+Verse%2Dbased+Test+Collection+for+Arabic+Question+Answering+on+the+Holy+Qur%E2%80%99an%2E+ACM+Transactions+on+Asian+and+Low%2DResource+Language+Infor)
11. Rana Malhas and Tamer Elsayed. 2022. Arabic Machine Reading Comprehension on the Holy Qur’an using CL-AraBERT. Information Processing & Management, 59(6):103068. [↗](https://doi.org/10.1016/j.ipm.2022.103068)
12. Rana Malhas, Watheq Mansour, and Tamer Elsayed. 2022. [find](https://scholar.google.com/scholar?q=Rana+Malhas%2C+Watheq+Mansour%2C+and+Tamer+Elsayed%2E+2022%2E)
13. Qur’an QA 2022: [Overview of the First Shared Task on Question Answering over the Holy Qur’an](https://md.research.elamur.ai/free/en/papers/malhas-2022-qur-qa-2022-overview-first/). In Proceedings of the 5th Workshop on Open-Source Arabic Corpora and Processing Tools (OSACT5) at the 13th Language Resources and Evaluation Conference (LREC 2022), pages 79–87.
14. Rana Malhas, Watheq Mansour, and Tamer Elsayed. 2023. [find](https://scholar.google.com/scholar?q=Rana+Malhas%2C+Watheq+Mansour%2C+and+Tamer+Elsayed%2E+2023%2E)
15. Qur’an QA 2023 [Shared Task: Overview of Passage Retrieval and Reading Comprehension Tasks over the Holy Qur’an](https://md.research.elamur.ai/free/en/papers/malhas-2023-qur-qa-2023-shared-task/). In Proceedings of the First Arabic Natural Language Processing Conference (ArabicNLP 2023), Singapore.
16. RANA R MALHAS. 2023. ARABIC QUESTION AN-SWERING ON THE HOLY QUR’AN. Ph.D. thesis. [find](https://scholar.google.com/scholar?q=RANA+R+MALHAS%2E+2023%2E+ARABIC+QUESTION+AN%2DSWERING+ON+THE+HOLY+QUR%E2%80%99AN%2E+Ph%2ED%2E+thesis%2E)
17. Ali Mostafa and Omar Mohamed. 2022. [Gof at qur’an qa 2022: Towards an efficient question answering for the holy qu’ran in the arabic language using deep learning-based approach](https://md.research.elamur.ai/free/en/papers/mostafa-2022-gof-qur-qa-2022-efficient/). pages 104–111. European Language Resources Association. Retrieval, SIGIR ’21, page 1503–1512, New York, NY, USA. Association for Computing Machinery.

*Updated 1 November 2026*