---
title: "Open-domain Arabic Conversational Question Answering with"
description: "Arabic has had no open-domain conversational question answering (CQA) system. 24 references, cited by 1 paper here."
authors: ["Mariam E. Hassib", "Nagwa El-Makky", "Marwan Torki"]
year: 2025
venue: "Proceedings of The Third Arabic Natural Language Processing Conference"
source: "https://aclanthology.org/2025.arabicnlp-main.7/"
doi: "10.18653/v1/2025.arabicnlp-main.7"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/hassib-2025-open-domain-arabic-conversational-question/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/hassib-2025-open-domain-arabic-conversational-question/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2025](https://md.research.elamur.ai/free/en/papers/2025/page/15/) / Open-domain Arabic Conversational Question Answering with Question Rewriting

# Open-domain Arabic Conversational Question Answering with Question Rewriting

**Authors:** Mariam E. Hassib, [Nagwa El-Makky](https://md.research.elamur.ai/free/en/authors/nagwa-el-makky/) and [Marwan Torki](https://md.research.elamur.ai/free/en/authors/marwan-torki/) · **Year:** 2025 · **Venue:** [Proceedings of The Third Arabic Natural Language Processing Conference](https://md.research.elamur.ai/free/en/venues/arabicnlp/) ([ArabicNLP 2025](https://md.research.elamur.ai/free/en/venues/arabicnlp/2025/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2025.arabicnlp-main.7/) · [PDF](https://aclanthology.org/2025.arabicnlp-main.7.pdf) · [DOI 10.18653/v1/2025.arabicnlp-main.7](https://doi.org/10.18653/v1/2025.arabicnlp-main.7) · [OpenAlex](https://openalex.org/W4416037669)

**Citations:** 0 citations (OpenAlex, 2026-10-14) · [1](https://md.research.elamur.ai/free/en/papers/hassib-2025-open-domain-arabic-conversational-question/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
Arabic has had no open-domain conversational question answering (CQA) system. Two things were missing: a suitable dataset and a way to handle follow-up questions that depend on earlier turns. The authors built AraQReCC by machine-translating, with Googletrans, the QuAC-derived part of the English QReCC benchmark. The result has about 9K conversations and 62K question–answer pairs. They filtered translations in two ways: COMET scores, and averaged quality ratings from several LLMs. They also combined both filters into a stricter "dual quality" subset.

The system is a pipeline. AraT5 rewrites each follow-up question into a self-contained one. BM25 or DPR then retrieves passages, and AraT5 generates the answer. Rewriting greatly improves retrieval, and DPR beats BM25 throughout. The best reported answer F1 is about 21.5% on test, against an upper bound of roughly 40%. The work gives Arabic its first CQA benchmark. It also offers a reusable recipe for vetting translated training data in low-resource settings.

## Key points
- **AraQReCC size:** 62,749 turns in total. The filtered subsets keep 11,509 turns (COMET), 48,641 (LLM Rating) and 9,691 (Dual Quality).
- **COMET thresholds** differ by split: at least 65% for training and at least 70% for development and test.
- **LLM judges:** GPT-4o, LLaMA 3.1 70B and LLaMA 3.1 405B each scored every conversation from 0 to 5. Conversations averaging 4 or more were kept. Agreement between humans and GPT-4o was substantial (κ = 0.725).
- **Question rewriting F1:**
  - Training on the full data gave 68.38 (dev) and 67.99 (test).
  - The smaller LLM-rated subset did slightly better, at 70.26 and 70.29.
- **Retrieval with rewriting:**
  - BM25 MRR rose from 5.01% to 32.02% with LLM-rated rewrites, and DPR reached 44.52%.
  - With gold rewrites, the ceilings were 46.20% (DPR) and 40.18% (BM25).
- **Answering:** best F1 was 21.51% with LLM-rated rewrites and 23.85% with gold rewrites, versus an upper bound of 40.22%.

## Interesting ideas
- Several different LLMs act as independent graders of each translated conversation, and only consensus-high items are kept; one human-vs-LLM agreement check (kappa) then justifies trusting the panel
- Filtering thresholds differ by split: training data tolerates noisier items to keep volume, while dev/test must pass a stricter bar so evaluation stays trustworthy
- Context dependence is handled by a separately trained step that turns a follow-up question into a standalone one before retrieval; this cleanly isolates the "what does 'he' refer to" problem from answering
- The evaluation is a ladder of conditions that pinpoints where errors arise: original question, predicted rewrite, gold rewrite, gold passage, and an extractive upper bound
- A smaller, quality-filtered subset slightly beat the full noisy set for rewriting, evidence that cleaning can outweigh sheer volume

## For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

- **I1** (learn): The same panel-plus-kappa audit could gate silver UD trees or LLM-teacher labels. Mova could keep only items where multiple witnesses score high, and report one human-agreement number per tier.
- **I2** (learn): Silver training sets could use a looser witness-agreement cutoff, while held-out evaluation sets demand near-unanimous agreement, so scores are not polluted by label noise.
- **I3** (frames): Mova could rewrite follow-up questions about a tale, replacing pronouns and ellipsis with explicit entities found by coreference. The world model then receives a fully specified query, and the rewrite itself serves as an explainable trace.
- **I4** (answer): Mova's fable QA could report the same staged oracles, such as gold tree, gold frames and gold world. Each failure could then be blamed on a specific pipeline stage rather than on the whole system.
- **I5** (learn): Before scaling silver data, Mova should compare a strictly filtered subset against the full pool. This would show whether more data or cleaner data is the better investment.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Alexandria University](https://md.research.elamur.ai/free/en/universities/alexandria-university/) — University, Alexandria, Egypt · 39 works · 287 citations

📚 Shelves (5): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 16,039 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 15,891 works · [🗄️ Information Systems](https://md.research.elamur.ai/free/en/shelves/computer-science/information-systems/) · 1,437 works

🧭 Topics (3): [Semitic languages](https://md.research.elamur.ai/free/en/topics/semitic-languages/) · 2,908 works · [🗨️ Dialogue systems](https://md.research.elamur.ai/free/en/topics/dialogue-systems/) · 624 works · [❓ Question answering](https://md.research.elamur.ai/free/en/topics/question-answering/) · 441 works

## Cite

Cite as: Mariam E. Hassib, Nagwa El-Makky and Marwan Torki. 2025. Open-domain Arabic Conversational Question Answering with Question Rewriting. Proceedings of The Third Arabic Natural Language Processing Conference.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{hassib2025open,
  title = {{Open-domain Arabic Conversational Question Answering with Question Rewriting}},
  author = {Mariam E. Hassib, Nagwa El-Makky and Marwan Torki},
  booktitle = {Proceedings of The Third Arabic Natural Language Processing Conference},
  year = {2025},
  doi = {10.18653/v1/2025.arabicnlp-main.7},
  url = {https://doi.org/10.18653/v1/2025.arabicnlp-main.7}
}
```

- [← previous On Reference (In-)Determinacy in Natural Language Inference](https://md.research.elamur.ai/free/en/papers/chen-2025-reference-determinacy-natural-language-inference/)
- [next → Osint at AraGenEval shared task: Fine-Tuned Modeling for Tracking Style Signatures and AI Generation in Arabic Texts](https://md.research.elamur.ai/free/en/papers/agrahari-2025-osint-arageneval-shared-task-fine/)
- [↑ back to papers of 2025](https://md.research.elamur.ai/free/en/papers/2025/page/15/)

## References

24 references, 15 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Ahmed Abdallah, Mahmoud El-Haj, and Mohammad Al-Tawfiq. 2024. [Arabicaqa: A comprehensive dataset for arabic question answering](https://md.research.elamur.ai/free/en/papers/abdallah-2024-arabicaqa-comprehensive-dataset-arabic-question/). arXiv preprint arXiv:2403.01234. [↗](https://arxiv.org/abs/2403.01234)
2. Xiachong Feng, Xiaocheng Feng, Libo Qin, Bing Qin, and Ting Liu. 2021. Language model as an annotator: Exploring dialogpt for dialogue summarization. arXiv preprint arXiv:2105.12544. [↗](https://arxiv.org/abs/2105.12544)
3. Raviteja Anantha, Svitlana Vakulenko, Zhucheng Tu, Shayne Longpre, Stephen Pulman, and Srinivas Chappidi. 2021. [Open-domain question answering goes conversational via question rewriting](https://md.research.elamur.ai/free/en/papers/anantha-2021-open-domain-question-answering-goes/). In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 520–534.
4. Stefano Iovine, Luca Fogliato, Matteo Gatta, Marco Giraudo, and Andrea Lanza. 2022. Cyclekqr: Unsupervised bidirectional keyword-question rewriting. arXiv preprint arXiv:2209.07663. [↗](https://arxiv.org/abs/2209.07663)
5. Jeff Johnson, Matthijs Douze, and Hervé Jégou. 2019. [Billion-scale similarity search with gpus](https://md.research.elamur.ai/free/en/papers/johnson-2017-billion-scale-similarity-search-gpus/). IEEE Transactions on Big Data, 7(3):535–547. [↗](https://doi.org/10.1109/tbdata.2019.2921572)
6. Wissam Antoun, Fady Baly, and Hazem Hajj. 2020. [Araelectra: Pre-training text discriminators for arabic language understanding](https://md.research.elamur.ai/free/en/papers/antoun-2021-araelectra-pre-training-text-discriminators/). arXiv preprint arXiv:2012.15516. [↗](https://arxiv.org/abs/2012.15516)
7. Vladimir Karpukhin, Barlas Oğuz, Sewon Min, Patrick Lewis, Ledell Wu, Sergey Edunov, Danqi Chen, and Wen-tau Yih. 2020. [Dense passage retrieval for open-domain question answering](https://md.research.elamur.ai/free/en/papers/karpukhin-2020-dense-passage-retrieval-open-domain/). arXiv preprint arXiv:2004.04906. [↗](https://arxiv.org/abs/2004.04906)
8. Adel Atef, Bassam Mattar, Sandra Sherif, Eman Elrefai, and Marwan Torki. 2020. Aqad: 17,000+ arabic questions for machine comprehension of text. In 2020 IEEE/ACS 17th International Conference on Computer Systems and Applications (AICCSA), pages 1–7. IEEE. [find](https://scholar.google.com/scholar?q=Adel+Atef%2C+Bassam+Mattar%2C+Sandra+Sherif%2C+Eman+Elrefai%2C+and+Marwan+Torki%2E+2020%2E+Aqad%3A+17%2C000%2B+arabic+questions+for+machine+comprehension+of+text%2E+In+2020+IEEE%2FACS+17th+International+Conference+on+Compu)
9. Tom Kwiatkowski, Jennimaria Palomaki, Olivia Redfield, Michael Collins, Ankur Parikh, Chris Alberti, Danielle Epstein, Illia Polosukhin, Jacob Devlin, Kenton Lee, et al. 2019. [Natural questions: a benchmark for question answering research](https://md.research.elamur.ai/free/en/papers/kwiatkowski-2019-natural-questions-benchmark-question-answering/). Transactions of the Association for Computational Linguistics, 7:453–466. [↗](https://doi.org/10.1162/tacl_a_00276)
10. Chen Chen, Wenliang Chen, Yang Li, Jiansheng Li, Jiaying Li, Peixin Shi, and Yanan Wang. 2022. Reinforced question rewriting for conversational question answering. arXiv preprint arXiv:2208.01257. [↗](https://arxiv.org/abs/2208.01257)
11. Xinbei Ma, Yeyun Gong, Pengcheng He, Hai Zhao, and Nan Duan. 2023. Query rewriting for retrieval-augmented large language models. arXiv preprint arXiv:2305.14283. [↗](https://arxiv.org/abs/2305.14283)
12. Eunsol Choi, He He, Mohit Iyyer, Mark Yatskar, Wen-tau Yih, Yejin Choi, Percy Liang, and Luke Zettlemoyer. 2018. [Quac: Question answering in context](https://md.research.elamur.ai/free/en/papers/choi-2018-quac-question-answering-context/). arXiv preprint arXiv:1808.07036. [↗](https://arxiv.org/abs/1808.07036)
13. Hussein Mozannar, Karl El Hajal, Elie Maamary, and Hazem Hajj. 2019. [Neural arabic question answering](https://md.research.elamur.ai/free/en/papers/mozannar-2019-neural-arabic-question-answering/). ACL 2019, page 108.
14. Jonathan H Clark, Eunsol Choi, Michael Collins, Dan Garrette, Tom Kwiatkowski, Vitaly Nikolaev, and Jennimaria Palomaki. 2020. [Tydi qa: A benchmark for information-seeking question answering in typologically diverse languages](https://md.research.elamur.ai/free/en/papers/clark-2020-tydi-qa-benchmark-information-seeking/). Transactions of the Association for Computational Linguistics, 8:454–470. [↗](https://doi.org/10.48550/arxiv.2003.05002)
15. Chen Qu, Liu Yang, Cen Chen, Minghui Qiu, W Bruce Croft, and Mohit Iyyer. 2020. Open-retrieval conversational question answering. In Proceedings of the 43rd International ACM SIGIR conference on research and development in Information Retrieval, pages 539–548. [find](https://scholar.google.com/scholar?q=Chen+Qu%2C+Liu+Yang%2C+Cen+Chen%2C+Minghui+Qiu%2C+W+Bruce+Croft%2C+and+Mohit+Iyyer%2E+2020%2E+Open%2Dretrieval+conversational+question+answering%2E+In+Proceedings+of+the+43rd+International+ACM+SIGIR+conference+on+resea)
16. Jeffrey Dalton, Chenyan Xiong, Vaibhav Kumar, and Jamie Callan. 2020. Cast-19: A dataset for conversational information seeking. In Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 1985–1988. [find](https://scholar.google.com/scholar?q=Jeffrey+Dalton%2C+Chenyan+Xiong%2C+Vaibhav+Kumar%2C+and+Jamie+Callan%2E+2020%2E+Cast%2D19%3A+A+dataset+for+conversational+information+seeking%2E+In+Proceedings+of+the+43rd+International+ACM+SIGIR+Conference+on+Resear)
17. Chen Qu, Liu Yang, Minghui Qiu, W Bruce Croft, Yongfeng Zhang, and Mohit Iyyer. 2019. Bert with history answer embedding for conversational question answering. In Proceedings of the 42nd international ACM SIGIR conference on research and development in information retrieval, pages 1133– 1136. [find](https://scholar.google.com/scholar?q=Chen+Qu%2C+Liu+Yang%2C+Minghui+Qiu%2C+W+Bruce+Croft%2C+Yongfeng+Zhang%2C+and+Mohit+Iyyer%2E+2019%2E+Bert+with+history+answer+embedding+for+conversational+question+answering%2E+In+Proceedings+of+the+42nd+international)
18. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018. [Bert: Pre-training of deep bidirectional transformers for language understanding](https://md.research.elamur.ai/free/en/papers/devlin-2019-bert-pre-training-deep-bidirectional/). arXiv preprint arXiv:1810.04805. [↗](https://arxiv.org/abs/1810.04805)
19. Siva Reddy, Danqi Chen, and Christopher D Manning. 2019. [Coqa: A conversational question answering challenge](https://md.research.elamur.ai/free/en/papers/reddy-2019-coqa-conversational-question-answering-challenge/). Transactions of the Association for Computational Linguistics, 7:249–266. [↗](https://doi.org/10.1162/tacl_a_00266)
20. Ahmed Elgohary, Denis Peskov, and Jordan Boyd-Graber. 2019. [Can you unpack that? learning to rewrite questions-in-context](https://md.research.elamur.ai/free/en/papers/elgohary-2019-can-you-unpack-that-learning/). Conference on Empirical Methods in Natural Language Processing.
21. Ricardo Rei, Craig Stewart, Ana C Farinha, and Alon Lavie. 2020. [Comet: A neural framework for mt evaluation](https://md.research.elamur.ai/free/en/papers/rei-2020-comet-neural-framework-mt-evaluation/). In Proceedings of EMNLP.
22. AbdelRahim Elmadany, Muhammad Abdul-Mageed, et al. 2022. [Arat5: Text-to-text transformers for arabic language generation](https://md.research.elamur.ai/free/en/papers/nagoudi-2022-arat5-text-text-transformers-arabic/). In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 628–647.
23. Stephen E Robertson, Steve Walker, Susan Jones, Micheline M Hancock-Beaulieu, Mike Gatford, et al. 1995. [Okapi at trec-3](https://md.research.elamur.ai/free/en/papers/robertson-1994-okapi-trec-3/). Nist Special Publication Sp, 109:109. similarity between the generated rewrite and the gold rewrite.
24. Ivan Sekulic, Krisztian Balog, and Fabio Crestani. 2024. Towards self-contained answers: Entity-based answer rewriting in conversational search. In Proceedings of the 2024 Conference on Human Information Interaction and Retrieval, pages 209–218. [↗](https://doi.org/10.1145/3627508.3638300)

*Updated 1 November 2026*