---
title: "HalluTruthQA-4K — Zighem; 2026"
description: "Arabic hallucination benchmarks usually give only a verdict for the whole response. 27 references, cited by 1 paper here."
authors: ["Salah Eddine Bekhouche; Abdessalam Bouchekif; Hichem Telli; Mohammed-En-Nadhir Zighem;", "Abdenour Hadid"]
year: 2026
venue: "arXiv:2608.03966"
source: "https://arxiv.org/abs/2608.03966"
doi: "10.48550/arxiv.2608.03966"
arxiv: "2608.03966"
licence: "CC BY 4.0 (arXiv)"
licence_url: "http://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/bekhouche-2026-hallutruthqa-4k-fine-grained-corpus/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/bekhouche-2026-hallutruthqa-4k-fine-grained-corpus/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2026](https://md.research.elamur.ai/free/en/papers/2026/page/2/) / HalluTruthQA-4K: A Fine-Grained Corpus and Annotation Process for Arabic Hallucination Detection and Truth Verification

# HalluTruthQA-4K: A Fine-Grained Corpus and Annotation Process for Arabic Hallucination Detection and Truth Verification

**Authors:** Salah Eddine Bekhouche; Abdessalam Bouchekif; Hichem Telli; Mohammed-En-Nadhir Zighem; and [Abdenour Hadid](https://md.research.elamur.ai/free/en/authors/abdenour-hadid/) · **Year:** 2026 · **Venue:** arXiv:2608.03966 · **Licence:** [CC BY 4.0 (arXiv)](http://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://arxiv.org/abs/2608.03966) · [PDF](https://arxiv.org/pdf/2608.03966) · [DOI 10.48550/arxiv.2608.03966](https://doi.org/10.48550/arxiv.2608.03966) · [arXiv 2608.03966](https://arxiv.org/abs/2608.03966) · [OpenAlex](https://openalex.org/W7172502394)

**Citations:** 0 citations (OpenAlex, 2026-10-14) · 1 in our library, of the 140,884 papers whose bibliographies we read

## Summary
Arabic hallucination benchmarks usually give only a verdict for the whole response. They do not mark where the mistake sits, why it is wrong, or what the correct fact is. HalluTruthQA-4K addresses this by enlarging the earlier HalluTruthQA benchmark from 2,400 to 4,000 expert-annotated Arabic QA items. There are 1,000 items in each of four knowledge-heavy domains: Islamic knowledge, history, science and geography.

Every answer was produced by Fanar-1-9B-Instruct under controlled generation. Each item is paired with a verified reference answer and six answer options. For hallucinated responses, annotators also marked the faulty text at the character level, wrote an explanation, and assigned macro- and micro-level error types. Experts annotated the items, a second pass verified them independently, and adjudication resolved disagreements.

The result supports four tasks on the same instances: binary detection, span localization, explanation, and multiple-choice verification. It comes with balanced splits and is the official data for Track 2 of HalluScoring 2026. Because all four layers are joined per item, a system's factual reliability can be examined much more finely than with a single label.

## Key points
- The corpus has 4,000 items: 1,789 hallucinated responses, 2,211 non-hallucinated ones, and 2,419 annotated erroneous spans. It is released on Hugging Face as `Bekhouche/HalluTruthQA-4K`.
- Each item contains a question, a Fanar-1-9B-Instruct response, a verified answer, and six options (one correct plus five plausible distractors). Hallucinated items add character-offset spans, a human explanation, and a hierarchical error type.
- HalluScoring 2026 Track 2.1 uses the Islamic portion, and Track 2.2 uses history, science and geography.
- Hallucinated responses average 36.15 words, compared with 26.79 for correct ones. The hallucination rate climbs from 20.2% for answers of five words or fewer to 59.3% for answers of 16–30 words. Questions longer than 20 words reach 77.7%, although that group is small.
- The authors list three limitations: only one generating model was used, only four domains are covered, and multiple-choice checking does not test open-ended correction.

## Interesting ideas
- Four annotation layers sit on one instance (is it wrong, where, why, what is right), so one item can tell apart a system that detects errors from one that can locate or fix them.
- Erroneous content is marked with exact character offsets around just the wrong segment, rather than labelling a whole quotation or sentence, which allows precise scoring of localization.
- Each question offers five plausible distractors next to the free-form answer, which separates knowing the fact from producing it fluently.
- Error rates are broken down by response and question length with bin counts attached, and the authors explicitly treat the trends as associations because the largest bin is sparse.
- Verification needs are spelled out per domain: chronology for history, numbers and definitions for science, borders and spatial relations for geography, and source attribution for religious texts.

## For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

- **I1** (answer): QA readers on fables and school tests could be scored the same way, checking the yes/no verdict, the proposition responsible, the justification, and the corrected answer separately.
- **I2** (back): when the text → graph → text compressor regenerates text, offsets of segments that disagree with the source graph could be logged to show exactly where meaning was lost.
- **I3** (answer): pairing each open question in our maths and fable tests with distractor options would show whether failures come from missing world knowledge or from answer formulation.
- **I4** (learn): parser and reader evaluations could report accuracy per sentence-length bin with n shown, so that small bins are not over-interpreted.
- **I5** (world): fact checks in the world model could be typed by fact kind, with different consistency tests for dates, measures, places and quoted sources.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

Fields: [Linguistics & languages](https://md.research.elamur.ai/free/en/shelves/linguistics/)

Topics: [🐪 Semitic languages](https://md.research.elamur.ai/free/en/topics/semitic-languages/)

## Institutions

- [University of the Basque Country](https://md.research.elamur.ai/free/en/universities/university-of-the-basque-country/) — University, Leioa, Spain · 34 works · 14,069 citations
- [Hamad bin Khalifa University](https://md.research.elamur.ai/free/en/universities/hamad-bin-khalifa-university/) — University, Doha, Qatar · 58 works · 11,454 citations

Also on the byline: [Universiti Malaysia Kelantan](https://ror.org/0463y2v87) · [University of Biskra](https://ror.org/05fr5y859).

## Cite

Cite as: Salah Eddine Bekhouche; Abdessalam Bouchekif; Hichem Telli; Mohammed-En-Nadhir Zighem; and Abdenour Hadid. 2026. HalluTruthQA-4K: A Fine-Grained Corpus and Annotation Process for Arabic Hallucination Detection and Truth Verification. arXiv:2608.03966.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@misc{zighem2026hallutruthqa,
  title = {{HalluTruthQA-4K: A Fine-Grained Corpus and Annotation Process for Arabic Hallucination Detection and Truth Verification}},
  author = {Salah Eddine Bekhouche; Abdessalam Bouchekif; Hichem Telli; Mohammed-En-Nadhir Zighem; and Abdenour Hadid},
  year = {2026},
  doi = {10.48550/arxiv.2608.03966},
  eprint = {2608.03966},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2608.03966}
}
```

- [← previous Gretino: A Greek and Latin Dataset to Benchmark Retrieval Systems in Classical Languages](https://md.research.elamur.ai/free/en/papers/toyin-2026-gretino-greek-latin-dataset-benchmark/)
- [next → Harf-Speech: A Clinically Aligned Framework for Arabic Phoneme-Level Speech Assessment](https://md.research.elamur.ai/free/en/papers/azad-2026-harf-speech-clinically-aligned-framework/)
- [↑ back to papers of 2026](https://md.research.elamur.ai/free/en/papers/2026/page/2/)

## References

27 references, 10 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Samir Abdaljalil, Parichit Sharma, Erchin Serpedin, and Hasan Kurban. Halluverse-M 3 : A multitask multilingual benchmark for hallucination in LLMs. arXiv preprint arXiv:2602.06920, 2026. [↗](https://arxiv.org/abs/2602.06920)
2. Ali Abdelaal, Mohammed Nader Al Haffar, Mahmoud Fawzi, and Walid Magdy. IslamicMMLU: A benchmark for evaluating LLMs on islamic knowledge. arXiv preprint arXiv:2603.23750, 2026. [↗](https://arxiv.org/abs/2603.23750)
3. Aisha Alansari and Hamzah Luqman. AraHalluEval: A fine-grained hallucination evaluation framework for Arabic LLMs. In Proceedings of The Second Arabic Natural Language Processing Conference, pages 148–161, Suzhou, China, November 2025. Association for Computational Linguistics. doi: 10.18653/v1/2025.arabicnlp-main.12. URL https://aclanthology.org/ 2025.arabicnlp-main.12/. [↗](https://doi.org/10.18653/v1/2025.arabicnlp-main.12)
4. Aisha Alansari and Hamzah Luqman. HalluScore: Large language model hallucination question answering benchmark. arXiv preprint arXiv:2605.17007, 2026a. [↗](https://arxiv.org/abs/2605.17007) [our card](https://md.research.elamur.ai/free/en/papers/alansari-2026-halluscore-large-language-model-hallucination/)
5. Aisha Alansari and Hamzah Luqman. Large language models hallucination: A comprehensive survey. arXiv preprint arXiv:2510.06265, 2026b. URL https://arxiv.org/abs/2510.06265. [↗](https://arxiv.org/abs/2510.06265)
6. Fakhraddin Alwajih, Abdellah El Mekki, Hamdy Mubarak, Majd Hawasly, Abubakr Mohamed, and Muhammad Abdul-Mageed. PalmX 2025: The first shared task on benchmarking LLMs on arabic and islamic culture. In Proceedings of The Third Arabic Natural Language Processing Conference: Shared Tasks, pages 774–789, 2025. [find](https://scholar.google.com/scholar?q=Fakhraddin+Alwajih%2C+Abdellah+El+Mekki%2C+Hamdy+Mubarak%2C+Majd+Hawasly%2C+Abubakr+Mohamed%2C+and+Muhammad+Abdul%2DMageed%2E+PalmX+2025%3A+The+first+shared+task+on+benchmarking+LLMs+on+arabic+and+islamic+culture%2E+In)
7. Gagan Bhatia, Hamdy Mubarak, Mustafa Jarrar, George Mikros, Fadi Zaraket, Mahmoud Alhirthani, Mutaz Al-Khatib, Logan Cochrane, Kareem Darwish, Rashid Yahiaoui, and Firoj Alam. From RAG to agentic RAG for faithful islamic question answering. arXiv preprint arXiv:2601.07528, 2026. [↗](https://arxiv.org/abs/2601.07528)
8. Abdessalam Bouchekif, Samer Rashwani, Emad Soliman Ali Mohamed, Mutaz Alkhatib, Heba Sbahi, Shahd Gaben, Wajdi Zaghouani, Aiman Erbad, and Mohammed Ghaly. QIAS 2025: Overview of the shared task on islamic inheritance reasoning and knowledge assessment. In Proceedings of The Third Arabic Natural Language Processing Conference: Shared Tasks, pages 851–860, Suzhou, China, November 2025. Association for Computational Linguistics. ISBN 979-8-89176-356-2. doi: 10.18653/v1/2025.arabicnlp-sharedtasks.117. URL https://aclanthology.org/2025.arabicnlp-sharedtasks.117/. [↗](https://doi.org/10.18653/v1/2025.arabicnlp-sharedtasks.117)
9. Abdessalam Bouchekif, Somaya Eltanbouly, Samer Rashwani, Shahd Gaben, Mutaz Al-Khatib, Heba Sbahi, Emad Mohamed, and Mohammed Ghaly. QIAS 2026: Overview of the shared task on islamic inheritance reasoning. arXiv preprint arXiv:2606.13756, 2026a. [↗](https://arxiv.org/abs/2606.13756)
10. Abdessalam Bouchekif, Shahd Gaben, Samer Rashwani, Somaya Eltanbouly, Mutaz Al-Khatib, Heba Sbahi, Mohammed Ghaly, and Emad Mohamed. MAWARITH: A dataset and benchmark for legal inheritance reasoning with LLMs. arXiv preprint arXiv:2603.07539, 2026b. [↗](https://arxiv.org/abs/2603.07539) [our card](https://md.research.elamur.ai/free/en/papers/bouchekif-2026-mawarith-dataset-benchmark-legal-inheritance/)
11. Abdessalam Bouchekif, Mohammed-En-Nadhir Zighem, Salah Eddine Bekhouche, Hichem Telli, Somaya Eltanbouly, Shahd Gaben, Heba Sbahi, Samer Rashwani, Mutaz Al-Khatib, Emad Mohamed, et al. HalluTruthQA: A fine-grained benchmark for hallucination detection, localization, and explanation in arabic question answering. arXiv preprint arXiv:2607.20219, 2026c. [↗](https://arxiv.org/abs/2607.20219)
12. Shiqi Chen, Yiran Zhao, Jinghan Zhang, I-Chun Chern, Siyang Gao, Pengfei Liu, and Junxian He. FELM: Benchmarking factuality evaluation of large language models. In Thirty-seventh Conference on Neural Information Processing Systems Datasets and Benchmarks Track, 2023. URL https://arxiv.org/abs/2310.00741. [↗](https://arxiv.org/abs/2310.00741)
13. Passant Elchafei and Mervat Abu-Elkheir. Hallucination detectives at SemEval-2025 task 3: Span-level hallucination detection for LLM-generated answers. In Proceedings of the 19th International Workshop on Semantic Evaluation (SemEval-2025), pages 601–606, Vienna, Austria, July 2025. Association for Computational Linguistics. URL https://aclanthology.org/2025.semeval-1.84/. [↗](https://aclanthology.org/2025.semeval-1.84/)
14. Fanar Team, Ummar Abbas, Mohammad Shahmeer Ahmad, Firoj Alam, Enes Altinisik, Ehsannedin Asgari, Yazan Boshmaf, Sabri Boughorbel, Sanjay Chawla, Shammur Chowdhury, Fahim Dalvi, Kareem Darwish, Nadir Durrani, Mohamed Elfeky, Ahmed Elmagarmid, Mohamed Eltabakh, Masoomali Fatehkia, Anastasios Fragkopoulos, Maram Hasanain, Majd Hawasly, Mus’ab Husaini, Soon-Gyo Jung, Ji Kim Lucas, Walid Magdy, Safa Messaoud, Abubakr Mohamed, Tasnim Mohiuddin, Basel Mousi, Hamdy Mubarak, Ahmad Musleh, Zan Naeem, Mourad Ouzzani, Dorde Popovic, Amin Sadeghi, Husrev Taha Sencar, Mohammed Shinoy, Omar Sinan, Yifan Zhang, Ahmed Ali, Yassine El Kheir, Xiaosong Ma, and Chaoyi Ruan. Fanar: An arabic-centric multimodal generative AI platform, 2025. URL https://arxiv.org/abs/2501.13944. [↗](https://arxiv.org/abs/2501.13944) [our card](https://md.research.elamur.ai/free/en/papers/abbas-2025-fanar-arabic-centric-multimodal-generative/)
15. Shahd Gaben, Heba Sbahi, Samer Rashwani, Abdessalam Bouchekif, Mutaz Al-Khatib, Emad Mohamed, Somaya Eltanbouly, and Mohammed Ghaly. Islamicturathbench: A multi-task, multi-discipline benchmark for evaluating large language models on the islamic scholarly tradition (turath). arXiv preprint arXiv:2608.04703, 2026. [↗](https://arxiv.org/abs/2608.04703)
16. Lei Huang, Weijiang Yu, Weitao Ma, Weihong Zhong, Zhangyin Feng, Haotian Wang, Qianglong Chen, Weihua Peng, Xiaocheng Feng, Bing Qin, and Ting Liu. A survey on hallucination in large language models: Principles, taxonomy, challenges, and open questions. ACM Transactions on Information Systems, 43(2), 2025a. doi: 10.1145/3703155. [↗](https://doi.org/10.1145/3703155) [our card](https://md.research.elamur.ai/free/en/papers/huang-2023-survey-hallucination-large-language-models/)
17. Lei Huang, Weijiang Yu, Weitao Ma, Weihong Zhong, Zhangyin Feng, Haotian Wang, Qianglong Chen, Weihua Peng, Xiaocheng Feng, Bing Qin, et al. A survey on hallucination in large language models: Principles, taxonomy, challenges, and open questions. ACM Transactions on Information Systems, 43(2):1–55, 2025b. [our card](https://md.research.elamur.ai/free/en/papers/huang-2023-survey-hallucination-large-language-models/)
18. Ziwei Ji, Nayeon Lee, Rita Frieske, Tiezheng Yu, Dan Su, Yan Xu, Etsuko Ishii, Ye Jin Bang, Andrea Madotto, and Pascale Fung. Survey of hallucination in natural language generation. ACM Computing Surveys, 55(12):1–38, 2023. doi: 10.1145/3571730. [↗](https://doi.org/10.1145/3571730) [our card](https://md.research.elamur.ai/free/en/papers/ji-2022-survey-hallucination-natural-language-generation/)
19. Junyi Li, Xiaoxue Cheng, Xin Zhao, Jian-Yun Nie, and Ji-Rong Wen. HaluEval: A large-scale hallucination evaluation benchmark for large language models. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 6449–6464, Singapore, December 2023. Association for Computational Linguistics. doi: 10.18653/v1/ 2023.emnlp-main.397. URL https://aclanthology.org/2023.emnlp-main.397/. [↗](https://aclanthology.org/2023.emnlp-main.397/) [our card](https://md.research.elamur.ai/free/en/papers/li-2023-halueval-large-scale-hallucination-evaluation/)
20. Stephanie Lin, Jacob Hilton, and Owain Evans. TruthfulQA: Measuring how models mimic human falsehoods. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 3214–3252, Dublin, Ireland, May 2022. Association for Computational Linguistics. doi: 10.18653/v1/2022.acl-long.229. URL https://aclanthology.org/2022.acl-long.229/. [↗](https://doi.org/10.18653/v1/2022.acl-long.229) [our card](https://md.research.elamur.ai/free/en/papers/lin-2022-truthfulqa-measuring-how-models-mimic/)
21. Sewon Min, Kalpesh Krishna, Xinxi Lyu, Mike Lewis, Wen-tau Yih, Pang Koh, Mohit Iyyer, Luke Zettlemoyer, and Hannaneh Hajishirzi. FActScore: Fine-grained atomic evaluation of factual precision in long form text generation. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 12076–12100, Singapore, December 2023. Association for Computational Linguistics. doi: 10.18653/v1/2023.emnlp-main.741. URL https://aclanthology.org/2023.emnlp-main.741/. [↗](https://doi.org/10.18653/v1/2023.emnlp-main.741) [our card](https://md.research.elamur.ai/free/en/papers/min-2023-factscore-fine-grained-atomic-evaluation/)
22. Abhika Mishra, Akari Asai, Vidhisha Balachandran, Yizhong Wang, Graham Neubig, Yulia Tsvetkov, and Hannaneh Hajishirzi. Fine-grained hallucination detection and editing for language models. arXiv preprint arXiv:2401.06855, 2024. [↗](https://arxiv.org/abs/2401.06855)
23. Marryam Yahya Mohammed, Sama Ayman Ali, Salma Khaled Ali, Ayad Abdul Majeed, and Ensaf Hussein Mohamed. Aftina: Enhancing stability and preventing hallucination in AI-based islamic fatwa generation using LLMs and RAG. Neural Computing and Applications, 37(25):20957–20982, 2025. [find](https://scholar.google.com/scholar?q=Marryam+Yahya+Mohammed%2C+Sama+Ayman+Ali%2C+Salma+Khaled+Ali%2C+Ayad+Abdul+Majeed%2C+and+Ensaf+Hussein+Mohamed%2E+Aftina%3A+Enhancing+stability+and+preventing+hallucination+in+AI%2Dbased+islamic+fatwa+generation+us)
24. Hamdy Mubarak, Hend Al-Khalifa, and Khaloud Suliman Alkhalefah. Halwasa: Quantify and analyze hallucinations in large language models: Arabic as a case study. In Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024), pages 8008–8015, Torino, Italia, May 2024. ELRA and ICCL. URL https://aclanthology.org/2024.lrec-main.705/. [↗](https://aclanthology.org/2024.lrec-main.705/)
25. Hamdy Mubarak, Rana Malhas, Watheq Mansour, Abubakr Mohamed, Mahmoud Fawzi, Majd Hawasly, Tamer Elsayed, Kareem Mohamed Darwish, and Walid Magdy. IslamicEval 2025: The first shared task of capturing LLMs hallucination in islamic content. In Proceedings of The Third Arabic Natural Language Processing Conference: Shared Tasks, pages 480–493, 2025. [find](https://scholar.google.com/scholar?q=Hamdy+Mubarak%2C+Rana+Malhas%2C+Watheq+Mansour%2C+Abubakr+Mohamed%2C+Mahmoud+Fawzi%2C+Majd+Hawasly%2C+Tamer+Elsayed%2C+Kareem+Mohamed+Darwish%2C+and+Walid+Magdy%2E+IslamicEval+2025%3A+The+first+shared+task+of+capturing+L)
26. Cheng Niu, Yuanhao Wu, Juno Zhu, Siliang Xu, KaShun Shum, Randy Zhong, Juntong Song, and Tong Zhang. RAGTruth: A hallucination corpus for developing trustworthy retrieval-augmented language models. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 10862–10878, Bangkok, Thailand, August 2024. Association for Computational Linguistics. doi: 10.18653/v1/2024.acl-long.585. URL https://aclanthology.org/2024.acl-long.585/. [↗](https://doi.org/10.18653/v1/2024.acl-long.585) [our card](https://md.research.elamur.ai/free/en/papers/niu-2024-ragtruth-hallucination-corpus-developing-trustworthy/)
27. Raul Vazquez, Timothee Mickus, Elaine Zosa, Teemu Vahtola, Jörg Tiedemann, Aman Sinha, Vincent Segonne, Fernando Sanchez-Vega, Alessandro Raganato, Jindřich Libovický, Jussi Karlgren, Shaoxiong Ji, Jindřich Helcl, Liane Guillou, Ona De Gibert, Jaione Bengoetxea, Joseph Apidianaki, and Marianna Apidianaki. SemEval-2025 task 3: Mu-SHROOM, the multilingual shared-task on hallucinations and related observable overgeneration mistakes. In Proceedings of the 19th International Workshop on Semantic Evaluation (SemEval-2025), pages 2472–2497, Vienna, Austria, July 2025. Association for Computational Linguistics. URL https://aclanthology.org/2025.semeval-1.322/. [↗](https://aclanthology.org/2025.semeval-1.322/)