---
title: "Sacred or Synthetic? Evaluating LLM Reliability — Atif 2025"
description: "People increasingly ask chatbots about Islamic law, yet no benchmark had tested whether LLMs answer such questions correctly for a specific Sunni legal…"
authors: ["Farah Atif", "Nursultan Askarbekuly", "Kareem Darwish", "Monojit Choudhury"]
year: 2025
venue: "arXiv:2508.08287"
source: "https://arxiv.org/abs/2508.08287"
doi: "10.48550/arxiv.2508.08287"
arxiv: "2508.08287"
licence: "arXiv — non-exclusive license to distribute"
licence_url: "http://arxiv.org/licenses/nonexclusive-distrib/1.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/atif-2025-sacred-synthetic-evaluating-llm-reliability/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/atif-2025-sacred-synthetic-evaluating-llm-reliability/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2025](https://md.research.elamur.ai/free/en/papers/2025/page/2/) / Sacred or Synthetic? Evaluating LLM Reliability and Abstention for Religious Questions

# Sacred or Synthetic? Evaluating LLM Reliability and Abstention for Religious Questions

**Authors:** Farah Atif, [Nursultan Askarbekuly](https://md.research.elamur.ai/free/en/authors/nursultan-askarbekuly/), [Kareem Darwish](https://md.research.elamur.ai/free/en/authors/kareem-darwish/), [Monojit Choudhury](https://md.research.elamur.ai/free/en/authors/monojit-choudhury/) · **Year:** 2025 · **Venue:** arXiv:2508.08287 · **Licence:** [arXiv — non-exclusive license to distribute](http://arxiv.org/licenses/nonexclusive-distrib/1.0/)

**Links:** [Publisher's page](https://arxiv.org/abs/2508.08287) · [PDF](https://arxiv.org/pdf/2508.08287) · [DOI 10.48550/arxiv.2508.08287](https://doi.org/10.48550/arxiv.2508.08287) · [arXiv 2508.08287](https://arxiv.org/abs/2508.08287) · [OpenAlex](https://openalex.org/W4415230710)

**Citations:** 2 citations (OpenAlex, 2026-10-14) · 6 in our library, of the 140884 papers whose bibliographies we read

## Summary
People increasingly ask chatbots about Islamic law, yet no benchmark had tested whether LLMs answer such questions correctly *for a specific Sunni legal school* (madhhab), or whether they decline when unsure. The authors built FiqhQA, a manually curated set of 960 question–answer items covering rulings of the Hanafi, Maliki, Shafi'i and Hanbali schools, in Arabic and English. Rulings come from the Kuwaiti Fiqh Encyclopedia and focus on acts of worship. GPT-4o drafted question–answer pairs and the English translations, and humans then checked them. Models were tested zero-shot and with a prompt that strongly pushes them to say "I don't know" when uncertain.

GPT-4o had the highest accuracy. Gemini and Fanar abstained better, which means fewer confident wrong answers. Every model performed worse in Arabic than in English. The work shows that correctness and knowing when to stay silent are separate abilities that need separate measurement. This matters most in high-stakes domains, where a fluent but wrong answer does real harm.

## Key points
- FiqhQA has 960 QA items across four Sunni madhhabs, in Arabic and English, and is released as `MBZUAI/FiqhQA` on Hugging Face.
- The source is the Kuwaiti Fiqh Encyclopedia, limited to Fiqh al-'Ibadat: Prayer, Fasting, Purification, Marriage and divorce, Almsgiving, and Pilgrimage. Inclusion criteria were set with a domain expert.
- Each item records whether the schools agree or disagree. It also gives a separate gold answer per school.
- The abstention prompt casts the model as a scholar and states that errors are very costly. It asks for "I Don't Know" at any doubt and requests a short, school-specific answer without scriptural proofs.
- GPT-4o led on accuracy, and Gemini and Fanar led on abstention. All models showed a drop in Arabic.
- Known limitation: terms like wajib and fard carry different weights in different schools, and the evaluation ignored this nuance.

## Interesting ideas
- Tag each benchmark item as consensus or contested across authorities, so scores on easy shared cases can be reported apart from the genuinely divergent ones. — For Mova (learn): Silver data could carry a flag showing whether witness parsers agreed, and evaluation could report accuracy on contested sentences separately instead of letting easy ones inflate the score.
- Treat "I don't know" as its own outcome next to right and wrong, so a confident error is penalised differently from an honest refusal. — For Mova (answer): Fable and school-test readers could return "unknown" when the world of the text holds no supporting fact, and evaluation would count wrong answers and abstentions separately.
- Ask one question under different stated perspectives, each with its own correct answer, to test whether a system answers relative to a source rather than globally. — For Mova (world): The world model could store claims attributed to holders ("X held that P"), so questions like "according to X" are answered from that holder's beliefs rather than from flat facts.
- Notice that the same label can carry different strength in different traditions, which quietly corrupts any comparison that assumes one shared meaning. — For Mova (tree): When converting between UD dialects or treebanks, Mova could keep an explicit mapping table for labels that look identical but are used differently, rather than merging them silently.
- Build the benchmark in two languages from one source so that a language-specific performance drop shows up directly. — For Mova (back): Translation through the meta-world could be checked on parallel items, comparing whether the graph extracted from each side yields the same answers.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

Fields: [Linguistics & languages](https://md.research.elamur.ai/free/en/shelves/linguistics/)

Topics: [Language models & pretraining](https://md.research.elamur.ai/free/en/topics/language-models/) · [Evaluation & benchmarks](https://md.research.elamur.ai/free/en/topics/evaluation-benchmarks/) · [Semitic languages](https://md.research.elamur.ai/free/en/topics/semitic-languages/)

## Institutions

- [Hamad bin Khalifa University](https://md.research.elamur.ai/free/en/universities/hamad-bin-khalifa-university/) — University, Doha, Qatar · 58 works · 11 454 citations
- [Mohamed bin Zayed University of Artificial Intelligence](https://md.research.elamur.ai/free/en/universities/mohamed-bin-zayed-university-of-artificial-intelligence/) — University, Abu Dhabi, United Arab Emirates · 33 works · 3 016 citations
- [Innopolis University](https://md.research.elamur.ai/free/en/universities/innopolis-university/) — University, Innopolis, Russia · 3 works · 10 citations

## Cite

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@misc{atif2025sacred,
  title = {{Sacred or Synthetic? Evaluating LLM Reliability and Abstention for Religious Questions}},
  author = {Atif, Farah and Askarbekuly, Nursultan and Darwish, Kareem and Choudhury, Monojit},
  year = {2025},
  doi = {10.48550/arxiv.2508.08287},
  eprint = {2508.08287},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2508.08287}
}
```

- [← previous From Guidelines to Practice: A New Paradigm for Arabic Language Model Evaluation](https://md.research.elamur.ai/free/en/papers/sibaee-2025-guidelines-practice-new-paradigm-arabic/)
- [next → Survey of Abstract Meaning Representation: Then, Now, Future](https://md.research.elamur.ai/free/en/papers/mansouri-2025-survey-abstract-meaning-representation-then/)
- [↑ back to papers of 2025](https://md.research.elamur.ai/free/en/papers/2025/page/2/)

## References

33 references, 14 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Abdelnasser, H.; Ragab, M.; Mohamed, R.; Mohamed, A.; Farouk, B.; El-Makky, N. M.; and Torki, M. 2014. Al-Bayan: an Arabic question answering system for the Holy Quran. In Proceedings of the EMNLP 2014 Workshop on Arabic Natural Language Processing (ANLP), 57–64. [find](https://scholar.google.com/scholar?q=Abdelnasser%2C+H%2E%3B+Ragab%2C+M%2E%3B+Mohamed%2C+R%2E%3B+Mohamed%2C+A%2E%3B+Farouk%2C+B%2E%3B+El%2DMakky%2C+N%2E+M%2E%3B+and+Torki%2C+M%2E+2014%2E+Al%2DBayan%3A+an+Arabic+question+answering+system+for+the+Holy+Quran%2E+In+Proceedings+of+the+EMNLP+201)
2. Ahdritz, G.; Qin, T.; Vyas, N.; Barak, B.; and Edelman, B. L. 2024. Distinguishing the knowable from the unknowable with language models. arXiv preprint arXiv:2402.03563. [↗](https://arxiv.org/abs/2402.03563)
3. Alqahtani, M.; and Atwell, E. 2018. Annotated corpus of arabic al-quran question and answer. [find](https://scholar.google.com/scholar?q=Alqahtani%2C+M%2E%3B+and+Atwell%2C+E%2E+2018%2E+Annotated+corpus+of+arabic+al%2Dquran+question+and+answer%2E)
4. Bai, G.; Liu, J.; Bu, X.; He, Y.; Liu, J.; Zhou, Z.; Lin, Z.; Su, W.; Ge, T.; Zheng, B.; et al. 2024. Mt-bench-101: A fine-grained benchmark for evaluating large language models in multi-turn dialogues. arXiv preprint arXiv:2402.14762. [↗](https://arxiv.org/abs/2402.14762) [our card](https://md.research.elamur.ai/free/en/papers/bai-2024-mt-bench-101-fine-grained/)
5. Bari, M. S.; Alnumay, Y.; Alzahrani, N. A.; Alotaibi, N. M.; Alyahya, H. A.; AlRashed, S.; Mirza, F. A.; Alsubaie, S. Z.; Alahmed, H. A.; Alabduljabbar, G.; et al. 2024. Allam: Large language models for arabic and english. arXiv preprint arXiv:2407.15390. [↗](https://arxiv.org/abs/2407.15390) [our card](https://md.research.elamur.ai/free/en/papers/bari-2024-allam-large-language-models-arabic/)
6. Dang, J.; Singh, S.; D’souza, D.; Ahmadian, A.; Salamanca, A.; Smith, M.; Peppin, A.; Hong, S.; Govindassamy, M.; Zhao, T.; et al. 2024. Aya expanse: Combining research breakthroughs for a new multilingual frontier. arXiv preprint arXiv:2412.04261. [↗](https://arxiv.org/abs/2412.04261) [our card](https://md.research.elamur.ai/free/en/papers/dang-2024-aya-expanse-combining-research-breakthroughs/)
7. Fernández, V. M. C.; de Mendonça, J. C. T.; Matteo, M. A.; and Tighe, M. R. P. 2025. Antiqua et Nova: Note on the Relationship Between Artificial Intelligence and Human Intelligence. Official note by the Dicastery for the Doctrine of the Faith and the Dicastery for Culture and Education. Issued in Rome on the Liturgical Memorial of Saint Thomas Aquinas, Doctor of the Church. [find](https://scholar.google.com/scholar?q=Ferna%CC%81ndez%2C+V%2E+M%2E+C%2E%3B+de+Mendonc%CC%A7a%2C+J%2E+C%2E+T%2E%3B+Matteo%2C+M%2E+A%2E%3B+and+Tighe%2C+M%2E+R%2E+P%2E+2025%2E+Antiqua+et+Nova%3A+Note+on+the+Relationship+Between+Artificial+Intelligence+and+Human+Intelligence%2E+Official+note+b)
8. Hurst, A.; Lerer, A.; Goucher, A. P.; Perelman, A.; Ramesh, A.; Clark, A.; Ostrow, A.; Welihinda, A.; Hayes, A.; Radford, A.; et al. 2024. Gpt-4o system card. arXiv preprint arXiv:2410.21276. [↗](https://arxiv.org/abs/2410.21276) [our card](https://md.research.elamur.ai/free/en/papers/openai-2024-gpt-4o-system-card/)
9. Kasai, J.; Sakaguchi, K.; Le Bras, R.; Asai, A.; Yu, X.; Radev, D.; Smith, N. A.; Choi, Y.; Inui, K.; et al. 2023. Realtime qa: What’s the answer right now? Advances in neural information processing systems, 36: 49025–49043. [find](https://scholar.google.com/scholar?q=Kasai%2C+J%2E%3B+Sakaguchi%2C+K%2E%3B+Le+Bras%2C+R%2E%3B+Asai%2C+A%2E%3B+Yu%2C+X%2E%3B+Radev%2C+D%2E%3B+Smith%2C+N%2E+A%2E%3B+Choi%2C+Y%2E%3B+Inui%2C+K%2E%3B+et+al%2E+2023%2E+Realtime+qa%3A+What%E2%80%99s+the+answer+right+now%3F+Advances+in+neural+information+processing+s) [our card](https://md.research.elamur.ai/free/en/papers/kasai-2022-realtime-qa-s-answer-right/)
10. Kim, M.; and Thorne, J. 2024. Epistemology of Language Models: Do Language Models Have Holistic Knowledge? arXiv preprint arXiv:2403.12862. [↗](https://arxiv.org/abs/2403.12862)
11. Kojima, T.; Gu, S. S.; Reid, M.; Matsuo, Y.; and Iwasawa, Y. 2022. Large language models are zero-shot reasoners. Advances in neural information processing systems, 35: 22199–22213. [find](https://scholar.google.com/scholar?q=Kojima%2C+T%2E%3B+Gu%2C+S%2E+S%2E%3B+Reid%2C+M%2E%3B+Matsuo%2C+Y%2E%3B+and+Iwasawa%2C+Y%2E+2022%2E+Large+language+models+are+zero%2Dshot+reasoners%2E+Advances+in+neural+information+processing+systems%2C+35%3A+22199%E2%80%9322213%2E) [our card](https://md.research.elamur.ai/free/en/papers/kojima-2022-large-language-models-zero-shot/)
12. Koto, F.; Li, H.; Shatnawi, S.; Doughman, J.; Sadallah, A. B.; Alraeesi, A.; Almubarak, K.; Alyafeai, Z.; Sengupta, N.; Shehata, S.; et al. 2024. Arabicmmlu: Assessing massive multitask language understanding in arabic. arXiv preprint arXiv:2402.12840. [↗](https://arxiv.org/abs/2402.12840) [our card](https://md.research.elamur.ai/free/en/papers/koto-2024-arabicmmlu-assessing-massive-multitask-language/)
13. Lin, S.; Hilton, J.; and Evans, O. 2021. Truthfulqa: Measuring how models mimic human falsehoods. arXiv preprint arXiv:2109.07958. [↗](https://arxiv.org/abs/2109.07958) [our card](https://md.research.elamur.ai/free/en/papers/lin-2022-truthfulqa-measuring-how-models-mimic/)
14. Liu, Y.; Iter, D.; Xu, Y.; Wang, S.; Xu, R.; and Zhu, C. 2023. G-eval: NLG evaluation using gpt-4 with better human alignment. arXiv preprint arXiv:2303.16634. [↗](https://arxiv.org/abs/2303.16634)
15. Madhusudhan, N.; Madhusudhan, S. T.; Yadav, V.; and Hashemi, M. 2024. Do llms know when to not answer? investigating abstention abilities of large language models. arXiv preprint arXiv:2407.16221. [↗](https://arxiv.org/abs/2407.16221)
16. Malhas, R.; and Elsayed, T. 2020. Ayatec: building a reusable verse-based test collection for arabic question answering on the holy qur’an. ACM Transactions on Asian and Low-Resource Language Information Processing (TAL-LIP), 19(6): 1–21. [find](https://scholar.google.com/scholar?q=Malhas%2C+R%2E%3B+and+Elsayed%2C+T%2E+2020%2E+Ayatec%3A+building+a+reusable+verse%2Dbased+test+collection+for+arabic+question+answering+on+the+holy+qur%E2%80%99an%2E+ACM+Transactions+on+Asian+and+Low%2DResource+Language+Informat)
17. Malhas, R.; Mansour, W.; and Elsayed, T. 2023. Qur‘an QA 2023 Shared Task: Overview of Passage Retrieval and Reading Comprehension Tasks over the Holy Qur‘an. In Sawaf, H.; El-Beltagy, S.; Zaghouani, W.; Magdy, W.; Abdelali, A.; Tomeh, N.; Abu Farha, I.; Habash, N.; Khalifa, S.; Keleg, A.; Haddad, H.; Zitouni, I.; Mrini, K.; and Almatham, R., eds., Proceedings of ArabicNLP 2023, 690–701. Singapore (Hybrid): Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Malhas%2C+R%2E%3B+Mansour%2C+W%2E%3B+and+Elsayed%2C+T%2E+2023%2E+Qur%E2%80%98an+QA+2023+Shared+Task%3A+Overview+of+Passage+Retrieval+and+Reading+Comprehension+Tasks+over+the+Holy+Qur%E2%80%98an%2E+In+Sawaf%2C+H%2E%3B+El%2DBeltagy%2C+S%2E%3B+Zaghouani%2C+)
18. Mohammed, M.; Amin, S.; and Aref, M. M. 2022. An english islamic articles dataset (eiad) for developing an islam-bot question answering chatbot. In 2022 5th International Conference on Computing and Informatics (ICCI), 303–309. IEEE. [find](https://scholar.google.com/scholar?q=Mohammed%2C+M%2E%3B+Amin%2C+S%2E%3B+and+Aref%2C+M%2E+M%2E+2022%2E+An+english+islamic+articles+dataset+%28eiad%29+for+developing+an+islam%2Dbot+question+answering+chatbot%2E+In+2022+5th+International+Conference+on+Computing+and+I)
19. Niu, C.; Wu, Y.; Zhu, J.; Xu, S.; Shum, K.; Zhong, R.; Song, J.; and Zhang, T. 2023. Ragtruth: A hallucination corpus for developing trustworthy retrieval-augmented language models. arXiv preprint arXiv:2401.00396. [↗](https://arxiv.org/abs/2401.00396) [our card](https://md.research.elamur.ai/free/en/papers/niu-2024-ragtruth-hallucination-corpus-developing-trustworthy/)
20. Patel, S.; Kane, H.; and Patel, R. 2023. Building Domain-Specific LLMs Faithful To The Islamic Worldview: Mirage or Technical Possibility? arXiv preprint arXiv:2312.06652. [↗](https://arxiv.org/abs/2312.06652)
21. Qamar, F.; Latif, S.; and Latif, R. 2024. A Benchmark Dataset with Larger Context for Non-Factoid Question Answering over Islamic Text. arXiv preprint arXiv:2409.09844. [↗](https://arxiv.org/abs/2409.09844)
22. Rajpurkar, P.; Zhang, J.; Lopyrev, K.; and Liang, P. 2016. Squad: 100,000+ questions for machine comprehension of text. arXiv preprint arXiv:1606.05250. [↗](https://arxiv.org/abs/1606.05250) [our card](https://md.research.elamur.ai/free/en/papers/rajpurkar-2016-squad-100-000-questions-machine/)
23. Rizqullah, M. R.; Purwarianti, A.; and Aji, A. F. 2023. QASiNa: Religious Domain Question Answering Using Sirah Nabawiyah. In 2023 10th International Conference on Advanced Informatics: Concept, Theory and Application (ICAICTA), 1–6. IEEE. [find](https://scholar.google.com/scholar?q=Rizqullah%2C+M%2E+R%2E%3B+Purwarianti%2C+A%2E%3B+and+Aji%2C+A%2E+F%2E+2023%2E+QASiNa%3A+Religious+Domain+Question+Answering+Using+Sirah+Nabawiyah%2E+In+2023+10th+International+Conference+on+Advanced+Informatics%3A+Concept%2C+Theor)
24. Röttger, P.; Kirk, H.; Vidgen, B.; Attanasio, G.; Bianchi, F.; and Hovy, D. 2024. XSTest: A Test Suite for Identifying Exaggerated Safety Behaviours in Large Language Models. In Duh, K.; Gomez, H.; and Bethard, S., eds., Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), 5377–5400. Mexico City, Mexico: Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Ro%CC%88ttger%2C+P%2E%3B+Kirk%2C+H%2E%3B+Vidgen%2C+B%2E%3B+Attanasio%2C+G%2E%3B+Bianchi%2C+F%2E%3B+and+Hovy%2C+D%2E+2024%2E+XSTest%3A+A+Test+Suite+for+Identifying+Exaggerated+Safety+Behaviours+in+Large+Language+Models%2E+In+Duh%2C+K%2E%3B+Gomez%2C+H%2E%3B+a) [our card](https://md.research.elamur.ai/free/en/papers/rottger-2023-xstest-test-suite-identifying-exaggerated/)
25. Song, J.; Wang, X.; Zhu, J.; Wu, Y.; Cheng, X.; Zhong, R.; and Niu, C. 2024. RAG-HAT: A Hallucination-Aware Tuning Pipeline for LLM in Retrieval-Augmented Generation. [find](https://scholar.google.com/scholar?q=Song%2C+J%2E%3B+Wang%2C+X%2E%3B+Zhu%2C+J%2E%3B+Wu%2C+Y%2E%3B+Cheng%2C+X%2E%3B+Zhong%2C+R%2E%3B+and+Niu%2C+C%2E+2024%2E+RAG%2DHAT%3A+A+Hallucination%2DAware+Tuning+Pipeline+for+LLM+in+Retrieval%2DAugmented+Generation%2E)
26. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing: Industry Track, 1548– 1558. [find](https://scholar.google.com/scholar?q=In+Proceedings+of+the+2024+Conference+on+Empirical+Methods+in+Natural+Language+Processing%3A+Industry+Track%2C+1548%E2%80%93+1558%2E)
27. Team, F.; Abbas, U.; Ahmad, M. S.; Alam, F.; Altinisik, E.; Asgari, E.; Boshmaf, Y.; Boughorbel, S.; Chawla, S.; Chowdhury, S.; et al. 2025. Fanar: An Arabic-Centric Multimodal Generative AI Platform. arXiv preprint arXiv:2501.13944. [↗](https://arxiv.org/abs/2501.13944) [our card](https://md.research.elamur.ai/free/en/papers/abbas-2025-fanar-arabic-centric-multimodal-generative/)
28. Team, G.; Anil, R.; Borgeaud, S.; Alayrac, J.-B.; Yu, J.; Soricut, R.; Schalkwyk, J.; Dai, A. M.; Hauth, A.; Millican, K.; et al. 2023. Gemini: a family of highly capable multimodal models. arXiv preprint arXiv:2312.11805. [↗](https://arxiv.org/abs/2312.11805) [our card](https://md.research.elamur.ai/free/en/papers/gemini-team-2023-gemini-family-highly-capable-multimodal/)
29. Team, G.; Riviere, M.; Pathak, S.; Sessa, P. G.; Hardin, C.; Bhupatiraju, S.; Hussenot, L.; Mesnard, T.; Shahriari, B.; Ramé, A.; et al. 2024. Gemma 2: Improving open language models at a practical size. arXiv preprint arXiv:2408.00118. [↗](https://arxiv.org/abs/2408.00118)
30. Wiharja, K. R. S.; Murdiansyah, D. T.; Romdlony, M. Z.; Ramdhani, T.; and Gandidi, M. R. 2022. A Questions Answering System on Hadith Knowledge Graph. Journal of ICT Research & Applications, 16(2). [find](https://scholar.google.com/scholar?q=Wiharja%2C+K%2E+R%2E+S%2E%3B+Murdiansyah%2C+D%2E+T%2E%3B+Romdlony%2C+M%2E+Z%2E%3B+Ramdhani%2C+T%2E%3B+and+Gandidi%2C+M%2E+R%2E+2022%2E+A+Questions+Answering+System+on+Hadith+Knowledge+Graph%2E+Journal+of+ICT+Research+%26+Applications%2C+16%282%29%2E)
31. Yuan, W.; Neubig, G.; and Liu, P. 2021. Bartscore: Evaluating generated text as text generation. Advances in neural information processing systems, 34: 27263–27277. [find](https://scholar.google.com/scholar?q=Yuan%2C+W%2E%3B+Neubig%2C+G%2E%3B+and+Liu%2C+P%2E+2021%2E+Bartscore%3A+Evaluating+generated+text+as+text+generation%2E+Advances+in+neural+information+processing+systems%2C+34%3A+27263%E2%80%9327277%2E) [our card](https://md.research.elamur.ai/free/en/papers/yuan-2021-bartscore-evaluating-generated-text-text/)
32. Zhang, T.; Kishore, V.; Wu, F.; Weinberger, K. Q.; and Artzi, Y. 2019. Bertscore: Evaluating text generation with bert. arXiv preprint arXiv:1904.09675. [↗](https://arxiv.org/abs/1904.09675)
33. Zhong, M.; Liu, Y.; Yin, D.; Mao, Y.; Jiao, Y.; Liu, P.; Zhu, C.; Ji, H.; and Han, J. 2022. Towards a unified multidimensional evaluator for text generation. arXiv preprint arXiv:2210.07197. [↗](https://arxiv.org/abs/2210.07197)