---
title: "DebateQA — paper by Xu et al. (2026) — summary and ideas"
description: "The authors build DebateQA, a benchmark of 2,941 debatable questions. 80 references, cited by 6 papers here."
authors: ["Rongwu Xu", "Xuan Qi", "Zehan Qi", "Wei Xu", "Zhijiang Guo"]
year: 2026
venue: "Findings of the Association for Computational Linguistics: EACL 2026"
source: "https://aclanthology.org/2026.findings-eacl.44/"
doi: "10.18653/v1/2026.findings-eacl.44"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/xu-2026-debateqa-evaluating-question-answering-debatable/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/xu-2026-debateqa-evaluating-question-answering-debatable/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2026](https://md.research.elamur.ai/free/en/papers/2026/page/2/) / DebateQA: Evaluating Question Answering on Debatable Knowledge

# DebateQA: Evaluating Question Answering on Debatable Knowledge

**Authors:** [Rongwu Xu](https://md.research.elamur.ai/free/en/authors/rongwu-xu/), [Xuan Qi](https://md.research.elamur.ai/free/en/authors/xuan-qi/), [Zehan Qi](https://md.research.elamur.ai/free/en/authors/zehan-qi/), [Wei Xu](https://md.research.elamur.ai/free/en/authors/wei-hong-xu/) and [Zhijiang Guo](https://md.research.elamur.ai/free/en/authors/zhijiang-guo/) · **Year:** 2026 · **Venue:** [Findings of the Association for Computational Linguistics: EACL 2026](https://md.research.elamur.ai/free/en/venues/findings/) ([Findings of the Association for Computational Linguistics 2026](https://md.research.elamur.ai/free/en/venues/findings/2026/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2026.findings-eacl.44/) · [PDF](https://aclanthology.org/2026.findings-eacl.44.pdf) · [DOI 10.18653/v1/2026.findings-eacl.44](https://doi.org/10.18653/v1/2026.findings-eacl.44) · [OpenAlex](https://openalex.org/W7140110514)

**Citations:** 1 citations (OpenAlex, 2026-10-14) · 6 in our library, of the 149,439 papers whose bibliographies we read

## Summary

Standard QA benchmarks assume one correct answer, so they cannot judge how chatbots handle contested questions such as how crop circles form. The authors build DebateQA, a benchmark of 2,941 debatable questions. Each question comes with several human-checked partial answers, and each partial answer covers one viewpoint as a short point-of-view statement (POV) plus a longer explanation (Explan).

The questions were drawn from DELPHI, ConflictingQA and manually collected web sources. Partial answers were produced in three stages:
- retrieving evidence from authoritative domains,
- extracting POVs with GPT-4,
- expanding them into grounded explanations.

Three annotators then reviewed the results.

Two metrics are proposed. Perspective Diversity (P.D.) measures how many viewpoints a response covers. Dispute Awareness (D.A.) checks whether the model signals that the issue is contested. Both agree well with human preferences and stay stable across evaluator backbones.

Testing 12 LLMs and RAG setups showed that models usually recognize controversy but differ widely in how comprehensively they answer. The benchmark offers a reusable way to measure neutrality and breadth in chatbot answers on contested topics.

## Key points

- **Data sources:** 2,281 controversial questions from DELPHI, 434 from ConflictingQA and 1,758 newly collected from the web. After deduplication 3,216 remained, and filtering for document support left 2,982 before the final 2,941.
- **Evidence:** Each retained question is grounded in 3–5 documents from trusted TLDs (e.g., .edu, .org, .gov). Average question–chunk cosine similarity is 0.56.
- **Annotation:** Partial answers were generated with gpt-4-turbo, then reviewed by three annotators. Harmful or hateful viewpoints were removed.
- **Model comparison:** Leading open-source LLMs match or beat some commercial models. RAG is not uniformly helpful but improves closed-source models.
- **Other gains:** Tuning sampling hyperparameters and using task-specific prompts further raise scores.
- **Stated limitations:** Reliance on institutional sources may underrepresent non-Western or marginal views, and new perspectives emerge over time.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [University of Cambridge](https://md.research.elamur.ai/free/en/universities/university-of-cambridge/) — University, Cambridge, United Kingdom · 243 works · 121,396 citations
- [Tsinghua University](https://md.research.elamur.ai/free/en/universities/tsinghua-university/) — University, Beijing, China · 333 works · 87,019 citations

📚 Shelves (6): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 16,039 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 15,891 works · [👁️ Computer Vision and Pattern Recognition](https://md.research.elamur.ai/free/en/shelves/computer-science/computer-vision-and-pattern-recognition/) · 2,869 works · [🗄️ Information Systems](https://md.research.elamur.ai/free/en/shelves/computer-science/information-systems/) · 1,437 works

🧭 Topics (1): [❓ Question answering](https://md.research.elamur.ai/free/en/topics/question-answering/) · 441 works

## Cite

Cite as: Rongwu Xu, Xuan Qi, Zehan Qi, Wei Xu and Zhijiang Guo. 2026. DebateQA: Evaluating Question Answering on Debatable Knowledge. Findings of the Association for Computational Linguistics: EACL 2026.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{xu2026debateqa,
  title = {{DebateQA: Evaluating Question Answering on Debatable Knowledge}},
  author = {Rongwu Xu, Xuan Qi, Zehan Qi, Wei Xu and Zhijiang Guo},
  booktitle = {Findings of the Association for Computational Linguistics: EACL 2026},
  year = {2026},
  doi = {10.18653/v1/2026.findings-eacl.44},
  url = {https://doi.org/10.18653/v1/2026.findings-eacl.44}
}
```

- [← previous CORBA: Contagious Recursive Blocking Attacks on Multi-Agent Systems Based on Large Language Models](https://md.research.elamur.ai/free/en/papers/zhou-2026-corba-contagious-recursive-blocking-attacks/)
- [next → DiagnosisArena: Benchmarking Diagnostic Reasoning for Large Language Models](https://md.research.elamur.ai/free/en/papers/zhu-2026-diagnosisarena-benchmarking-diagnostic-reasoning-large/)
- [↑ back to papers of 2026](https://md.research.elamur.ai/free/en/papers/2026/page/2/)

## References

80 references, 42 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Marah I Abdin, Sam Ade Jacobs, Ammar Ahmad Awan, Jyoti Aneja, Ahmed Awadallah, Hany Awadalla, Nguyen Bach, Amit Bahree, Arash Bakhtiari, Harkirat S. Behl, Alon Benhaim, Misha Bilenko, Johan Bjorck, Sébastien Bubeck, Martin Cai, Caio César Teodoro Mendes, Weizhu Chen, Vishrav Chaudhary, Parul Chopra,… — [Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phone](https://md.research.elamur.ai/free/en/papers/abdin-2024-phi-3-technical-report-highly/)
2. Anthropic. 2024. Introducing Claude 3.5 Sonnet. Anthropic website. [find](https://scholar.google.com/scholar?q=Anthropic%2E+2024%2E+Introducing+Claude+3%2E5+Sonnet%2E+Anthropic+website%2E)
3. Ansar Aynetdinov and Alan Akbik. 2024. Semscore: Automated evaluation of instruction-tuned llms based on semantic textual similarity. CoRR, abs/2401.17072. — [arXiv:2401.17072](https://arxiv.org/abs/2401.17072) [find](https://scholar.google.com/scholar?q=Ansar+Aynetdinov+and+Alan+Akbik%2E+2024%2E+Semscore%3A+Automated+evaluation+of+instruction%2Dtuned+llms+based+on+semantic+textual+similarity%2E+CoRR%2C+abs%2F2401%2E17072%2E+%E2%80%94+%5BarXiv%3A2401%2E17072%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2)
4. Yuntao Bai, Andy Jones, Kamal Ndousse, Amanda Askell, Anna Chen, Nova DasSarma, Dawn Drain, Stanislav Fort, Deep Ganguli, Tom Henighan, Nicholas Joseph, Saurav Kadavath, Jackson Kernion, Tom Conerly, Sheer El Showk, Nelson Elhage, Zac Hatfield-Dodds, Danny Hernandez, Tristan Hume, and 12 others. 202… [find](https://scholar.google.com/scholar?q=Yuntao+Bai%2C+Andy+Jones%2C+Kamal+Ndousse%2C+Amanda+Askell%2C+Anna+Chen%2C+Nova+DasSarma%2C+Dawn+Drain%2C+Stanislav+Fort%2C+Deep+Ganguli%2C+Tom+Henighan%2C+Nicholas+Joseph%2C+Saurav+Kadavath%2C+Jackson+Kernion%2C+Tom+Conerly%2C+)
5. Roy Bar-Haim, Dalia Krieger, Orith Toledo-Ronen, Lilach Edelstein, Yonatan Bilu, Alon Halfon, Yoav Katz, Amir Menczel, Ranit Aharonov, and Noam Slonim. 2019. From surrogacy to adoption; from bitcoin to cryptocurrency: Debate topic expansion. In Proceedings of the 57th Conference of the Association f… [find](https://scholar.google.com/scholar?q=Roy+Bar%2DHaim%2C+Dalia+Krieger%2C+Orith+Toledo%2DRonen%2C+Lilach+Edelstein%2C+Yonatan+Bilu%2C+Alon+Halfon%2C+Yoav+Katz%2C+Amir+Menczel%2C+Ranit+Aharonov%2C+and+Noam+Slonim%2E+2019%2E+From+surrogacy+to+adoption%3B+from+bitcoin+t)
6. Johannes Bjerva, Nikita Bhutani, Behzad Golshan, Wang-Chiew Tan, and Isabelle Augenstein. 2020. Subjqa: A dataset for subjectivity and review comprehension. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, EMNLP 2020, Online, November 16-20, 2020, pages 5480… [find](https://scholar.google.com/scholar?q=Johannes+Bjerva%2C+Nikita+Bhutani%2C+Behzad+Golshan%2C+Wang%2DChiew+Tan%2C+and+Isabelle+Augenstein%2E+2020%2E+Subjqa%3A+A+dataset+for+subjectivity+and+review+comprehension%2E+In+Proceedings+of+the+2020+Conference+on+Em)
7. Michael Brady. 2009. Curiosity and the value of truth. Epistemic value, pages 265–283. [find](https://scholar.google.com/scholar?q=Michael+Brady%2E+2009%2E+Curiosity+and+the+value+of+truth%2E+Epistemic+value%2C+pages+265%E2%80%93283%2E)
8. Jianlv Chen, Shitao Xiao, Peitian Zhang, Kun Luo, Defu Lian, and Zheng Liu. 2024. [Bge m3-embedding: Multi-lingual, multi-functionality, multi-granularity text embeddings through self-knowledge distillation](https://md.research.elamur.ai/free/en/papers/chen-2024-m3-embedding-multi-linguality/). Preprint, arXiv:2402.03216.
9. Kai Chen, Zihao He, Rong-Ching Chang, Jonathan May, and Kristina Lerman. 2023. Anger breeds controversy: Analyzing controversy and emotions on reddit. In Social, Cultural, and Behavioral Modeling - 16th International Conference, SBP-BRiMS 2023, Pittsburgh, PA, USA, September 20-22, 2023, Proceedings… [find](https://scholar.google.com/scholar?q=Kai+Chen%2C+Zihao+He%2C+Rong%2DChing+Chang%2C+Jonathan+May%2C+and+Kristina+Lerman%2E+2023%2E+Anger+breeds+controversy%3A+Analyzing+controversy+and+emotions+on+reddit%2E+In+Social%2C+Cultural%2C+and+Behavioral+Modeling+%2D+16)
10. David Cheng-Han Chiang and Hung-yi Lee. 2023. Can large language models be an alternative to human evaluations? In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), ACL 2023, Toronto, Canada, July 9-14, 2023, pages 15607–15631. Associati… [find](https://scholar.google.com/scholar?q=David+Cheng%2DHan+Chiang+and+Hung%2Dyi+Lee%2E+2023%2E+Can+large+language+models+be+an+alternative+to+human+evaluations%3F+In+Proceedings+of+the+61st+Annual+Meeting+of+the+Association+for+Computational+Linguisti)
11. Yoonjung Choi, Yuchul Jung, and Sung-Hyon Myaeng. 2010. Identifying controversial issues and their subtopics in news articles. In Intelligence and Security Informatics, Pacific Asia Workshop, PAISI 2010, Hyderabad, India, June 21, 2010. Proceedings, volume 6122 of Lecture Notes in Computer Science,… [find](https://scholar.google.com/scholar?q=Yoonjung+Choi%2C+Yuchul+Jung%2C+and+Sung%2DHyon+Myaeng%2E+2010%2E+Identifying+controversial+issues+and+their+subtopics+in+news+articles%2E+In+Intelligence+and+Security+Informatics%2C+Pacific+Asia+Workshop%2C+PAISI+20)
12. Aakanksha Chowdhery, Sharan Narang, Jacob Devlin, Maarten Bosma, Gaurav Mishra, Adam Roberts, Paul Barham, Hyung Won Chung, Charles Sutton, Sebastian Gehrmann, Parker Schuh, Kensen Shi, Sasha Tsvyashchenko, Joshua Maynez, Abhishek Rao, Parker Barnes, Yi Tay, Noam Shazeer, Vinodkumar Prabhakaran, and… — [PaLM: Scaling Language Modeling with Pathways](https://md.research.elamur.ai/free/en/papers/chowdhery-2022-palm-scaling-language-modeling-pathways/)
13. Alexander R. Fabbri, Chien-Sheng Wu, Wenhao Liu, and Caiming Xiong. 2022. [Qafacteval: Improved qa-based factual consistency evaluation for summarization](https://md.research.elamur.ai/free/en/papers/fabbri-2022-qafacteval-improved-qa-based-factual/). In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies,…
14. Angela Fan, Yacine Jernite, Ethan Perez, David Grangier, Jason Weston, and Michael Auli. 2019. [ELI5: long form question answering](https://md.research.elamur.ai/free/en/papers/fan-2019-eli5-long-form-question-answering/). In Proceedings of the 57th Conference of the Association for Computational Linguistics, ACL 2019, Florence, Italy, July 28- August 2, 2019, Volume 1: Long Papers, pages…
15. Joseph L Fleiss, Bruce Levin, Myunghee Cho Paik, and 1 others. 1981. The measurement of interrater agreement. Statistical methods for rates and proportions, 2(212-236):22–23. [find](https://scholar.google.com/scholar?q=Joseph+L+Fleiss%2C+Bruce+Levin%2C+Myunghee+Cho+Paik%2C+and+1+others%2E+1981%2E+The+measurement+of+interrater+agreement%2E+Statistical+methods+for+rates+and+proportions%2C+2%28212%2D236%29%3A22%E2%80%9323%2E)
16. Tianyu Gao, Howard Yen, Jiatong Yu, and Danqi Chen. 2023. [Enabling large language models to generate text with citations](https://md.research.elamur.ai/free/en/papers/gao-2023-enabling-large-language-models-generate/). In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, EMNLP 2023, Singapore, December 6-10, 2023, pages 6465–6488. Association for Computatio…
17. Maarten Grootendorst. 2022. [Bertopic: Neural topic modeling with a class-based TF-IDF procedure](https://md.research.elamur.ai/free/en/papers/grootendorst-2022-bertopic-neural-topic-modeling-class/). CoRR, abs/2203.05794.
18. Jurgen Habermas. 1991. The structural transformation of the public sphere: An inquiry into a category of bourgeois society. MIT press. [find](https://scholar.google.com/scholar?q=Jurgen+Habermas%2E+1991%2E+The+structural+transformation+of+the+public+sphere%3A+An+inquiry+into+a+category+of+bourgeois+society%2E+MIT+press%2E)
19. James A Hanley and Barbara J McNeil. 1982. The meaning and use of the area under a receiver operating characteristic (roc) curve. Radiology, 143(1):29– 36. [find](https://scholar.google.com/scholar?q=James+A+Hanley+and+Barbara+J+McNeil%2E+1982%2E+The+meaning+and+use+of+the+area+under+a+receiver+operating+characteristic+%28roc%29+curve%2E+Radiology%2C+143%281%29%3A29%E2%80%93+36%2E)
20. Jack Hessel and Lillian Lee. 2019. Something’s brewing! early prediction of controversy-causing posts from discussion features. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, volume 1. [find](https://scholar.google.com/scholar?q=Jack+Hessel+and+Lillian+Lee%2E+2019%2E+Something%E2%80%99s+brewing%21+early+prediction+of+controversy%2Dcausing+posts+from+discussion+features%2E+In+Proceedings+of+the+2019+Conference+of+the+North+American+Chapter+of+t)
21. Albert O. Hirschman. 1945. National Power and the Structure of Foreign Trade. University of California Press. [find](https://scholar.google.com/scholar?q=Albert+O%2E+Hirschman%2E+1945%2E+National+Power+and+the+Structure+of+Foreign+Trade%2E+University+of+California+Press%2E)
22. Ari Holtzman, Jan Buys, Li Du, Maxwell Forbes, and Yejin Choi. 2020a. [The curious case of neural text degeneration](https://md.research.elamur.ai/free/en/papers/holtzman-2019-curious-case-neural-text-degeneration/). In 8th International Conference on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia, April 26-30, 2020. OpenReview.net.
23. Ari Holtzman, Jan Buys, Li Du, Maxwell Forbes, and Yejin Choi. 2020b. [The curious case of neural text degeneration](https://md.research.elamur.ai/free/en/papers/holtzman-2019-curious-case-neural-text-degeneration/). In ICLR.
24. Shengding Hu, Yuge Tu, Xu Han, Chaoqun He, Ganqu Cui, Xiang Long, Zhi Zheng, Yewei Fang, Yuxiang Huang, Weilin Zhao, Xinrong Zhang, Zhen Leng Thai, Kai Zhang, Chongyi Wang, Yuan Yao, Chenyang Zhao, Jie Zhou, Jie Cai, Zhongwu Zhai, and 6 others. 2024. [Minicpm: Unveiling the potential of small languag…](https://md.research.elamur.ai/free/en/papers/hu-2024-minicpm-unveiling-potential-small-language/)
25. Vladimir Karpukhin, Barlas Oguz, Sewon Min, Patrick S. H. Lewis, Ledell Wu, Sergey Edunov, Danqi Chen, and Wen-tau Yih. 2020. [Dense passage retrieval for open-domain question answering](https://md.research.elamur.ai/free/en/papers/karpukhin-2020-dense-passage-retrieval-open-domain/). In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, EMNLP 2020, Online, Nov…
26. Maurice G Kendall. 1938. A new measure of rank correlation. Biometrika, 30(1-2):81–93. [find](https://scholar.google.com/scholar?q=Maurice+G+Kendall%2E+1938%2E+A+new+measure+of+rank+correlation%2E+Biometrika%2C+30%281%2D2%29%3A81%E2%80%9393%2E)
27. Kalpesh Krishna, Yapei Chang, John Wieting, and Mohit Iyyer. 2022. Rankgen: Improving text generation with large ranking models. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, EMNLP 2022, Abu Dhabi, United Arab Emirates, December 7-11, 2022, pages 199–232.… [find](https://scholar.google.com/scholar?q=Kalpesh+Krishna%2C+Yapei+Chang%2C+John+Wieting%2C+and+Mohit+Iyyer%2E+2022%2E+Rankgen%3A+Improving+text+generation+with+large+ranking+models%2E+In+Proceedings+of+the+2022+Conference+on+Empirical+Methods+in+Natural+L)
28. Tom Kwiatkowski, Jennimaria Palomaki, Olivia Redfield, Michael Collins, Ankur P. Parikh, Chris Alberti, Danielle Epstein, Illia Polosukhin, Jacob Devlin, Kenton Lee, Kristina Toutanova, Llion Jones, Matthew Kelcey, Ming-Wei Chang, Andrew M. Dai, Jakob Uszkoreit, Quoc Le, and Slav Petrov. 2019. [Natur…](https://md.research.elamur.ai/free/en/papers/kwiatkowski-2019-natural-questions-benchmark-question-answering/)
29. Brian W Matthews. 1975. Comparison of the predicted and observed secondary structure of t4 phage lysozyme. Biochimica et Biophysica Acta (BBA)-Protein Structure, 405(2):442–451. [find](https://scholar.google.com/scholar?q=Brian+W+Matthews%2E+1975%2E+Comparison+of+the+predicted+and+observed+secondary+structure+of+t4+phage+lysozyme%2E+Biochimica+et+Biophysica+Acta+%28BBA%29%2DProtein+Structure%2C+405%282%29%3A442%E2%80%93451%2E)
30. Meta. 2024. Build the future of AI with Meta Llama 3. Meta AI website. [find](https://scholar.google.com/scholar?q=Meta%2E+2024%2E+Build+the+future+of+AI+with+Meta+Llama+3%2E+Meta+AI+website%2E)
31. Nathan Lambert, Jacob Morrison, Valentina Pyatkin, Shengyi Huang, Hamish Ivison, Faeze Brahman, Lester James V. Miranda, Alisa Liu, Nouha Dziri, Shane Lyu, Yuling Gu, Saumya Malik, Victoria Graf, Jena D. Hwang, Jiangjiang Yang, Ronan Le Bras, Oyvind Tafjord, Chris Wilhelm, Luca Soldaini, and 4 other… — [Tulu 3: Pushing Frontiers in Open Language Model Post-Training](https://md.research.elamur.ai/free/en/papers/lambert-2024-tulu-3-pushing-frontiers-open/)
32. Mike Lewis, Yinhan Liu, Naman Goyal, Marjan Ghazvininejad, Abdelrahman Mohamed, Omer Levy, Veselin Stoyanov, and Luke Zettlemoyer. 2020a. [BART: denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension](https://md.research.elamur.ai/free/en/papers/lewis-2020-bart-denoising-sequence-sequence-pre/). In Proceedings of the 58th Annual Meeting of th…
33. Patrick S. H. Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen-tau Yih, Tim Rocktäschel, Sebastian Riedel, and Douwe Kiela. 2020b. [Retrieval-augmented generation for knowledge-intensive NLP tasks](https://md.research.elamur.ai/free/en/papers/lewis-2020-retrieval-augmented-generation-knowledge-intensive/). In Advances in Neural Informati…
34. Zehan Li, Xin Zhang, Yanzhao Zhang, Dingkun Long, Pengjun Xie, and Meishan Zhang. 2023. [Towards general text embeddings with multi-stage contrastive learning](https://md.research.elamur.ai/free/en/papers/li-2023-towards-general-text-embeddings-multi/). CoRR, abs/2308.03281.
35. Chin-Yew Lin. 2004. [Rouge: A package for automatic evaluation of summaries](https://md.research.elamur.ai/free/en/papers/lin-2004-rouge-package-automatic-evaluation-summaries/). In Text summarization branches out, pages 74–81.
36. Sewon Min, Kalpesh Krishna, Xinxi Lyu, Mike Lewis, Wen-tau Yih, Pang Wei Koh, Mohit Iyyer, Luke Zettlemoyer, and Hannaneh Hajishirzi. 2023. [Factscore: Fine-grained atomic evaluation of factual precision in long form text generation](https://md.research.elamur.ai/free/en/papers/min-2023-factscore-fine-grained-atomic-evaluation/). In Proceedings of the 2023 Conference on Empirical Methods in Natur…
37. Sewon Min, Julian Michael, Hannaneh Hajishirzi, and Luke Zettlemoyer. 2020. Ambigqa: Answering ambiguous open-domain questions. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, EMNLP 2020, Online, November 16-20, 2020, pages 5783–5797. Association for Comput… [find](https://scholar.google.com/scholar?q=Sewon+Min%2C+Julian+Michael%2C+Hannaneh+Hajishirzi%2C+and+Luke+Zettlemoyer%2E+2020%2E+Ambigqa%3A+Answering+ambiguous+open%2Ddomain+questions%2E+In+Proceedings+of+the+2020+Conference+on+Empirical+Methods+in+Natural+La)
38. Thomas Misco. 2011. Teaching about controversial issues: Rationale, practice, and need for inquiry. Int’l J. Educ. L. & Pol’y, 7:13. [find](https://scholar.google.com/scholar?q=Thomas+Misco%2E+2011%2E+Teaching+about+controversial+issues%3A+Rationale%2C+practice%2C+and+need+for+inquiry%2E+Int%E2%80%99l+J%2E+Educ%2E+L%2E+%26+Pol%E2%80%99y%2C+7%3A13%2E)
39. Ella Neeman, Roee Aharoni, Or Honovich, Leshem Choshen, Idan Szpektor, and Omri Abend. 2023. [Disentqa: Disentangling parametric and contextual knowledge with counterfactual question answering](https://md.research.elamur.ai/free/en/papers/neeman-2023-disentqa-disentangling-parametric-contextual-knowledge/). In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long…
40. OpenAI. 2023a. Gpt-3.5 turbo. OpenAI Website. [find](https://scholar.google.com/scholar?q=OpenAI%2E+2023a%2E+Gpt%2D3%2E5+turbo%2E+OpenAI+Website%2E)
41. Stephanie Lin, Jacob Hilton, and Owain Evans. 2022. [Truthfulqa: Measuring how models mimic human falsehoods](https://md.research.elamur.ai/free/en/papers/lin-2022-truthfulqa-measuring-how-models-mimic/). In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), ACL 2022, Dublin, Ireland, May 22-27, 2022, pages 3214–3252. Association fo…
42. Yang Liu, Dan Iter, Yichong Xu, Shuohang Wang, Ruochen Xu, and Chenguang Zhu. 2023. [G-eval: NLG evaluation using gpt-4 with better human alignment](https://md.research.elamur.ai/free/en/papers/liu-2023-g-eval-nlg-evaluation-gpt/). In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, EMNLP 2023, Singapore, December 6-10, 2023, pages 2511–2522.…
43. Nancy Lowry and David W Johnson. 1981. Effects of controversy on epistemic curiosity, achievement, and attitudes. The Journal of Social Psychology, 115(1):31–43. [find](https://scholar.google.com/scholar?q=Nancy+Lowry+and+David+W+Johnson%2E+1981%2E+Effects+of+controversy+on+epistemic+curiosity%2C+achievement%2C+and+attitudes%2E+The+Journal+of+Social+Psychology%2C+115%281%29%3A31%E2%80%9343%2E)
44. OpenAI. 2023b. [GPT-4 technical report](https://md.research.elamur.ai/free/en/papers/openai-2023-gpt-4-technical-report/). abs/2303.08774.
45. CoRR, OpenAI. 2024a. Gpt-4o mini: advancing cost-efficient intelligence. OpenAI Website. [find](https://scholar.google.com/scholar?q=CoRR%2C+OpenAI%2E+2024a%2E+Gpt%2D4o+mini%3A+advancing+cost%2Defficient+intelligence%2E+OpenAI+Website%2E)
46. OpenAI. 2024b. Hello GPT-4o. OpenAI website. [find](https://scholar.google.com/scholar?q=OpenAI%2E+2024b%2E+Hello+GPT%2D4o%2E+OpenAI+website%2E)
47. Long Ouyang, Jeff Wu, Xu Jiang, and et al. 2022a. [Training language models to follow instructions with human feedback](https://md.research.elamur.ai/free/en/papers/ouyang-2022-training-language-models-follow-instructions/). In NeurIPS.
48. Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll L. Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, John Schulman, Jacob Hilton, Fraser Kelton, Luke Miller, Maddie Simens, Amanda Askell, Peter Welinder, Paul F. Christiano, Jan Leike, and Ryan Lowe. 2022b… — [Training language models to follow instructions with human feedback](https://md.research.elamur.ai/free/en/papers/ouyang-2022-training-language-models-follow-instructions/)
49. Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. [Bleu: a method for automatic evaluation of machine translation](https://md.research.elamur.ai/free/en/papers/papineni-2002-bleu-method-automatic-evaluation-machine/). In Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics, July 6-12, 2002, Philadelphia, PA, USA, pages 311–318. ACL.
50. Qwen. 2024a. Hello Qwen2. QwenLM Blog. [find](https://scholar.google.com/scholar?q=Qwen%2E+2024a%2E+Hello+Qwen2%2E+QwenLM+Blog%2E)
51. Qwen. 2024b. Introducing Qwen1.5. QwenLM Blog. [find](https://scholar.google.com/scholar?q=Qwen%2E+2024b%2E+Introducing+Qwen1%2E5%2E+QwenLM+Blog%2E)
52. Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, and 1 others. 2019. Language models are unsupervised multitask learners. OpenAI blog, 1(8):9. [find](https://scholar.google.com/scholar?q=Alec+Radford%2C+Jeffrey+Wu%2C+Rewon+Child%2C+David+Luan%2C+Dario+Amodei%2C+Ilya+Sutskever%2C+and+1+others%2E+2019%2E+Language+models+are+unsupervised+multitask+learners%2E+OpenAI+blog%2C+1%288%29%3A9%2E)
53. Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D. Manning, Stefano Ermon, and Chelsea Finn. 2023a. [Direct preference optimization: Your language model is secretly a reward model](https://md.research.elamur.ai/free/en/papers/rafailov-2023-direct-preference-optimization-language-model/). In Advances in Neural Information Processing Systems 36: Annual Conference on Neural Information Processing S…
54. Rafael Rafailov, Sherry Sharma, Eric Mitchell, and Chelsea Finn. 2023b. [Direct preference optimization: Your language model is secretly a reward model](https://md.research.elamur.ai/free/en/papers/rafailov-2023-direct-preference-optimization-language-model/). In NeurIPS.
55. Pranav Rajpurkar, Jian Zhang, Konstantin Lopyrev, and Percy Liang. 2016. [Squad: 100, 000+ questions for machine comprehension of text](https://md.research.elamur.ai/free/en/papers/rajpurkar-2016-squad-100-000-questions-machine/). In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, EMNLP 2016, Austin, Texas, USA, November 1-4, 2016, pages 2383–2392. The A…
56. Nils Reimers and Iryna Gurevych. 2019a. [Sentence-bert: Sentence embeddings using siamese bert-networks](https://md.research.elamur.ai/free/en/papers/reimers-2019-sentence-bert-sentence-embeddings-siamese/). In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, EMNLP-IJCNLP 2019, Hong Kong, C…
57. Nils Reimers and Iryna Gurevych. 2019b. [Sentence-bert: Sentence embeddings using siamese bert-networks](https://md.research.elamur.ai/free/en/papers/reimers-2019-sentence-bert-sentence-embeddings-siamese/). In EMNLP.
58. Stephen A. Rhoades. 1993. The herfindahl-hirschman index. Federal Reserve Bulletin, 79(3):188–189. [find](https://scholar.google.com/scholar?q=Stephen+A%2E+Rhoades%2E+1993%2E+The+herfindahl%2Dhirschman+index%2E+Federal+Reserve+Bulletin%2C+79%283%29%3A188%E2%80%93189%2E)
59. Noah Shinn, Federico Cassano, Ashwin Gopinath, Karthik Narasimhan, and Shunyu Yao. 2023. [Reflexion: language agents with verbal reinforcement learning](https://md.research.elamur.ai/free/en/papers/shinn-2023-reflexion-language-agents-verbal-reinforcement/). In Advances in Neural Information Processing Systems 36: Annual Conference on Neural Information Processing Systems 2023, NeurIPS 2023, New Orleans,…
60. Noam Slonim, Yonatan Bilu, Carlos Alzate, Roy Bar-Haim, Ben Bogin, Francesca Bonin, Leshem Choshen, Edo Cohen-Karlik, Lena Dankin, Lilach Edelstein, Liat Ein-Dor, Roni Friedman-Melamed, Assaf Gavron, Ariel Gera, Martin Gleize, Shai Gretz, Dan Gutfreund, Alon Halfon, Daniel Hershcovich, and 34 others… [find](https://scholar.google.com/scholar?q=Noam+Slonim%2C+Yonatan+Bilu%2C+Carlos+Alzate%2C+Roy+Bar%2DHaim%2C+Ben+Bogin%2C+Francesca+Bonin%2C+Leshem+Choshen%2C+Edo+Cohen%2DKarlik%2C+Lena+Dankin%2C+Lilach+Edelstein%2C+Liat+Ein%2DDor%2C+Roni+Friedman%2DMelamed%2C+Assaf+Gavron%2C+)
61. Manfred Stede, Jodi Schneider, and Graeme Hirst. 2019. Argumentation mining. Springer. [find](https://scholar.google.com/scholar?q=Manfred+Stede%2C+Jodi+Schneider%2C+and+Graeme+Hirst%2E+2019%2E+Argumentation+mining%2E+Springer%2E)
62. Ivan Stelmakh, Yi Luan, Bhuwan Dhingra, and Ming-Wei Chang. 2022a. [ASQA: factoid questions meet long-form answers](https://md.research.elamur.ai/free/en/papers/stelmakh-2022-asqa-factoid-questions-meet-long/). In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, EMNLP 2022, Abu Dhabi, United Arab Emirates, December 7-11, 2022, pages 8273–8288. Association…
63. Ivan Stelmakh, Yi Luan, Bhuwan Dhingra, and Ming-Wei Chang. 2022b. [ASQA: factoid questions meet long-form answers](https://md.research.elamur.ai/free/en/papers/stelmakh-2022-asqa-factoid-questions-meet-long/). In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, EMNLP 2022, Abu Dhabi, United Arab Emirates, December 7-11, 2022, pages 8273–8288. Association…
64. David Q. Sun, Artem Abzaliev, Hadas Kotek, Christopher Klein, Zidi Xiu, and Jason D. Williams. 2023. DELPHI: data for evaluating llms’ performance in handling controversial issues. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing: EMNLP 2023 - Industry Track,… [find](https://scholar.google.com/scholar?q=David+Q%2E+Sun%2C+Artem+Abzaliev%2C+Hadas+Kotek%2C+Christopher+Klein%2C+Zidi+Xiu%2C+and+Jason+D%2E+Williams%2E+2023%2E+DELPHI%3A+data+for+evaluating+llms%E2%80%99+performance+in+handling+controversial+issues%2E+In+Proceedings+of+t)
65. Haochen Tan, Zhijiang Guo, Zhan Shi, Lu Xu, Zhili Liu, Xiaoguang Li, Yasheng Wang, Lifeng Shang, Qun Liu, and Linqi Song. 2024. [PROXYQA: an alternative framework for evaluating long-form text generation with large language models](https://md.research.elamur.ai/free/en/papers/tan-2024-proxyqa-alternative-framework-evaluating-long/). CoRR, abs/2401.15042.
66. Gemma Team. 2024. Gemma 2: Improving open language models at a practical size. Technical Report. [find](https://scholar.google.com/scholar?q=Gemma+Team%2E+2024%2E+Gemma+2%3A+Improving+open+language+models+at+a+practical+size%2E+Technical+Report%2E)
67. Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, Dan Bikel, Lukas Blecher, Cristian Canton-Ferrer, Moya Chen, Guillem Cucurull, David Esiobu, Jude Fernandes, Jeremy Fu, Wenyin Fu, and 49 others… — [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://md.research.elamur.ai/free/en/papers/touvron-2023-llama-2-open-foundation-fine/)
68. Lewis Tunstall, Edward Beeching, Nathan Lambert, Nazneen Rajani, Kashif Rasul, Younes Belkada, Shengyi Huang, Leandro von Werra, Clémentine Fourrier, Nathan Habib, Nathan Sarrazin, Omar Sanseviero, Alexander M. Rush, and Thomas Wolf. 2023. Zephyr: Direct distillation of LM alignment. CoRR, abs/2310.… [find](https://scholar.google.com/scholar?q=Lewis+Tunstall%2C+Edward+Beeching%2C+Nathan+Lambert%2C+Nazneen+Rajani%2C+Kashif+Rasul%2C+Younes+Belkada%2C+Shengyi+Huang%2C+Leandro+von+Werra%2C+Cl%C3%A9mentine+Fourrier%2C+Nathan+Habib%2C+Nathan+Sarrazin%2C+Omar+Sanseviero%2C+Al)
69. Alexander Wan, Eric Wallace, and Dan Klein. 2024. What evidence do language models find convincing? CoRR, abs/2402.11782. [find](https://scholar.google.com/scholar?q=Alexander+Wan%2C+Eric+Wallace%2C+and+Dan+Klein%2E+2024%2E+What+evidence+do+language+models+find+convincing%3F+CoRR%2C+abs%2F2402%2E11782%2E)
70. Bjorn Gert Jan Wansink, Jacob Timmer, and Larike Henriette Bronkhorst. 2023. Navigating multiple perspectives in discussing controversial topics: Boundary crossing in the classroom. Education Sciences, 13(9):938. [find](https://scholar.google.com/scholar?q=Bjorn+Gert+Jan+Wansink%2C+Jacob+Timmer%2C+and+Larike+Henriette+Bronkhorst%2E+2023%2E+Navigating+multiple+perspectives+in+discussing+controversial+topics%3A+Boundary+crossing+in+the+classroom%2E+Education+Sciences)
71. Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed H. Chi, Quoc V. Le, and Denny Zhou. 2022. [Chain-of-thought prompting elicits reasoning in large language models](https://md.research.elamur.ai/free/en/papers/wei-2022-chain-thought-prompting-elicits-reasoning/). In Advances in Neural Information Processing Systems 35: Annual Conference on Neural Information Processi…
72. Jerry Wei, Chengrun Yang, Xinying Song, Yifeng Lu, Nathan Hu, Dustin Tran, Daiyi Peng, Ruibo Liu, Da Huang, Cosmo Du, and Quoc V. Le. 2024. [Long-form factuality in large language models](https://md.research.elamur.ai/free/en/papers/wei-2024-long-form-factuality-large-language/). CoRR, abs/2403.18802.
73. Orion Weller, Marc Marone, Nathaniel Weir, Dawn J. Lawrie, Daniel Khashabi, and Benjamin Van Durme. 2024. ["according to](https://md.research.elamur.ai/free/en/papers/weller-2024-according-prompting-language-models-improves/). . . ": Prompting language models improves quoting from pre-training data. In Proceedings of the 18th Conference of the European Chapter of the Association for Computational Lingu…
74. Erik Olin Wright. 1998. The debate on classes, volume 20. Verso. [find](https://scholar.google.com/scholar?q=Erik+Olin+Wright%2E+1998%2E+The+debate+on+classes%2C+volume+20%2E+Verso%2E)
75. Fangyuan Xu, Yixiao Song, Mohit Iyyer, and Eunsol Choi. 2023. [A critical evaluation of evaluations for long-form question answering](https://md.research.elamur.ai/free/en/papers/xu-2023-critical-evaluation-evaluations-long-form/). In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), ACL 2023, Toronto, Canada, July 9-14, 2023, pages 3…
76. Rongwu Xu, Zi’an Zhou, Tianwei Zhang, Zehan Qi, Su Yao, Ke Xu, Wei Xu, and Han Qiu. 2024. Walking in others’ shoes: How perspective-taking guides large language models in reducing toxicity and bias. arXiv preprint arXiv:2407.15366. [find](https://scholar.google.com/scholar?q=Rongwu+Xu%2C+Zi%E2%80%99an+Zhou%2C+Tianwei+Zhang%2C+Zehan+Qi%2C+Su+Yao%2C+Ke+Xu%2C+Wei+Xu%2C+and+Han+Qiu%2E+2024%2E+Walking+in+others%E2%80%99+shoes%3A+How+perspective%2Dtaking+guides+large+language+models+in+reducing+toxicity+and+bias%2E+a)
77. Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik R. Narasimhan, and Yuan Cao. 2023. [React: Synergizing reasoning and acting in language models](https://md.research.elamur.ai/free/en/papers/yao-2022-react-synergizing-reasoning-acting-language/). In The Eleventh International Conference on Learning Representations, ICLR 2023, Kigali, Rwanda, May 1-5, 2023. OpenReview.net.
78. Weizhe Yuan, Graham Neubig, and Pengfei Liu. 2021. [Bartscore: Evaluating generated text as text generation](https://md.research.elamur.ai/free/en/papers/yuan-2021-bartscore-evaluating-generated-text-text/). In Advances in Neural Information Processing Systems 34: Annual Conference on Neural Information Processing Systems 2021, NeurIPS 2021, December 6-14, 2021, virtual, pages 27263–27277.
79. Jerrold H. Zar. 2005. Spearman rank correlation. [find](https://scholar.google.com/scholar?q=Jerrold+H%2E+Zar%2E+2005%2E+Spearman+rank+correlation%2E)
80. Yue Zhang, Yafu Li, Leyang Cui, Deng Cai, Lemao Liu, Tingchen Fu, Xinting Huang, Enbo Zhao, Yu Zhang, Yulong Chen, Longyue Wang, Anh Tuan Luu, Wei Bi, Freda Shi, and Shuming Shi. 2023. [Siren’s song in the AI ocean: A survey on hallucination in large language models](https://md.research.elamur.ai/free/en/papers/zhang-2023-siren-s-song-ai-ocean/). CoRR, abs/2309.01219.

*Updated 31 October 2026*