--- title: "WildGraphBench — paper by Wang et al. (2026)" description: "From these, the authors build about 1,100 questions of three kinds: single-fact lookups, multi-fact… 15.30 F1 on the People. 2 references." authors: ["Pengyu Wang", "Benfeng Xu", "Licheng Zhang", "Shaohan Wang", "Mingxuan Du", "Chiwei Zhu", "Zhendong Mao"] year: 2026 venue: "Findings of the Association for Computational Linguistics: ACL 2026" source: "https://aclanthology.org/2026.findings-acl.679/" doi: "10.18653/v1/2026.findings-acl.679" licence: "CC BY 4.0 (ACL Anthology)" licence_url: "https://creativecommons.org/licenses/by/4.0/" summary_licence: "Elamur Cosmic License" canonical: "https://research.elamur.ai/free/en/papers/wang-2026-wildgraphbench-benchmarking-graphrag-wild-source/" --- [Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/wang-2026-wildgraphbench-benchmarking-graphrag-wild-source/) [home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2026](https://md.research.elamur.ai/free/en/papers/2026/page/19/) / WildGraphBench: Benchmarking GraphRAG with Wild-Source Corpora # WildGraphBench: Benchmarking GraphRAG with Wild-Source Corpora **Authors:** [Pengyu Wang](https://md.research.elamur.ai/free/en/authors/pengyu-wang/), [Benfeng Xu](https://md.research.elamur.ai/free/en/authors/benfeng-xu/), [Licheng Zhang](https://md.research.elamur.ai/free/en/authors/licheng-zhang/), [Shaohan Wang](https://md.research.elamur.ai/free/en/authors/shaohan-wang/), [Mingxuan Du](https://md.research.elamur.ai/free/en/authors/mingxuan-du/), [Chiwei Zhu](https://md.research.elamur.ai/free/en/authors/chiwei-zhu/) and [Zhendong Mao](https://md.research.elamur.ai/free/en/authors/zhendong-mao/) · **Year:** 2026 · **Venue:** [Findings of the Association for Computational Linguistics: ACL 2026](https://md.research.elamur.ai/free/en/venues/findings/) ([Findings of the Association for Computational Linguistics 2026](https://md.research.elamur.ai/free/en/venues/findings/2026/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/) **Links:** [Publisher's page](https://aclanthology.org/2026.findings-acl.679/) · [PDF](https://aclanthology.org/2026.findings-acl.679.pdf) · [DOI 10.18653/v1/2026.findings-acl.679](https://doi.org/10.18653/v1/2026.findings-acl.679) · [OpenAlex](https://openalex.org/W7166895228) **Citations:** 1 citations (OpenAlex, 2026-10-14) ## Summary Most GraphRAG benchmarks use short, pre-cleaned passages. That makes it hard to judge how graph-based retrieval copes with long, messy, mixed-source document collections, the setting where it should help most. WildGraphBench fills this gap using Wikipedia. Articles from 12 top-level topics are sampled. The external pages each article cites (mostly news, plus magazines, research sites, PDFs and books) form the retrieval corpus. The article's citation-backed sentences serve as gold facts. From these, the authors build about 1,100 questions of three kinds: - single-fact lookups, - multi-fact questions that combine several references, - section-level summaries. Answers are scored against the gold statements. QA uses accuracy, and summaries use statement-level precision, recall and F1. The authors test flat RAG baselines and several GraphRAG systems. Graph methods add little on single-fact queries but clearly help when facts must be combined across a moderate number of sources. All methods score poorly on summaries, partly because graph aggregation favours high-level claims over specific details. The benchmark exposes realistic weaknesses that curated datasets hide. ## Key points - **Corpus and gold data:** Wikipedia reference pages make up the corpus, and about 78% of them are news sources. Citation-linked Wikipedia statements serve as ground truth. The 12 topics are Wikipedia's top-level content categories, with "Reference" excluded. - **Systems evaluated:** NaiveRAG, BM25, Fast-GraphRAG, HippoRAG2, MS GraphRAG (local/global), LightRAG and LinearRAG. Indexing costs vary widely: on the Culture domain, MS GraphRAG takes about 21.5 h to index, compared with seconds for the flat baselines. - **Model swap test:** Replacing gpt-4o-mini with gpt-5-mini in HippoRAG2 (Culture domain) gives large QA gains: - single-fact: 62.79 → 80.23 - multi-fact: 24.32 → 40.54 - summary F1: only 6.68 → 8.33 This points to retrieval as the bottleneck for summaries. - **Retrieval budget and human reference:** For summaries, F1 rises with retrieval top-k up to k = 8 and then falls as noise increases. Human annotators reach 15.30 F1 on the People subset. - **Error analysis and judge reliability:** - Multi-fact errors are 65% generation-side and 35% retrieval-side. - Half of summary errors come from gold evidence that was never retrieved. - The LLM judge agrees with human checks 89.8%, 96.0% and 98.0% of the time across the three question types. *Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.* ## Institutions - [University of Science and Technology of China](https://md.research.elamur.ai/free/en/universities/university-of-science-and-technology-of-china/) — University, Hefei, China · 108 works · 112,698 citations 📚 Shelves (6): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 16,039 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 15,891 works · [👁️ Computer Vision and Pattern Recognition](https://md.research.elamur.ai/free/en/shelves/computer-science/computer-vision-and-pattern-recognition/) · 2,869 works · [🗄️ Information Systems](https://md.research.elamur.ai/free/en/shelves/computer-science/information-systems/) · 1,437 works ## Cite Cite as: Pengyu Wang, Benfeng Xu, Licheng Zhang, Shaohan Wang, Mingxuan Du, Chiwei Zhu and Zhendong Mao. 2026. WildGraphBench: Benchmarking GraphRAG with Wild-Source Corpora. Findings of the Association for Computational Linguistics: ACL 2026. The paper itself, not this page. BibTeX from the library's record of the work. ```bibtex @inproceedings{wang2026wildgraphbench, title = {{WildGraphBench: Benchmarking GraphRAG with Wild-Source Corpora}}, author = {Pengyu Wang, Benfeng Xu, Licheng Zhang, Shaohan Wang, Mingxuan Du, Chiwei Zhu and Zhendong Mao}, booktitle = {Findings of the Association for Computational Linguistics: ACL 2026}, year = {2026}, doi = {10.18653/v1/2026.findings-acl.679}, url = {https://doi.org/10.18653/v1/2026.findings-acl.679} } ``` - [← previous WikiSeeker: Rethinking the Role of Vision-Language Models in Knowledge-Based Visual Question Answering](https://md.research.elamur.ai/free/en/papers/zhu-2026-wikiseeker-rethinking-role-vision-language/) - [next → Word Alignment-Based Evaluation of Uniform Meaning Representations](https://md.research.elamur.ai/free/en/papers/zeman-2026-word-alignment-based-evaluation-uniform/) - [↑ back to papers of 2026](https://md.research.elamur.ai/free/en/papers/2026/page/19/) ## References 2 references, 1 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet. 1. We thank the anonymous reviewers for their constructive feedback, which helped improve the clarity and rigor of this work. We also thank the human annotators who participated in the evaluation and error analysis. This research is supported by Artificial Intelligence-National Science and Technology M… [find](https://scholar.google.com/scholar?q=We+thank+the+anonymous+reviewers+for+their+constructive+feedback%2C+which+helped+improve+the+clarity+and+rigor+of+this+work%2E+We+also+thank+the+human+annotators+who+participated+in+the+evaluation+and+err) 2. Danqi Chen, Adam Fisch, Jason Weston, and Antoine Bordes. 2017. [Reading Wikipedia to answer open-domain questions](https://md.research.elamur.ai/free/en/papers/chen-2017-reading-wikipedia-answer-open-domain/). In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1870–1879, Vancouver, Canada. Association for Computational Lin… *Updated 1 November 2026*