---
title: "Reasoning Graph-Structured Question Answering — Yone 2026"
description: "Benchmarks for multi-hop question answering, such as HotpotQA, 2WikiMultiHopQA, MuSiQue and GSM8K, test multi-step reasoning."
authors: ["Khin Yone", "Devasha Trivedi", "Anish Pahilajani", "Jincen Shuai", "Samyak Rajesh Jain", "Ryan Rossi", "Nesreen K. Ahmed", "Franck Dernoncourt", "Yu Wang", "Namyong Park"]
year: 2026
venue: "Proceedings of the Fifteenth Language Resources and Evaluation Conference"
source: "https://aclanthology.org/2026.lrec-1.414/"
doi: "10.63317/4zjpmtqxxtx4"
licence: "CC BY-NC 4.0 (ELRA, LREC proceedings)"
licence_url: "https://creativecommons.org/licenses/by-nc/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/yone-2026-reasoning-graph-structured-question-answering/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/yone-2026-reasoning-graph-structured-question-answering/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2026](https://md.research.elamur.ai/free/en/papers/2026/page/8/) / Reasoning Graph-Structured Question Answering: Datasets and Insights from LLM Benchmarking

# Reasoning Graph-Structured Question Answering: Datasets and Insights from LLM Benchmarking

**Authors:** Khin Yone, Devasha Trivedi, Anish Pahilajani, Jincen Shuai, Samyak Rajesh Jain, [Ryan Rossi](https://md.research.elamur.ai/free/en/authors/ryan-rossi/), [Nesreen K. Ahmed](https://md.research.elamur.ai/free/en/authors/nesreen-k-ahmed/), [Franck Dernoncourt](https://md.research.elamur.ai/free/en/authors/franck-dernoncourt/), [Yu Wang](https://md.research.elamur.ai/free/en/authors/yu-wang/), Namyong Park · **Year:** 2026 · **Venue:** Proceedings of the Fifteenth Language Resources and Evaluation Conference · **Licence:** [CC BY-NC 4.0 (ELRA, LREC proceedings)](https://creativecommons.org/licenses/by-nc/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2026.lrec-1.414/) · [PDF](https://aclanthology.org/2026.lrec-1.414.pdf) · [DOI 10.63317/4zjpmtqxxtx4](https://doi.org/10.63317/4zjpmtqxxtx4) · [OpenAlex](https://openalex.org/W7160012515)

**Citations:** 0 citations (openalex-snapshot · 2026-10-14)

## Summary
Benchmarks for multi-hop question answering, such as HotpotQA, 2WikiMultiHopQA, MuSiQue and GSM8K, test multi-step reasoning. However, they do not explicitly record the reasoning path behind each answer, so it is hard to study how reasoning structure affects LLMs. The authors address this with GRS-QA. This resource adds a directed reasoning graph to QA instances drawn from those four sources, using the DARG-processed version of GSM8K. In each graph, nodes are evidence sentences or intermediate equations, and edges capture which step depends on which. Besides these gold ("positive") graphs, the dataset contains perturbed or negative variants that add irrelevant nodes or random edges. With this resource, the authors compare two prompting strategies: supplying a reasoning graph as context, and showing graphs as worked examples. Example-based prompting consistently performed better. Larger models handled graph structure in textual QA better than smaller ones. Textual tasks reacted more strongly to graph perturbations than math tasks. The work gives a common structural formalism for analysing multi-hop reasoning across domains.

## Key points
- **How graphs are built:** GRS-QA unifies graph annotations from HotpotQA, MuSiQue, 2WikiMultiHopQA and DARG-processed GSM8K.
  - HotpotQA "bridge" questions get an edge between their two supporting sentences, while "comparison" questions have no edges.
  - For GSM8K, an edge links two equations when one equation's result is used as an operand in the other.
- **Graph types:** The textual sources yield 12 distinct graph structures across comparison, bridge, compositional and bridge-comparison categories (e.g., B-2-1, C-4-1, CO-4-3, BC-5-1). GSM8K graphs are counted separately because their structures are too varied.
- **Negative and perturbed graphs:** These are created by adding extra context nodes or random edges. For GSM8K, DARG supplies width- and depth-based perturbations.
- **Data loss:** 4,507 training instances from 2WikiMultiHopQA were dropped because inconsistent entity strings prevented building positive graphs.
- **Availability:** The dataset and code are released under CC BY 4.0 at github.com/kyone138/grs-qa.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

Fields: [Linguistics & languages](https://md.research.elamur.ai/free/en/shelves/linguistics/)

## Cite

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{yone2026reasoning,
  title = {{Reasoning Graph-Structured Question Answering: Datasets and Insights from LLM Benchmarking}},
  author = {Yone, Khin and Trivedi, Devasha and Pahilajani, Anish and Shuai, Jincen and Jain, Samyak Rajesh and Rossi, Ryan and Ahmed, Nesreen K. and Dernoncourt, Franck and Wang, Yu and Park, Namyong},
  booktitle = {Proceedings of the Fifteenth Language Resources and Evaluation Conference},
  year = {2026},
  doi = {10.63317/4zjpmtqxxtx4},
  url = {https://doi.org/10.63317/4zjpmtqxxtx4}
}
```

- [← previous Reasoning as Supportive Context for Machine Translation: A Case Study on Hindi to Bengali Language Pair](https://md.research.elamur.ai/free/en/papers/singh-2026-reasoning-as-supportive-context-machine/)
- [next → Rebelòt: Datasets and Token-Level Language Identification for Lombard-Italian-English Code-Mixing](https://md.research.elamur.ai/free/en/papers/signoroni-2026-rebelot-datasets-token-level-language/)
- [↑ back to papers of 2026](https://md.research.elamur.ai/free/en/papers/2026/page/8/)

## References

25 references, 13 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Marah Abdin, Jyoti Aneja, Hany Awadalla, Ahmed Awadallah, Ammar Ahmad Awan, Nguyen Bach, Amit Bahree, Arash Bakhtiari, Jianmin Bao, Harkirat Behl, Alon Benhaim, Misha Bilenko, Johan Bjorck, Sébastien Bubeck, Martin Cai, Qin Cai, Vishrav Chaudhary, Dong Chen, Dongdong Chen, Weizhu Chen, Yen-Chun Chen… [find](https://scholar.google.com/scholar?q=Marah+Abdin%2C+Jyoti+Aneja%2C+Hany+Awadalla%2C+Ahmed+Awadallah%2C+Ammar+Ahmad+Awan%2C+Nguyen+Bach%2C+Amit+Bahree%2C+Arash+Bakhtiari%2C+Jianmin+Bao%2C+Harkirat+Behl%2C+Alon+Benhaim%2C+Misha+Bilenko%2C+Johan+Bjorck%2C+S%C3%A9bastien+) [our card](https://md.research.elamur.ai/free/en/papers/abdin-2024-phi-3-technical-report-highly/)
2. Farah Atif, Ola El Khatib, and Djellel Difallah. 2023. Beamqa: Multi-hop knowledge graph question answering with sequence-to-sequence prediction and beam search. [find](https://scholar.google.com/scholar?q=Farah+Atif%2C+Ola+El+Khatib%2C+and+Djellel+Difallah%2E+2023%2E+Beamqa%3A+Multi%2Dhop+knowledge+graph+question+answering+with+sequence%2Dto%2Dsequence+prediction+and+beam+search%2E)
3. Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, Christopher Hesse, and John Schulman. 2021. Training verifiers to solve math word problems. [find](https://scholar.google.com/scholar?q=Karl+Cobbe%2C+Vineet+Kosaraju%2C+Mohammad+Bavarian%2C+Mark+Chen%2C+Heewoo+Jun%2C+Lukasz+Kaiser%2C+Matthias+Plappert%2C+Jerry+Tworek%2C+Jacob+Hilton%2C+Reiichiro+Nakano%2C+Christopher+Hesse%2C+and+John+Schulman%2E+2021%2E+Train) [our card](https://md.research.elamur.ai/free/en/papers/cobbe-2021-training-verifiers-solve-math-word/)
4. Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, Amy Yang, Angela Fan, Anirudh Goyal, Anthony Hartshorn, Aobo Yang, Archi Mitra, Archie Sravankumar, Artem Korenev, Arthur Hinsvark, Arun Rao,… [find](https://scholar.google.com/scholar?q=Aaron+Grattafiori%2C+Abhimanyu+Dubey%2C+Abhinav+Jauhri%2C+Abhinav+Pandey%2C+Abhishek+Kadian%2C+Ahmad+Al%2DDahle%2C+Aiesha+Letman%2C+Akhil+Mathur%2C+Alan+Schelten%2C+Alex+Vaughan%2C+Amy+Yang%2C+Angela+Fan%2C+Anirudh+Goyal%2C+Anth)
5. Jie Huang and Kevin Chen-Chuan Chang. 2023. Towards reasoning in large language models: A survey. [find](https://scholar.google.com/scholar?q=Jie+Huang+and+Kevin+Chen%2DChuan+Chang%2E+2023%2E+Towards+reasoning+in+large+language+models%3A+A+survey%2E) [our card](https://md.research.elamur.ai/free/en/papers/huang-2023-reasoning-large-language-models-survey/)
6. Gautier Izacard and Edouard Grave. 2021. Leveraging passage retrieval with generative models for open domain question answering. [find](https://scholar.google.com/scholar?q=Gautier+Izacard+and+Edouard+Grave%2E+2021%2E+Leveraging+passage+retrieval+with+generative+models+for+open+domain+question+answering%2E) [our card](https://md.research.elamur.ai/free/en/papers/izacard-2021-leveraging-passage-retrieval-generative-models/)
7. Vladimir Karpukhin, Barlas Oğuz, Sewon Min, Patrick Lewis, Ledell Wu, Sergey Edunov, Danqi Chen, and Wen tau Yih. 2020. Dense passage retrieval for open-domain question answering. [find](https://scholar.google.com/scholar?q=Vladimir+Karpukhin%2C+Barlas+O%C4%9Fuz%2C+Sewon+Min%2C+Patrick+Lewis%2C+Ledell+Wu%2C+Sergey+Edunov%2C+Danqi+Chen%2C+and+Wen+tau+Yih%2E+2020%2E+Dense+passage+retrieval+for+open%2Ddomain+question+answering%2E) [our card](https://md.research.elamur.ai/free/en/papers/karpukhin-2020-dense-passage-retrieval-open-domain/)
8. Jaehyung Kim, Jaehyun Nam, Sangwoo Mo, Jongjin Park, Sang-Woo Lee, Minjoon Seo, Jung-Woo Ha, and Jinwoo Shin. 2024. Sure: Summarizing retrievals using answer candidates for open-domain qa of llms. [find](https://scholar.google.com/scholar?q=Jaehyung+Kim%2C+Jaehyun+Nam%2C+Sangwoo+Mo%2C+Jongjin+Park%2C+Sang%2DWoo+Lee%2C+Minjoon+Seo%2C+Jung%2DWoo+Ha%2C+and+Jinwoo+Shin%2E+2024%2E+Sure%3A+Summarizing+retrievals+using+answer+candidates+for+open%2Ddomain+qa+of+llms%2E)
9. Kenton Lee, Ming-Wei Chang, and Kristina Toutanova. 2019. Latent retrieval for weakly supervised open domain question answering. [find](https://scholar.google.com/scholar?q=Kenton+Lee%2C+Ming%2DWei+Chang%2C+and+Kristina+Toutanova%2E+2019%2E+Latent+retrieval+for+weakly+supervised+open+domain+question+answering%2E) [our card](https://md.research.elamur.ai/free/en/papers/lee-2019-latent-retrieval-weakly-supervised-open/)
10. OpenAI. 2024. Gpt-4o-mini: Advancing cost-efficient intelligence. [find](https://scholar.google.com/scholar?q=OpenAI%2E+2024%2E+Gpt%2D4o%2Dmini%3A+Advancing+cost%2Defficient+intelligence%2E)
11. Yu Wang, Nedim Lipka, Ryan A Rossi, Alexa Siu, Ruiyi Zhang, and Tyler Derr. 2024. Knowledge graph prompting for multi-document question answering. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 19206–19214. [find](https://scholar.google.com/scholar?q=Yu+Wang%2C+Nedim+Lipka%2C+Ryan+A+Rossi%2C+Alexa+Siu%2C+Ruiyi+Zhang%2C+and+Tyler+Derr%2E+2024%2E+Knowledge+graph+prompting+for+multi%2Ddocument+question+answering%2E+In+Proceedings+of+the+AAAI+Conference+on+Artificial+I)
12. Yu Wang, V.srinivasan@samsung.com V.srinivasan@samsung.com, and Hongxia Jin. 2022. A new concept of knowledge based question answering (KBQA) system for multi-hop reasoning. In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human La… [find](https://scholar.google.com/scholar?q=Yu+Wang%2C+V%2Esrinivasan%40samsung%2Ecom+V%2Esrinivasan%40samsung%2Ecom%2C+and+Hongxia+Jin%2E+2022%2E+A+new+concept+of+knowledge+based+question+answering+%28KBQA%29+system+for+multi%2Dhop+reasoning%2E+In+Proceedings+of+the+2022)
13. Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. 2022. Chain-of-thought prompting elicits reasoning in large language models. Advances in neural information processing systems, 35:24824–24837. [find](https://scholar.google.com/scholar?q=Jason+Wei%2C+Xuezhi+Wang%2C+Dale+Schuurmans%2C+Maarten+Bosma%2C+Fei+Xia%2C+Ed+Chi%2C+Quoc+V+Le%2C+Denny+Zhou%2C+et+al%2E+2022%2E+Chain%2Dof%2Dthought+prompting+elicits+reasoning+in+large+language+models%2E+Advances+in+neural+i) [our card](https://md.research.elamur.ai/free/en/papers/wei-2022-chain-thought-prompting-elicits-reasoning/)
14. Yike Wu, Nan Hu, Sheng Bi, Guilin Qi, Jie Ren, Anhuan Xie, and Wei Song. 2023. Retrieve-rewrite-answer: A kg-to-text enhanced llms framework for knowledge graph question answering. [find](https://scholar.google.com/scholar?q=Yike+Wu%2C+Nan+Hu%2C+Sheng+Bi%2C+Guilin+Qi%2C+Jie+Ren%2C+Anhuan+Xie%2C+and+Wei+Song%2E+2023%2E+Retrieve%2Drewrite%2Danswer%3A+A+kg%2Dto%2Dtext+enhanced+llms+framework+for+knowledge+graph+question+answering%2E)
15. An Yang, Beichen Zhang, Binyuan Hui, Bofei Gao, Bowen Yu, Chengpeng Li, Dayiheng Liu, Jianhong Tu, Jingren Zhou, Junyang Lin, Keming Lu, Mingfeng Xue, Runji Lin, Tianyu Liu, Xingzhang Ren, and Zhenru Zhang. 2024. Qwen2.5-math technical report: Toward mathematical expert model via self-improvement. [find](https://scholar.google.com/scholar?q=An+Yang%2C+Beichen+Zhang%2C+Binyuan+Hui%2C+Bofei+Gao%2C+Bowen+Yu%2C+Chengpeng+Li%2C+Dayiheng+Liu%2C+Jianhong+Tu%2C+Jingren+Zhou%2C+Junyang+Lin%2C+Keming+Lu%2C+Mingfeng+Xue%2C+Runji+Lin%2C+Tianyu+Liu%2C+Xingzhang+Ren%2C+and+Zhenru+) [our card](https://md.research.elamur.ai/free/en/papers/yang-2024-qwen2-5-math-technical-report/)
16. Zhehao Zhang, Jiaao Chen, and Diyi Yang. 2024. Darg: Dynamic evaluation of large language models via adaptive reasoning graph. [find](https://scholar.google.com/scholar?q=Zhehao+Zhang%2C+Jiaao+Chen%2C+and+Diyi+Yang%2E+2024%2E+Darg%3A+Dynamic+evaluation+of+large+language+models+via+adaptive+reasoning+graph%2E)
17. Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric P. Xing, Hao Zhang, Joseph E. Gonzalez, and Ion Stoica. 2023. Judging llm-as-a-judge with mt-bench and chatbot arena. 9. [find](https://scholar.google.com/scholar?q=Lianmin+Zheng%2C+Wei%2DLin+Chiang%2C+Ying+Sheng%2C+Siyuan+Zhuang%2C+Zhanghao+Wu%2C+Yonghao+Zhuang%2C+Zi+Lin%2C+Zhuohan+Li%2C+Dacheng+Li%2C+Eric+P%2E+Xing%2C+Hao+Zhang%2C+Joseph+E%2E+Gonzalez%2C+and+Ion+Stoica%2E+2023%2E+Judging+llm%2Das)
18. Language Resource References Karl Cobbe and Vineet Kosaraju and Mohammad Bavarian and Mark Chen and Heewoo Jun and Lukasz Kaiser and Matthias Plappert and Jerry Tworek and Jacob Hilton and Reiichiro Nakano and Christopher Hesse and John Schulman. 2021. Training Verifiers to Solve Math Word Problems. [find](https://scholar.google.com/scholar?q=Language+Resource+References+Karl+Cobbe+and+Vineet+Kosaraju+and+Mohammad+Bavarian+and+Mark+Chen+and+Heewoo+Jun+and+Lukasz+Kaiser+and+Matthias+Plappert+and+Jerry+Tworek+and+Jacob+Hilton+and+Reiichiro+N) [our card](https://md.research.elamur.ai/free/en/papers/cobbe-2021-training-verifiers-solve-math-word/)
19. Ho, Xanh and Duong Nguyen, Anh-Khoa and Sugawara, Saku and Aizawa, Akiko. 2020. Constructing A Multi-hop QA Dataset for Comprehensive Evaluation of Reasoning Steps. International Committee on Computational Linguistics. [find](https://scholar.google.com/scholar?q=Ho%2C+Xanh+and+Duong+Nguyen%2C+Anh%2DKhoa+and+Sugawara%2C+Saku+and+Aizawa%2C+Akiko%2E+2020%2E+Constructing+A+Multi%2Dhop+QA+Dataset+for+Comprehensive+Evaluation+of+Reasoning+Steps%2E+International+Committee+on+Computat) [our card](https://md.research.elamur.ai/free/en/papers/ho-2020-constructing-multi-hop-qa-dataset/)
20. Peter Jansen, Elizabeth Wainwright, Steven Marmorstein, and Clayton Morrison. 2018. WorldTree: A corpus of explanation graphs for elementary science questions supporting multi-hop inference. In Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018), Miy… [find](https://scholar.google.com/scholar?q=Peter+Jansen%2C+Elizabeth+Wainwright%2C+Steven+Marmorstein%2C+and+Clayton+Morrison%2E+2018%2E+WorldTree%3A+A+corpus+of+explanation+graphs+for+elementary+science+questions+supporting+multi%2Dhop+inference%2E+In+Procee) [our card](https://md.research.elamur.ai/free/en/papers/jansen-2018-worldtree-corpus-explanation-graphs-elementary/)
21. Ming Nie, Renyuan Peng, Chunwei Wang, Xinyue Cai, Jianhua Han, Hang Xu, and Li Zhang. 2023. Reason2drive: Towards interpretable and chain-based reasoning for autonomous driving. [find](https://scholar.google.com/scholar?q=Ming+Nie%2C+Renyuan+Peng%2C+Chunwei+Wang%2C+Xinyue+Cai%2C+Jianhua+Han%2C+Hang+Xu%2C+and+Li+Zhang%2E+2023%2E+Reason2drive%3A+Towards+interpretable+and+chain%2Dbased+reasoning+for+autonomous+driving%2E)
22. Trivedi, Harsh and Balasubramanian, Niranjan and Khot, Tushar and Sabharwal, Ashish. 2022. MuSiQue: Multihop Questions via Single-hop Question Composition. MIT Press One Broadway, 12th Floor, Cambridge, Massachusetts 02142, USA . . . . [find](https://scholar.google.com/scholar?q=Trivedi%2C+Harsh+and+Balasubramanian%2C+Niranjan+and+Khot%2C+Tushar+and+Sabharwal%2C+Ashish%2E+2022%2E+MuSiQue%3A+Multihop+Questions+via+Single%2Dhop+Question+Composition%2E+MIT+Press+One+Broadway%2C+12th+Floor%2C+Cambridg) [our card](https://md.research.elamur.ai/free/en/papers/trivedi-2022-musique-multihop-questions-single-hop/)
23. Yang, Zhilin and Qi, Peng and Zhang, Saizheng and Bengio, Yoshua and Cohen, William W and Salakhutdinov, Ruslan and Manning, Christopher D. 2018. HotpotQA: A dataset for diverse, explainable multi-hop question answering. [find](https://scholar.google.com/scholar?q=Yang%2C+Zhilin+and+Qi%2C+Peng+and+Zhang%2C+Saizheng+and+Bengio%2C+Yoshua+and+Cohen%2C+William+W+and+Salakhutdinov%2C+Ruslan+and+Manning%2C+Christopher+D%2E+2018%2E+HotpotQA%3A+A+dataset+for+diverse%2C+explainable+multi%2Dhop) [our card](https://md.research.elamur.ai/free/en/papers/yang-2018-hotpotqa-dataset-diverse-explainable-multi/)
24. Michihiro Yasunaga, Hongyu Ren, Antoine Bosselut, Percy Liang, and Jure Leskovec. 2021. QA-GNN: Reasoning with language models and knowledge graphs for question answering. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Lang… [find](https://scholar.google.com/scholar?q=Michihiro+Yasunaga%2C+Hongyu+Ren%2C+Antoine+Bosselut%2C+Percy+Liang%2C+and+Jure+Leskovec%2E+2021%2E+QA%2DGNN%3A+Reasoning+with+language+models+and+knowledge+graphs+for+question+answering%2E+In+Proceedings+of+the+2021+C)
25. Zhehao Zhang and Jiaao Chen and Diyi Yang. 2024. DARG: Dynamic Evaluation of Large Language Models via Adaptive Reasoning Graph. [find](https://scholar.google.com/scholar?q=Zhehao+Zhang+and+Jiaao+Chen+and+Diyi+Yang%2E+2024%2E+DARG%3A+Dynamic+Evaluation+of+Large+Language+Models+via+Adaptive+Reasoning+Graph%2E)