---
title: "GR1: Reinforcement-Enhanced LLM for Geoscience Reasoning"
description: "This paper tests whether it also helps in geoscience, where questions combine factual recall, multi-step inference and often… 37 references."
authors: ["Yule Xie", "Jiaxin Ding", "Cheng Deng", "Shiqing Gao", "Junran Zhang", "Sibo Zhang", "Zeyuan Wang", "Ke Wu", "Xin Ding", "Luoyi Fu", "Meng Jin", "Xinbing Wang"]
year: 2026
venue: "Findings of the Association for Computational Linguistics: ACL 2026"
source: "https://aclanthology.org/2026.findings-acl.1140/"
doi: "10.18653/v1/2026.findings-acl.1140"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/xie-2026-gr1-reinforcement-enhanced-llm-geoscience/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/xie-2026-gr1-reinforcement-enhanced-llm-geoscience/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2026](https://md.research.elamur.ai/free/en/papers/2026/page/35/) / GR1: Reinforcement-Enhanced LLM for Geoscience Reasoning

# GR1: Reinforcement-Enhanced LLM for Geoscience Reasoning

**Authors:** Yule Xie, Jiaxin Ding, [Cheng Deng](https://md.research.elamur.ai/free/en/authors/cheng-deng/), Shiqing Gao, Junran Zhang, Sibo Zhang, Zeyuan Wang, Ke Wu, Xin Ding, Luoyi Fu, Meng Jin and Xinbing Wang · **Year:** 2026 · **Venue:** [Findings of the Association for Computational Linguistics: ACL 2026](https://md.research.elamur.ai/free/en/venues/findings/) ([Findings of the Association for Computational Linguistics 2026](https://md.research.elamur.ai/free/en/venues/findings/2026/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2026.findings-acl.1140/) · [PDF](https://aclanthology.org/2026.findings-acl.1140.pdf) · [DOI 10.18653/v1/2026.findings-acl.1140](https://doi.org/10.18653/v1/2026.findings-acl.1140) · [OpenAlex](https://openalex.org/W7166863554)

**Citations:** 0 citations (OpenAlex, 2026-10-14)

## Summary
Reinforcement learning has improved LLM reasoning mostly in mathematics. This paper tests whether it also helps in geoscience, where questions combine factual recall, multi-step inference and often visual material such as maps or cross-sections.

The authors built GeoMC-10K, a benchmark of 10,000 multiple-choice questions. The items come from school exams and professional certification sources, and roughly a third of them depend on images. Training a text-only RL model on those image-based items requires a text version of each figure. For this they designed GeoM2T, a pipeline of three cooperating LLM agents. It rewrites figures as text while trying to keep each question answerable without making it easier.

They then trained LLaMA-3.1-8B and Qwen-3-8B with GRPO. Besides rewarding correct answers, the training adds a factual reward that gives partial credit for accurate scientific statements. The resulting models, called GR1, gain substantially in accuracy, most of all on reasoning-heavy questions. The 8B Qwen variant beats larger general-purpose and geoscience-specific models, and the gains carry over to out-of-distribution benchmarks. The work provides a verifiable resource and a baseline for RL in scientific domains.

## Key points
- **GeoMC-10K sources:** NYSED Earth Science Regents, China's Gaokao, AP tests, Engineering Geology MCQs, ASBOG and the Geoscience Concept Inventory (GCI). Of the 10,000 items, 3,381 are multimodal.
- **GeoM2T pipeline:** an Extractor drafts a text description of the image, a Critic checks whether the question can be answered from it, and an Advisor proposes revisions. The cycle repeats until the draft is complete without leaking the answer.
- **Main accuracy results:**
  - LLaMA-3.1-8B rose from 62.64% to 68.57% (+5.93 ppt).
  - Qwen-3-8B rose from 65.59% to 78.91% (+13.32 ppt).
  - On reasoning-oriented questions, Qwen improved by 20.53 ppt.
- **Factual reward:** it adds about 1.6–1.8 ppt of accuracy, which is roughly 17.2% of the total gain.
- **Out-of-distribution tests:**
  - On IESO (Olympiad questions, 2013–2019), GR1-Qwen reached 74.85%, compared with 53.27% for the base model.
  - On GeOpen (open-ended postgraduate questions, scored by GPT-4.1-mini as judge), GR1-Qwen reached 94.83%, compared with 58.50%.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [University of Edinburgh](https://md.research.elamur.ai/free/en/universities/university-of-edinburgh/) — University, Edinburgh, United Kingdom · 381 works · 80,747 citations
- [Shanghai Jiao Tong University](https://md.research.elamur.ai/free/en/universities/shanghai-jiao-tong-university/) — University, Shanghai, China · 212 works · 19,544 citations

📚 Shelves (5): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 18,614 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 17,965 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 17,450 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 17,296 works · [👁️ Computer Vision and Pattern Recognition](https://md.research.elamur.ai/free/en/shelves/computer-science/computer-vision-and-pattern-recognition/) · 3,486 works

🧭 Topics (2): [🧠 Language models & pretraining](https://md.research.elamur.ai/free/en/topics/language-models/) · 3,894 works · [🧮 Reasoning](https://md.research.elamur.ai/free/en/topics/reasoning/) · 1,685 works

## Cite

Cite as: Yule Xie, Jiaxin Ding, Cheng Deng, Shiqing Gao, Junran Zhang, Sibo Zhang, Zeyuan Wang, Ke Wu, Xin Ding, Luoyi Fu, Meng Jin and Xinbing Wang. 2026. GR1: Reinforcement-Enhanced LLM for Geoscience Reasoning. Findings of the Association for Computational Linguistics: ACL 2026.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{xie2026gr1,
  title = {{GR1: Reinforcement-Enhanced LLM for Geoscience Reasoning}},
  author = {Yule Xie, Jiaxin Ding, Cheng Deng, Shiqing Gao, Junran Zhang, Sibo Zhang, Zeyuan Wang, Ke Wu, Xin Ding, Luoyi Fu, Meng Jin and Xinbing Wang},
  booktitle = {Findings of the Association for Computational Linguistics: ACL 2026},
  year = {2026},
  doi = {10.18653/v1/2026.findings-acl.1140},
  url = {https://doi.org/10.18653/v1/2026.findings-acl.1140}
}
```

- [← previous GQLBench: A Large-Scale Cross-Domain, Cross-Dialect Benchmark for NL2GQL](https://md.research.elamur.ai/free/en/papers/su-2026-gqlbench-large-scale-cross-domain/)
- [next → GrACE: A Generative Approach to Better Confidence Elicitation and Efficient Test-Time Scaling in Large Language Models](https://md.research.elamur.ai/free/en/papers/zhang-2026-grace-generative-approach-better-confidence/)
- [↑ back to papers of 2026](https://md.research.elamur.ai/free/en/papers/2026/page/35/)

## References

37 references, 17 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Alon Albalak, Duy Phung, Nathan Lile, Rafael Rafailov, Kanishk Gandhi, Louis Castricato, Anikait Singh, Chase Blagden, Violet Xiang, Dakota Mahan, and et al. 2025. Big-math: A large-scale, high-quality math dataset for reinforcement learning in language models. arXiv preprint arXiv:2502.17387. [find](https://scholar.google.com/scholar?q=Alon+Albalak%2C+Duy+Phung%2C+Nathan+Lile%2C+Rafael+Rafailov%2C+Kanishk+Gandhi%2C+Louis+Castricato%2C+Anikait+Singh%2C+Chase+Blagden%2C+Violet+Xiang%2C+Dakota+Mahan%2C+and+et+al%2E+2025%2E+Big%2Dmath%3A+A+large%2Dscale%2C+high%2Dqualit)
2. Aida Amini, Saadia Gabriel, Shanchuan Lin, Rik Koncel-Kedziorski, Yejin Choi, and Hannaneh Hajishirzi. 2019. [MathQA: Towards interpretable math word problem solving with operation-based formalisms](https://md.research.elamur.ai/free/en/papers/amini-2019-mathqa-interpretable-math-word-problem/). Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational…
3. Yonatan Bisk, Rowan Zellers, Jianfeng Gao, Yejin Choi, and et al. 2020. [Piqa: Reasoning about physical commonsense in natural language](https://md.research.elamur.ai/free/en/papers/bisk-2019-piqa-reasoning-about-physical-commonsense/). Proceedings of the AAAI conference on artificial intelligence, 34(05):7432–7439.
4. David Chan, Austin Myers, Sudheendra Vijayanarasimhan, David Ross, and John Canny. 2023. Ic3: Image captioning by committee consensus. Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 8975–9003. [find](https://scholar.google.com/scholar?q=David+Chan%2C+Austin+Myers%2C+Sudheendra+Vijayanarasimhan%2C+David+Ross%2C+and+John+Canny%2E+2023%2E+Ic3%3A+Image+captioning+by+committee+consensus%2E+Proceedings+of+the+2023+Conference+on+Empirical+Methods+in+Natura)
5. Hardy Chen, Haoqin Tu, Fali Wang, Hui Liu, Xianfeng Tang, Xinya Du, Yuyin Zhou, and Cihang Xie. 2025a. [Sft or rl? an early investigation into training r1-like reasoning large vision-language models](https://md.research.elamur.ai/free/en/papers/chen-2025-sft-rl-early-investigation-training/). arXiv preprint arXiv:2504.11468.
6. Junying Chen, Zhenyang Cai, Ke Ji, Xidong Wang, Wanlong Liu, Rongsheng Wang, Jianye Hou, and Benyou Wang. 2024. [Huatuogpt-o1, towards medical complex reasoning with llms](https://md.research.elamur.ai/free/en/papers/chen-2024-huatuogpt-o1-towards-medical-complex/). arXiv preprint arXiv:2412.18925.
7. Xilun Chen, Ilia Kulikov, Vincent-Pierre Berges, Barlas Oğuz, Rulin Shao, Gargi Ghosh, Jason Weston, and Wen-tau Yih. 2025b. Learning to reason for factuality. arXiv preprint arXiv:2508.05618. — [arXiv:2508.05618](https://arxiv.org/abs/2508.05618) [find](https://scholar.google.com/scholar?q=Xilun+Chen%2C+Ilia+Kulikov%2C+Vincent%2DPierre+Berges%2C+Barlas+Og%CC%86uz%2C+Rulin+Shao%2C+Gargi+Ghosh%2C+Jason+Weston%2C+and+Wen%2Dtau+Yih%2E+2025b%2E+Learning+to+reason+for+factuality%2E+arXiv+preprint+arXiv%3A2508%2E05618%2E+%E2%80%94+%5BarX)
8. Paul F Christiano, Jan Leike, Tom Brown, Miljan Martic, Shane Legg, and Dario Amodei. 2017. Deep reinforcement learning from human preferences. Advances in neural information processing systems, 30. [find](https://scholar.google.com/scholar?q=Paul+F+Christiano%2C+Jan+Leike%2C+Tom+Brown%2C+Miljan+Martic%2C+Shane+Legg%2C+and+Dario+Amodei%2E+2017%2E+Deep+reinforcement+learning+from+human+preferences%2E+Advances+in+neural+information+processing+systems%2C+30%2E)
9. Christopher Clark, Kenton Lee, Ming-Wei Chang, Tom Kwiatkowski, Michael Collins, and Kristina Toutanova. 2019. [Boolq: Exploring the surprising difficulty of natural yes/no questions](https://md.research.elamur.ai/free/en/papers/clark-2019-boolq-exploring-surprising-difficulty-natural/). arXiv preprint arXiv:1905.10044.
10. Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, Christopher Hesse, and John Schulman. 2021. [Training verifiers to solve math word problems](https://md.research.elamur.ai/free/en/papers/cobbe-2021-training-verifiers-solve-math-word/). arXiv preprint arXiv:2110.14168.
11. Cheng Deng, Tianhang Zhang, Zhongmou He, Qiyuan Chen, Yuanyuan Shi, Yi Xu, Luoyi Fu, Weinan Zhang, Xinbing Wang, Chenghu Zhou, Zhouhan Lin, and Junxian He. 2024. K2: A foundation language model for geoscience knowledge understanding and utilization. Proceedings of the 17th ACM International Conferen… [find](https://scholar.google.com/scholar?q=Cheng+Deng%2C+Tianhang+Zhang%2C+Zhongmou+He%2C+Qiyuan+Chen%2C+Yuanyuan+Shi%2C+Yi+Xu%2C+Luoyi+Fu%2C+Weinan+Zhang%2C+Xinbing+Wang%2C+Chenghu+Zhou%2C+Zhouhan+Lin%2C+and+Junxian+He%2E+2024%2E+K2%3A+A+foundation+language+model+for+ge)
12. Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, and et al. 2024. [The llama 3 herd of models](https://md.research.elamur.ai/free/en/papers/grattafiori-2024-llama-3-herd-models/). arXiv e-prints, pages arXiv–2407.
13. Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. 2020. [Measuring massive multitask language understanding](https://md.research.elamur.ai/free/en/papers/hendrycks-2020-measuring-massive-multitask-language-understanding/). arXiv preprint arXiv:2009.03300.
14. Yushi Hu, Hang Hua, Zhengyuan Yang, Weijia Shi, Noah A Smith, and Jiebo Luo. 2023. Promptcap: Prompt-guided image captioning for vqa with gpt-3. Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 2963–2975. [find](https://scholar.google.com/scholar?q=Yushi+Hu%2C+Hang+Hua%2C+Zhengyuan+Yang%2C+Weijia+Shi%2C+Noah+A+Smith%2C+and+Jiebo+Luo%2E+2023%2E+Promptcap%3A+Prompt%2Dguided+image+captioning+for+vqa+with+gpt%2D3%2E+Proceedings+of+the+IEEE%2FCVF+International+Conference+on)
15. James Kirkpatrick, Razvan Pascanu, Neil Rabinowitz, Joel Veness, Guillaume Desjardins, Andrei A Rusu, Kieran Milan, John Quan, Tiago Ramalho, Agnieszka Grabska-Barwinska, and et al. 2017. Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences, 114(13):… [find](https://scholar.google.com/scholar?q=James+Kirkpatrick%2C+Razvan+Pascanu%2C+Neil+Rabinowitz%2C+Joel+Veness%2C+Guillaume+Desjardins%2C+Andrei+A+Rusu%2C+Kieran+Milan%2C+John+Quan%2C+Tiago+Ramalho%2C+Agnieszka+Grabska%2DBarwinska%2C+and+et+al%2E+2017%2E+Overcoming+c)
16. Sergey Levine. 2018. Reinforcement learning and control as probabilistic inference: Tutorial and review. arXiv preprint arXiv:1805.00909. — [arXiv:1805.00909](https://arxiv.org/abs/1805.00909) [find](https://scholar.google.com/scholar?q=Sergey+Levine%2E+2018%2E+Reinforcement+learning+and+control+as+probabilistic+inference%3A+Tutorial+and+review%2E+arXiv+preprint+arXiv%3A1805%2E00909%2E+%E2%80%94+%5BarXiv%3A1805%2E00909%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F1805%2E00909%29)
17. Aitor Lewkowycz, Anders Andreassen, David Dohan, Ethan Dyer, Henryk Michalewski, Vinay Ramasesh, Ambrose Slone, Cem Anil, Imanol Schlag, Theo Gutman-Solo, and et al. 2022. [Solving quantitative reasoning problems with language models](https://md.research.elamur.ai/free/en/papers/lewkowycz-2022-solving-quantitative-reasoning-problems-language/). Advances in neural information processing systems, 35:3843– 3857.
18. Zhouhan Lin, Cheng Deng, Le Zhou, Tianhang Zhang, Yi Xu, Yutong Xu, Zhongmou He, Yuanyuan Shi, Beiya Dai, Yunchong Song, and et al. 2023. Geogalactica: A scientific large language model in geoscience. arXiv preprint arXiv:2401.00434. — [arXiv:2401.00434](https://arxiv.org/abs/2401.00434) [find](https://scholar.google.com/scholar?q=Zhouhan+Lin%2C+Cheng+Deng%2C+Le+Zhou%2C+Tianhang+Zhang%2C+Yi+Xu%2C+Yutong+Xu%2C+Zhongmou+He%2C+Yuanyuan+Shi%2C+Beiya+Dai%2C+Yunchong+Song%2C+and+et+al%2E+2023%2E+Geogalactica%3A+A+scientific+large+language+model+in+geoscience%2E)
19. Pan Lu, Swaroop Mishra, Tony Xia, Liang Qiu, Kai-Wei Chang, Song-Chun Zhu, Oyvind Tafjord, Peter Clark, and Ashwin Kalyan. 2022. Learn to explain: Multimodal reasoning via thought chains for science question answering. Advances in neural information processing systems. [find](https://scholar.google.com/scholar?q=Pan+Lu%2C+Swaroop+Mishra%2C+Tony+Xia%2C+Liang+Qiu%2C+Kai%2DWei+Chang%2C+Song%2DChun+Zhu%2C+Oyvind+Tafjord%2C+Peter+Clark%2C+and+Ashwin+Kalyan%2E+2022%2E+Learn+to+explain%3A+Multimodal+reasoning+via+thought+chains+for+science+q)
20. Yun Luo, Zhen Yang, Fandong Meng, Yafu Li, Jie Zhou, and Yue Zhang. 2023. An empirical study of catastrophic forgetting in large language models during continual fine-tuning. arXiv preprint arXiv:2308.08747. [find](https://scholar.google.com/scholar?q=Yun+Luo%2C+Zhen+Yang%2C+Fandong+Meng%2C+Yafu+Li%2C+Jie+Zhou%2C+and+Yue+Zhang%2E+2023%2E+An+empirical+study+of+catastrophic+forgetting+in+large+language+models+during+continual+fine%2Dtuning%2E+arXiv+preprint+arXiv%3A2308)
21. Yan Ma, Steffi Chern, Xuyang Shen, Yiran Zhong, and Pengfei Liu. 2025. Rethinking rl scaling for vision language models: A transparent, from-scratch framework and comprehensive evaluation scheme. arXiv preprint arXiv:2504.02587. — [arXiv:2504.02587](https://arxiv.org/abs/2504.02587) [find](https://scholar.google.com/scholar?q=Yan+Ma%2C+Steffi+Chern%2C+Xuyang+Shen%2C+Yiran+Zhong%2C+and+Pengfei+Liu%2E+2025%2E+Rethinking+rl+scaling+for+vision+language+models%3A+A+transparent%2C+from%2Dscratch+framework+and+comprehensive+evaluation+scheme%2E+arXi)
22. Shiwen Ni, Xiangtao Kong, Chengming Li, Xiping Hu, Ruifeng Xu, Jia Zhu, and Min Yang. 2025. Training on the benchmark is not all you need. Proceedings of the AAAI Conference on Artificial Intelligence. [find](https://scholar.google.com/scholar?q=Shiwen+Ni%2C+Xiangtao+Kong%2C+Chengming+Li%2C+Xiping+Hu%2C+Ruifeng+Xu%2C+Jia+Zhu%2C+and+Min+Yang%2E+2025%2E+Training+on+the+benchmark+is+not+all+you+need%2E+Proceedings+of+the+AAAI+Conference+on+Artificial+Intelligence)
23. Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, and et al. 2022. [Training language models to follow instructions with human feedback](https://md.research.elamur.ai/free/en/papers/ouyang-2022-training-language-models-follow-instructions/). Advances in neural information processing systems, 35:27730–27744.
24. D Paperno, G Kruszewski, A Lazaridou, QN Pham, Raffaella Bernardi, S Pezzelle, M Baroni, G Boleda, and R Fernández. 2016. [The lambada dataset: Word prediction requiring a broad discourse context](https://md.research.elamur.ai/free/en/papers/paperno-2016-lambada-dataset-word-prediction-requiring/). 54th Annual Meeting of the Association for Computational Linguistics, ACL 2016-Long Papers, 3:1525–1534.
25. Team Qwen. 2024. Qwq: Reflect deeply on the boundaries of the unknown. Hugging Face. [find](https://scholar.google.com/scholar?q=Team+Qwen%2E+2024%2E+Qwq%3A+Reflect+deeply+on+the+boundaries+of+the+unknown%2E+Hugging+Face%2E)
26. Team Qwen. 2025. [Qwen3 technical report](https://md.research.elamur.ai/free/en/papers/yang-2025-qwen3-technical-report/). Preprint, arXiv:2505.09388.
27. Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D Manning, Stefano Ermon, and Chelsea Finn. 2023. [Direct preference optimization: Your language model is secretly a reward model](https://md.research.elamur.ai/free/en/papers/rafailov-2023-direct-preference-optimization-language-model/). Advances in neural information processing systems, 36:53728–53741.
28. John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. 2017. Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347. [find](https://scholar.google.com/scholar?q=John+Schulman%2C+Filip+Wolski%2C+Prafulla+Dhariwal%2C+Alec+Radford%2C+and+Oleg+Klimov%2E+2017%2E+Proximal+policy+optimization+algorithms%2E+arXiv+preprint+arXiv%3A1707%2E06347%2E)
29. Science Custom Publishing. 2024. Beyond boundaries: Geogpt’s evolution in geosciences. Science. Sponsored letter. [find](https://scholar.google.com/scholar?q=Science+Custom+Publishing%2E+2024%2E+Beyond+boundaries%3A+Geogpt%E2%80%99s+evolution+in+geosciences%2E+Science%2E+Sponsored+letter%2E)
30. Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, YK Li, Yang Wu, and et al. 2024. [Deepseekmath: Pushing the limits of mathematical reasoning in open language models](https://md.research.elamur.ai/free/en/papers/shao-2024-deepseekmath-pushing-limits-mathematical-reasoning/). arXiv preprint arXiv:2402.03300.
31. Noah Shinn, Federico Cassano, Ashwin Gopinath, Karthik Narasimhan, and Shunyu Yao. 2023. [Reflexion: Language agents with verbal reinforcement learning](https://md.research.elamur.ai/free/en/papers/shinn-2023-reflexion-language-agents-verbal-reinforcement/). Advances in Neural Information Processing Systems, 36:8634–8652.
32. Yixiao Song, Yekyung Kim, and Mohit Iyyer. 2024. [VeriScore: Evaluating the factuality of verifiable claims in long-form text generation](https://md.research.elamur.ai/free/en/papers/song-2024-veriscore-evaluating-factuality-verifiable-claims/). Findings of the Association for Computational Linguistics: EMNLP 2024.
33. Engineering Geology MCQs. We collect about 3,000 professional-level engineering geology questions from Sanfoundry. These items span subjects including soil mechanics, rock classification, groundwater flow, and metamorphic processes, making it a challenge for LLMs. Both this dataset and the previous… [find](https://scholar.google.com/scholar?q=Engineering+Geology+MCQs%2E+We+collect+about+3%2C000+professional%2Dlevel+engineering+geology+questions+from+Sanfoundry%2E+These+items+span+subjects+including+soil+mechanics%2C+rock+classification%2C+groundwater+)
34. Rowan Zellers, Ari Holtzman, Yonatan Bisk, Ali Farhadi, and Yejin Choi. 2019. Hellaswag: Can a machine really finish your sentence? Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 4791–4800. [find](https://scholar.google.com/scholar?q=Rowan+Zellers%2C+Ari+Holtzman%2C+Yonatan+Bisk%2C+Ali+Farhadi%2C+and+Yejin+Choi%2E+2019%2E+Hellaswag%3A+Can+a+machine+really+finish+your+sentence%3F+Proceedings+of+the+57th+Annual+Meeting+of+the+Association+for+Comput)
35. Ruohong Zhang, Bowen Zhang, Yanghao Li, Haotian Zhang, Zhiqing Sun, Zhe Gan, Yinfei Yang, Ruoming Pang, and Yiming Yang. 2024. [Improve vision language model chain-of-thought reasoning](https://md.research.elamur.ai/free/en/papers/zhang-2025-improve-vision-language-model-chain/). arXiv preprint arXiv:2410.16198.
36. ASBOG Exam Outline Questions. We extract the sample multiple-choice questions provided at the end of the official ASBOG exam specifications (Association of State Boards of Geology). Although only a few dozen items are included, they cover key professional topics and are representative under relevant… [find](https://scholar.google.com/scholar?q=ASBOG+Exam+Outline+Questions%2E+We+extract+the+sample+multiple%2Dchoice+questions+provided+at+the+end+of+the+official+ASBOG+exam+specifications+%28Association+of+State+Boards+of+Geology%29%2E+Although+only+a+fe)
37. Xiaotian Zhang, Yuan Wang, Zhaopeng Feng, Ruizhe Chen, Zhijie Zhou, Yan Zhang, Hongxia Xu, Jian Wu, and Zuozhu Liu. 2025. Med-u1: Incentivizing unified medical reasoning in llms via large-scale reinforcement learning. arXiv preprint arXiv:2506.12307. — [arXiv:2506.12307](https://arxiv.org/abs/2506.12307) [find](https://scholar.google.com/scholar?q=Xiaotian+Zhang%2C+Yuan+Wang%2C+Zhaopeng+Feng%2C+Ruizhe+Chen%2C+Zhijie+Zhou%2C+Yan+Zhang%2C+Hongxia+Xu%2C+Jian+Wu%2C+and+Zuozhu+Liu%2E+2025%2E+Med%2Du1%3A+Incentivizing+unified+medical+reasoning+in+llms+via+large%2Dscale+reinfo)

*Updated 2 November 2026*