---
title: "LogicPro — paper by Jiang et al. (2025) — summary and ideas"
description: "Large language models often fail at multi-step logical reasoning when a task is posed in plain text… 40 references, cited by 7 papers here."
authors: ["Jin Jiang", "Yuchen Yan", "Yang Liu", "Jianing Wang", "Shuai Peng", "Xunliang Cai", "Yixin Cao", "Mengdi Zhang", "Liangcai Gao"]
year: 2025
venue: "Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)"
source: "https://aclanthology.org/2025.acl-long.1270/"
doi: "10.18653/v1/2025.acl-long.1270"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/jiang-2025-logicpro-improving-complex-logical-reasoning/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/jiang-2025-logicpro-improving-complex-logical-reasoning/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2025](https://md.research.elamur.ai/free/en/papers/2025/page/9/) / LogicPro: Improving Complex Logical Reasoning via Program-Guided Learning

# LogicPro: Improving Complex Logical Reasoning via Program-Guided Learning

**Authors:** [Jin Jiang](https://md.research.elamur.ai/free/en/authors/jin-jiang/), [Yuchen Yan](https://md.research.elamur.ai/free/en/authors/yuchen-yan/), [Yang Liu](https://md.research.elamur.ai/free/en/authors/yang-liu/), [Jianing Wang](https://md.research.elamur.ai/free/en/authors/jianing-wang/), Shuai Peng, [Xunliang Cai](https://md.research.elamur.ai/free/en/authors/xunliang-cai/), [Yixin Cao](https://md.research.elamur.ai/free/en/authors/yixin-cao/), [Mengdi Zhang](https://md.research.elamur.ai/free/en/authors/mengdi-zhang/) and Liangcai Gao · **Year:** 2025 · **Venue:** [Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)](https://md.research.elamur.ai/free/en/venues/acl/) ([ACL 2025](https://md.research.elamur.ai/free/en/venues/acl/2025/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2025.acl-long.1270/) · [PDF](https://aclanthology.org/2025.acl-long.1270.pdf) · [DOI 10.18653/v1/2025.acl-long.1270](https://doi.org/10.18653/v1/2025.acl-long.1270) · [OpenAlex](https://openalex.org/W4412889878)

**Citations:** 1 citations (OpenAlex, 2026-10-14) · [7](https://md.research.elamur.ai/free/en/papers/jiang-2025-logicpro-improving-complex-logical-reasoning/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
Large language models often fail at multi-step logical reasoning when a task is posed in plain text, even when the same task is easy to solve with code. Earlier work on synthetic logic data mostly relied on propositional logic or formal languages. LogicPro uses algorithmic programming problems instead. Starting from 2,360 official LeetCode problems with reference Python solutions, the authors prompt an LLM to turn each problem plus a concrete test case into a story-like text reasoning question. They then run test-case-specific code to get both the correct answer and the values of key intermediate variables. Those execution traces guide an LLM in writing a step-by-step natural-language solution. Filtering steps remove questions that disagree with their code or cannot be solved. The result is a 540K-example training set with verified answers and grounded reasoning chains. Models fine-tuned on it improve on several out-of-distribution benchmarks and beat many existing reasoning datasets. This shows that code execution is a scalable, verifiable source of logic supervision.

## Key points
- Seeds: 2,360 LeetCode problems with standard Python solutions. GPT-4 generated test cases by producing 150 per problem in three sampling rounds, then merging, deduplicating and filtering them, for up to 300 test cases per problem.
- Pipeline: (1) rewrite the problem and test case into a text reasoning question with added background, (2) run the code to extract the answer and intermediate variables, (3) generate a program-guided textual reasoning process.
- Data flow: 699K questions were synthesized. A Problem-Program Consistency Check removed 5K+ of them, and a Solvability Check also filtered entries, leaving 595K. The final dataset has 540K examples.
- Reported gains cover BBH 27, LogicBench, DROP, AR-LSAT and GSM8K across multiple models. Code and data are at github.com/jiangjin1999/LogicPro.
- Proposed extensions include more seed problems from Luogu, ACM contests and other online judges. The authors also suggest using answers and intermediate variables as outcome and process signals for reinforcement learning.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 16,039 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 15,891 works

🧭 Topics (1): [🧮 Reasoning](https://md.research.elamur.ai/free/en/topics/reasoning/) · 1,426 works

## Cite

Cite as: Jin Jiang, Yuchen Yan, Yang Liu, Jianing Wang, Shuai Peng, Xunliang Cai, Yixin Cao, Mengdi Zhang and Liangcai Gao. 2025. LogicPro: Improving Complex Logical Reasoning via Program-Guided Learning. Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers).

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{jiang2025logicpro,
  title = {{LogicPro: Improving Complex Logical Reasoning via Program-Guided Learning}},
  author = {Jin Jiang, Yuchen Yan, Yang Liu, Jianing Wang, Shuai Peng, Xunliang Cai, Yixin Cao, Mengdi Zhang and Liangcai Gao},
  booktitle = {Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2025},
  doi = {10.18653/v1/2025.acl-long.1270},
  url = {https://doi.org/10.18653/v1/2025.acl-long.1270}
}
```

- [← previous LiTEx: A Linguistic Taxonomy of Explanations for Understanding Within-Label Variation in Natural Language Inference](https://md.research.elamur.ai/free/en/papers/hong-2025-litex-linguistic-taxonomy-explanations-understanding/)
- [next → LuxEmbedder: A Cross-Lingual Approach to Enhanced Luxembourgish Sentence Embeddings](https://md.research.elamur.ai/free/en/papers/philippy-2025-luxembedder-cross-lingual-approach-enhanced/)
- [↑ back to papers of 2025](https://md.research.elamur.ai/free/en/papers/2025/page/9/)

## References

40 references, 17 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Bo Adler, Niket Agarwal, Ashwath Aithal, Dong H Anh, Pallab Bhattacharya, Annika Brundyn, Jared Casper, Bryan Catanzaro, Sharon Clay, Jonathan Cohen, et al. 2024. [Nemotron-4 340b technical report](https://md.research.elamur.ai/free/en/papers/adler-2024-nemotron-4-340b-technical-report/). arXiv preprint arXiv:2406.11704.
2. Xiao Bi, Deli Chen, Guanting Chen, Shanhuang Chen, Damai Dai, Chengqi Deng, Honghui Ding, Kai Dong, Qiushi Du, Zhe Fu, et al. 2024. Deepseek llm: Scaling open-source language models with longtermism. arXiv preprint arXiv:2401.02954. [find](https://scholar.google.com/scholar?q=Xiao+Bi%2C+Deli+Chen%2C+Guanting+Chen%2C+Shanhuang+Chen%2C+Damai+Dai%2C+Chengqi+Deng%2C+Honghui+Ding%2C+Kai+Dong%2C+Qiushi+Du%2C+Zhe+Fu%2C+et+al%2E+2024%2E+Deepseek+llm%3A+Scaling+open%2Dsource+language+models+with+longtermism%2E+)
3. Xiancai Chen, Zhengwei Tao, Kechi Zhang, Changzhi Zhou, Wanli Gu, Yuanpeng He, Mengdi Zhang, Xunliang Cai, Haiyan Zhao, and Zhi Jin. 2025. Revisit self-debugging with self-generated tests for code generation. arXiv preprint arXiv:2501.12793. [find](https://scholar.google.com/scholar?q=Xiancai+Chen%2C+Zhengwei+Tao%2C+Kechi+Zhang%2C+Changzhi+Zhou%2C+Wanli+Gu%2C+Yuanpeng+He%2C+Mengdi+Zhang%2C+Xunliang+Cai%2C+Haiyan+Zhao%2C+and+Zhi+Jin%2E+2025%2E+Revisit+self%2Ddebugging+with+self%2Dgenerated+tests+for+code+gen)
4. Peter Clark, Oyvind Tafjord, and Kyle Richardson. 2021. [Transformers as soft reasoners over language](https://md.research.elamur.ai/free/en/papers/clark-2020-transformers-soft-reasoners-over-language/). In Proceedings of the Twenty-Ninth International Conference on International Joint Conferences on Artificial Intelligence, pages 3882–3890.
5. Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, et al. 2021. [Training verifiers to solve math word problems](https://md.research.elamur.ai/free/en/papers/cobbe-2021-training-verifiers-solve-math-word/). arXiv preprint arXiv:2110.14168.
6. Bhavana Dalvi, Peter Jansen, Oyvind Tafjord, Zhengnan Xie, Hannah Smith, Leighanna Pipatanangkura, and Peter Clark. 2021. [Explaining answers with entailment trees](https://md.research.elamur.ai/free/en/papers/dalvi-2021-explaining-answers-entailment-trees/). In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 7358–7370.
7. Dheeru Dua, Yizhong Wang, Pradeep Dasigi, Gabriel Stanovsky, Sameer Singh, and Matt Gardner. 2019. [Drop: A reading comprehension benchmark requiring discrete reasoning over paragraphs](https://md.research.elamur.ai/free/en/papers/dua-2019-drop-reading-comprehension-benchmark-requiring/). In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistic…
8. Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, et al. 2024. [The llama 3 herd of models](https://md.research.elamur.ai/free/en/papers/grattafiori-2024-llama-3-herd-models/). arXiv preprint arXiv:2407.21783.
9. Luyu Gao, Aman Madaan, Shuyan Zhou, Uri Alon, Pengfei Liu, Yiming Yang, Jamie Callan, and Graham Neubig. 2023. Pal: Program-aided language models. In International Conference on Machine Learning, pages 10764–10799. PMLR. [find](https://scholar.google.com/scholar?q=Luyu+Gao%2C+Aman+Madaan%2C+Shuyan+Zhou%2C+Uri+Alon%2C+Pengfei+Liu%2C+Yiming+Yang%2C+Jamie+Callan%2C+and+Graham+Neubig%2E+2023%2E+Pal%3A+Program%2Daided+language+models%2E+In+International+Conference+on+Machine+Learning%2C+page)
10. Simeng Han, Hailey Schoelkopf, Yilun Zhao, Zhenting Qi, Martin Riddell, Wenfei Zhou, James Coady, David Peng, Yujie Qiao, Luke Benson, et al. 2022. [Folio: Natural language reasoning with first-order logic](https://md.research.elamur.ai/free/en/papers/han-2022-folio-natural-language-reasoning-first/). arXiv preprint arXiv:2209.00840.
11. Yilun Hao, Yang Zhang, and Chuchu Fan. 2024. Planning anything with rigor: General-purpose zero-shot planning with llm-based formalized programming. arXiv preprint arXiv:2410.12112. — [arXiv:2410.12112](https://arxiv.org/abs/2410.12112) [find](https://scholar.google.com/scholar?q=Yilun+Hao%2C+Yang+Zhang%2C+and+Chuchu+Fan%2E+2024%2E+Planning+anything+with+rigor%3A+General%2Dpurpose+zero%2Dshot+planning+with+llm%2Dbased+formalized+programming%2E+arXiv+preprint+arXiv%3A2410%2E12112%2E+%E2%80%94+%5BarXiv%3A2410%2E1211)
12. Ioannis Hatzilygeroudis and Jim Prentzas. 2004. Neuro-symbolic approaches for knowledge representation in expert systems. International Journal of Hybrid Intelligent Systems, 1(3-4):111–126. [find](https://scholar.google.com/scholar?q=Ioannis+Hatzilygeroudis+and+Jim+Prentzas%2E+2004%2E+Neuro%2Dsymbolic+approaches+for+knowledge+representation+in+expert+systems%2E+International+Journal+of+Hybrid+Intelligent+Systems%2C+1%283%2D4%29%3A111%E2%80%93126%2E)
13. Joy He-Yueya, Gabriel Poesia, Rose Wang, and Noah Goodman. 2023. Solving math word problems by combining language models with symbolic solvers. In The 3rd Workshop on Mathematical Reasoning and AI at NeurIPS’23. [find](https://scholar.google.com/scholar?q=Joy+He%2DYueya%2C+Gabriel+Poesia%2C+Rose+Wang%2C+and+Noah+Goodman%2E+2023%2E+Solving+math+word+problems+by+combining+language+models+with+symbolic+solvers%2E+In+The+3rd+Workshop+on+Mathematical+Reasoning+and+AI+at+)
14. Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardt. 2020. hendrycks2measuring. Sort, 2(4):0–6. [find](https://scholar.google.com/scholar?q=Dan+Hendrycks%2C+Collin+Burns%2C+Saurav+Kadavath%2C+Akul+Arora%2C+Steven+Basart%2C+Eric+Tang%2C+Dawn+Song%2C+and+Jacob+Steinhardt%2E+2020%2E+hendrycks2measuring%2E+Sort%2C+2%284%29%3A0%E2%80%936%2E)
15. Yiming Huang, Xiao Liu, Yeyun Gong, Zhibin Gou, Yelong Shen, Nan Duan, and Weizhu Chen. 2024. Key-point-driven data synthesis with its enhancement on mathematical reasoning. arXiv preprint arXiv:2403.02333. [find](https://scholar.google.com/scholar?q=Yiming+Huang%2C+Xiao+Liu%2C+Yeyun+Gong%2C+Zhibin+Gou%2C+Yelong+Shen%2C+Nan+Duan%2C+and+Weizhu+Chen%2E+2024%2E+Key%2Dpoint%2Ddriven+data+synthesis+with+its+enhancement+on+mathematical+reasoning%2E+arXiv+preprint+arXiv%3A2403%2E)
16. Mehran Kazemi, Quan Yuan, Deepti Bhatia, Najoung Kim, Xin Xu, Vaiva Imbrasaite, and Deepak Ramachandran. 2024. Boardgameqa: A dataset for natural language reasoning with contradictory information. Advances in Neural Information Processing Systems, 36. [find](https://scholar.google.com/scholar?q=Mehran+Kazemi%2C+Quan+Yuan%2C+Deepti+Bhatia%2C+Najoung+Kim%2C+Xin+Xu%2C+Vaiva+Imbrasaite%2C+and+Deepak+Ramachandran%2E+2024%2E+Boardgameqa%3A+A+dataset+for+natural+language+reasoning+with+contradictory+information%2E+Adv)
17. Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph E. Gonzalez, Hao Zhang, and Ion Stoica. 2023. [Efficient memory management for large language model serving with pagedattention](https://md.research.elamur.ai/free/en/papers/kwon-2023-efficient-memory-management-large-language/). In Proceedings of the ACM SIGOPS 29th Symposium on Operating Systems Principles.
18. Zenan Li, Zhi Zhou, Yuan Yao, Xian Zhang, Yu-Feng Li, Chun Cao, Fan Yang, and Xiaoxing Ma. 2024. Neuro-symbolic data generation for math reasoning. In The Thirty-eighth Annual Conference on Neural Information Processing Systems. [find](https://scholar.google.com/scholar?q=Zenan+Li%2C+Zhi+Zhou%2C+Yuan+Yao%2C+Xian+Zhang%2C+Yu%2DFeng+Li%2C+Chun+Cao%2C+Fan+Yang%2C+and+Xiaoxing+Ma%2E+2024%2E+Neuro%2Dsymbolic+data+generation+for+math+reasoning%2E+In+The+Thirty%2Deighth+Annual+Conference+on+Neural+Inf)
19. Aixin Liu, Bei Feng, Bin Wang, Bingxuan Wang, Bo Liu, Chenggang Zhao, Chengqi Dengr, Chong Ruan, Damai Dai, Daya Guo, et al. 2024a. Deepseek-v2: A strong, economical, and efficient mixture-of-experts language model. arXiv preprint arXiv:2405.04434. [find](https://scholar.google.com/scholar?q=Aixin+Liu%2C+Bei+Feng%2C+Bin+Wang%2C+Bingxuan+Wang%2C+Bo+Liu%2C+Chenggang+Zhao%2C+Chengqi+Dengr%2C+Chong+Ruan%2C+Damai+Dai%2C+Daya+Guo%2C+et+al%2E+2024a%2E+Deepseek%2Dv2%3A+A+strong%2C+economical%2C+and+efficient+mixture%2Dof%2Dexperts+)
20. Hanmeng Liu, Zhiyang Teng, Leyang Cui, Chaoli Zhang, Qiji Zhou, and Yue Zhang. 2023. Logicot: Logical chain-of-thought instruction tuning. In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 2908–2921. [find](https://scholar.google.com/scholar?q=Hanmeng+Liu%2C+Zhiyang+Teng%2C+Leyang+Cui%2C+Chaoli+Zhang%2C+Qiji+Zhou%2C+and+Yue+Zhang%2E+2023%2E+Logicot%3A+Logical+chain%2Dof%2Dthought+instruction+tuning%2E+In+Findings+of+the+Association+for+Computational+Linguistics%3A)
21. Ruibo Liu, Jerry Wei, Fangyu Liu, Chenglei Si, Yanzhe Zhang, Jinmeng Rao, Steven Zheng, Daiyi Peng, Diyi Yang, Denny Zhou, et al. 2024b. [Best practices and lessons learned on synthetic data](https://md.research.elamur.ai/free/en/papers/liu-2024-best-practices-lessons-learned-synthetic/) for language models. arXiv preprint arXiv:2404.07503.
22. Donald W Loveland. 2016. Automated theorem proving: A logical basis. Elsevier. [find](https://scholar.google.com/scholar?q=Donald+W+Loveland%2E+2016%2E+Automated+theorem+proving%3A+A+logical+basis%2E+Elsevier%2E)
23. Zimu Lu, Aojun Zhou, Houxing Ren, Ke Wang, Weikang Shi, Junting Pan, Mingjie Zhan, and Hongsheng Li. 2024. Mathgenie: Generating synthetic data with question back-translation for enhancing mathematical reasoning of llms. arXiv preprint arXiv:2402.16352. [find](https://scholar.google.com/scholar?q=Zimu+Lu%2C+Aojun+Zhou%2C+Houxing+Ren%2C+Ke+Wang%2C+Weikang+Shi%2C+Junting+Pan%2C+Mingjie+Zhan%2C+and+Hongsheng+Li%2E+2024%2E+Mathgenie%3A+Generating+synthetic+data+with+question+back%2Dtranslation+for+enhancing+mathematica)
24. Haipeng Luo, Qingfeng Sun, Can Xu, Pu Zhao, Jian-guang Lou, Chongyang Tao, Xiubo Geng, Qingwei Lin, Shifeng Chen, and Dongmei Zhang. 2023a. [Wizardmath: Empowering mathematical reasoning for large language models via reinforced evol-instruct](https://md.research.elamur.ai/free/en/papers/luo-2023-wizardmath-empowering-mathematical-reasoning-large/). arXiv preprint arXiv:2308.09583.
25. Ziyang Luo, Can Xu, Pu Zhao, Qingfeng Sun, Xiubo Geng, Wenxiang Hu, Chongyang Tao, Jing Ma, Qingwei Lin, and Daxin Jiang. 2023b. Wizardcoder: Empowering code large language models with evol-instruct. In The Twelfth International Conference on Learning Representations. [find](https://scholar.google.com/scholar?q=Ziyang+Luo%2C+Can+Xu%2C+Pu+Zhao%2C+Qingfeng+Sun%2C+Xiubo+Geng%2C+Wenxiang+Hu%2C+Chongyang+Tao%2C+Jing+Ma%2C+Qingwei+Lin%2C+and+Daxin+Jiang%2E+2023b%2E+Wizardcoder%3A+Empowering+code+large+language+models+with+evol%2Dinstruct%2E+)
26. Terufumi Morishita, Gaku Morio, Atsuki Yamaguchi, and Yasuhiro Sogawa. 2023. Learning deductive reasoning from synthetic corpus based on formal logic. In Proceedings of the 40th International Conference on Machine Learning, pages 25254–25274. [find](https://scholar.google.com/scholar?q=Terufumi+Morishita%2C+Gaku+Morio%2C+Atsuki+Yamaguchi%2C+and+Yasuhiro+Sogawa%2E+2023%2E+Learning+deductive+reasoning+from+synthetic+corpus+based+on+formal+logic%2E+In+Proceedings+of+the+40th+International+Conferen)
27. Terufumi Morishita, Gaku Morio, Atsuki Yamaguchi, and Yasuhiro Sogawa. 2024. Enhancing reasoning capabilities of llms via principled synthetic logic corpus. In The Thirty-eighth Annual Conference on Neural Information Processing Systems. [find](https://scholar.google.com/scholar?q=Terufumi+Morishita%2C+Gaku+Morio%2C+Atsuki+Yamaguchi%2C+and+Yasuhiro+Sogawa%2E+2024%2E+Enhancing+reasoning+capabilities+of+llms+via+principled+synthetic+logic+corpus%2E+In+The+Thirty%2Deighth+Annual+Conference+on+N)
28. Liangming Pan, Alon Albalak, Xinyi Wang, and William Yang Wang. 2023. [Logic-lm: Empowering large language models with symbolic solvers for faithful logical reasoning](https://md.research.elamur.ai/free/en/papers/pan-2023-logic-lm-empowering-large-language/). In The 2023 Conference on Empirical Methods in Natural Language Processing.
29. Mihir Parmar, Nisarg Patel, Neeraj Varshney, Mutsumi Nakamura, Man Luo, Santosh Mashetty, Arindam Mitra, and Chitta Baral. 2024. [Logicbench: Towards systematic evaluation of logical reasoning ability of large language models](https://md.research.elamur.ai/free/en/papers/parmar-2024-logicbench-systematic-evaluation-logical-reasoning/). In Proceedings of the 62nd Annual Meeting of the Association for Computati…
30. Nisarg Patel, Mohith Kulkarni, Mihir Parmar, Aashna Budhiraja, Mutsumi Nakamura, Neeraj Varshney, and Chitta Baral. 2024. [Multi-logieval: Towards evaluating multi-step logical reasoning ability of large language models](https://md.research.elamur.ai/free/en/papers/patel-2024-multi-logieval-evaluating-multi-step/). In Proceedings of the 2024 Conference on Empirical Methods in Natural Language P…
31. Mohammed Saeed, Naser Ahmadi, Preslav Nakov, and Paolo Papotti. 2021. Rulebert: Teaching soft rules to pre-trained language models. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 1460–1476. [find](https://scholar.google.com/scholar?q=Mohammed+Saeed%2C+Naser+Ahmadi%2C+Preslav+Nakov%2C+and+Paolo+Papotti%2E+2021%2E+Rulebert%3A+Teaching+soft+rules+to+pre%2Dtrained+language+models%2E+In+Proceedings+of+the+2021+Conference+on+Empirical+Methods+in+Natura)
32. Yunfan Shao, Linyang Li, Yichuan Ma, Peiji Li, Demin Song, Qinyuan Cheng, Shimin Li, Xiaonan Li, Pengyu Wang, Qipeng Guo, et al. 2024. Case2code: Learning inductive reasoning with synthetic data. CoRR. [find](https://scholar.google.com/scholar?q=Yunfan+Shao%2C+Linyang+Li%2C+Yichuan+Ma%2C+Peiji+Li%2C+Demin+Song%2C+Qinyuan+Cheng%2C+Shimin+Li%2C+Xiaonan+Li%2C+Pengyu+Wang%2C+Qipeng+Guo%2C+et+al%2E+2024%2E+Case2code%3A+Learning+inductive+reasoning+with+synthetic+data%2E+CoRR)
33. Koustuv Sinha, Shagun Sodhani, Jin Dong, Joelle Pineau, and William L Hamilton. 2019. Clutrr: A diagnostic benchmark for inductive reasoning from text. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Lang… [find](https://scholar.google.com/scholar?q=Koustuv+Sinha%2C+Shagun+Sodhani%2C+Jin+Dong%2C+Joelle+Pineau%2C+and+William+L+Hamilton%2E+2019%2E+Clutrr%3A+A+diagnostic+benchmark+for+inductive+reasoning+from+text%2E+In+Proceedings+of+the+2019+Conference+on+Empiric)
34. Mirac Suzgun, Nathan Scales, Nathanael Schärli, Sebastian Gehrmann, Yi Tay, Hyung Won Chung, Aakanksha Chowdhery, Quoc Le, Ed Chi, Denny Zhou, et al. 2023. [Challenging big-bench tasks and whether chain-of-thought can solve them](https://md.research.elamur.ai/free/en/papers/suzgun-2023-challenging-big-bench-tasks-whether/). In Findings of the Association for Computational Linguistics: ACL 2023,…
35. Oyvind Tafjord, Bhavana Dalvi, and Peter Clark. 2021. [Proofwriter: Generating implications, proofs, and abductive statements over natural language](https://md.research.elamur.ai/free/en/papers/tafjord-2021-proofwriter-generating-implications-proofs-abductive/). In Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021, pages 3621–3634.
36. Zhengyang Tang, Xingxing Zhang, Benyou Wang, and Furu Wei. 2024. Mathscale: Scaling instruction tuning for mathematical reasoning. In Forty-first International Conference on Machine Learning. [find](https://scholar.google.com/scholar?q=Zhengyang+Tang%2C+Xingxing+Zhang%2C+Benyou+Wang%2C+and+Furu+Wei%2E+2024%2E+Mathscale%3A+Scaling+instruction+tuning+for+mathematical+reasoning%2E+In+Forty%2Dfirst+International+Conference+on+Machine+Learning%2E)
37. Teknium. 2023. Openhermes 2.5: An open dataset of synthetic data for generalist llm assistants. [find](https://scholar.google.com/scholar?q=Teknium%2E+2023%2E+Openhermes+2%2E5%3A+An+open+dataset+of+synthetic+data+for+generalist+llm+assistants%2E)
38. Jidong Tian, Yitian Li, Wenqing Chen, Liqiang Xiao, Hao He, and Yaohui Jin. 2021. [Diagnosing the first-order logical reasoning ability through logicnli](https://md.research.elamur.ai/free/en/papers/tian-2021-diagnosing-first-order-logical-reasoning/). In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 3738–3747. language models. In The Twelfth Interna…
39. Weihao Zeng, Can Xu, Yingxiu Zhao, Jian-Guang Lou, and Weizhu Chen. 2024. [Automatic instruction evolving for large language models](https://md.research.elamur.ai/free/en/papers/zeng-2024-automatic-instruction-evolving-large-language/). arXiv preprint arXiv:2406.00770.
40. Wanjun Zhong, Siyuan Wang, Duyu Tang, Zenan Xu, Daya Guo, Jiahai Wang, Jian Yin, Ming Zhou, and Nan Duan. 2021. Ar-lsat: Investigating analytical reasoning of text. arXiv preprint arXiv:2104.06598. [find](https://scholar.google.com/scholar?q=Wanjun+Zhong%2C+Siyuan+Wang%2C+Duyu+Tang%2C+Zenan+Xu%2C+Daya+Guo%2C+Jiahai+Wang%2C+Jian+Yin%2C+Ming+Zhou%2C+and+Nan+Duan%2E+2021%2E+Ar%2Dlsat%3A+Investigating+analytical+reasoning+of+text%2E+arXiv+preprint+arXiv%3A2104%2E06598%2E)

*Updated 30 October 2026*