---
title: "Large Language Models Are Neurosymbolic Reasoners (2024)"
description: "The authors build a zero-shot LLM agent, mainly GPT-3.5. 43 references, cited by 15 papers here."
authors: ["Meng Fang", "Shilong Deng", "Yudi Zhang", "Zijing Shi", "Ling Chen", "Mykola Pechenizkiy", "Jun Wang"]
year: 2024
venue: "arXiv:2401.09334"
source: "https://arxiv.org/abs/2401.09334"
doi: "10.48550/arxiv.2401.09334"
arxiv: "2401.09334"
licence: "CC BY 4.0 (arXiv)"
licence_url: "http://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/fang-2024-large-language-models-neurosymbolic/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/fang-2024-large-language-models-neurosymbolic/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2024](https://md.research.elamur.ai/free/en/papers/2024/page/10/) / Large Language Models Are Neurosymbolic Reasoners

# Large Language Models Are Neurosymbolic Reasoners

**Authors:** [Meng Fang](https://md.research.elamur.ai/free/en/authors/meng-fang/), Shilong Deng, [Yudi Zhang](https://md.research.elamur.ai/free/en/authors/yudi-zhang/), [Zijing Shi](https://md.research.elamur.ai/free/en/authors/zijing-shi/), [Ling Chen](https://md.research.elamur.ai/free/en/authors/ling-chen/), [Mykola Pechenizkiy](https://md.research.elamur.ai/free/en/authors/mykola-pechenizkiy/) and [Jun Wang](https://md.research.elamur.ai/free/en/authors/jun-wang/) · **Year:** 2024 · **Venue:** [arXiv:2401.09334](https://md.research.elamur.ai/free/en/venues/proceedings-of-the-aaai-conference-on-artificial-intelligence/) ([Proceedings of the AAAI Conference on Artificial Intelligence 2024](https://md.research.elamur.ai/free/en/venues/proceedings-of-the-aaai-conference-on-artificial-intelligence/2024/)) · **Publisher:** [Association for the Advancement of Artificial Intelligence](https://md.research.elamur.ai/free/en/publishers/association-for-the-advancement-of-artificial-intelligence/) · **Licence:** [CC BY 4.0 (arXiv)](http://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://arxiv.org/abs/2401.09334) · [PDF](https://arxiv.org/pdf/2401.09334) · [DOI 10.48550/arxiv.2401.09334](https://doi.org/10.48550/arxiv.2401.09334) · [arXiv 2401.09334](https://arxiv.org/abs/2401.09334) · [OpenAlex](https://openalex.org/W4393160365)

**Citations:** 19 citations (OpenAlex, 2026-10-14) · [15](https://md.research.elamur.ai/free/en/papers/fang-2024-large-language-models-neurosymbolic/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
The paper asks whether general-purpose LLMs can serve as symbolic reasoners without task-specific training. It tests this in text-based games, which are modelled as POMDPs and require skills such as arithmetic, map navigation, sorting, and common-sense object placement. The authors build a zero-shot LLM agent, mainly GPT-3.5. At each step, the prompt gives the agent its role, the current observation, and a combined list of valid actions from the game environment and an external symbolic module such as a calculator, navigator, sorter, or knowledge lookup. Constrained prompts guide which action to choose.

On the four benchmark tasks from Wang et al. (2022b), the agent averages 0.88 normalized test score. It clearly beats a DRRN reinforcement learning baseline. It comes close to a T5-based Behavior Cloned Transformer with symbolic modules, which scores slightly higher but depends on gold expert trajectories. The work suggests that pairing LLMs with external tools can replace costly RL or imitation training for symbolic decision-making.

## Key points
- **Tasks:** The four benchmarks are TWC (Text World Common Sense), Arithmetic, Sorting, and MapReader. The baselines are DRRN and a T5 Behavior Cloned Transformer, each run with and without symbolic modules.
- **GPT-3.5 test scores:** Arithmetic 1.00, MapReader 0.86 (about 15 steps on average), Sorting 0.71, TWC 0.94, for an average of 0.88.
- **Failure modes:** On MapReader, the agent forgets routes and has to re-query them. On Sorting, it loses track of the ascending order because of limited memory.
- **Prompt ablation:** Removing the constrained prompts lowers the average to 0.67. Sorting drops most sharply, to 0.35.
- **GPT-4 comparison:** GPT-4 raises the average to 0.91, with MapReader at 0.99 and Sorting at 0.93. However, TWC falls to 0.71 and takes more steps.
- **Code:** github.com/hyintell/LLMSymbolic.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [University College London](https://md.research.elamur.ai/free/en/universities/university-college-london/) — University, London, United Kingdom · 119 works · 23,546 citations
- [University of Technology Sydney](https://md.research.elamur.ai/free/en/universities/university-of-technology-sydney/) — University, Sydney, Australia · 20 works · 12,942 citations
- [University of Liverpool](https://md.research.elamur.ai/free/en/universities/university-of-liverpool/) — University, Liverpool, United Kingdom · 19 works · 840 citations
- [Eindhoven University of Technology](https://md.research.elamur.ai/free/en/universities/eindhoven-university-of-technology/) — University, Eindhoven, Netherlands · 11 works · 956 citations

📚 Shelves (2): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 19,503 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 18,805 works

🧭 Topics (1): [🧠 Language models & pretraining](https://md.research.elamur.ai/free/en/topics/language-models/) · 4,145 works

## Cite

Cite as: Meng Fang, Shilong Deng, Yudi Zhang, Zijing Shi, Ling Chen, Mykola Pechenizkiy and Jun Wang. 2024. Large Language Models Are Neurosymbolic Reasoners. arXiv:2401.09334.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@misc{fang2024large,
  title = {{Large Language Models Are Neurosymbolic Reasoners}},
  author = {Meng Fang, Shilong Deng, Yudi Zhang, Zijing Shi, Ling Chen, Mykola Pechenizkiy and Jun Wang},
  year = {2024},
  doi = {10.48550/arxiv.2401.09334},
  eprint = {2401.09334},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2401.09334}
}
```

- [← previous Language Varieties of Italy: Technology Challenges and Opportunities](https://md.research.elamur.ai/free/en/papers/ramponi-2024-language-varieties-italy-technology-challenges/)
- [next → mCoT: Multilingual Instruction Tuning for Reasoning Consistency in Language Models](https://md.research.elamur.ai/free/en/papers/lai-2024-mcot-multilingual-instruction-tuning-reasoning/)
- [↑ back to papers of 2024](https://md.research.elamur.ai/free/en/papers/2024/page/10/)

## References

43 references, 13 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Discussion. Our results demonstrate that the incorporation of external symbolic modules by the LLM agent leads to enhanced average accuracy compared to other baselines. This capability is achieved by leveraging the underlying patterns present in the training data. Instead of relying on symbolic thin… [find](https://scholar.google.com/scholar?q=Discussion%2E+Our+results+demonstrate+that+the+incorporation+of+external+symbolic+modules+by+the+LLM+agent+leads+to+enhanced+average+accuracy+compared+to+other+baselines%2E+This+capability+is+achieved+by+)
2. Ammanabrolu, P.; and Hausknecht, M. 2020. Graph constrained reinforcement learning for natural language action spaces. In International Conference on Learning Representations. [find](https://scholar.google.com/scholar?q=Ammanabrolu%2C+P%2E%3B+and+Hausknecht%2C+M%2E+2020%2E+Graph+constrained+reinforcement+learning+for+natural+language+action+spaces%2E+In+International+Conference+on+Learning+Representations%2E)
3. Ammanabrolu, P.; Urbanek, J.; Li, M.; Szlam, A.; Rocktäschel, T.; and Weston, J. 2021. How to Motivate Your Dragon: Teaching Goal-Driven Agents to Speak and Act in Fantasy Worlds. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: H… [find](https://scholar.google.com/scholar?q=Ammanabrolu%2C+P%2E%3B+Urbanek%2C+J%2E%3B+Li%2C+M%2E%3B+Szlam%2C+A%2E%3B+Rockta%CC%88schel%2C+T%2E%3B+and+Weston%2C+J%2E+2021%2E+How+to+Motivate+Your+Dragon%3A+Teaching+Goal%2DDriven+Agents+to+Speak+and+Act+in+Fantasy+Worlds%2E+In+Proceedings+of+t)
4. Basu, K.; Murugesan, K.; Atzeni, M.; Kapanipathi, P.; Talamadupula, K.; Klinger, T.; Campbell, M.; Sachan, M.; and Gupta, G. 2022. A hybrid neuro-symbolic approach for text-based games using inductive logic programming. In Combining Learning and Reasoning: Programming Languages, Formalisms, and Repr… [find](https://scholar.google.com/scholar?q=Basu%2C+K%2E%3B+Murugesan%2C+K%2E%3B+Atzeni%2C+M%2E%3B+Kapanipathi%2C+P%2E%3B+Talamadupula%2C+K%2E%3B+Klinger%2C+T%2E%3B+Campbell%2C+M%2E%3B+Sachan%2C+M%2E%3B+and+Gupta%2C+G%2E+2022%2E+A+hybrid+neuro%2Dsymbolic+approach+for+text%2Dbased+games+using+inductive)
5. Brooks, E.; Walls, L.; Lewis, R. L.; and Singh, S. 2022. In-context policy iteration. arXiv preprint arXiv:2210.03821. — [arXiv:2210.03821](https://arxiv.org/abs/2210.03821) [find](https://scholar.google.com/scholar?q=Brooks%2C+E%2E%3B+Walls%2C+L%2E%3B+Lewis%2C+R%2E+L%2E%3B+and+Singh%2C+S%2E+2022%2E+In%2Dcontext+policy+iteration%2E+arXiv+preprint+arXiv%3A2210%2E03821%2E+%E2%80%94+%5BarXiv%3A2210%2E03821%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2210%2E03821%29)
6. Brown, T.; Mann, B.; Ryder, N.; Subbiah, M.; Kaplan, J. D.; Dhariwal, P.; Neelakantan, A.; Shyam, P.; Sastry, G.; Askell, A.; et al. 2020. [Language models are few-shot learners](https://md.research.elamur.ai/free/en/papers/brown-2020-language-models-few-shot-learners/). In Advances in Neural Information Processing Systems.
7. Cambria, E.; Liu, Q.; Decherchi, S.; Xing, F.; and Kwok, K. 2022. SenticNet 7: A commonsense-based neurosymbolic AI framework for explainable sentiment analysis. In Proceedings of the Thirteenth Language Resources and Evaluation Conference, 3829–3839. [find](https://scholar.google.com/scholar?q=Cambria%2C+E%2E%3B+Liu%2C+Q%2E%3B+Decherchi%2C+S%2E%3B+Xing%2C+F%2E%3B+and+Kwok%2C+K%2E+2022%2E+SenticNet+7%3A+A+commonsense%2Dbased+neurosymbolic+AI+framework+for+explainable+sentiment+analysis%2E+In+Proceedings+of+the+Thirteenth+Langu)
8. Chen, L.; Lu, K.; Rajeswaran, A.; Lee, K.; Grover, A.; Laskin, M.; Abbeel, P.; Srinivas, A.; and Mordatch, I. 2021. Decision transformer: Reinforcement learning via sequence modeling. In Advances in Neural Information Processing Systems. Côté, M.-A.; Kádár, A.; Yuan, X.; Kybartas, B.; Barnes, T.… [find](https://scholar.google.com/scholar?q=Chen%2C+L%2E%3B+Lu%2C+K%2E%3B+Rajeswaran%2C+A%2E%3B+Lee%2C+K%2E%3B+Grover%2C+A%2E%3B+Laskin%2C+M%2E%3B+Abbeel%2C+P%2E%3B+Srinivas%2C+A%2E%3B+and+Mordatch%2C+I%2E+2021%2E+Decision+transformer%3A+Reinforcement+learning+via+sequence+modeling%2E+In+Advances+in+N)
9. He, J.; Ostendorf, M.; He, X.; Chen, J.; Gao, J.; Li, L.; and Deng, L. 2016. Deep Reinforcement Learning with a Combinatorial Action Space for Predicting Popular Reddit Threads. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, 1838–1848. [find](https://scholar.google.com/scholar?q=He%2C+J%2E%3B+Ostendorf%2C+M%2E%3B+He%2C+X%2E%3B+Chen%2C+J%2E%3B+Gao%2C+J%2E%3B+Li%2C+L%2E%3B+and+Deng%2C+L%2E+2016%2E+Deep+Reinforcement+Learning+with+a+Combinatorial+Action+Space+for+Predicting+Popular+Reddit+Threads%2E+In+Proceedings+of+the+)
10. Hendrycks, D.; Mazeika, M.; Zou, A.; Patel, S.; Zhu, C.; Navarro, J.; Song, D.; Li, B.; and Steinhardt, J. 2021. What would jiminy cricket do? Towards agents that behave morally. In Advances in Neural Information Processing Systems. [find](https://scholar.google.com/scholar?q=Hendrycks%2C+D%2E%3B+Mazeika%2C+M%2E%3B+Zou%2C+A%2E%3B+Patel%2C+S%2E%3B+Zhu%2C+C%2E%3B+Navarro%2C+J%2E%3B+Song%2C+D%2E%3B+Li%2C+B%2E%3B+and+Steinhardt%2C+J%2E+2021%2E+What+would+jiminy+cricket+do%3F+Towards+agents+that+behave+morally%2E+In+Advances+in+Neural)
11. Huang, W.; Abbeel, P.; Pathak, D.; and Mordatch, I. 2022a. [Language models as zero-shot planners: Extracting actionable knowledge for embodied agents](https://md.research.elamur.ai/free/en/papers/huang-2022-language-models-zero-shot-planners/). In International Conference on Machine Learning.
12. Huang, W.; Xia, F.; Xiao, T.; Chan, H.; Liang, J.; Florence, P.; Zeng, A.; Tompson, J.; Mordatch, I.; Chebotar, Y.; et al. 2022b. Inner monologue: Embodied reasoning through planning with language models. arXiv preprint arXiv:2207.05608. [find](https://scholar.google.com/scholar?q=Huang%2C+W%2E%3B+Xia%2C+F%2E%3B+Xiao%2C+T%2E%3B+Chan%2C+H%2E%3B+Liang%2C+J%2E%3B+Florence%2C+P%2E%3B+Zeng%2C+A%2E%3B+Tompson%2C+J%2E%3B+Mordatch%2C+I%2E%3B+Chebotar%2C+Y%2E%3B+et+al%2E+2022b%2E+Inner+monologue%3A+Embodied+reasoning+through+planning+with+language+mod)
13. Jansen, P.; and Cote, M.-a. 2023. TextWorldExpress: Simulating Text Games at One Million Steps Per Second. In Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics: System Demonstrations, 169–177. [find](https://scholar.google.com/scholar?q=Jansen%2C+P%2E%3B+and+Cote%2C+M%2E%2Da%2E+2023%2E+TextWorldExpress%3A+Simulating+Text+Games+at+One+Million+Steps+Per+Second%2E+In+Proceedings+of+the+17th+Conference+of+the+European+Chapter+of+the+Association+for+Computat)
14. Kimura, D.; Chaudhury, S.; Ono, M.; Tatsubori, M.; Agravante, D. J.; Munawar, A.; Wachi, A.; Kohita, R.; and Gray, A. 2021a. LOA: Logical Optimal Actions for Text-based Interaction Games. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th Internat… [find](https://scholar.google.com/scholar?q=Kimura%2C+D%2E%3B+Chaudhury%2C+S%2E%3B+Ono%2C+M%2E%3B+Tatsubori%2C+M%2E%3B+Agravante%2C+D%2E+J%2E%3B+Munawar%2C+A%2E%3B+Wachi%2C+A%2E%3B+Kohita%2C+R%2E%3B+and+Gray%2C+A%2E+2021a%2E+LOA%3A+Logical+Optimal+Actions+for+Text%2Dbased+Interaction+Games%2E+In+Proceedin)
15. Kimura, D.; Ono, M.; Chaudhury, S.; Kohita, R.; Wachi, A.; Agravante, D. J.; Tatsubori, M.; Munawar, A.; and Gray, A. 2021b. Neuro-Symbolic Reinforcement Learning with First-Order Logic. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, 3505– 3511. [find](https://scholar.google.com/scholar?q=Kimura%2C+D%2E%3B+Ono%2C+M%2E%3B+Chaudhury%2C+S%2E%3B+Kohita%2C+R%2E%3B+Wachi%2C+A%2E%3B+Agravante%2C+D%2E+J%2E%3B+Tatsubori%2C+M%2E%3B+Munawar%2C+A%2E%3B+and+Gray%2C+A%2E+2021b%2E+Neuro%2DSymbolic+Reinforcement+Learning+with+First%2DOrder+Logic%2E+In+Proceeding)
16. Kwon, M.; Xie, S. M.; Bullard, K.; and Sadigh, D. 2023. [Reward design with language models](https://md.research.elamur.ai/free/en/papers/kwon-2023-reward-design-language-models/). In International Conference on Learning Representations.
17. Lample, G.; and Charton, F. 2020. [Deep Learning For Symbolic Mathematics](https://md.research.elamur.ai/free/en/papers/lample-2019-deep-learning-symbolic-mathematics/). In International Conference on Learning Representations.
18. Liang, J.; Huang, W.; Xia, F.; Xu, P.; Hausman, K.; Ichter, B.; Florence, P.; and Zeng, A. 2023. Code as policies: Language model programs for embodied control. In 2023 IEEE International Conference on Robotics and Automation, 9493–9500. [find](https://scholar.google.com/scholar?q=Liang%2C+J%2E%3B+Huang%2C+W%2E%3B+Xia%2C+F%2E%3B+Xu%2C+P%2E%3B+Hausman%2C+K%2E%3B+Ichter%2C+B%2E%3B+Florence%2C+P%2E%3B+and+Zeng%2C+A%2E+2023%2E+Code+as+policies%3A+Language+model+programs+for+embodied+control%2E+In+2023+IEEE+International+Conference+o)
19. Liu, G.; Adhikari, A.; Farahmand, A.-m.; and Poupart, P. 2021. Learning object-oriented dynamics for planning from text. In International Conference on Learning Representations. [find](https://scholar.google.com/scholar?q=Liu%2C+G%2E%3B+Adhikari%2C+A%2E%3B+Farahmand%2C+A%2E%2Dm%2E%3B+and+Poupart%2C+P%2E+2021%2E+Learning+object%2Doriented+dynamics+for+planning+from+text%2E+In+International+Conference+on+Learning+Representations%2E)
20. Madaan, A.; Tandon, N.; Gupta, P.; Hallinan, S.; Gao, L.; Wiegreffe, S.; Alon, U.; Dziri, N.; Prabhumoye, S.; Yang, Y.; et al. 2023. Self-refine: Iterative refinement with self-feedback. arXiv preprint arXiv:2303.17651. [find](https://scholar.google.com/scholar?q=Madaan%2C+A%2E%3B+Tandon%2C+N%2E%3B+Gupta%2C+P%2E%3B+Hallinan%2C+S%2E%3B+Gao%2C+L%2E%3B+Wiegreffe%2C+S%2E%3B+Alon%2C+U%2E%3B+Dziri%2C+N%2E%3B+Prabhumoye%2C+S%2E%3B+Yang%2C+Y%2E%3B+et+al%2E+2023%2E+Self%2Drefine%3A+Iterative+refinement+with+self%2Dfeedback%2E+arXiv+preprin)
21. Min, S.; Lewis, M.; Zettlemoyer, L.; and Hajishirzi, H. 2022. MetaICL: Learning to Learn In Context. In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, 2791–2809. [find](https://scholar.google.com/scholar?q=Min%2C+S%2E%3B+Lewis%2C+M%2E%3B+Zettlemoyer%2C+L%2E%3B+and+Hajishirzi%2C+H%2E+2022%2E+MetaICL%3A+Learning+to+Learn+In+Context%2E+In+Proceedings+of+the+2022+Conference+of+the+North+American+Chapter+of+the+Association+for+Computat)
22. Mnih, V.; Kavukcuoglu, K.; Silver, D.; Graves, A.; Antonoglou, I.; Wierstra, D.; and Riedmiller, M. 2013. Playing atari with deep reinforcement learning. arXiv preprint arXiv:1312.5602. [find](https://scholar.google.com/scholar?q=Mnih%2C+V%2E%3B+Kavukcuoglu%2C+K%2E%3B+Silver%2C+D%2E%3B+Graves%2C+A%2E%3B+Antonoglou%2C+I%2E%3B+Wierstra%2C+D%2E%3B+and+Riedmiller%2C+M%2E+2013%2E+Playing+atari+with+deep+reinforcement+learning%2E+arXiv+preprint+arXiv%3A1312%2E5602%2E)
23. Murugesan, K.; Atzeni, M.; Kapanipathi, P.; Shukla, P.; Kumaravel, S.; Tesauro, G.; Talamadupula, K.; Sachan, M.; and Campbell, M. 2021. Text-based rl agents with commonsense knowledge: New challenges, environments and baselines. In Proceedings of the AAAI Conference on Artificial Intelligence, volu… [find](https://scholar.google.com/scholar?q=Murugesan%2C+K%2E%3B+Atzeni%2C+M%2E%3B+Kapanipathi%2C+P%2E%3B+Shukla%2C+P%2E%3B+Kumaravel%2C+S%2E%3B+Tesauro%2C+G%2E%3B+Talamadupula%2C+K%2E%3B+Sachan%2C+M%2E%3B+and+Campbell%2C+M%2E+2021%2E+Text%2Dbased+rl+agents+with+commonsense+knowledge%3A+New+challenges)
24. Narasimhan, K.; Kulkarni, T.; and Barzilay, R. 2015. [Language Understanding for Text-based Games using Deep Reinforcement Learning](https://md.research.elamur.ai/free/en/papers/narasimhan-2015-language-understanding-text-based-games/). In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, 1–11.
25. OpenAI. 2023. [GPT-4 Technical Report](https://md.research.elamur.ai/free/en/papers/openai-2023-gpt-4-technical-report/). ArXiv, abs/2303.08774. Osborne, P.; Nõmm, H.; and Freitas, A. 2022. A survey of text games for reinforcement learning informed by natural language. Transactions of the Association for Computational Linguistics, 873–887.
26. Poesia, G.; Dong, W.; and Goodman, N. 2021. Contrastive reinforcement learning of symbolic reasoning domains. In Advances in Neural Information Processing Systems. [find](https://scholar.google.com/scholar?q=Poesia%2C+G%2E%3B+Dong%2C+W%2E%3B+and+Goodman%2C+N%2E+2021%2E+Contrastive+reinforcement+learning+of+symbolic+reasoning+domains%2E+In+Advances+in+Neural+Information+Processing+Systems%2E)
27. Qian, J.; Wang, H.; Li, Z.; Li, S.; and Yan, X. 2023. Limitations of Language Models in Arithmetic and Symbolic Induction. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 9285–9298. [find](https://scholar.google.com/scholar?q=Qian%2C+J%2E%3B+Wang%2C+H%2E%3B+Li%2C+Z%2E%3B+Li%2C+S%2E%3B+and+Yan%2C+X%2E+2023%2E+Limitations+of+Language+Models+in+Arithmetic+and+Symbolic+Induction%2E+In+Proceedings+of+the+61st+Annual+Meeting+of+the+Association+for+Computationa)
28. Raffel, C.; Shazeer, N.; Roberts, A.; Lee, K.; Narang, S.; Matena, M.; Zhou, Y.; Li, W.; and Liu, P. J. 2020. [Exploring the limits of transfer learning with a unified text-to-text transformer](https://md.research.elamur.ai/free/en/papers/raffel-2019-exploring-limits-transfer-learning-unified/). The Journal of machine learning Research, 21(1): 5485–5551.
29. Ryu, D.; Shareghi, E.; Fang, M.; Xu, Y.; Pan, S.; and Haf, R. 2022. Fire Burns, Sword Cuts: Commonsense Inductive Bias for Exploration in Text-based Games. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), 515–522. [find](https://scholar.google.com/scholar?q=Ryu%2C+D%2E%3B+Shareghi%2C+E%2E%3B+Fang%2C+M%2E%3B+Xu%2C+Y%2E%3B+Pan%2C+S%2E%3B+and+Haf%2C+R%2E+2022%2E+Fire+Burns%2C+Sword+Cuts%3A+Commonsense+Inductive+Bias+for+Exploration+in+Text%2Dbased+Games%2E+In+Proceedings+of+the+60th+Annual+Meeting+of)
30. Shi, Z.; Fang, M.; Xu, Y.; Chen, L.; and Du, Y. 2022. Stay moral and explore: Learn to behave morally in text-based games. In International Conference on Learning Representations. [find](https://scholar.google.com/scholar?q=Shi%2C+Z%2E%3B+Fang%2C+M%2E%3B+Xu%2C+Y%2E%3B+Chen%2C+L%2E%3B+and+Du%2C+Y%2E+2022%2E+Stay+moral+and+explore%3A+Learn+to+behave+morally+in+text%2Dbased+games%2E+In+International+Conference+on+Learning+Representations%2E)
31. Shinn, N.; Labash, B.; and Gopinath, A. 2023. [Reflexion: an autonomous agent with dynamic memory and self-reflection](https://md.research.elamur.ai/free/en/papers/shinn-2023-reflexion-language-agents-verbal-reinforcement/). arXiv preprint arXiv:2303.11366.
32. Singh, I.; Blukis, V.; Mousavian, A.; Goyal, A.; Xu, D.; Tremblay, J.; Fox, D.; Thomason, J.; and Garg, A. 2023. [Progprompt: Generating situated robot task plans using large language models](https://md.research.elamur.ai/free/en/papers/singh-2022-progprompt-generating-situated-robot-task/). In 2023 IEEE International Conference on Robotics and Automation, 11523–11530.
33. Sutton, R. S.; and Barto, A. G. 2018. Reinforcement learning: An introduction. MIT press. [find](https://scholar.google.com/scholar?q=Sutton%2C+R%2E+S%2E%3B+and+Barto%2C+A%2E+G%2E+2018%2E+Reinforcement+learning%3A+An+introduction%2E+MIT+press%2E)
34. Vemprala, S.; Bonatti, R.; Bucker, A.; and Kapoor, A. 2023. ChatGPT for Robotics: Design Principles and Model Abilities. Technical Report MSR-TR-2023-8, Microsoft. Wang, R.; Jansen, P.; Côté, M.-A.; and Ammanabrolu, P. 2022a. ScienceWorld: Is your Agent Smarter than a 5th Grader? In Proceedings of… [find](https://scholar.google.com/scholar?q=Vemprala%2C+S%2E%3B+Bonatti%2C+R%2E%3B+Bucker%2C+A%2E%3B+and+Kapoor%2C+A%2E+2023%2E+ChatGPT+for+Robotics%3A+Design+Principles+and+Model+Abilities%2E+Technical+Report+MSR%2DTR%2D2023%2D8%2C+Microsoft%2E+Wang%2C+R%2E%3B+Jansen%2C+P%2E%3B+Co%CC%82te%CC%81%2C+M%2E%2DA%2E%3B)
35. Wei, J.; Wang, X.; Schuurmans, D.; Bosma, M.; Xia, F.; Chi, E. H.; Le, Q. V.; Zhou, D.; et al. 2022. [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://md.research.elamur.ai/free/en/papers/wei-2022-chain-thought-prompting-elicits-reasoning/). In Advances in Neural Information Processing Systems.
36. Xu, Y.; Fang, M.; Chen, L.; Du, Y.; and Zhang, C. 2021. Generalization in Text-based Games via Hierarchical Reinforcement Learning. In Findings of the Association for Computational Linguistics: EMNLP 2021, 1343–1353. [find](https://scholar.google.com/scholar?q=Xu%2C+Y%2E%3B+Fang%2C+M%2E%3B+Chen%2C+L%2E%3B+Du%2C+Y%2E%3B+and+Zhang%2C+C%2E+2021%2E+Generalization+in+Text%2Dbased+Games+via+Hierarchical+Reinforcement+Learning%2E+In+Findings+of+the+Association+for+Computational+Linguistics%3A+EMNLP+)
37. Xu, Y.; Fang, M.; Chen, L.; Du, Y.; Zhou, J.; and Zhang, C. 2022. Perceiving the World: Question-guided Reinforcement Learning for Text-based Games. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics, 538–560. [find](https://scholar.google.com/scholar?q=Xu%2C+Y%2E%3B+Fang%2C+M%2E%3B+Chen%2C+L%2E%3B+Du%2C+Y%2E%3B+Zhou%2C+J%2E%3B+and+Zhang%2C+C%2E+2022%2E+Perceiving+the+World%3A+Question%2Dguided+Reinforcement+Learning+for+Text%2Dbased+Games%2E+In+Proceedings+of+the+60th+Annual+Meeting+of+the+As)
38. Xu, Y.; Fang, M.; Chen, L.; Du, Y.; Zhou, J. T.; and Zhang, C. 2020. Deep reinforcement learning with stacked hierarchical attention for text-based games. In Advances in Neural Information Processing Systems. [find](https://scholar.google.com/scholar?q=Xu%2C+Y%2E%3B+Fang%2C+M%2E%3B+Chen%2C+L%2E%3B+Du%2C+Y%2E%3B+Zhou%2C+J%2E+T%2E%3B+and+Zhang%2C+C%2E+2020%2E+Deep+reinforcement+learning+with+stacked+hierarchical+attention+for+text%2Dbased+games%2E+In+Advances+in+Neural+Information+Processing+)
39. Yang, S.; Nachum, O.; Du, Y.; Wei, J.; Abbeel, P.; and Schuurmans, D. 2023. [Foundation models for decision making: Problems, methods, and opportunities](https://md.research.elamur.ai/free/en/papers/yang-2023-foundation-models-decision-making-problems/). arXiv preprint arXiv:2303.04129.
40. Yao, S.; Rao, R.; Hausknecht, M.; and Narasimhan, K. 2020. [Keep CALM and Explore: Language Models for Action Generation in Text-based Games](https://md.research.elamur.ai/free/en/papers/yao-2020-keep-calm-explore-language-models/). In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, 8736–8754.
41. Yao, S.; Zhao, J.; Yu, D.; Du, N.; Shafran, I.; Narasimhan, K.; and Cao, Y. 2022. [React: Synergizing reasoning and acting in language models](https://md.research.elamur.ai/free/en/papers/yao-2022-react-synergizing-reasoning-acting-language/). arXiv preprint arXiv:2210.03629.
42. Yin, X.; and May, J. 2019. Comprehensible context-driven text game playing. In IEEE Conference on Games, 1–8. Yuan, X.; Côté, M.-A.; Sordoni, A.; Laroche, R.; Combes, R. T. d.; Hausknecht, M.; and Trischler, A. 2018. Counting to explore and generalize in text-based games. arXiv preprint arXiv:1806… — [arXiv:1806.11525](https://arxiv.org/abs/1806.11525) [find](https://scholar.google.com/scholar?q=Yin%2C+X%2E%3B+and+May%2C+J%2E+2019%2E+Comprehensible+context%2Ddriven+text+game+playing%2E+In+IEEE+Conference+on+Games%2C+1%E2%80%938%2E+Yuan%2C+X%2E%3B+Co%CC%82te%CC%81%2C+M%2E%2DA%2E%3B+Sordoni%2C+A%2E%3B+Laroche%2C+R%2E%3B+Combes%2C+R%2E+T%2E+d%2E%3B+Hausknecht%2C+M%2E%3B+and+T)
43. Zellers, R.; Holtzman, A.; Peters, M.; Mottaghi, R.; Kembhavi, A.; Farhadi, A.; and Choi, Y. 2021. PIGLeT: Language Grounding Through Neuro-Symbolic Interaction in a 3D World. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint… [find](https://scholar.google.com/scholar?q=Zellers%2C+R%2E%3B+Holtzman%2C+A%2E%3B+Peters%2C+M%2E%3B+Mottaghi%2C+R%2E%3B+Kembhavi%2C+A%2E%3B+Farhadi%2C+A%2E%3B+and+Choi%2C+Y%2E+2021%2E+PIGLeT%3A+Language+Grounding+Through+Neuro%2DSymbolic+Interaction+in+a+3D+World%2E+In+Proceedings+of+the+59)

*Updated 3 November 2026*