---
title: "Collaborating with language models for embodied reasoning (2023)"
description: "Large language models reason well and adapt to new tasks through in-context learning, but they… 13 references, cited by 27 papers here."
authors: ["Ishita Dasgupta", "Christine Kaeser-Chen", "Kenneth Marino", "Arun Ahuja", "Sheila Babayan", "Felix Hill", "Rob Fergus"]
year: 2023
venue: "arXiv:2302.00763"
source: "https://arxiv.org/abs/2302.00763"
doi: "10.48550/arxiv.2302.00763"
arxiv: "2302.00763"
licence: "CC BY 4.0 (arXiv)"
licence_url: "http://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/dasgupta-2023-collaborating-language-models-embodied-reasoning/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/dasgupta-2023-collaborating-language-models-embodied-reasoning/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2023](https://md.research.elamur.ai/free/en/papers/2023/page/12/) / Collaborating with language models for embodied reasoning

# Collaborating with language models for embodied reasoning

**Authors:** [Ishita Dasgupta](https://md.research.elamur.ai/free/en/authors/ishita-dasgupta/), Christine Kaeser-Chen, [Kenneth Marino](https://md.research.elamur.ai/free/en/authors/kenneth-marino/), [Arun Ahuja](https://md.research.elamur.ai/free/en/authors/arun-ahuja/), Sheila Babayan, [Felix Hill](https://md.research.elamur.ai/free/en/authors/felix-hill/) and [Rob Fergus](https://md.research.elamur.ai/free/en/authors/rob-fergus/) · **Year:** 2023 · **Venue:** arXiv:2302.00763 · **Licence:** [CC BY 4.0 (arXiv)](http://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://arxiv.org/abs/2302.00763) · [PDF](https://arxiv.org/pdf/2302.00763) · [DOI 10.48550/arxiv.2302.00763](https://doi.org/10.48550/arxiv.2302.00763) · [arXiv 2302.00763](https://arxiv.org/abs/2302.00763) · [OpenAlex](https://openalex.org/W4319165647)

**Citations:** 9 citations (OpenAlex, 2026-10-14) · [27](https://md.research.elamur.ai/free/en/papers/dasgupta-2023-collaborating-language-models-embodied-reasoning/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
Large language models reason well and adapt to new tasks through in-context learning, but they cannot perceive or act in an environment. RL agents can act, yet they need large amounts of data and generalize poorly. The authors combine the two in a three-module architecture. A pre-trained Chinchilla model acts as the Planner and turns a task description into simple text instructions. An RL-trained Actor carries these out in a partially observable 2D grid-world (PycoLab). A Reporter converts the outcomes back into text for the Planner. The tasks require explicit information gathering: the agent examines objects to uncover a hidden "secret property" and then decides which object to pick up. With five few-shot demonstrations and only pre-trained components, the system solves these tasks zero-shot, while a pure RL baseline does poorly. The 70B Planner is much more accurate than the 7B one and copes better with noisy reports. The authors also show that a Reporter can be learned from scratch with RL using episode reward. This is an early step toward replacing hand-coded feedback when grounding language models in embodied settings.

## Key points
- **Architecture:** The Planner is Chinchilla (7B or 70B), prompted with 5 solved example dialogues. The Actor is pre-trained only on "pick up X" and "examine X" instructions, sees an egocentric RGB view limited to 5 squares, and can examine or pick up objects.
- **Conditional task:** Given complete information, the 7B Planner chooses the right object 58% of the time and the 70B Planner 96% of the time. A pure RL baseline stays weak even after 100M learner frames.
- **Search task:** Here the agent must examine several objects in turn. The agents surprisingly score higher than on the single-examine task, likely because the flexible number of steps lets the Planner recover from Actor mistakes.
- **Noisy Reporter:** Irrelevant movement messages were injected 20% of the time. This reduced accuracy only moderately for 70B, which coped by repeating or cycling instructions. Showing a repeat strategy in the prompt fully restored 70B performance, while 7B recovered only partly.
- **Trained Reporter:** On a visual conditional task (is an object near the wall?), a Reporter with a binary report head was trained from scratch with RL and improved task performance. The work appeared at the NeurIPS 2022 LaReL workshop.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 19,503 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 18,805 works · [⚙️ Engineering](https://md.research.elamur.ai/free/en/shelves/engineering/) · 206 works · [⚙️ Control and Systems Engineering](https://md.research.elamur.ai/free/en/shelves/engineering/control-and-systems-engineering/) · 72 works

🧭 Topics (2): [🧠 Language models & pretraining](https://md.research.elamur.ai/free/en/topics/language-models/) · 4,145 works · [🧮 Reasoning](https://md.research.elamur.ai/free/en/topics/reasoning/) · 1,948 works

## Cite

Cite as: Ishita Dasgupta, Christine Kaeser-Chen, Kenneth Marino, Arun Ahuja, Sheila Babayan, Felix Hill and Rob Fergus. 2023. Collaborating with language models for embodied reasoning. arXiv:2302.00763.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@misc{dasgupta2023collaborating,
  title = {{Collaborating with language models for embodied reasoning}},
  author = {Ishita Dasgupta, Christine Kaeser-Chen, Kenneth Marino, Arun Ahuja, Sheila Babayan, Felix Hill and Rob Fergus},
  year = {2023},
  doi = {10.48550/arxiv.2302.00763},
  eprint = {2302.00763},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2302.00763}
}
```

- [← previous An Emulator for Fine-Tuning Large Language Models using Small Language Models](https://md.research.elamur.ai/free/en/papers/mitchell-2023-emulator-fine-tuning-large-language/)
- [next → Efficient Language Model Training through Cross-Lingual and Progressive Transfer Learning](https://md.research.elamur.ai/free/en/papers/ostendorff-2023-efficient-language-model-training-through/)
- [↑ back to papers of 2023](https://md.research.elamur.ai/free/en/papers/2023/page/12/)

## References

13 references, 5 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. M. Ahn, A. Brohan, N. Brown, Y. Chebotar, O. Cortes, B. David, C. Finn, K. Gopalakrishnan, K. Hausman, A. Herzog, et al. [Do as i can, not as i say: Grounding language in robotic affordances](https://md.research.elamur.ai/free/en/papers/ahn-2022-i-i-say-grounding-language/). arXiv preprint arXiv:2204.01691, 2022.
2. T. B. Brown, B. Mann, N. Ryder, M. Subbiah, J. Kaplan, P. Dhariwal, A. Neelakantan, P. Shyam, G. Sastry, A. Askell, S. Agarwal, A. Herbert-Voss, G. Krueger, T. J. Henighan, R. Child, A. Ramesh, D. M. Ziegler, J. Wu, C. Winter, C. Hesse, M. Chen, E. Sigler, M. Litwin, S. Gray, B. Chess, J. Clark, C.… — [Language Models are Few-Shot Learners](https://md.research.elamur.ai/free/en/papers/brown-2020-language-models-few-shot-learners/)
3. L. Espeholt, H. Soyer, R. Munos, K. Simonyan, V. Mnih, T. Ward, Y. Doron, V. Firoiu, T. Harley, I. Dunning, S. Legg, and K. Kavukcuoglu. IMPALA: scalable distributed deep-rl with importance weighted actor-learner architectures. CoRR, abs/1802.01561, 2018. URL [http://arxiv.org/](http://arxiv.org/) abs/1802.01561. [↗](http://arxiv.org/)
4. J. Hoffmann, S. Borgeaud, A. Mensch, E. Buchatskaya, T. Cai, E. Rutherford, D. de Las Casas, L. A. Hendricks, J. Welbl, A. Clark, T. Hennigan, E. Noland, K. Millican, G. van den Driessche, B. Damoc, A. Guy, S. Osindero, K. Simonyan, E. Elsen, J. W. Rae, O. Vinyals, and L. Sifre. Training compute-opt… — [Training Compute-Optimal Large Language Models](https://md.research.elamur.ai/free/en/papers/hoffmann-2022-training-compute-optimal-large-language/)
5. W. Huang, F. Xia, T. Xiao, H. Chan, J. Liang, P. Florence, A. Zeng, J. Tompson, I. Mordatch, Y. Chebotar, et al. Inner monologue: Embodied reasoning through planning with language models. arXiv preprint arXiv:2207.05608, 2022. [find](https://scholar.google.com/scholar?q=W%2E+Huang%2C+F%2E+Xia%2C+T%2E+Xiao%2C+H%2E+Chan%2C+J%2E+Liang%2C+P%2E+Florence%2C+A%2E+Zeng%2C+J%2E+Tompson%2C+I%2E+Mordatch%2C+Y%2E+Chebotar%2C+et+al%2E+Inner+monologue%3A+Embodied+reasoning+through+planning+with+language+models%2E+arXiv+prepri)
6. R. Kirk, A. Zhang, E. Grefenstette, and T. Rocktaschel. A survey of generalisation in deep reinforcement learning. ArXiv, abs/2111.09794, 2021. — [arXiv:2111.09794](https://arxiv.org/abs/2111.09794) [find](https://scholar.google.com/scholar?q=R%2E+Kirk%2C+A%2E+Zhang%2C+E%2E+Grefenstette%2C+and+T%2E+Rocktaschel%2E+A+survey+of+generalisation+in+deep+reinforcement+learning%2E+ArXiv%2C+abs%2F2111%2E09794%2C+2021%2E+%E2%80%94+%5BarXiv%3A2111%2E09794%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2111%2E09794%29)
7. T. D. Kulkarni, K. Narasimhan, A. Saeedi, and J. Tenenbaum. Hierarchical deep reinforcement learning: Integrating temporal abstraction and intrinsic motivation. Advances in neural information processing systems, 29, 2016. [find](https://scholar.google.com/scholar?q=T%2E+D%2E+Kulkarni%2C+K%2E+Narasimhan%2C+A%2E+Saeedi%2C+and+J%2E+Tenenbaum%2E+Hierarchical+deep+reinforcement+learning%3A+Integrating+temporal+abstraction+and+intrinsic+motivation%2E+Advances+in+neural+information+processi)
8. M. Mitchell. [Abstraction and analogy-making in artificial intelligence](https://md.research.elamur.ai/free/en/papers/mitchell-2021-abstraction-analogy-making-artificial-intelligence/). Annals of the New York Academy of Sciences, 1505(1):79–101, 2021.
9. A. Radford, J. Wu, R. Child, D. Luan, D. Amodei, and I. Sutskever. Language models are unsupervised multitask learners. 2019. [find](https://scholar.google.com/scholar?q=A%2E+Radford%2C+J%2E+Wu%2C+R%2E+Child%2C+D%2E+Luan%2C+D%2E+Amodei%2C+and+I%2E+Sutskever%2E+Language+models+are+unsupervised+multitask+learners%2E+2019%2E)
10. J. W. Rae, S. Borgeaud, T. Cai, K. Millican, J. Hoffmann, F. Song, J. Aslanides, S. Henderson, R. Ring, S. Young, E. Rutherford, T. Hennigan, J. Menick, A. Cassirer, R. Powell, G. van den Driessche, L. A. Hendricks, M. Rauh, P.-S. Huang, A. Glaese, J. Welbl, S. Dathathri, S. Huang, J. Uesato, J. F.… [find](https://scholar.google.com/scholar?q=J%2E+W%2E+Rae%2C+S%2E+Borgeaud%2C+T%2E+Cai%2C+K%2E+Millican%2C+J%2E+Hoffmann%2C+F%2E+Song%2C+J%2E+Aslanides%2C+S%2E+Henderson%2C+R%2E+Ring%2C+S%2E+Young%2C+E%2E+Rutherford%2C+T%2E+Hennigan%2C+J%2E+Menick%2C+A%2E+Cassirer%2C+R%2E+Powell%2C+G%2E+van+den+Driessche%2C+L)
11. J. Russin, R. C. O’Reilly, and Y. Bengio. Deep learning needs a prefrontal cortex. Work Bridging AI Cogn Sci, 107:603–616, 2020. [find](https://scholar.google.com/scholar?q=J%2E+Russin%2C+R%2E+C%2E+O%E2%80%99Reilly%2C+and+Y%2E+Bengio%2E+Deep+learning+needs+a+prefrontal+cortex%2E+Work+Bridging+AI+Cogn+Sci%2C+107%3A603%E2%80%93616%2C+2020%2E)
12. R. S. Sutton, D. Precup, and S. Singh. Between mdps and semi-mdps: A framework for temporal abstraction in reinforcement learning. Artificial intelligence, 112(1-2):181–211, 1999. [find](https://scholar.google.com/scholar?q=R%2E+S%2E+Sutton%2C+D%2E+Precup%2C+and+S%2E+Singh%2E+Between+mdps+and+semi%2Dmdps%3A+A+framework+for+temporal+abstraction+in+reinforcement+learning%2E+Artificial+intelligence%2C+112%281%2D2%29%3A181%E2%80%93211%2C+1999%2E)
13. A. Zeng, A. Wong, S. Welker, K. Choromanski, F. Tombari, A. Purohit, M. Ryoo, V. Sindhwani, J. Lee, V. Vanhoucke, et al. [Socratic models: Composing zero-shot multimodal reasoning with language](https://md.research.elamur.ai/free/en/papers/zeng-2022-socratic-models-composing-zero-shot/). arXiv preprint arXiv:2204.00598, 2022.

*Updated 3 November 2026*