---
title: "ProgPrompt: Generating Situated Robot Task — Singh 2022"
description: "Large language models can propose robot task plans, but current approaches have two weaknesses."
authors: ["Ishika Singh", "Valts Blukis", "Arsalan Mousavian", "Ankit Goyal", "Danfei Xu", "Jonathan Tremblay", "Dieter Fox", "Jesse Thomason", "Animesh Garg"]
year: 2022
venue: "arXiv:2209.11302"
source: "https://arxiv.org/abs/2209.11302"
doi: "10.48550/arxiv.2209.11302"
arxiv: "2209.11302"
licence: "arXiv — non-exclusive licence to distribute"
licence_url: "http://arxiv.org/licenses/nonexclusive-distrib/1.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/singh-2022-progprompt-generating-situated-robot-task/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/singh-2022-progprompt-generating-situated-robot-task/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2022](https://md.research.elamur.ai/free/en/papers/2022/page/4/) / ProgPrompt: Generating Situated Robot Task Plans using Large Language Models

# ProgPrompt: Generating Situated Robot Task Plans using Large Language Models

**Authors:** Ishika Singh, Valts Blukis, Arsalan Mousavian, Ankit Goyal, Danfei Xu, Jonathan Tremblay, [Dieter Fox](https://md.research.elamur.ai/free/en/authors/dieter-fox/), [Jesse Thomason](https://md.research.elamur.ai/free/en/authors/jesse-thomason/), Animesh Garg · **Year:** 2022 · **Venue:** arXiv:2209.11302 · **Licence:** [arXiv — non-exclusive licence to distribute](http://arxiv.org/licenses/nonexclusive-distrib/1.0/)

**Links:** [Publisher's page](https://arxiv.org/abs/2209.11302) · [PDF](https://arxiv.org/pdf/2209.11302) · [DOI 10.48550/arxiv.2209.11302](https://doi.org/10.48550/arxiv.2209.11302) · [arXiv 2209.11302](https://arxiv.org/abs/2209.11302) · [OpenAlex](https://openalex.org/W4383108457)

**Citations:** 583 citations (openalex-snapshot · 2026-10-14) · [127](#cited-by) in our library, of the 140673 papers whose bibliographies we read

## Summary
Large language models can propose robot task plans, but current approaches have two weaknesses. They either have to score every possible next action, or they produce free-form text that may name actions or objects the robot cannot use in its current scene. ProgPrompt addresses this by writing the prompt as a Python-like program. The prompt imports the robot's available actions as functions and lists the objects present in the environment. It also includes a few complete example tasks, written as functions whose bodies are sequences of action calls, with natural-language comments describing each step. Given a new task name as a function header, GPT-3 completes the body and so produces a plan that can be run directly. The generated plans include assertions that check preconditions, such as being near the microwave or holding the salmon. When a check fails, recovery actions run, which gives the plan a form of feedback from the environment. The authors used ablations to work out which prompt elements and generation constraints matter. They report state-of-the-art success rates on household tasks in the VirtualHome simulator and also deployed the method on a real robot arm doing tabletop tasks.

## Key points
- **Prompt structure:** the prompt has three parts: an `import` line listing action primitives with their arguments (e.g. `grab`, `putin`, `open`, `switchon`), a list of the objects in the scene, and example task functions such as `throw_away_lime()`.
- **State feedback:** `assert(...)`/`else:` statements check things like whether the robot is near an object, whether it holds an object, or whether a container is open. A separate GPT-3 prompt decides whether each assertion is true, based on a text description of the current scene state.
- **Comments help:** adding natural-language step comments to the example programs raised the task success rate of the generated plans.
- **Comparison with related work:** unlike SayCan, ProgPrompt does not need to list and score every admissible action. Unlike Huang et al. and Inner Monologue, it avoids proposing objects that are not in the scene.
- **Evaluation:** the method was tested on household tasks in the VirtualHome simulator, where it reached state-of-the-art success rates. It was also run on a physical robot arm for tabletop sorting tasks. The excerpt I was given leaves out the results tables, so this summary has no exact scores.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

Fields: [Other](https://md.research.elamur.ai/free/en/shelves/other/)

## Cited by

Cited by

127 papers of our library cite it (of the 140673 whose bibliographies we read); 10 of them have a page here, 117 not yet.

- [Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context](https://md.research.elamur.ai/free/en/papers/gemini-2024-team-gemini-1-5-unlocking-multimodal/)
- [ToolkenGPT: Augmenting Frozen Language Models with Massive Tools via Tool Embeddings](https://md.research.elamur.ai/free/en/papers/hao-2023-toolkengpt-augmenting-frozen-language-models/)
- [Challenges and Applications of Large Language Models](https://md.research.elamur.ai/free/en/papers/kaddour-2023-challenges-applications-large-language-models/)
- [LLM+P: Empowering Large Language Models with Optimal Planning Proficiency](https://md.research.elamur.ai/free/en/papers/liu-2023-llm-p-empowering-large-language/)
- [A Comprehensive Overview of Large Language Models](https://md.research.elamur.ai/free/en/papers/naveed-2023-comprehensive-overview-large-language-models/)
- [Tool Learning with Foundation Models](https://md.research.elamur.ai/free/en/papers/qin-2023-tool-learning-foundation-models/)
- [Large Language Models for Data Annotation and Synthesis: A Survey](https://md.research.elamur.ai/free/en/papers/tan-2024-large-language-models-data-annotation/)
- [Describe, Explain, Plan and Select: Interactive Planning with Large Language Models Enables Open-World Multi-Task Agents](https://md.research.elamur.ai/free/en/papers/wang-2023-describe-explain-plan-select-interactive/)
- [Executable Code Actions Elicit Better LLM Agents](https://md.research.elamur.ai/free/en/papers/wang-2024-executable-code-actions-elicit-better/)
- [A Survey of Large Language Models](https://md.research.elamur.ai/free/en/papers/zhao-2023-survey-large-language-models/)

## Cite

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@misc{singh2022progprompt,
  title = {{ProgPrompt: Generating Situated Robot Task Plans using Large Language Models}},
  author = {Singh, Ishika and Blukis, Valts and Mousavian, Arsalan and Goyal, Ankit and Xu, Danfei and Tremblay, Jonathan and Fox, Dieter and Thomason, Jesse and Garg, Animesh},
  year = {2022},
  doi = {10.48550/arxiv.2209.11302},
  eprint = {2209.11302},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2209.11302}
}
```

- [← previous Proceedings of the Universal Dependencies Brazilian Festival](https://md.research.elamur.ai/free/en/papers/anon-2022-proceedings-universal-dependencies-brazilian-festival/)
- [next → Prompt for Extraction? PAIE: Prompting Argument Interaction for Event Argument Extraction](https://md.research.elamur.ai/free/en/papers/ma-2022-prompt-extraction-paie-prompting-argument/)
- [↑ back to papers of 2022](https://md.research.elamur.ai/free/en/papers/2022/page/4/)

## References

39 references, 8 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. W. Huang, F. Xia, T. Xiao, H. Chan, J. Liang, P. Florence, A. Zeng, J. Tompson, I. Mordatch, Y. Chebotar, P. Sermanet, N. Brown, T. Jackson, L. Luu, S. Levine, K. Hausman, and B. Ichter, “Inner monologue: Embodied reasoning through planning with language models,” in arXiv preprint arXiv:2207.05608,… [find](https://scholar.google.com/scholar?q=W%2E+Huang%2C+F%2E+Xia%2C+T%2E+Xiao%2C+H%2E+Chan%2C+J%2E+Liang%2C+P%2E+Florence%2C+A%2E+Zeng%2C+J%2E+Tompson%2C+I%2E+Mordatch%2C+Y%2E+Chebotar%2C+P%2E+Sermanet%2C+N%2E+Brown%2C+T%2E+Jackson%2C+L%2E+Luu%2C+S%2E+Levine%2C+K%2E+Hausman%2C+and+B%2E+Ichter%2C+%E2%80%9CInner+monolo)
2. W. Huang, P. Abbeel, D. Pathak, and I. Mordatch, “Language models as zero-shot planners: Extracting actionable knowledge for embodied agents,” arXiv preprint arXiv:2201.07207, 2022. [find](https://scholar.google.com/scholar?q=W%2E+Huang%2C+P%2E+Abbeel%2C+D%2E+Pathak%2C+and+I%2E+Mordatch%2C+%E2%80%9CLanguage+models+as+zero%2Dshot+planners%3A+Extracting+actionable+knowledge+for+embodied+agents%2C%E2%80%9D+arXiv+preprint+arXiv%3A2201%2E07207%2C+2022%2E) [our card](https://md.research.elamur.ai/free/en/papers/huang-2022-language-models-zero-shot-planners/)
3. A. Zeng, M. Attarian, B. Ichter, K. Choromanski, A. Wong, S. Welker, F. Tombari, A. Purohit, M. Ryoo, V. Sindhwani, J. Lee, V. Vanhoucke, and P. Florence, “Socratic models: Composing zero-shot multimodal reasoning with language,” arXiv, 2022. [find](https://scholar.google.com/scholar?q=A%2E+Zeng%2C+M%2E+Attarian%2C+B%2E+Ichter%2C+K%2E+Choromanski%2C+A%2E+Wong%2C+S%2E+Welker%2C+F%2E+Tombari%2C+A%2E+Purohit%2C+M%2E+Ryoo%2C+V%2E+Sindhwani%2C+J%2E+Lee%2C+V%2E+Vanhoucke%2C+and+P%2E+Florence%2C+%E2%80%9CSocratic+models%3A+Composing+zero%2Dshot+multimo) [our card](https://md.research.elamur.ai/free/en/papers/zeng-2022-socratic-models-composing-zero-shot/)
4. M. Ahn, A. Brohan, N. Brown, Y. Chebotar, O. Cortes, B. David, C. Finn, K. Gopalakrishnan, K. Hausman, A. Herzog, D. Ho, J. Hsu, J. Ibarz, B. Ichter, A. Irpan, E. Jang, R. J. Ruano, K. Jeffrey, S. Jesmonth, N. J. Joshi, R. Julian, D. Kalashnikov, Y. Kuang, K.-H. Lee, S. Levine, Y. Lu, L. Luu, C. Par… [find](https://scholar.google.com/scholar?q=M%2E+Ahn%2C+A%2E+Brohan%2C+N%2E+Brown%2C+Y%2E+Chebotar%2C+O%2E+Cortes%2C+B%2E+David%2C+C%2E+Finn%2C+K%2E+Gopalakrishnan%2C+K%2E+Hausman%2C+A%2E+Herzog%2C+D%2E+Ho%2C+J%2E+Hsu%2C+J%2E+Ibarz%2C+B%2E+Ichter%2C+A%2E+Irpan%2C+E%2E+Jang%2C+R%2E+J%2E+Ruano%2C+K%2E+Jeffrey%2C+S%2E+Jes) [our card](https://md.research.elamur.ai/free/en/papers/ahn-2022-i-i-say-grounding-language/)
5. R. E. Fikes and N. J. Nilsson, “Strips: A new approach to the application of theorem proving to problem solving,” in Proceedings of the 2nd International Joint Conference on Artificial Intelligence, ser. IJCAI’71. San Francisco, CA, USA: Morgan Kaufmann Publishers Inc., 1971, p. 608–620. [find](https://scholar.google.com/scholar?q=R%2E+E%2E+Fikes+and+N%2E+J%2E+Nilsson%2C+%E2%80%9CStrips%3A+A+new+approach+to+the+application+of+theorem+proving+to+problem+solving%2C%E2%80%9D+in+Proceedings+of+the+2nd+International+Joint+Conference+on+Artificial+Intelligence%2C+s)
6. C. R. Garrett, T. Lozano-Pérez, and L. P. Kaelbling, “Pddlstream: Integrating symbolic planners and blackbox samplers via optimistic adaptive planning,” Proceedings of the International Conference on Automated Planning and Scheduling, vol. 30, no. 1, pp. 440–448, Jun. 2020. [find](https://scholar.google.com/scholar?q=C%2E+R%2E+Garrett%2C+T%2E+Lozano%2DPe%CC%81rez%2C+and+L%2E+P%2E+Kaelbling%2C+%E2%80%9CPddlstream%3A+Integrating+symbolic+planners+and+blackbox+samplers+via+optimistic+adaptive+planning%2C%E2%80%9D+Proceedings+of+the+International+Conference+on)
7. Y. Jiang, S. Zhang, P. Khandelwal, and P. Stone, “Task planning in robotics: an empirical comparison of pddl-based and asp-based systems,” 2018. [find](https://scholar.google.com/scholar?q=Y%2E+Jiang%2C+S%2E+Zhang%2C+P%2E+Khandelwal%2C+and+P%2E+Stone%2C+%E2%80%9CTask+planning+in+robotics%3A+an+empirical+comparison+of+pddl%2Dbased+and+asp%2Dbased+systems%2C%E2%80%9D+2018%2E)
8. X. Puig, K. Ra, M. Boben, J. Li, T. Wang, S. Fidler, and A. Torralba, “Virtualhome: Simulating household activities via programs,” in 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2018, pp. 8494–8502. [find](https://scholar.google.com/scholar?q=X%2E+Puig%2C+K%2E+Ra%2C+M%2E+Boben%2C+J%2E+Li%2C+T%2E+Wang%2C+S%2E+Fidler%2C+and+A%2E+Torralba%2C+%E2%80%9CVirtualhome%3A+Simulating+household+activities+via+programs%2C%E2%80%9D+in+2018+IEEE%2FCVF+Conference+on+Computer+Vision+and+Pattern+Recognitio)
9. M. Shridhar, J. Thomason, D. Gordon, Y. Bisk, W. Han, R. Mottaghi, L. Zettlemoyer, and D. Fox, “ALFRED: A Benchmark for Interpreting Grounded Instructions for Everyday Tasks,” in The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2020. [find](https://scholar.google.com/scholar?q=M%2E+Shridhar%2C+J%2E+Thomason%2C+D%2E+Gordon%2C+Y%2E+Bisk%2C+W%2E+Han%2C+R%2E+Mottaghi%2C+L%2E+Zettlemoyer%2C+and+D%2E+Fox%2C+%E2%80%9CALFRED%3A+A+Benchmark+for+Interpreting+Grounded+Instructions+for+Everyday+Tasks%2C%E2%80%9D+in+The+IEEE+Conference+o) [our card](https://md.research.elamur.ai/free/en/papers/shridhar-2019-alfred-benchmark-interpreting-grounded-instructions/)
10. J. A. Baier, F. Bacchus, and S. A. McIlraith, “A heuristic search approach to planning with temporally extended preferences,” in Proceedings of the 20th International Joint Conference on Artifical Intelligence, ser. IJCAI’07. San Francisco, CA, USA: Morgan Kaufmann Publishers Inc., 2007, p. 1808–181… [find](https://scholar.google.com/scholar?q=J%2E+A%2E+Baier%2C+F%2E+Bacchus%2C+and+S%2E+A%2E+McIlraith%2C+%E2%80%9CA+heuristic+search+approach+to+planning+with+temporally+extended+preferences%2C%E2%80%9D+in+Proceedings+of+the+20th+International+Joint+Conference+on+Artifical+Int)
11. J. Hoffmann, “Ff: The fast-forward planning system,” AI Magazine, vol. 22, no. 3, p. 57, Sep. 2001. [find](https://scholar.google.com/scholar?q=J%2E+Hoffmann%2C+%E2%80%9CFf%3A+The+fast%2Dforward+planning+system%2C%E2%80%9D+AI+Magazine%2C+vol%2E+22%2C+no%2E+3%2C+p%2E+57%2C+Sep%2E+2001%2E)
12. M. Helmert, “The fast downward planning system,” J. Artif. Int. Res., vol. 26, no. 1, p. 191–246, jul 2006. [find](https://scholar.google.com/scholar?q=M%2E+Helmert%2C+%E2%80%9CThe+fast+downward+planning+system%2C%E2%80%9D+J%2E+Artif%2E+Int%2E+Res%2E%2C+vol%2E+26%2C+no%2E+1%2C+p%2E+191%E2%80%93246%2C+jul+2006%2E)
13. D. Bryce and S. Kambhampati, “A tutorial on planning graph based reachability heuristics,” AI Magazine, vol. 28, no. 1, p. 47, Mar. 2007. [find](https://scholar.google.com/scholar?q=D%2E+Bryce+and+S%2E+Kambhampati%2C+%E2%80%9CA+tutorial+on+planning+graph+based+reachability+heuristics%2C%E2%80%9D+AI+Magazine%2C+vol%2E+28%2C+no%2E+1%2C+p%2E+47%2C+Mar%2E+2007%2E)
14. B. Eysenbach, R. R. Salakhutdinov, and S. Levine, “Search on the replay buffer: Bridging planning and reinforcement learning,” in Advances in Neural Information Processing Systems, H. Wallach, H. Larochelle, A. Beygelzimer, F. d'Alché-Buc, E. Fox, and R. Garnett, Eds., vol. 32. Curran Associates, I… [find](https://scholar.google.com/scholar?q=B%2E+Eysenbach%2C+R%2E+R%2E+Salakhutdinov%2C+and+S%2E+Levine%2C+%E2%80%9CSearch+on+the+replay+buffer%3A+Bridging+planning+and+reinforcement+learning%2C%E2%80%9D+in+Advances+in+Neural+Information+Processing+Systems%2C+H%2E+Wallach%2C+H%2E+Laro)
15. D. Xu, S. Nair, Y. Zhu, J. Gao, A. Garg, L. Fei-Fei, and S. Savarese, “Neural task programming: Learning to generalize across hierarchical tasks,” in 2018 IEEE International Conference on Robotics and Automation (ICRA), 2018, pp. 3795–3802. [find](https://scholar.google.com/scholar?q=D%2E+Xu%2C+S%2E+Nair%2C+Y%2E+Zhu%2C+J%2E+Gao%2C+A%2E+Garg%2C+L%2E+Fei%2DFei%2C+and+S%2E+Savarese%2C+%E2%80%9CNeural+task+programming%3A+Learning+to+generalize+across+hierarchical+tasks%2C%E2%80%9D+in+2018+IEEE+International+Conference+on+Robotics+and)
16. D. Xu, R. Martı́n-Martı́n, D.-A. Huang, Y. Zhu, S. Savarese, and L. F. Fei-Fei, “Regression planning networks,” in Advances in Neural Information Processing Systems, H. Wallach, H. Larochelle, A. Beygelzimer, F. d'Alché-Buc, E. Fox, and R. Garnett, Eds., vol. 32. Curran Associates, Inc., 2019. [find](https://scholar.google.com/scholar?q=D%2E+Xu%2C+R%2E+Mart%C4%B1%CC%81n%2DMart%C4%B1%CC%81n%2C+D%2E%2DA%2E+Huang%2C+Y%2E+Zhu%2C+S%2E+Savarese%2C+and+L%2E+F%2E+Fei%2DFei%2C+%E2%80%9CRegression+planning+networks%2C%E2%80%9D+in+Advances+in+Neural+Information+Processing+Systems%2C+H%2E+Wallach%2C+H%2E+Larochelle%2C+A%2E+Beyg)
17. T. Silver, R. Chitnis, N. Kumar, W. McClinton, T. Lozano-Perez, L. P. Kaelbling, and J. Tenenbaum, “Inventing relational state and action abstractions for effective and efficient bilevel planning,” in The Multi-disciplinary Conference on Reinforcement Learning and Decision Making (RLDM), 2022. [find](https://scholar.google.com/scholar?q=T%2E+Silver%2C+R%2E+Chitnis%2C+N%2E+Kumar%2C+W%2E+McClinton%2C+T%2E+Lozano%2DPerez%2C+L%2E+P%2E+Kaelbling%2C+and+J%2E+Tenenbaum%2C+%E2%80%9CInventing+relational+state+and+action+abstractions+for+effective+and+efficient+bilevel+planning%2C%E2%80%9D+in)
18. D. Shah, A. T. Toshev, S. Levine, and brian ichter, “Value function spaces: Skill-centric state abstractions for long-horizon reasoning,” in International Conference on Learning Representations, 2022. [find](https://scholar.google.com/scholar?q=D%2E+Shah%2C+A%2E+T%2E+Toshev%2C+S%2E+Levine%2C+and+brian+ichter%2C+%E2%80%9CValue+function+spaces%3A+Skill%2Dcentric+state+abstractions+for+long%2Dhorizon+reasoning%2C%E2%80%9D+in+International+Conference+on+Learning+Representations%2C+2022%2E)
19. A. Srinivas, A. Jabri, P. Abbeel, S. Levine, and C. Finn, “Universal planning networks: Learning generalizable representations for visuomotor control,” in Proceedings of the 35th International Conference on Machine Learning, ser. Proceedings of Machine Learning Research, J. Dy and A. Krause, Eds., v… [find](https://scholar.google.com/scholar?q=A%2E+Srinivas%2C+A%2E+Jabri%2C+P%2E+Abbeel%2C+S%2E+Levine%2C+and+C%2E+Finn%2C+%E2%80%9CUniversal+planning+networks%3A+Learning+generalizable+representations+for+visuomotor+control%2C%E2%80%9D+in+Proceedings+of+the+35th+International+Confere)
20. T. Kurutach, A. Tamar, G. Yang, S. J. Russell, and P. Abbeel, “Learning plannable representations with causal infogan,” in Advances in Neural Information Processing Systems, S. Bengio, H. Wallach, H. Larochelle, K. Grauman, N. Cesa-Bianchi, and R. Garnett, Eds., vol. 31. Curran Associates, Inc., 201… [find](https://scholar.google.com/scholar?q=T%2E+Kurutach%2C+A%2E+Tamar%2C+G%2E+Yang%2C+S%2E+J%2E+Russell%2C+and+P%2E+Abbeel%2C+%E2%80%9CLearning+plannable+representations+with+causal+infogan%2C%E2%80%9D+in+Advances+in+Neural+Information+Processing+Systems%2C+S%2E+Bengio%2C+H%2E+Wallach%2C+H%2E+)
21. A. Akakzia, C. Colas, P.-Y. Oudeyer, M. CHETOUANI, and O. Sigaud, “Grounding language to autonomously-acquired skills via goal generation,” in International Conference on Learning Representations, 2021. [find](https://scholar.google.com/scholar?q=A%2E+Akakzia%2C+C%2E+Colas%2C+P%2E%2DY%2E+Oudeyer%2C+M%2E+CHETOUANI%2C+and+O%2E+Sigaud%2C+%E2%80%9CGrounding+language+to+autonomously%2Dacquired+skills+via+goal+generation%2C%E2%80%9D+in+International+Conference+on+Learning+Representations%2C+202)
22. S. Nair and C. Finn, “Hierarchical foresight: Self-supervised learning of long-horizon tasks via visual subgoal generation,” in International Conference on Learning Representations, 2020. [find](https://scholar.google.com/scholar?q=S%2E+Nair+and+C%2E+Finn%2C+%E2%80%9CHierarchical+foresight%3A+Self%2Dsupervised+learning+of+long%2Dhorizon+tasks+via+visual+subgoal+generation%2C%E2%80%9D+in+International+Conference+on+Learning+Representations%2C+2020%2E)
23. Y. Jiang, S. S. Gu, K. P. Murphy, and C. Finn, “Language as an abstraction for hierarchical deep reinforcement learning,” in Advances in Neural Information Processing Systems, H. Wallach, H. Larochelle, A. Beygelzimer, F. d'Alché-Buc, E. Fox, and R. Garnett, Eds., vol. 32. Curran Associates, Inc.,… [find](https://scholar.google.com/scholar?q=Y%2E+Jiang%2C+S%2E+S%2E+Gu%2C+K%2E+P%2E+Murphy%2C+and+C%2E+Finn%2C+%E2%80%9CLanguage+as+an+abstraction+for+hierarchical+deep+reinforcement+learning%2C%E2%80%9D+in+Advances+in+Neural+Information+Processing+Systems%2C+H%2E+Wallach%2C+H%2E+Larochell)
24. S. Mirchandani, S. Karamcheti, and D. Sadigh, “Ella: Exploration through learned language abstraction,” in Advances in Neural Information Processing Systems, M. Ranzato, A. Beygelzimer, Y. Dauphin, P. Liang, and J. W. Vaughan, Eds., vol. 34. Curran Associates, Inc., 2021, pp. 29 529–29 540. [find](https://scholar.google.com/scholar?q=S%2E+Mirchandani%2C+S%2E+Karamcheti%2C+and+D%2E+Sadigh%2C+%E2%80%9CElla%3A+Exploration+through+learned+language+abstraction%2C%E2%80%9D+in+Advances+in+Neural+Information+Processing+Systems%2C+M%2E+Ranzato%2C+A%2E+Beygelzimer%2C+Y%2E+Dauphin%2C+P%2E)
25. P. Sharma, A. Torralba, and J. Andreas, “Skill induction and planning with latent language,” in Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). Dublin, Ireland: Association for Computational Linguistics, May 2022, pp. 1713–1726. [find](https://scholar.google.com/scholar?q=P%2E+Sharma%2C+A%2E+Torralba%2C+and+J%2E+Andreas%2C+%E2%80%9CSkill+induction+and+planning+with+latent+language%2C%E2%80%9D+in+Proceedings+of+the+60th+Annual+Meeting+of+the+Association+for+Computational+Linguistics+%28Volume+1%3A+Long+)
26. Y. Zhu, J. Tremblay, S. Birchfield, and Y. Zhu, “Hierarchical planning for long-horizon manipulation with geometric and symbolic scene graphs,” 2020. [find](https://scholar.google.com/scholar?q=Y%2E+Zhu%2C+J%2E+Tremblay%2C+S%2E+Birchfield%2C+and+Y%2E+Zhu%2C+%E2%80%9CHierarchical+planning+for+long%2Dhorizon+manipulation+with+geometric+and+symbolic+scene+graphs%2C%E2%80%9D+2020%2E)
27. T. B. Brown, B. Mann, N. Ryder, M. Subbiah, J. Kaplan, P. Dhariwal, A. Neelakantan, P. Shyam, G. Sastry, A. Askell, S. Agarwal, A. Herbert-Voss, G. Krueger, T. Henighan, R. Child, A. Ramesh, D. M. Ziegler, J. Wu, C. Winter, C. Hesse, M. Chen, E. Sigler, M. Litwin, S. Gray, B. Chess, J. Clark, C. Ber… [find](https://scholar.google.com/scholar?q=T%2E+B%2E+Brown%2C+B%2E+Mann%2C+N%2E+Ryder%2C+M%2E+Subbiah%2C+J%2E+Kaplan%2C+P%2E+Dhariwal%2C+A%2E+Neelakantan%2C+P%2E+Shyam%2C+G%2E+Sastry%2C+A%2E+Askell%2C+S%2E+Agarwal%2C+A%2E+Herbert%2DVoss%2C+G%2E+Krueger%2C+T%2E+Henighan%2C+R%2E+Child%2C+A%2E+Ramesh%2C+D%2E+M%2E+Zie)
28. M. Chen, J. Tworek, H. Jun, Q. Yuan, H. P. d. O. Pinto, J. Kaplan, H. Edwards, Y. Burda, N. Joseph, G. Brockman, A. Ray, R. Puri, G. Krueger, M. Petrov, H. Khlaaf, G. Sastry, P. Mishkin, B. Chan, S. Gray, N. Ryder, M. Pavlov, A. Power, L. Kaiser, M. Bavarian, C. Winter, P. Tillet, F. P. Such, D. Cum… [find](https://scholar.google.com/scholar?q=M%2E+Chen%2C+J%2E+Tworek%2C+H%2E+Jun%2C+Q%2E+Yuan%2C+H%2E+P%2E+d%2E+O%2E+Pinto%2C+J%2E+Kaplan%2C+H%2E+Edwards%2C+Y%2E+Burda%2C+N%2E+Joseph%2C+G%2E+Brockman%2C+A%2E+Ray%2C+R%2E+Puri%2C+G%2E+Krueger%2C+M%2E+Petrov%2C+H%2E+Khlaaf%2C+G%2E+Sastry%2C+P%2E+Mishkin%2C+B%2E+Chan%2C+S%2E+G) [our card](https://md.research.elamur.ai/free/en/papers/chen-2021-evaluating-large-language-models-trained/)
29. T. Luong, H. Pham, and C. D. Manning, “Effective approaches to attention-based neural machine translation,” in Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing. Lisbon, Portugal: Association for Computational Linguistics, Sept. 2015, pp. 1412–1421. [find](https://scholar.google.com/scholar?q=T%2E+Luong%2C+H%2E+Pham%2C+and+C%2E+D%2E+Manning%2C+%E2%80%9CEffective+approaches+to+attention%2Dbased+neural+machine+translation%2C%E2%80%9D+in+Proceedings+of+the+2015+Conference+on+Empirical+Methods+in+Natural+Language+Processing%2E+L)
30. S. Wiseman, S. Shieber, and A. Rush, “Challenges in data-to-document generation,” in Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing. Copenhagen, Denmark: Association for Computational Linguistics, Sept. 2017, pp. 2253–2263. [find](https://scholar.google.com/scholar?q=S%2E+Wiseman%2C+S%2E+Shieber%2C+and+A%2E+Rush%2C+%E2%80%9CChallenges+in+data%2Dto%2Ddocument+generation%2C%E2%80%9D+in+Proceedings+of+the+2017+Conference+on+Empirical+Methods+in+Natural+Language+Processing%2E+Copenhagen%2C+Denmark%3A+Associ) [our card](https://md.research.elamur.ai/free/en/papers/wiseman-2017-challenges-data-document-generation/)
31. A. Holtzman, J. Buys, L. Du, M. Forbes, and Y. Choi, “The curious case of neural text degeneration,” in International Conference on Learning Representations, 2020. [find](https://scholar.google.com/scholar?q=A%2E+Holtzman%2C+J%2E+Buys%2C+L%2E+Du%2C+M%2E+Forbes%2C+and+Y%2E+Choi%2C+%E2%80%9CThe+curious+case+of+neural+text+degeneration%2C%E2%80%9D+in+International+Conference+on+Learning+Representations%2C+2020%2E)
32. P. Jansen, “Visually-grounded planning without vision: Language models infer detailed plans from high-level instructions,” in Findings of the Association for Computational Linguistics: EMNLP 2020. Online: Association for Computational Linguistics, Nov. 2020, pp. 4412–4417. [find](https://scholar.google.com/scholar?q=P%2E+Jansen%2C+%E2%80%9CVisually%2Dgrounded+planning+without+vision%3A+Language+models+infer+detailed+plans+from+high%2Dlevel+instructions%2C%E2%80%9D+in+Findings+of+the+Association+for+Computational+Linguistics%3A+EMNLP+2020%2E+Onl)
33. S. Li, X. Puig, C. Paxton, Y. Du, C. Wang, L. Fan, T. Chen, D.-A. Huang, E. Akyürek, A. Anandkumar, J. Andreas, I. Mordatch, A. Torralba, and Y. Zhu, “Pre-trained language models for interactive decision-making,” 2022. [find](https://scholar.google.com/scholar?q=S%2E+Li%2C+X%2E+Puig%2C+C%2E+Paxton%2C+Y%2E+Du%2C+C%2E+Wang%2C+L%2E+Fan%2C+T%2E+Chen%2C+D%2E%2DA%2E+Huang%2C+E%2E+Akyu%CC%88rek%2C+A%2E+Anandkumar%2C+J%2E+Andreas%2C+I%2E+Mordatch%2C+A%2E+Torralba%2C+and+Y%2E+Zhu%2C+%E2%80%9CPre%2Dtrained+language+models+for+interactive+deci)
34. R. Patel and E. Pavlick, “Mapping language models to grounded conceptual spaces,” in International Conference on Learning Representations, 2022. [find](https://scholar.google.com/scholar?q=R%2E+Patel+and+E%2E+Pavlick%2C+%E2%80%9CMapping+language+models+to+grounded+conceptual+spaces%2C%E2%80%9D+in+International+Conference+on+Learning+Representations%2C+2022%2E)
35. P. Liu, W. Yuan, J. Fu, Z. Jiang, H. Hayashi, and G. Neubig, “Pre-train, prompt, and predict: A systematic survey of prompting methods in natural language processing,” 2021. [find](https://scholar.google.com/scholar?q=P%2E+Liu%2C+W%2E+Yuan%2C+J%2E+Fu%2C+Z%2E+Jiang%2C+H%2E+Hayashi%2C+and+G%2E+Neubig%2C+%E2%80%9CPre%2Dtrain%2C+prompt%2C+and+predict%3A+A+systematic+survey+of+prompting+methods+in+natural+language+processing%2C%E2%80%9D+2021%2E) [our card](https://md.research.elamur.ai/free/en/papers/liu-2021-pre-train-prompt-predict-systematic/)
36. J. Wei, X. Wang, D. Schuurmans, M. Bosma, B. Ichter, F. Xia, E. Chi, Q. Le, and D. Zhou, “Chain of thought prompting elicits reasoning in large language models,” 2022. [find](https://scholar.google.com/scholar?q=J%2E+Wei%2C+X%2E+Wang%2C+D%2E+Schuurmans%2C+M%2E+Bosma%2C+B%2E+Ichter%2C+F%2E+Xia%2C+E%2E+Chi%2C+Q%2E+Le%2C+and+D%2E+Zhou%2C+%E2%80%9CChain+of+thought+prompting+elicits+reasoning+in+large+language+models%2C%E2%80%9D+2022%2E) [our card](https://md.research.elamur.ai/free/en/papers/wei-2022-chain-thought-prompting-elicits-reasoning/)
37. M. Danielczuk, A. Mousavian, C. Eppner, and D. Fox, “Object rearrangement using learned implicit collision functions,” IEEE International Conference on Robotics and Automation (ICRA), 2021. [find](https://scholar.google.com/scholar?q=M%2E+Danielczuk%2C+A%2E+Mousavian%2C+C%2E+Eppner%2C+and+D%2E+Fox%2C+%E2%80%9CObject+rearrangement+using+learned+implicit+collision+functions%2C%E2%80%9D+IEEE+International+Conference+on+Robotics+and+Automation+%28ICRA%29%2C+2021%2E)
38. M. Sundermeyer, A. Mousavian, R. Triebel, and D. Fox, “Contact-graspnet: Efficient 6-dof grasp generation in cluttered scenes,” in 2021 IEEE International Conference on Robotics and Automation (ICRA), 2021, pp. 13 438–13 444. [find](https://scholar.google.com/scholar?q=M%2E+Sundermeyer%2C+A%2E+Mousavian%2C+R%2E+Triebel%2C+and+D%2E+Fox%2C+%E2%80%9CContact%2Dgraspnet%3A+Efficient+6%2Ddof+grasp+generation+in+cluttered+scenes%2C%E2%80%9D+in+2021+IEEE+International+Conference+on+Robotics+and+Automation+%28ICRA%29%2C)
39. X. Gu, T.-Y. Lin, W. Kuo, and Y. Cui, “Open-vocabulary object detection via vision and language knowledge distillation,” in International Conference on Learning Representations, 2022. [find](https://scholar.google.com/scholar?q=X%2E+Gu%2C+T%2E%2DY%2E+Lin%2C+W%2E+Kuo%2C+and+Y%2E+Cui%2C+%E2%80%9COpen%2Dvocabulary+object+detection+via+vision+and+language+knowledge+distillation%2C%E2%80%9D+in+International+Conference+on+Learning+Representations%2C+2022%2E)