---
title: "Pre-Trained Language Models for Interactive Decision-Making"
description: "The authors propose LID, which turns a policy's inputs (the goal, the history of past observations and… 49 references, cited by 71 papers here."
authors: ["Shuang Li", "Xavier Puig", "Chris Paxton", "Yilun Du", "Clinton Wang", "Linxi Fan", "Tao Chen", "De-An Huang", "Ekin Akyurek", "Anima Anandkumar", "Jacob Andreas", "Igor Mordatch", "Antonio Torralba", "Yuke Zhu"]
year: 2022
venue: "arXiv:2202.01771"
source: "https://arxiv.org/abs/2202.01771"
doi: "10.48550/arxiv.2202.01771"
arxiv: "2202.01771"
licence: "arXiv — non-exclusive license to distribute"
licence_url: "http://arxiv.org/licenses/nonexclusive-distrib/1.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/li-2022-pre-trained-language-models-interactive/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/li-2022-pre-trained-language-models-interactive/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2022](https://md.research.elamur.ai/free/en/papers/2022/page/4/) / Pre-Trained Language Models for Interactive Decision-Making

# Pre-Trained Language Models for Interactive Decision-Making

**Authors:** [Shuang Li](https://md.research.elamur.ai/free/en/authors/shuang-li/), Xavier Puig, Chris Paxton, [Yilun Du](https://md.research.elamur.ai/free/en/authors/yilun-du/), Clinton Wang, [Linxi Fan](https://md.research.elamur.ai/free/en/authors/linxi-fan/), [Tao Chen](https://md.research.elamur.ai/free/en/authors/tao-chen/), [De-An Huang](https://md.research.elamur.ai/free/en/authors/de-an-huang/), [Ekin Akyurek](https://md.research.elamur.ai/free/en/authors/ekin-akyurek/), [Anima Anandkumar](https://md.research.elamur.ai/free/en/authors/anima-anandkumar/), [Jacob Andreas](https://md.research.elamur.ai/free/en/authors/jacob-andreas/), [Igor Mordatch](https://md.research.elamur.ai/free/en/authors/igor-mordatch/), [Antonio Torralba](https://md.research.elamur.ai/free/en/authors/antonio-b-torralba/) and [Yuke Zhu](https://md.research.elamur.ai/free/en/authors/yuke-zhu/) · **Year:** 2022 · **Venue:** arXiv:2202.01771 · **Licence:** [arXiv — non-exclusive license to distribute](http://arxiv.org/licenses/nonexclusive-distrib/1.0/)

**Links:** [Publisher's page](https://arxiv.org/abs/2202.01771) · [PDF](https://arxiv.org/pdf/2202.01771) · [DOI 10.48550/arxiv.2202.01771](https://doi.org/10.48550/arxiv.2202.01771) · [arXiv 2202.01771](https://arxiv.org/abs/2202.01771) · [OpenAlex](https://openalex.org/W4221159132)

**Citations:** 53 citations (OpenAlex, 2026-10-14) · [71](https://md.research.elamur.ai/free/en/papers/li-2022-pre-trained-language-models-interactive/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
The paper asks whether language models trained only on text can serve as a backbone for sequential decision-making, even when the task involves no language. The authors propose LID, which turns a policy's inputs (the goal, the history of past observations and actions, and the current partial observation) into a sequence of embeddings. These go into a transformer initialized from a pre-trained LM, and a task-specific head then predicts the next action. The approach is tested in VirtualHome and BabyAI, where goals and observations appear as scene graphs, grids or language instructions.

Training uses behavior cloning on expert demonstrations. When no expert data is available, it uses Active Data Gathering (ADG): the agent explores, relabels failed trajectories in hindsight with sub-goals they did reach, and learns from them. LM initialization clearly improves generalization to unseen combinations of goals, states and objects. Ablations show that sequential inputs and pre-trained weights matter, while phrasing inputs as natural language does not. This suggests LM pre-training provides representations useful well beyond text.

## Key points
- With behavior cloning in VirtualHome, LID yields about 20% more successful policies on in-distribution tasks. On novel tasks it raises completion rates by 43.6%.
- ADG trains the LM-based policy without pre-collected demonstrations. It beats RL methods and exceeds the best baseline by 25.1% on combinatorial generalization.
- Ablations:
  - Removing sequential structure (No-Seq) sharply hurts novel-task performance.
  - Training from scratch (No-Pretrain) lowers novel-task success by 11.2%.
  - Freezing LM weights (No-FT) degrades both evaluation settings.
- Encoding inputs as natural strings versus arbitrary sequences barely changes final performance. Language encodings mainly help data efficiency.
- Failures split into grounding errors (acting on the wrong object) and policy errors (not finding or not using targets). ADG's relabeling also depends on hand-designed rules.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Google](https://md.research.elamur.ai/free/en/companies/google/) — Company, Mountain View, United States · 492 works · 440,463 citations
- [Massachusetts Institute of Technology](https://md.research.elamur.ai/free/en/universities/massachusetts-institute-of-technology/) — University, Cambridge, United States · 243 works · 76,506 citations
- [The University of Texas at Austin](https://md.research.elamur.ai/free/en/universities/the-university-of-texas-at-austin/) — University, Austin, United States · 119 works · 57,950 citations
- [Nvidia](https://md.research.elamur.ai/free/en/companies/nvidia/) — Company, Santa Clara, United States · 37 works · 12,656 citations
- [Nvidia (United Kingdom)](https://md.research.elamur.ai/free/en/companies/nvidia-united-kingdom/) — Company, Reading, United Kingdom · 27 works · 11,141 citations
- [California Institute of Technology](https://md.research.elamur.ai/free/en/universities/california-institute-of-technology/) — University, Pasadena, United States · 11 works · 1,138 citations

📚 Shelves (2): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 18,614 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 17,965 works

🧭 Topics (1): [🧠 Language models & pretraining](https://md.research.elamur.ai/free/en/topics/language-models/) · 3,894 works

## Cite

Cite as: Shuang Li, Xavier Puig, Chris Paxton, Yilun Du, Clinton Wang, Linxi Fan, Tao Chen, De-An Huang, Ekin Akyurek, Anima Anandkumar, Jacob Andreas, Igor Mordatch, Antonio Torralba and Yuke Zhu. 2022. Pre-Trained Language Models for Interactive Decision-Making. arXiv:2202.01771.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@misc{li2022pre,
  title = {{Pre-Trained Language Models for Interactive Decision-Making}},
  author = {Shuang Li, Xavier Puig, Chris Paxton, Yilun Du, Clinton Wang, Linxi Fan, Tao Chen, De-An Huang, Ekin Akyurek, Anima Anandkumar, Jacob Andreas, Igor Mordatch, Antonio Torralba and Yuke Zhu},
  year = {2022},
  doi = {10.48550/arxiv.2202.01771},
  eprint = {2202.01771},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2202.01771}
}
```

- [← previous An Improved Baseline for Sentence-level Relation Extraction](https://md.research.elamur.ai/free/en/papers/zhou-2022-improved-baseline-sentence-level-relation/)
- [next → The case for 4-bit precision: k-bit Inference Scaling Laws](https://md.research.elamur.ai/free/en/papers/dettmers-2022-case-4-bit-precision-k/)
- [↑ back to papers of 2022](https://md.research.elamur.ai/free/en/papers/2022/page/4/)

## References

49 references, 18 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. P. Ammanabrolu and M. O. Riedl. Playing text-adventure games with graph-based deep reinforcement learning. arXiv preprint arXiv:1812.01628, 2018. [find](https://scholar.google.com/scholar?q=P%2E+Ammanabrolu+and+M%2E+O%2E+Riedl%2E+Playing+text%2Dadventure+games+with+graph%2Dbased+deep+reinforcement+learning%2E+arXiv+preprint+arXiv%3A1812%2E01628%2C+2018%2E)
2. J. Andreas and D. Klein. Learning with latent language. In North American Association for Computational Linguistics, 2022. [find](https://scholar.google.com/scholar?q=J%2E+Andreas+and+D%2E+Klein%2E+Learning+with+latent+language%2E+In+North+American+Association+for+Computational+Linguistics%2C+2022%2E)
3. M. Andrychowicz, F. Wolski, A. Ray, J. Schneider, R. Fong, P. Welinder, B. McGrew, J. Tobin, P. Abbeel, and W. Zaremba. [Hindsight experience replay](https://md.research.elamur.ai/free/en/papers/andrychowicz-2017-hindsight-experience-replay/). arXiv preprint arXiv:1707.01495, 2017.
4. M. L. Athul Paul Jacob and J. Andreas. Multitasking inhibits semantic drift. In North American Association for Computational Linguistics, 2021. [find](https://scholar.google.com/scholar?q=M%2E+L%2E+Athul+Paul+Jacob+and+J%2E+Andreas%2E+Multitasking+inhibits+semantic+drift%2E+In+North+American+Association+for+Computational+Linguistics%2C+2021%2E)
5. G. Brockman, V. Cheung, L. Pettersson, J. Schneider, J. Schulman, J. Tang, and W. Zaremba. Openai gym, 2016. [find](https://scholar.google.com/scholar?q=G%2E+Brockman%2C+V%2E+Cheung%2C+L%2E+Pettersson%2C+J%2E+Schneider%2C+J%2E+Schulman%2C+J%2E+Tang%2C+and+W%2E+Zaremba%2E+Openai+gym%2C+2016%2E)
6. T. B. Brown, B. Mann, N. Ryder, M. Subbiah, J. Kaplan, P. Dhariwal, A. Neelakantan, P. Shyam, G. Sastry, A. Askell, et al. [Language models are few-shot learners](https://md.research.elamur.ai/free/en/papers/brown-2020-language-models-few-shot-learners/). arXiv preprint arXiv:2005.14165, 2020.
7. L. Chen, K. Lu, A. Rajeswaran, K. Lee, A. Grover, M. Laskin, P. Abbeel, A. Srinivas, and I. Mordatch. Decision transformer: Reinforcement learning via sequence modeling. arXiv preprint arXiv:2106.01345, 2021. [find](https://scholar.google.com/scholar?q=L%2E+Chen%2C+K%2E+Lu%2C+A%2E+Rajeswaran%2C+K%2E+Lee%2C+A%2E+Grover%2C+M%2E+Laskin%2C+P%2E+Abbeel%2C+A%2E+Srinivas%2C+and+I%2E+Mordatch%2E+Decision+transformer%3A+Reinforcement+learning+via+sequence+modeling%2E+arXiv+preprint+arXiv%3A2106%2E0134)
8. S. Deerwester, S. T. Dumais, G. W. Furnas, T. K. Landauer, and R. Harshman. Indexing by latent semantic analysis. Journal of the American society for information science, 41(6):391–407, 1990. [find](https://scholar.google.com/scholar?q=S%2E+Deerwester%2C+S%2E+T%2E+Dumais%2C+G%2E+W%2E+Furnas%2C+T%2E+K%2E+Landauer%2C+and+R%2E+Harshman%2E+Indexing+by+latent+semantic+analysis%2E+Journal+of+the+American+society+for+information+science%2C+41%286%29%3A391%E2%80%93407%2C+1990%2E)
9. J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova. [Bert: Pre-training of deep bidirectional transformers for language understanding](https://md.research.elamur.ai/free/en/papers/devlin-2019-bert-pre-training-deep-bidirectional/). arXiv preprint arXiv:1810.04805, 2018.
10. A. Dosovitskiy, L. Beyer, A. Kolesnikov, D. Weissenborn, X. Zhai, T. Unterthiner, M. Dehghani, M. Minderer, G. Heigold, S. Gelly, et al. [An image is worth 16x16 words: Transformers for image recognition at scale](https://md.research.elamur.ai/free/en/papers/dosovitskiy-2020-image-worth-16x16-words-transformers/). arXiv preprint arXiv:2010.11929, 2020.
11. S. T. Dumais. Latent semantic analysis. Annual review of information science and technology, 38(1):188–230, 2004. [find](https://scholar.google.com/scholar?q=S%2E+T%2E+Dumais%2E+Latent+semantic+analysis%2E+Annual+review+of+information+science+and+technology%2C+38%281%29%3A188%E2%80%93230%2C+2004%2E)
12. D. Ghosh, A. Gupta, A. Reddy, J. Fu, C. Devin, B. Eysenbach, and S. Levine. Learning to reach goals via iterated supervised learning. arXiv preprint arXiv:1912.06088, 2019. — [arXiv:1912.06088](https://arxiv.org/abs/1912.06088) [find](https://scholar.google.com/scholar?q=D%2E+Ghosh%2C+A%2E+Gupta%2C+A%2E+Reddy%2C+J%2E+Fu%2C+C%2E+Devin%2C+B%2E+Eysenbach%2C+and+S%2E+Levine%2E+Learning+to+reach+goals+via+iterated+supervised+learning%2E+arXiv+preprint+arXiv%3A1912%2E06088%2C+2019%2E+%E2%80%94+%5BarXiv%3A1912%2E06088%5D%28https%3A)
13. F. Hill, S. Mokra, N. Wong, and T. Harley. Human instruction-following with deep reinforcement learning via transfer-learning from text. arXiv preprint arXiv:2005.09382, 2020. — [arXiv:2005.09382](https://arxiv.org/abs/2005.09382) [find](https://scholar.google.com/scholar?q=F%2E+Hill%2C+S%2E+Mokra%2C+N%2E+Wong%2C+and+T%2E+Harley%2E+Human+instruction%2Dfollowing+with+deep+reinforcement+learning+via+transfer%2Dlearning+from+text%2E+arXiv+preprint+arXiv%3A2005%2E09382%2C+2020%2E+%E2%80%94+%5BarXiv%3A2005%2E09382%5D%28htt)
14. S. Hochreiter and J. Schmidhuber. Long short-term memory. Neural computation, 9(8):1735– 1780, 1997. [find](https://scholar.google.com/scholar?q=S%2E+Hochreiter+and+J%2E+Schmidhuber%2E+Long+short%2Dterm+memory%2E+Neural+computation%2C+9%288%29%3A1735%E2%80%93+1780%2C+1997%2E)
15. W. Huang, P. Abbeel, D. Pathak, and I. Mordatch. [Language models as zero-shot planners: Extracting actionable knowledge for embodied agents](https://md.research.elamur.ai/free/en/papers/huang-2022-language-models-zero-shot-planners/). arXiv preprint arXiv:2201.07207, 2022.
16. D. Y.-T. Hui, M. Chevalier-Boisvert, D. Bahdanau, and Y. Bengio. Babyai 1.1, 2020. [find](https://scholar.google.com/scholar?q=D%2E+Y%2E%2DT%2E+Hui%2C+M%2E+Chevalier%2DBoisvert%2C+D%2E+Bahdanau%2C+and+Y%2E+Bengio%2E+Babyai+1%2E1%2C+2020%2E)
17. E. Jang, A. Irpan, M. Khansari, D. Kappler, F. Ebert, C. Lynch, S. Levine, and C. Finn. Bc-z: Zero-shot task generalization with robotic imitation learning. In Conference on Robot Learning, pages 991–1002. PMLR, 2022. [find](https://scholar.google.com/scholar?q=E%2E+Jang%2C+A%2E+Irpan%2C+M%2E+Khansari%2C+D%2E+Kappler%2C+F%2E+Ebert%2C+C%2E+Lynch%2C+S%2E+Levine%2C+and+C%2E+Finn%2E+Bc%2Dz%3A+Zero%2Dshot+task+generalization+with+robotic+imitation+learning%2E+In+Conference+on+Robot+Learning%2C+pages+991%E2%80%93)
18. N. S. Keskar, B. McCann, L. R. Varshney, C. Xiong, and R. Socher. [Ctrl: A conditional transformer language model for controllable generation](https://md.research.elamur.ai/free/en/papers/keskar-2019-ctrl-conditional-transformer-language-model/). arXiv preprint arXiv:1909.05858, 2019.
19. N. Kitaev, S. Cao, and D. Klein. [Multilingual constituency parsing with self-attention and pre-training](https://md.research.elamur.ai/free/en/papers/kitaev-2019-multilingual-constituency-parsing-self-attention/). arXiv preprint arXiv:1812.11760, 2018.
20. R. E. Korf. Planning as search: A quantitative approach. Artificial intelligence, 33(1):65–88, 1987. [find](https://scholar.google.com/scholar?q=R%2E+E%2E+Korf%2E+Planning+as+search%3A+A+quantitative+approach%2E+Artificial+intelligence%2C+33%281%29%3A65%E2%80%9388%2C+1987%2E)
21. T. P. Lillicrap, J. J. Hunt, A. Pritzel, N. Heess, T. Erez, Y. Tassa, D. Silver, and D. Wierstra. Continuous control with deep reinforcement learning. arXiv preprint arXiv:1509.02971, 2015. [find](https://scholar.google.com/scholar?q=T%2E+P%2E+Lillicrap%2C+J%2E+J%2E+Hunt%2C+A%2E+Pritzel%2C+N%2E+Heess%2C+T%2E+Erez%2C+Y%2E+Tassa%2C+D%2E+Silver%2C+and+D%2E+Wierstra%2E+Continuous+control+with+deep+reinforcement+learning%2E+arXiv+preprint+arXiv%3A1509%2E02971%2C+2015%2E)
22. J. Lu, D. Batra, D. Parikh, and S. Lee. [Vilbert: Pretraining task-agnostic visiolinguistic representations for vision-and-language tasks](https://md.research.elamur.ai/free/en/papers/lu-2019-vilbert-pretraining-task-agnostic-visiolinguistic/). arXiv preprint arXiv:1908.02265, 2019.
23. K. Lu, A. Grover, P. Abbeel, and I. Mordatch. Pretrained transformers as universal computation engines. arXiv preprint arXiv:2103.05247, 2021. — [arXiv:2103.05247](https://arxiv.org/abs/2103.05247) [find](https://scholar.google.com/scholar?q=K%2E+Lu%2C+A%2E+Grover%2C+P%2E+Abbeel%2C+and+I%2E+Mordatch%2E+Pretrained+transformers+as+universal+computation+engines%2E+arXiv+preprint+arXiv%3A2103%2E05247%2C+2021%2E+%E2%80%94+%5BarXiv%3A2103%2E05247%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2103%2E05247%29)
24. A. Majumdar, A. Shrivastava, S. Lee, P. Anderson, D. Parikh, and D. Batra. Improving vision-and-language navigation with image-text pairs from the web. In European Conference on Computer Vision, pages 259–274. Springer, 2020. [find](https://scholar.google.com/scholar?q=A%2E+Majumdar%2C+A%2E+Shrivastava%2C+S%2E+Lee%2C+P%2E+Anderson%2C+D%2E+Parikh%2C+and+D%2E+Batra%2E+Improving+vision%2Dand%2Dlanguage+navigation+with+image%2Dtext+pairs+from+the+web%2E+In+European+Conference+on+Computer+Vision%2C+pages)
25. T. Mikolov, I. Sutskever, K. Chen, G. S. Corrado, and J. Dean. [Distributed representations of words and phrases and their compositionality](https://md.research.elamur.ai/free/en/papers/mikolov-2013-distributed-representations-words-phrases-their/). In Advances in neural information processing systems, pages 3111–3119, 2013.
26. V. Mnih, A. P. Badia, M. Mirza, A. Graves, T. Lillicrap, T. Harley, D. Silver, and K. Kavukcuoglu. Asynchronous methods for deep reinforcement learning. In International conference on machine learning, pages 1928–1937. PMLR, 2016. [find](https://scholar.google.com/scholar?q=V%2E+Mnih%2C+A%2E+P%2E+Badia%2C+M%2E+Mirza%2C+A%2E+Graves%2C+T%2E+Lillicrap%2C+T%2E+Harley%2C+D%2E+Silver%2C+and+K%2E+Kavukcuoglu%2E+Asynchronous+methods+for+deep+reinforcement+learning%2E+In+International+conference+on+machine+learning)
27. V. Mnih, K. Kavukcuoglu, D. Silver, A. Graves, I. Antonoglou, D. Wierstra, and M. Riedmiller. Playing atari with deep reinforcement learning. arXiv preprint arXiv:1312.5602, 2013. [find](https://scholar.google.com/scholar?q=V%2E+Mnih%2C+K%2E+Kavukcuoglu%2C+D%2E+Silver%2C+A%2E+Graves%2C+I%2E+Antonoglou%2C+D%2E+Wierstra%2C+and+M%2E+Riedmiller%2E+Playing+atari+with+deep+reinforcement+learning%2E+arXiv+preprint+arXiv%3A1312%2E5602%2C+2013%2E)
28. J. Pennington, R. Socher, and C. D. Manning. [Glove: Global vectors for word representation](https://md.research.elamur.ai/free/en/papers/pennington-2014-glove-global-vectors-word-representation/). In Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP), pages 1532–1543, 2014.
29. M. E. Peters, M. Neumann, M. Iyyer, M. Gardner, C. Clark, K. Lee, and L. Zettlemoyer. [Deep contextualized word representations](https://md.research.elamur.ai/free/en/papers/peters-2018-deep-contextualized-word-representations/). arXiv preprint arXiv:1802.05365, 2018.
30. E. A. Platanios, A. Pauls, S. Roy, Y. Zhang, A. Kyte, A. Guo, S. Thomson, J. Krishnamurthy, J. Wolfe, J. Andreas, et al. Value-agnostic conversational semantic parsing. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Confere… [find](https://scholar.google.com/scholar?q=E%2E+A%2E+Platanios%2C+A%2E+Pauls%2C+S%2E+Roy%2C+Y%2E+Zhang%2C+A%2E+Kyte%2C+A%2E+Guo%2C+S%2E+Thomson%2C+J%2E+Krishnamurthy%2C+J%2E+Wolfe%2C+J%2E+Andreas%2C+et+al%2E+Value%2Dagnostic+conversational+semantic+parsing%2E+In+Proceedings+of+the+59th+Annu)
31. X. Puig, K. Ra, M. Boben, J. Li, T. Wang, S. Fidler, and A. Torralba. Virtualhome: Simulating household activities via programs. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 8494–8502, 2018. [find](https://scholar.google.com/scholar?q=X%2E+Puig%2C+K%2E+Ra%2C+M%2E+Boben%2C+J%2E+Li%2C+T%2E+Wang%2C+S%2E+Fidler%2C+and+A%2E+Torralba%2E+Virtualhome%3A+Simulating+household+activities+via+programs%2E+In+Proceedings+of+the+IEEE+Conference+on+Computer+Vision+and+Pattern+Re)
32. X. Puig, T. Shu, S. Li, Z. Wang, J. B. Tenenbaum, S. Fidler, and A. Torralba. Watch-and-help: A challenge for social perception and human-ai collaboration. arXiv preprint arXiv:2010.09890, 2020. — [arXiv:2010.09890](https://arxiv.org/abs/2010.09890) [find](https://scholar.google.com/scholar?q=X%2E+Puig%2C+T%2E+Shu%2C+S%2E+Li%2C+Z%2E+Wang%2C+J%2E+B%2E+Tenenbaum%2C+S%2E+Fidler%2C+and+A%2E+Torralba%2E+Watch%2Dand%2Dhelp%3A+A+challenge+for+social+perception+and+human%2Dai+collaboration%2E+arXiv+preprint+arXiv%3A2010%2E09890%2C+2020%2E+%E2%80%94+%5Bar)
33. A. Radford, K. Narasimhan, T. Salimans, and I. Sutskever. Improving language understanding by generative pre-training. 2018. [find](https://scholar.google.com/scholar?q=A%2E+Radford%2C+K%2E+Narasimhan%2C+T%2E+Salimans%2C+and+I%2E+Sutskever%2E+Improving+language+understanding+by+generative+pre%2Dtraining%2E+2018%2E)
34. A. Radford, J. Wu, R. Child, D. Luan, D. Amodei, I. Sutskever, et al. Language models are unsupervised multitask learners. OpenAI blog, 1(8):9, 2019. [find](https://scholar.google.com/scholar?q=A%2E+Radford%2C+J%2E+Wu%2C+R%2E+Child%2C+D%2E+Luan%2C+D%2E+Amodei%2C+I%2E+Sutskever%2C+et+al%2E+Language+models+are+unsupervised+multitask+learners%2E+OpenAI+blog%2C+1%288%29%3A9%2C+2019%2E)
35. A. Raffin, A. Hill, A. Gleave, A. Kanervisto, M. Ernestus, and N. Dormann. Stable-baselines3: Reliable reinforcement learning implementations. Journal of Machine Learning Research, 22(268):1–8, 2021. [find](https://scholar.google.com/scholar?q=A%2E+Raffin%2C+A%2E+Hill%2C+A%2E+Gleave%2C+A%2E+Kanervisto%2C+M%2E+Ernestus%2C+and+N%2E+Dormann%2E+Stable%2Dbaselines3%3A+Reliable+reinforcement+learning+implementations%2E+Journal+of+Machine+Learning+Research%2C+22%28268%29%3A1%E2%80%938%2C+2021%2E)
36. M. Reid, Y. Yamada, and S. S. Gu. Can wikipedia help offline reinforcement learning? arXiv preprint arXiv:2201.12122, 2022. — [arXiv:2201.12122](https://arxiv.org/abs/2201.12122) [find](https://scholar.google.com/scholar?q=M%2E+Reid%2C+Y%2E+Yamada%2C+and+S%2E+S%2E+Gu%2E+Can+wikipedia+help+offline+reinforcement+learning%3F+arXiv+preprint+arXiv%3A2201%2E12122%2C+2022%2E+%E2%80%94+%5BarXiv%3A2201%2E12122%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2201%2E12122%29)
37. J. Schulman, F. Wolski, P. Dhariwal, A. Radford, and O. Klimov. Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347, 2017. [find](https://scholar.google.com/scholar?q=J%2E+Schulman%2C+F%2E+Wolski%2C+P%2E+Dhariwal%2C+A%2E+Radford%2C+and+O%2E+Klimov%2E+Proximal+policy+optimization+algorithms%2E+arXiv+preprint+arXiv%3A1707%2E06347%2C+2017%2E)
38. P. Sharma, A. Torralba, and J. Andreas. Skill induction and planning with latent language. In Association for Computational Linguistics, 2022. [find](https://scholar.google.com/scholar?q=P%2E+Sharma%2C+A%2E+Torralba%2C+and+J%2E+Andreas%2E+Skill+induction+and+planning+with+latent+language%2E+In+Association+for+Computational+Linguistics%2C+2022%2E)
39. B. Shen, F. Xia, C. Li, R. Martín-Martín, L. Fan, G. Wang, S. Buch, C. D’Arpino, S. Srivastava, L. P. Tchapmi, et al. igibson, a simulation environment for interactive tasks in large realisticscenes. arXiv preprint arXiv:2012.02924, 2020. — [arXiv:2012.02924](https://arxiv.org/abs/2012.02924) [find](https://scholar.google.com/scholar?q=B%2E+Shen%2C+F%2E+Xia%2C+C%2E+Li%2C+R%2E+Mart%C3%ADn%2DMart%C3%ADn%2C+L%2E+Fan%2C+G%2E+Wang%2C+S%2E+Buch%2C+C%2E+D%E2%80%99Arpino%2C+S%2E+Srivastava%2C+L%2E+P%2E+Tchapmi%2C+et+al%2E+igibson%2C+a+simulation+environment+for+interactive+tasks+in+large+realisticscenes%2E+)
40. M. Shridhar, J. Thomason, D. Gordon, Y. Bisk, W. Han, R. Mottaghi, L. Zettlemoyer, and D. Fox. [Alfred: A benchmark for interpreting grounded instructions for everyday tasks](https://md.research.elamur.ai/free/en/papers/shridhar-2019-alfred-benchmark-interpreting-grounded-instructions/). In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10740–10749, 2020.
41. E. Todorov, T. Erez, and Y. Tassa. Mujoco: A physics engine for model-based control. In 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems, pages 5026–5033. IEEE, 2012. [find](https://scholar.google.com/scholar?q=E%2E+Todorov%2C+T%2E+Erez%2C+and+Y%2E+Tassa%2E+Mujoco%3A+A+physics+engine+for+model%2Dbased+control%2E+In+2012+IEEE%2FRSJ+International+Conference+on+Intelligent+Robots+and+Systems%2C+pages+5026%E2%80%935033%2E+IEEE%2C+2012%2E)
42. M. Tsimpoukelli, J. Menick, S. Cabi, S. Eslami, O. Vinyals, and F. Hill. [Multimodal few-shot learning with frozen language models](https://md.research.elamur.ai/free/en/papers/tsimpoukelli-2021-multimodal-few-shot-learning-frozen/). arXiv preprint arXiv:2106.13884, 2021.
43. A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, L. Kaiser, and I. Polosukhin. [Attention is all you need](https://md.research.elamur.ai/free/en/papers/vaswani-2017-attention-all-need/). arXiv preprint arXiv:1706.03762, 2017.
44. J. Vig. [A multiscale visualization of attention in the transformer model](https://md.research.elamur.ai/free/en/papers/vig-2019-multiscale-visualization-attention-transformer-model/). In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics: System Demonstrations, pages 37–42, Florence, Italy, July 2019. Association for Computational Linguistics.
45. B. Wang, R. Shin, X. Liu, O. Polozov, and M. Richardson. [Rat-sql: Relation-aware schema encoding and linking for text-to-sql parsers](https://md.research.elamur.ai/free/en/papers/wang-2020-rat-sql-relation-aware-schema/). arXiv preprint arXiv:1911.04942, 2019.
46. T. Wolf, L. Debut, V. Sanh, J. Chaumond, C. Delangue, A. Moi, P. Cistac, T. Rault, R. Louf, M. Funtowicz, et al. Huggingface’s [transformers: State-of-the-art natural language processing](https://md.research.elamur.ai/free/en/papers/wolf-2020-transformers-state-art-natural-language/). arXiv preprint arXiv:1910.03771, 2019.
47. Z. Yang, Z. Dai, Y. Yang, J. Carbonell, R. R. Salakhutdinov, and Q. V. Le. [Xlnet: Generalized autoregressive pretraining for language understanding](https://md.research.elamur.ai/free/en/papers/yang-2019-xlnet-generalized-autoregressive-pretraining-language/). In H. Wallach, H. Larochelle, A. Beygelzimer, F. d'Alché-Buc, E. Fox, and R. Garnett, editors, Advances in Neural Information Processing Systems, volum…
48. Z. Yang, N. Garcia, C. Chu, M. Otani, Y. Nakashima, and H. Takemura. Bert representations for video question answering. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pages 1556–1565, 2020. [find](https://scholar.google.com/scholar?q=Z%2E+Yang%2C+N%2E+Garcia%2C+C%2E+Chu%2C+M%2E+Otani%2C+Y%2E+Nakashima%2C+and+H%2E+Takemura%2E+Bert+representations+for+video+question+answering%2E+In+Proceedings+of+the+IEEE%2FCVF+Winter+Conference+on+Applications+of+Computer+Vis)
49. J. Zhu, Y. Xia, L. Wu, D. He, T. Qin, W. Zhou, H. Li, and T.-Y. Liu. Incorporating bert into neural machine translation. arXiv preprint arXiv:2002.06823, 2020. [find](https://scholar.google.com/scholar?q=J%2E+Zhu%2C+Y%2E+Xia%2C+L%2E+Wu%2C+D%2E+He%2C+T%2E+Qin%2C+W%2E+Zhou%2C+H%2E+Li%2C+and+T%2E%2DY%2E+Liu%2E+Incorporating+bert+into+neural+machine+translation%2E+arXiv+preprint+arXiv%3A2002%2E06823%2C+2020%2E)

*Updated 2 November 2026*