---
title: "Minigrid & Miniworld — paper by Chevalier-Boisvert et al. (2023)"
description: "The authors present Minigrid, a collection of 2D GridWorld tasks, and Miniworld, a set of 3D tasks… 29 references, cited by 21 papers here."
authors: ["Maxime Chevalier-Boisvert", "Bolun Dai", "Mark Towers", "Rodrigo de Lazcano", "Lucas Willems", "Salem Lahlou", "Suman Pal", "Pablo Samuel Castro", "Jordan Terry"]
year: 2023
venue: "arXiv:2306.13831"
source: "https://arxiv.org/abs/2306.13831"
doi: "10.48550/arxiv.2306.13831"
arxiv: "2306.13831"
licence: "CC BY 4.0 (arXiv)"
licence_url: "http://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/chevalier-boisvert-2023-minigrid-miniworld-modular-customizable-reinforcement/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/chevalier-boisvert-2023-minigrid-miniworld-modular-customizable-reinforcement/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2023](https://md.research.elamur.ai/free/en/papers/2023/page/13/) / Minigrid & Miniworld: Modular & Customizable Reinforcement Learning Environments for Goal-Oriented Tasks

# Minigrid & Miniworld: Modular & Customizable Reinforcement Learning Environments for Goal-Oriented Tasks

**Authors:** [Maxime Chevalier-Boisvert](https://md.research.elamur.ai/free/en/authors/maxime-chevalier-boisvert/), Bolun Dai, [Mark Towers](https://md.research.elamur.ai/free/en/authors/mark-towers/), Rodrigo de Lazcano, [Lucas Willems](https://md.research.elamur.ai/free/en/authors/lucas-willems/), [Salem Lahlou](https://md.research.elamur.ai/free/en/authors/salem-lahlou/), Suman Pal, [Pablo Samuel Castro](https://md.research.elamur.ai/free/en/authors/pablo-samuel-castro/) and [Jordan Terry](https://md.research.elamur.ai/free/en/authors/jordan-terry/) · **Year:** 2023 · **Venue:** arXiv:2306.13831 · **Licence:** [CC BY 4.0 (arXiv)](http://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://arxiv.org/abs/2306.13831) · [PDF](https://arxiv.org/pdf/2306.13831) · [DOI 10.48550/arxiv.2306.13831](https://doi.org/10.48550/arxiv.2306.13831) · [arXiv 2306.13831](https://arxiv.org/abs/2306.13831) · [OpenAlex](https://openalex.org/W4382319558)

**Citations:** 4 citations (OpenAlex, 2026-10-14) · [21](https://md.research.elamur.ai/free/en/papers/chevalier-boisvert-2023-minigrid-miniworld-modular-customizable-reinforcement/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
Popular RL benchmark suites such as Gym and dm_control mostly test motor control or visual perception. They are poorly suited to studying goal-directed reasoning or instruction following, and adding new objectives to them is hard. The authors present Minigrid, a collection of 2D GridWorld tasks, and Miniworld, a set of 3D tasks made of connected rooms, and describe their design for the first time in a paper. Both libraries run on the Gymnasium interface and keep dependencies to a minimum: NumPy for Minigrid, Pyglet for Miniworld's simplified 2.5D renderer. This keeps the code small enough to read and modify in full. New tasks are defined through a short world-generation function. The paper also surveys how the community has used the libraries, for example in curriculum learning, exploration, and meta- or transfer learning. Two case studies use the shared API: a PPO agent trained in Minigrid is moved into a matching Miniworld task, and humans are tested on transfer between the two observation types. The main takeaway is that heavily customized experiments need very little code, which makes these environments practical testbeds for decision-making research.

## Key points
- Minigrid observations combine a partial top-down image, the agent's heading, and a text "mission". Agents choose from 7 discrete actions. Miniworld instead gives an 80×60 RGB first-person view and adds a move-back action, for 8 actions in total. Rewards are sparse by default in both.
- Installation is via `pip install minigrid` / `pip install miniworld`. Both integrate directly with Stable-Baselines3 and ship extra wrappers, such as stochastic actions and alternative observation spaces.
- Together, the two GitHub repositories have about 2400 stars and 620 forks.
- The agent-transfer case study, with custom environments, wrapper, feature extractor, and SB3 PPO training, took only 101–138 lines of code. The human study used MiniGrid-FourRooms-v0 and MiniWorld-FourRooms-v0 and required roughly three hours of coding.
- The authors name two limitations: the minimal creation API restricts which environments can be built, and the pure-Python implementation runs slower than C++ game engines.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [New York University](https://md.research.elamur.ai/free/en/universities/new-york-university/) — University, New York, United States · 233 works · 104,003 citations
- [Mila - Quebec Artificial Intelligence Institute](https://md.research.elamur.ai/free/en/labs/mila-quebec-artificial-intelligence-institute/) — Research institute, Montreal, Canada · 73 works · 14,704 citations
- [University of Southampton](https://md.research.elamur.ai/free/en/universities/university-of-southampton/) — University, Southampton, United Kingdom · 14 works · 2,633 citations

📚 Shelves (2): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 19,503 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 18,805 works

## Cite

Cite as: Maxime Chevalier-Boisvert, Bolun Dai, Mark Towers, Rodrigo de Lazcano, Lucas Willems, Salem Lahlou, Suman Pal, Pablo Samuel Castro and Jordan Terry. 2023. Minigrid & Miniworld: Modular & Customizable Reinforcement Learning Environments for Goal-Oriented Tasks. arXiv:2306.13831.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@misc{chevalierboisvert2023minigrid,
  title = {{Minigrid \& Miniworld: Modular \& Customizable Reinforcement Learning Environments for Goal-Oriented Tasks}},
  author = {Maxime Chevalier-Boisvert, Bolun Dai, Mark Towers, Rodrigo {de Lazcano}, Lucas Willems, Salem Lahlou, Suman Pal, Pablo Samuel Castro and Jordan Terry},
  year = {2023},
  doi = {10.48550/arxiv.2306.13831},
  eprint = {2306.13831},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2306.13831}
}
```

- [← previous Getting from Generative AI to Trustworthy AI: What LLMs might learn from Cyc](https://md.research.elamur.ai/free/en/papers/lenat-2023-getting-generative-ai-trustworthy/)
- [next → Moderating New Waves of Online Hate with Chain-of-Thought Reasoning in Large Language Models](https://md.research.elamur.ai/free/en/papers/vishwamitra-2023-moderating-new-waves-online-hate/)
- [↑ back to papers of 2023](https://md.research.elamur.ai/free/en/papers/2023/page/13/)

## References

29 references, 5 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. C. Bamford. Griddly: A platform for AI research in games. Software Impacts, 8:100066, 2021. [find](https://scholar.google.com/scholar?q=C%2E+Bamford%2E+Griddly%3A+A+platform+for+AI+research+in+games%2E+Software+Impacts%2C+8%3A100066%2C+2021%2E)
2. C. Beattie, J. Z. Leibo, D. Teplyashin, T. Ward, M. Wainwright, H. Küttler, A. Lefrancq, S. Green, V. Valdés, A. Sadik, J. Schrittwieser, K. Anderson, S. York, M. Cant, A. Cain, A. Bolton, S. Gaffney, H. King, D. Hassabis, S. Legg, and S. Petersen. [DeepMind Lab](https://md.research.elamur.ai/free/en/papers/beattie-2016-deepmind-lab/). CoRR, abs/1612.03801, 2016.
3. G. Brockman, V. Cheung, L. Pettersson, J. Schneider, J. Schulman, J. Tang, and W. Zaremba. OpenAI Gym. CoRR, abs/1606.01540, 2016. [find](https://scholar.google.com/scholar?q=G%2E+Brockman%2C+V%2E+Cheung%2C+L%2E+Pettersson%2C+J%2E+Schneider%2C+J%2E+Schulman%2C+J%2E+Tang%2C+and+W%2E+Zaremba%2E+OpenAI+Gym%2E+CoRR%2C+abs%2F1606%2E01540%2C+2016%2E)
4. M. Chevalier-Boisvert, D. Bahdanau, S. Lahlou, L. Willems, C. Saharia, T. H. Nguyen, and Y. Bengio. [BabyAI: A platform to study the sample efficiency of grounded language learning](https://md.research.elamur.ai/free/en/papers/chevalier-boisvert-2018-babyai-platform-study-sample-efficiency/). In Proceedings of International Conference on Learning Representations, New Orleans, LA, May 2019.
5. M. Dennis, N. Jaques, E. Vinitsky, A. M. Bayen, S. Russell, A. Critch, and S. Levine. Emergent complexity and zero-shot transfer via unsupervised environment design. In Proceedings of Advances in Neural Information Processing Systems 33, Virtual, December 2020. [find](https://scholar.google.com/scholar?q=M%2E+Dennis%2C+N%2E+Jaques%2C+E%2E+Vinitsky%2C+A%2E+M%2E+Bayen%2C+S%2E+Russell%2C+A%2E+Critch%2C+and+S%2E+Levine%2E+Emergent+complexity+and+zero%2Dshot+transfer+via+unsupervised+environment+design%2E+In+Proceedings+of+Advances+in+Neur)
6. A. Gupta, V. Kumar, C. Lynch, S. Levine, and K. Hausman. Relay Policy Learning: Solving long-horizon tasks via imitation and reinforcement learning. In Proceedings of the Conference on Robot Learning, Virtual, pages 1025–1037, October 2020. [find](https://scholar.google.com/scholar?q=A%2E+Gupta%2C+V%2E+Kumar%2C+C%2E+Lynch%2C+S%2E+Levine%2C+and+K%2E+Hausman%2E+Relay+Policy+Learning%3A+Solving+long%2Dhorizon+tasks+via+imitation+and+reinforcement+learning%2E+In+Proceedings+of+the+Conference+on+Robot+Learning%2C)
7. R. L. Gutierrez and M. Leonetti. Information-theoretic task selection for meta-reinforcement learning. In Proceedings of Advances in Neural Information Processing Systems 33, Virtual, December 2020. [find](https://scholar.google.com/scholar?q=R%2E+L%2E+Gutierrez+and+M%2E+Leonetti%2E+Information%2Dtheoretic+task+selection+for+meta%2Dreinforcement+learning%2E+In+Proceedings+of+Advances+in+Neural+Information+Processing+Systems+33%2C+Virtual%2C+December+2020%2E)
8. M. Hutsebaut-Buysse, K. Mets, and S. Latré. Pre-trained word embeddings for goal-conditional transfer learning in reinforcement learning. CoRR, abs/2007.05196, 2020. — [arXiv:2007.05196](https://arxiv.org/abs/2007.05196) [find](https://scholar.google.com/scholar?q=M%2E+Hutsebaut%2DBuysse%2C+K%2E+Mets%2C+and+S%2E+Latr%C3%A9%2E+Pre%2Dtrained+word+embeddings+for+goal%2Dconditional+transfer+learning+in+reinforcement+learning%2E+CoRR%2C+abs%2F2007%2E05196%2C+2020%2E+%E2%80%94+%5BarXiv%3A2007%2E05196%5D%28https%3A%2F%2Farxiv)
9. M. Igl, K. Ciosek, Y. Li, S. Tschiatschek, C. Zhang, S. Devlin, and K. Hofmann. Generalization in reinforcement learning with selective noise injection and information bottleneck. In Proceedings of Advances in Neural Information Processing Systems 32, Vancouver, Canada, pages 13956–13968, December 2… [find](https://scholar.google.com/scholar?q=M%2E+Igl%2C+K%2E+Ciosek%2C+Y%2E+Li%2C+S%2E+Tschiatschek%2C+C%2E+Zhang%2C+S%2E+Devlin%2C+and+K%2E+Hofmann%2E+Generalization+in+reinforcement+learning+with+selective+noise+injection+and+information+bottleneck%2E+In+Proceedings+of+Ad)
10. L. P. Kaelbling, M. L. Littman, and A. R. Cassandra. Partially observable markov decision processes for artificial intelligence. In I. Wachsmuth, C.-R. Rollinger, and W. Brauer, editors, KI-95: Advances in Artificial Intelligence, pages 1–17, Berlin, Heidelberg, 1995. Springer Berlin Heidelberg. ISB… [find](https://scholar.google.com/scholar?q=L%2E+P%2E+Kaelbling%2C+M%2E+L%2E+Littman%2C+and+A%2E+R%2E+Cassandra%2E+Partially+observable+markov+decision+processes+for+artificial+intelligence%2E+In+I%2E+Wachsmuth%2C+C%2E%2DR%2E+Rollinger%2C+and+W%2E+Brauer%2C+editors%2C+KI%2D95%3A+Advanc)
11. M. Kempka, M. Wydmuch, G. Runc, J. Toczek, and W. Jaskowski. ViZDoom: A Doom-based AI research platform for visual reinforcement learning. In Proceedings of IEEE Conference on Computational Intelligence and Games, Santorini, Greece, pages 1–8. IEEE, September 2016. [find](https://scholar.google.com/scholar?q=M%2E+Kempka%2C+M%2E+Wydmuch%2C+G%2E+Runc%2C+J%2E+Toczek%2C+and+W%2E+Jaskowski%2E+ViZDoom%3A+A+Doom%2Dbased+AI+research+platform+for+visual+reinforcement+learning%2E+In+Proceedings+of+IEEE+Conference+on+Computational+Intelligen)
12. E. Z. Liu, A. Raghunathan, P. Liang, and C. Finn. Decoupling exploration and exploitation for meta-reinforcement learning without sacrifices. In Proceedings of the 38th International Conference on Machine Learning, Virtual Event, volume 139, pages 6925–6935. PMLR, July 2021. [find](https://scholar.google.com/scholar?q=E%2E+Z%2E+Liu%2C+A%2E+Raghunathan%2C+P%2E+Liang%2C+and+C%2E+Finn%2E+Decoupling+exploration+and+exploitation+for+meta%2Dreinforcement+learning+without+sacrifices%2E+In+Proceedings+of+the+38th+International+Conference+on+Mac)
13. V. Makoviychuk, L. Wawrzyniak, Y. Guo, M. Lu, K. Storey, M. Macklin, D. Hoeller, N. Rudin, A. Allshire, A. Handa, and G. State. Isaac Gym: High performance GPU based physics simulation for robot learning. In Proceedings of the Neural Information Processing Systems Track on Datasets and Benchmarks, V… — [Isaac Gym: High Performance GPU-Based Physics Simulation For Robot Learning](https://md.research.elamur.ai/free/en/papers/makoviychuk-2021-isaac-gym-high-performance-gpu/)
14. A. N. Mavor-Parker, K. A. Young, C. Barry, and L. D. Griffin. How to stay curious while avoiding noisy tvs using aleatoric uncertainty estimation. In Proceedings of International Conference on Machine Learning, Baltimore, MD, volume 162, pages 15220–15240. PMLR, July 2022. [find](https://scholar.google.com/scholar?q=A%2E+N%2E+Mavor%2DParker%2C+K%2E+A%2E+Young%2C+C%2E+Barry%2C+and+L%2E+D%2E+Griffin%2E+How+to+stay+curious+while+avoiding+noisy+tvs+using+aleatoric+uncertainty+estimation%2E+In+Proceedings+of+International+Conference+on+Machine)
15. V. Mnih, K. Kavukcuoglu, D. Silver, A. A. Rusu, J. Veness, M. G. Bellemare, A. Graves, M. A. Riedmiller, A. Fidjeland, G. Ostrovski, S. Petersen, C. Beattie, A. Sadik, I. Antonoglou, H. King, D. Kumaran, D. Wierstra, S. Legg, and D. Hassabis. Human-level control through deep reinforcement learning.… [find](https://scholar.google.com/scholar?q=V%2E+Mnih%2C+K%2E+Kavukcuoglu%2C+D%2E+Silver%2C+A%2E+A%2E+Rusu%2C+J%2E+Veness%2C+M%2E+G%2E+Bellemare%2C+A%2E+Graves%2C+M%2E+A%2E+Riedmiller%2C+A%2E+Fidjeland%2C+G%2E+Ostrovski%2C+S%2E+Petersen%2C+C%2E+Beattie%2C+A%2E+Sadik%2C+I%2E+Antonoglou%2C+H%2E+King%2C+D%2E+Kumar)
16. J. Parker-Holder, M. Jiang, M. Dennis, M. Samvelyan, J. N. Foerster, E. Grefenstette, and T. Rocktäschel. Evolving curricula with regret-based environment design. In Proceedings of International Conference on Machine Learning, Baltimore, MD, volume 162, pages 17473– 17498. PMLR, July 2022. [find](https://scholar.google.com/scholar?q=J%2E+Parker%2DHolder%2C+M%2E+Jiang%2C+M%2E+Dennis%2C+M%2E+Samvelyan%2C+J%2E+N%2E+Foerster%2C+E%2E+Grefenstette%2C+and+T%2E+Rockt%C3%A4schel%2E+Evolving+curricula+with+regret%2Dbased+environment+design%2E+In+Proceedings+of+International+Confe)
17. A. Paszke, S. Gross, F. Massa, A. Lerer, J. Bradbury, G. Chanan, T. Killeen, Z. Lin, N. Gimelshein, L. Antiga, A. Desmaison, A. Köpf, E. Z. Yang, Z. DeVito, M. Raison, A. Tejani, S. Chilamkurthy, B. Steiner, L. Fang, J. Bai, and S. Chintala. Pytorch: An imperative style, high-performance deep learni… — [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://md.research.elamur.ai/free/en/papers/paszke-2019-pytorch-imperative-style-high-performance/)
18. I. Radosavovic, T. Xiao, S. James, P. Abbeel, J. Malik, and T. Darrell. Real-world robot learning with masked visual pre-training. In Proceedings of the Conference on Robot Learning, Auckland, New Zealand, pages 416–426, 2022. [find](https://scholar.google.com/scholar?q=I%2E+Radosavovic%2C+T%2E+Xiao%2C+S%2E+James%2C+P%2E+Abbeel%2C+J%2E+Malik%2C+and+T%2E+Darrell%2E+Real%2Dworld+robot+learning+with+masked+visual+pre%2Dtraining%2E+In+Proceedings+of+the+Conference+on+Robot+Learning%2C+Auckland%2C+New+Zea)
19. A. Raffin, A. Hill, A. Gleave, A. Kanervisto, M. Ernestus, and N. Dormann. Stable-baselines3: Reliable reinforcement learning implementations. Journal of Machine Learning Research, 22: 268:1–268:8, 2021. [find](https://scholar.google.com/scholar?q=A%2E+Raffin%2C+A%2E+Hill%2C+A%2E+Gleave%2C+A%2E+Kanervisto%2C+M%2E+Ernestus%2C+and+N%2E+Dormann%2E+Stable%2Dbaselines3%3A+Reliable+reinforcement+learning+implementations%2E+Journal+of+Machine+Learning+Research%2C+22%3A+268%3A1%E2%80%93268%3A8%2C+20)
20. J. Schulman, F. Wolski, P. Dhariwal, A. Radford, and O. Klimov. Proximal policy optimization algorithms. CoRR, abs/1707.06347, 2017. [find](https://scholar.google.com/scholar?q=J%2E+Schulman%2C+F%2E+Wolski%2C+P%2E+Dhariwal%2C+A%2E+Radford%2C+and+O%2E+Klimov%2E+Proximal+policy+optimization+algorithms%2E+CoRR%2C+abs%2F1707%2E06347%2C+2017%2E)
21. Y. Seo, L. Chen, J. Shin, H. Lee, P. Abbeel, and K. Lee. State entropy maximization with random encoders for efficient exploration. In Proceedings of the 38th International Conference on Machine Learning, Virtual Event, volume 139, pages 9443–9454. PMLR, July 2021. [find](https://scholar.google.com/scholar?q=Y%2E+Seo%2C+L%2E+Chen%2C+J%2E+Shin%2C+H%2E+Lee%2C+P%2E+Abbeel%2C+and+K%2E+Lee%2E+State+entropy+maximization+with+random+encoders+for+efficient+exploration%2E+In+Proceedings+of+the+38th+International+Conference+on+Machine+Learn)
22. D. Silver, A. Huang, C. J. Maddison, A. Guez, L. Sifre, G. van den Driessche, J. Schrittwieser, I. Antonoglou, V. Panneershelvam, M. Lanctot, S. Dieleman, D. Grewe, J. Nham, N. Kalchbrenner, I. Sutskever, T. P. Lillicrap, M. Leach, K. Kavukcuoglu, T. Graepel, and D. Hassabis. Mastering the game of g… [find](https://scholar.google.com/scholar?q=D%2E+Silver%2C+A%2E+Huang%2C+C%2E+J%2E+Maddison%2C+A%2E+Guez%2C+L%2E+Sifre%2C+G%2E+van+den+Driessche%2C+J%2E+Schrittwieser%2C+I%2E+Antonoglou%2C+V%2E+Panneershelvam%2C+M%2E+Lanctot%2C+S%2E+Dieleman%2C+D%2E+Grewe%2C+J%2E+Nham%2C+N%2E+Kalchbrenner%2C+I%2E+Sutske)
23. D. Silver, T. Hubert, J. Schrittwieser, I. Antonoglou, M. Lai, A. Guez, M. Lanctot, L. Sifre, D. Kumaran, T. Graepel, T. P. Lillicrap, K. Simonyan, and D. Hassabis. [Mastering chess and shogi by self-play with a general reinforcement learning algorithm](https://md.research.elamur.ai/free/en/papers/al-2017-mastering-chess-shogi-self-play/). CoRR, abs/1712.01815, 2017.
24. S. Sukhbaatar, A. Szlam, G. Synnaeve, S. Chintala, and R. Fergus. Mazebase: A sandbox for learning from games. CoRR, abs/1511.07401, 2015. [find](https://scholar.google.com/scholar?q=S%2E+Sukhbaatar%2C+A%2E+Szlam%2C+G%2E+Synnaeve%2C+S%2E+Chintala%2C+and+R%2E+Fergus%2E+Mazebase%3A+A+sandbox+for+learning+from+games%2E+CoRR%2C+abs%2F1511%2E07401%2C+2015%2E)
25. R. S. Sutton and A. G. Barto. Reinforcement Learning: An Introduction. MIT Press, Cambridge, MA, 1998. [find](https://scholar.google.com/scholar?q=R%2E+S%2E+Sutton+and+A%2E+G%2E+Barto%2E+Reinforcement+Learning%3A+An+Introduction%2E+MIT+Press%2C+Cambridge%2C+MA%2C+1998%2E)
26. E. Todorov, T. Erez, and Y. Tassa. MuJoCo: A physics engine for model-based control. In Proceedings of IEEE/RSJ International Conference on Intelligent Robots and Systems, Vilamoura, Portugal, pages 5026–5033, October 2012. [find](https://scholar.google.com/scholar?q=E%2E+Todorov%2C+T%2E+Erez%2C+and+Y%2E+Tassa%2E+MuJoCo%3A+A+physics+engine+for+model%2Dbased+control%2E+In+Proceedings+of+IEEE%2FRSJ+International+Conference+on+Intelligent+Robots+and+Systems%2C+Vilamoura%2C+Portugal%2C+pages+5)
27. S. Tunyasuvunakool, A. Muldal, Y. Doron, S. Liu, S. Bohez, J. Merel, T. Erez, T. Lillicrap, N. Heess, and Y. Tassa. dm_control: Software and tasks for continuous control. Software Impacts, 6:100022, 2020. [find](https://scholar.google.com/scholar?q=S%2E+Tunyasuvunakool%2C+A%2E+Muldal%2C+Y%2E+Doron%2C+S%2E+Liu%2C+S%2E+Bohez%2C+J%2E+Merel%2C+T%2E+Erez%2C+T%2E+Lillicrap%2C+N%2E+Heess%2C+and+Y%2E+Tassa%2E+dm%5Fcontrol%3A+Software+and+tasks+for+continuous+control%2E+Software+Impacts%2C+6%3A100022%2C+2)
28. A. Wachi, Y. Wei, and Y. Sui. Safe policy optimization with local generalized linear function approximations. In Proceedings of Advances in Neural Information Processing Systems 34, Virtual, pages 20759–20771, December 2021. [find](https://scholar.google.com/scholar?q=A%2E+Wachi%2C+Y%2E+Wei%2C+and+Y%2E+Sui%2E+Safe+policy+optimization+with+local+generalized+linear+function+approximations%2E+In+Proceedings+of+Advances+in+Neural+Information+Processing+Systems+34%2C+Virtual%2C+pages+207)
29. T. Zhang, H. Xu, X. Wang, Y. Wu, K. Keutzer, J. E. Gonzalez, and Y. Tian. Noveld: A simple yet effective exploration criterion. In Proceedings of Advances in Neural Information Processing Systems 34, Virtual, pages 25217–25230, December 2021. [find](https://scholar.google.com/scholar?q=T%2E+Zhang%2C+H%2E+Xu%2C+X%2E+Wang%2C+Y%2E+Wu%2C+K%2E+Keutzer%2C+J%2E+E%2E+Gonzalez%2C+and+Y%2E+Tian%2E+Noveld%3A+A+simple+yet+effective+exploration+criterion%2E+In+Proceedings+of+Advances+in+Neural+Information+Processing+Systems+34%2C+)

*Updated 3 November 2026*