--- title: "MLGO — paper by Trofin et al. (2021) — summary and ideas" description: "The authors present MLGO, a set of design principles and an implementation for bringing learned… 7 references, cited by 10 papers here." authors: ["Mircea Trofin", "Yundi Qian", "Eugene Brevdo", "Zinan Lin", "Krzysztof Choromanski", "David Li"] year: 2021 venue: "arXiv:2101.04808" source: "https://arxiv.org/abs/2101.04808" doi: "10.48550/arxiv.2101.04808" arxiv: "2101.04808" licence: "CC BY 4.0 (arXiv)" licence_url: "http://creativecommons.org/licenses/by/4.0/" summary_licence: "Elamur Cosmic License" canonical: "https://research.elamur.ai/free/en/papers/trofin-2021-mlgo-machine-learning-guided-compiler/" --- [Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/trofin-2021-mlgo-machine-learning-guided-compiler/) [home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2021](https://md.research.elamur.ai/free/en/papers/2021/page/6/) / MLGO: a Machine Learning Guided Compiler Optimizations Framework # MLGO: a Machine Learning Guided Compiler Optimizations Framework **Authors:** Mircea Trofin, Yundi Qian, [Eugene Brevdo](https://md.research.elamur.ai/free/en/authors/eugene-brevdo/), [Zinan Lin](https://md.research.elamur.ai/free/en/authors/zinan-lin/), [Krzysztof Choromanski](https://md.research.elamur.ai/free/en/authors/krzysztof-choromanski/) and David Li · **Year:** 2021 · **Venue:** arXiv:2101.04808 · **Licence:** [CC BY 4.0 (arXiv)](http://creativecommons.org/licenses/by/4.0/) **Links:** [Publisher's page](https://arxiv.org/abs/2101.04808) · [PDF](https://arxiv.org/pdf/2101.04808) · [DOI 10.48550/arxiv.2101.04808](https://doi.org/10.48550/arxiv.2101.04808) · [arXiv 2101.04808](https://arxiv.org/abs/2101.04808) · [OpenAlex](https://openalex.org/W3120378074) **Citations:** 32 citations (OpenAlex, 2026-10-14) · [10](https://md.research.elamur.ai/free/en/papers/trofin-2021-mlgo-machine-learning-guided-compiler/cited-by/) in our library, of the 149,439 papers whose bibliographies we read ## Summary Academic research has repeatedly shown that machine learning can outperform hand-written compiler heuristics, yet production-grade compilers had not adopted it. The authors present MLGO, a set of design principles and an implementation for bringing learned policies into LLVM without disrupting how developers build and ship software. Training runs offline on a large corpus of IR modules. The instrumented compiler logs features and decisions, and these logs feed the learning algorithm. The finished model is compiled into the compiler as native code, so builds stay deterministic and need no new dependencies. As a pilot, they replaced the inlining-for-size decision heuristic with a neural policy trained by Policy Gradient and by Evolution Strategies. Compared with LLVM -Oz, the learned policy shrinks binaries by up to 7%. A model trained on one corpus also transfers to unrelated software and still helps after four months of changes to both code and compiler. This matters because it suggests ML-driven optimization can be deployed in industry without frequent retraining. ## Key points - MLGO is described as the first complete integration of ML into a complex pass of a real-world compiler. It ships in the main LLVM repository as a build-time opt-in. - Two training approaches were compared: Policy Gradient (reinforcement learning) and Evolution Strategies (black-box optimization). - The learned inlining-for-size policy achieves up to 7% size reduction over LLVM -Oz, with results reported on several real-world targets and on SPEC 2006. - Generalization was tested in two ways. Policies were applied to software outside the training corpus, and they were re-evaluated four months later on newer code and a newer LLVM (Clang selfhost 4ca60915bcc). In the later test the gains shrank somewhat but stayed clearly positive. - The authors plan to extend MLGO to inlining for speed and to register allocation. For these tasks they intend to use profile-based reward approximations instead of noisy benchmark runs. The code is at github.com/google/ml-compiler-opt. *Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.* ## Institutions - [Google](https://md.research.elamur.ai/free/en/companies/google/) — Company, Mountain View, United States · 533 works · 443,884 citations - [Carnegie Mellon University](https://md.research.elamur.ai/free/en/universities/carnegie-mellon-university/) — University, Pittsburgh, United States · 633 works · 195,217 citations 📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 19,503 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 18,805 works · [🗄️ Information Systems](https://md.research.elamur.ai/free/en/shelves/computer-science/information-systems/) · 1,888 works · [🔌 Hardware and Architecture](https://md.research.elamur.ai/free/en/shelves/computer-science/hardware-and-architecture/) · 183 works ## Cite Cite as: Mircea Trofin, Yundi Qian, Eugene Brevdo, Zinan Lin, Krzysztof Choromanski and David Li. 2021. MLGO: a Machine Learning Guided Compiler Optimizations Framework. arXiv:2101.04808. The paper itself, not this page. BibTeX from the library's record of the work. ```bibtex @misc{trofin2021mlgo, title = {{MLGO: a Machine Learning Guided Compiler Optimizations Framework}}, author = {Mircea Trofin, Yundi Qian, Eugene Brevdo, Zinan Lin, Krzysztof Choromanski and David Li}, year = {2021}, doi = {10.48550/arxiv.2101.04808}, eprint = {2101.04808}, archivePrefix = {arXiv}, url = {https://doi.org/10.48550/arxiv.2101.04808} } ``` - [← previous Facebook AI's WMT21 News Translation Task Submission](https://md.research.elamur.ai/free/en/papers/tran-2021-facebook-ai-s-wmt21-news/) - [next → Pragmatic competence of pre-trained language models through the lens of discourse connectives](https://md.research.elamur.ai/free/en/papers/pandia-2021-pragmatic-competence-pre-trained-language/) - [↑ back to papers of 2021](https://md.research.elamur.ai/free/en/papers/2021/page/6/) ## References 7 references, 2 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet. 1. Martín Abadi, Paul Barham, Jianmin Chen, Zhifeng Chen, Andy Davis, Jeffrey Dean, Matthieu Devin, Sanjay Ghemawat, Geoffrey Irving, Michael Isard, et al. 2016. [Tensorflow: A system for large-scale machine learning](https://md.research.elamur.ai/free/en/papers/abadi-2016-tensorflow-system-large-scale-machine/). In 12th {USENIX} symposium on operating systems design and implementation ({OSDI} 16).… 2. Andrew Adams, Karima Ma, Luke Anderson, Riyadh Baghdadi, Tzu-mao Li, Michael Charbi, Benoit Steiner, Steven Johnson, Kayvon Fa-tahlian, Fredo Durand, and Jonathan Regan-kelley. 2019. Learning to optimize Halide with tree search and random programs (ACM Transactions on Graphics). 2–12. [find](https://scholar.google.com/scholar?q=Andrew+Adams%2C+Karima+Ma%2C+Luke+Anderson%2C+Riyadh+Baghdadi%2C+Tzu%2Dmao+Li%2C+Michael+Charbi%2C+Benoit+Steiner%2C+Steven+Johnson%2C+Kayvon+Fa%2Dtahlian%2C+Fredo+Durand%2C+and+Jonathan+Regan%2Dkelley%2E+2019%2E+Learning+to+optim) 3. Uri Alon, Meital Zilberstein, Omer Levy, and Eran Yahav. 2019. code2vec: Learning distributed representations of code. Proceedings of the ACM on Programming Languages 3, POPL (2019), 1–29. [find](https://scholar.google.com/scholar?q=Uri+Alon%2C+Meital+Zilberstein%2C+Omer+Levy%2C+and+Eran+Yahav%2E+2019%2E+code2vec%3A+Learning+distributed+representations+of+code%2E+Proceedings+of+the+ACM+on+Programming+Languages+3%2C+POPL+%282019%29%2C+1%E2%80%9329%2E) 4. Michael Bain and Claude Sammut. 1995. A Framework for Behavioural Cloning.. In Machine Intelligence 15. 103–129. [find](https://scholar.google.com/scholar?q=Michael+Bain+and+Claude+Sammut%2E+1995%2E+A+Framework+for+Behavioural+Cloning%2E%2E+In+Machine+Intelligence+15%2E+103%E2%80%93129%2E) 5. Rudy Bunel, Alban Desmaison, M Pawan Kumar, Philip HS Torr, and Pushmeet Kohli. 2016. Learning to superoptimize programs. arXiv preprint arXiv:1611.01787 (2016). [find](https://scholar.google.com/scholar?q=Rudy+Bunel%2C+Alban+Desmaison%2C+M+Pawan+Kumar%2C+Philip+HS+Torr%2C+and+Pushmeet+Kohli%2E+2016%2E+Learning+to+superoptimize+programs%2E+arXiv+preprint+arXiv%3A1611%2E01787+%282016%29%2E) 6. John Cavazos and Michael FP O’Boyle. 2005. Automatic tuning of inlining heuristics. In SC’05: Proceedings of the 2005 ACM/IEEE Conference on Supercomputing. IEEE, 14–14. [find](https://scholar.google.com/scholar?q=John+Cavazos+and+Michael+FP+O%E2%80%99Boyle%2E+2005%2E+Automatic+tuning+of+inlining+heuristics%2E+In+SC%E2%80%9905%3A+Proceedings+of+the+2005+ACM%2FIEEE+Conference+on+Supercomputing%2E+IEEE%2C+14%E2%80%9314%2E) 7. Tianqi Chen, Lianmin Zheng, Eddie Q. Yan, Ziheng Jiang, Thierry Moreau, Luis Ceze, Carlos Guestrin, and Arvind Krishnamurthy. 2018. [Learning to Optimize Tensor Programs](https://md.research.elamur.ai/free/en/papers/chen-2019-learning-optimize-tensor-programs/). CoRR abs/1805.08166 (2018). arXiv:1805.08166 http://arxiv.org/abs/1805.08166 ML Techniques [↗](http://arxiv.org/abs/1805.08166) *Updated 3 November 2026*