---
title: "Learning to Optimize Tensor Programs — Chen 2019"
description: "Deep learning systems depend on fast tensor operators such as matrix multiplication and convolution. 43 references, cited by 92 papers here."
authors: ["Tianqi Chen", "Lianmin Zheng", "Eddie Yan", "Ziheng Jiang", "Thierry Moreau", "Luis Ceze", "Carlos Guestrin", "Arvind Krishnamurthy"]
year: 2019
venue: "arXiv:1805.08166"
source: "https://arxiv.org/abs/1805.08166"
doi: "10.48550/arxiv.1805.08166"
arxiv: "1805.08166"
licence: "arXiv — non-exclusive licence to distribute"
licence_url: "https://arxiv.org/licenses/nonexclusive-distrib/1.0/license.html"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/chen-2019-learning-optimize-tensor-programs/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/chen-2019-learning-optimize-tensor-programs/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2019](https://md.research.elamur.ai/free/en/papers/2019/page/3/) / Learning to Optimize Tensor Programs

# Learning to Optimize Tensor Programs

**Authors:** [Tianqi Chen](https://md.research.elamur.ai/free/en/authors/tianqi-chen/), [Lianmin Zheng](https://md.research.elamur.ai/free/en/authors/lianmin-zheng/), [Eddie Yan](https://md.research.elamur.ai/free/en/authors/eddie-yan/), [Ziheng Jiang](https://md.research.elamur.ai/free/en/authors/ziheng-jiang/), [Thierry Moreau](https://md.research.elamur.ai/free/en/authors/thierry-moreau/), [Luis Ceze](https://md.research.elamur.ai/free/en/authors/luis-ceze/), [Carlos Guestrin](https://md.research.elamur.ai/free/en/authors/carlos-guestrin/) and [Arvind Krishnamurthy](https://md.research.elamur.ai/free/en/authors/arvind-krishnamurthy/) · **Year:** 2019 · **Venue:** arXiv:1805.08166 · **Licence:** [arXiv — non-exclusive licence to distribute](https://arxiv.org/licenses/nonexclusive-distrib/1.0/license.html)

**Links:** [Publisher's page](https://arxiv.org/abs/1805.08166) · [PDF](https://arxiv.org/pdf/1805.08166) · [DOI 10.48550/arxiv.1805.08166](https://doi.org/10.48550/arxiv.1805.08166) · [arXiv 1805.08166](https://arxiv.org/abs/1805.08166) · [OpenAlex](https://openalex.org/W2804500013)

**Citations:** 177 citations (OpenAlex, 2026-10-14) · [92](https://md.research.elamur.ai/free/en/papers/chen-2019-learning-optimize-tensor-programs/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
Deep learning systems depend on fast tensor operators such as matrix multiplication and convolution. Today these come mostly from hand-tuned vendor libraries like cuDNN, which cover only a narrow set of hardware and make every new device expensive to support. The authors treat operator implementation as a search problem. An operator is written as an index expression, and the goal is to find the schedule (loop order, tiling, threading, etc.) whose compiled code runs fastest on real hardware.

Their framework, AutoTVM, trains a statistical cost model that predicts run time from the generated low-level program. This model steers exploration of a space containing billions of program variants. Because a full system needs many similar operators, the model is transferred across workloads, which speeds up tuning by 2–10×.

Without relying on any external operator library, AutoTVM performs as well as or better than cuDNN, TFLite and the ARM Compute Library across three platforms: a GPU for servers, a CPU for embedded devices, and a GPU for mobile devices. End-to-end inference gets 1.2–3.8× faster. This matters because it cuts the engineering cost of moving DL models to new hardware.

## Key points
- The task is framed as minimizing measured run time over a schedule space, with schedules built from Halide-style transformation primitives. It resembles hyper-parameter optimization but differs in three ways:
  - each experiment is cheap, taking seconds;
  - programs have rich internal structure the model can use;
  - there are many similar operators to tune.
- Hardware and baselines:
  - NVIDIA TITAN X vs. cuDNN v7 and TensorComprehensions;
  - ARM Cortex-A53 vs. TFLite;
  - ARM Mali-T860 vs. ARM Compute Library v18.03.
- End-to-end workloads were ResNet-18, MobileNet, LSTM Language Model, DQN and DCGAN, compared against MXNet v1.1 and Tensorflow v1.7 (including XLA) on GPU. Speedups ranged from 1.2× to 3.8×, partly thanks to operator fusion that fixed libraries cannot offer.
- Three program representations were compared:
  - configuration-space features worked only within a single domain;
  - flattened AST features transferred across convolution workloads;
  - a context relation representation also transferred across operator types.
- A preliminary experiment showed transfer from a Mali GPU to a Cortex-A53 CPU. The code is published at tvm.ai.

## Interesting ideas
- **Learned cost model as a cheap stand-in for real measurement.** A learned predictor ranks candidates, and only the promising ones are actually run on hardware. This pattern fits any setting where trials are cheap but the search space is enormous, such as tuning data pipelines or tokenizer settings.
- **Match the feature representation to how far you want to transfer.** Simple configuration features suffice within one task, while more invariant structural features are needed to cross task types. This is a useful principle when deciding how abstract features should be for cross-domain transfer in general.
- **Reuse models across many related jobs.** Instead of tuning each operator shape from scratch, a model trained on earlier workloads warm-starts new ones. The same idea could speed up repeated tuning across similar datasets or languages.
- **Exploit problem structure rather than treating it as a black box.** Feeding the model the actual program structure, instead of opaque parameters, made the search far more effective. Domain-aware surrogates may likewise beat generic Bayesian optimization in other areas.
- **Evaluate by wall-clock time, not by trial count.** Plotting speedup against real tuning seconds, against random search and other baselines, fairly accounts for the overhead of training the model itself. This is a good evaluation habit for any optimization method that adds its own compute cost.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

📚 Shelves (3): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 18,614 works · [👁️ Computer Vision and Pattern Recognition](https://md.research.elamur.ai/free/en/shelves/computer-science/computer-vision-and-pattern-recognition/) · 3,486 works · [🔌 Hardware and Architecture](https://md.research.elamur.ai/free/en/shelves/computer-science/hardware-and-architecture/) · 167 works

## Cite

Cite as: Tianqi Chen, Lianmin Zheng, Eddie Yan, Ziheng Jiang, Thierry Moreau, Luis Ceze, Carlos Guestrin and Arvind Krishnamurthy. 2019. Learning to Optimize Tensor Programs. arXiv:1805.08166.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@misc{chen2019learning,
  title = {{Learning to Optimize Tensor Programs}},
  author = {Tianqi Chen, Lianmin Zheng, Eddie Yan, Ziheng Jiang, Thierry Moreau, Luis Ceze, Carlos Guestrin and Arvind Krishnamurthy},
  year = {2019},
  doi = {10.48550/arxiv.1805.08166},
  eprint = {1805.08166},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.1805.08166}
}
```

- [← previous Representing Schema Structure with Graph Neural Networks for Text-to-SQL Parsing](https://md.research.elamur.ai/free/en/papers/bogin-2019-representing-schema-structure-graph-neural/)
- [next → Handling Divergent Reference Texts when Evaluating Table-to-Text Generation](https://md.research.elamur.ai/free/en/papers/dhingra-2019-handling-divergent-reference-texts-when/)
- [↑ back to papers of 2019](https://md.research.elamur.ai/free/en/papers/2019/page/3/)

## References

43 references, 13 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Martin Abadi, Paul Barham, Jianmin Chen, Zhifeng Chen, Andy Davis, Jeffrey Dean, Matthieu Devin, Sanjay Ghemawat, Geoffrey Irving, Michael Isard, Manjunath Kudlur, Josh Levenberg, Rajat Monga, Sherry Moore, Derek G. Murray, Benoit Steiner, Paul Tucker, Vijay Vasudevan, Pete Warden, Martin 1 Accordin… — [TensorFlow: A system for large-scale machine learning](https://md.research.elamur.ai/free/en/papers/abadi-2016-tensorflow-system-large-scale-machine/)
2. Amit Agarwal, Eldar Akchurin, Chris Basoglu, Guoguo Chen, Scott Cyphers, Jasha Droppo, Adam Eversole, Brian Guenter, Mark Hillebrand, Ryan Hoens, Xuedong Huang, Zhiheng Huang, Vladimir Ivanov, Alexey Kamenev, Philipp Kranen, Oleksii Kuchaiev, Wolfgang Manousek, Avner May, Bhaskar Mitra, Olivier Nano… [find](https://scholar.google.com/scholar?q=Amit+Agarwal%2C+Eldar+Akchurin%2C+Chris+Basoglu%2C+Guoguo+Chen%2C+Scott+Cyphers%2C+Jasha+Droppo%2C+Adam+Eversole%2C+Brian+Guenter%2C+Mark+Hillebrand%2C+Ryan+Hoens%2C+Xuedong+Huang%2C+Zhiheng+Huang%2C+Vladimir+Ivanov%2C+Alexey+)
3. Miltiadis Allamanis, Marc Brockschmidt, and Mahmoud Khademi. [Learning to represent programs with graphs](https://md.research.elamur.ai/free/en/papers/allamanis-2017-learning-represent-programs-graphs/). In International Conference on Learning Representations, 2018.
4. Frédéric Bastien, Pascal Lamblin, Razvan Pascanu, James Bergstra, Ian J. Goodfellow, Arnaud Bergeron, Nicolas Bouchard, and Yoshua Bengio. [Theano: new features and speed improvements](https://md.research.elamur.ai/free/en/papers/bastien-2012-theano-new-features-speed-improvements/). Deep Learning and Unsupervised Feature Learning NIPS 2012 Workshop, 2012.
5. Uday Bondhugula, Albert Hartono, J. Ramanujam, and P. Sadayappan. A practical automatic polyhedral parallelizer and locality optimizer. In Proceedings of the 29th ACM SIGPLAN Conference on Programming Language Design and Implementation, PLDI ’08, pages 101–113. ACM, 2008. [find](https://scholar.google.com/scholar?q=Uday+Bondhugula%2C+Albert+Hartono%2C+J%2E+Ramanujam%2C+and+P%2E+Sadayappan%2E+A+practical+automatic+polyhedral+parallelizer+and+locality+optimizer%2E+In+Proceedings+of+the+29th+ACM+SIGPLAN+Conference+on+Programming)
6. Chris Burges, Tal Shaked, Erin Renshaw, Ari Lazier, Matt Deeds, Nicole Hamilton, and Greg Hullender. [Learning to rank using gradient descent](https://md.research.elamur.ai/free/en/papers/burges-2005-learning-rank-using-gradient/). In Proceedings of the 22Nd International Conference on Machine Learning, ICML ’05, pages 89–96, New York, NY, USA, 2005. ACM.
7. Tianqi Chen and Carlos Guestrin. XGBoost: A scalable tree boosting system. In Proceedings of the 22Nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD ’16, pages 785–794, New York, NY, USA, 2016. ACM. [find](https://scholar.google.com/scholar?q=Tianqi+Chen+and+Carlos+Guestrin%2E+XGBoost%3A+A+scalable+tree+boosting+system%2E+In+Proceedings+of+the+22Nd+ACM+SIGKDD+International+Conference+on+Knowledge+Discovery+and+Data+Mining%2C+KDD+%E2%80%9916%2C+pages+785%E2%80%93794)
8. Tianqi Chen, Mu Li, Yutian Li, Min Lin, Naiyan Wang, Minjie Wang, Tianjun Xiao, Bing Xu, Chiyuan Zhang, , and Zheng Zhang. [MXNet: A flexible and efficient machine learning library for heterogeneous distributed systems](https://md.research.elamur.ai/free/en/papers/chen-2015-mxnet-flexible-efficient-machine-learning/). In Neural Information Processing Systems, Workshop on Machine Learning Systems (L…
9. Tianqi Chen, Thierry Moreau, Ziheng Jiang, Lianmin Zheng, Eddie Yan, Meghan Cowan, Haichen Shen, Leyuan Wang, Yuwei Hu, Luis Ceze, Carlos Guestrin, and Arvind Krishnamurthy. [Tvm: An automated end-to-end optimizing compiler for deep learning](https://md.research.elamur.ai/free/en/papers/chen-2018-tvm-automated-end-end-optimizing/). In 13th USENIX Symposium on Operating Systems Design and I…
10. Xinyun Chen, Chang Liu, and Dawn Song. Tree-to-tree neural networks for program translation. CoRR, abs/1802.03691, 2018. [find](https://scholar.google.com/scholar?q=Xinyun+Chen%2C+Chang+Liu%2C+and+Dawn+Song%2E+Tree%2Dto%2Dtree+neural+networks+for+program+translation%2E+CoRR%2C+abs%2F1802%2E03691%2C+2018%2E)
11. J.H. Friedman. Greedy function approximation: a gradient boosting machine. Annals of Statistics, 29(5):1189–1232, 2001. [find](https://scholar.google.com/scholar?q=J%2EH%2E+Friedman%2E+Greedy+function+approximation%3A+a+gradient+boosting+machine%2E+Annals+of+Statistics%2C+29%285%29%3A1189%E2%80%931232%2C+2001%2E)
12. M. Frigo and S. G. Johnson. Fftw: an adaptive software architecture for the fft. In Acoustics, Speech and Signal Processing, 1998. Proceedings of the 1998 IEEE International Conference on, volume 3, pages 1381–1384 vol.3, May 1998. [find](https://scholar.google.com/scholar?q=M%2E+Frigo+and+S%2E+G%2E+Johnson%2E+Fftw%3A+an+adaptive+software+architecture+for+the+fft%2E+In+Acoustics%2C+Speech+and+Signal+Processing%2C+1998%2E+Proceedings+of+the+1998+IEEE+International+Conference+on%2C+volume+3%2C+p)
13. Daniel Golovin, Benjamin Solnik, Subhodeep Moitra, Greg Kochanski, John Karro, and D. Sculley. Google vizier: A service for black-box optimization. In Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD ’17, pages 1487–1495. ACM, 2017. [find](https://scholar.google.com/scholar?q=Daniel+Golovin%2C+Benjamin+Solnik%2C+Subhodeep+Moitra%2C+Greg+Kochanski%2C+John+Karro%2C+and+D%2E+Sculley%2E+Google+vizier%3A+A+service+for+black%2Dbox+optimization%2E+In+Proceedings+of+the+23rd+ACM+SIGKDD+International+)
14. Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Identity mappings in deep residual networks. arXiv preprint arXiv:1603.05027, 2016. [find](https://scholar.google.com/scholar?q=Kaiming+He%2C+Xiangyu+Zhang%2C+Shaoqing+Ren%2C+and+Jian+Sun%2E+Identity+mappings+in+deep+residual+networks%2E+arXiv+preprint+arXiv%3A1603%2E05027%2C+2016%2E)
15. Troels Henriksen, Niels G. W. Serup, Martin Elsman, Fritz Henglein, and Cosmin E. Oancea. Futhark: Purely functional gpu-programming with nested parallelism and in-place array updates. In Proceedings of the 38th ACM SIGPLAN Conference on Programming Language Design and Implementation, PLDI 2017, pag… [find](https://scholar.google.com/scholar?q=Troels+Henriksen%2C+Niels+G%2E+W%2E+Serup%2C+Martin+Elsman%2C+Fritz+Henglein%2C+and+Cosmin+E%2E+Oancea%2E+Futhark%3A+Purely+functional+gpu%2Dprogramming+with+nested+parallelism+and+in%2Dplace+array+updates%2E+In+Proceedings+)
16. Andrew G. Howard, Menglong Zhu, Bo Chen, Dmitry Kalenichenko, Weijun Wang, Tobias Weyand, Marco Andreetto, and Hartwig Adam. [Mobilenets: Efficient convolutional neural networks for mobile vision applications](https://md.research.elamur.ai/free/en/papers/howard-2017-mobilenets-efficient-convolutional-neural-networks/). CoRR, abs/1704.04861, 2017.
17. Frank Hutter, Holger H. Hoos, and Kevin Leyton-Brown. Sequential model-based optimization for general algorithm configuration. In Proceedings of the 5th International Conference on Learning and Intelligent Optimization, LION’05, pages 507–523, Berlin, Heidelberg, 2011. Springer-Verlag. [find](https://scholar.google.com/scholar?q=Frank+Hutter%2C+Holger+H%2E+Hoos%2C+and+Kevin+Leyton%2DBrown%2E+Sequential+model%2Dbased+optimization+for+general+algorithm+configuration%2E+In+Proceedings+of+the+5th+International+Conference+on+Learning+and+Intell)
18. Frank Hutter, Lin Xu, Holger Hoos, and Kevin Leyton-Brown. Algorithm runtime prediction: Methods and evaluation (extended abstract). In Proceedings of the Twenty-Fourth International Joint Conference on Artificial Intelligence, IJCAI 2015, Buenos Aires, Argentina, July 25-31, 2015, pages 4197–4201,… [find](https://scholar.google.com/scholar?q=Frank+Hutter%2C+Lin+Xu%2C+Holger+Hoos%2C+and+Kevin+Leyton%2DBrown%2E+Algorithm+runtime+prediction%3A+Methods+and+evaluation+%28extended+abstract%29%2E+In+Proceedings+of+the+Twenty%2DFourth+International+Joint+Conference+)
19. S. Kirkpatrick, C. D. Gelatt, and M. P. Vecchi. 220(4598):671–680, 1983. [find](https://scholar.google.com/scholar?q=S%2E+Kirkpatrick%2C+C%2E+D%2E+Gelatt%2C+and+M%2E+P%2E+Vecchi%2E+220%284598%29%3A671%E2%80%93680%2C+1983%2E)
20. Fredrik Kjolstad, Shoaib Kamil, Stephen Chou, David Lugato, and Saman Amarasinghe. The tensor algebra compiler. Proc. ACM Program. Lang., 1(OOPSLA):77:1–77:29, October 2017. [find](https://scholar.google.com/scholar?q=Fredrik+Kjolstad%2C+Shoaib+Kamil%2C+Stephen+Chou%2C+David+Lugato%2C+and+Saman+Amarasinghe%2E+The+tensor+algebra+compiler%2E+Proc%2E+ACM+Program%2E+Lang%2E%2C+1%28OOPSLA%29%3A77%3A1%E2%80%9377%3A29%2C+October+2017%2E)
21. Tim Kraska, Alex Beutel, Ed H. Chi, Jeffrey Dean, and Neoklis Polyzotis. The case for learned index structures. CoRR, abs/1712.01208, 2017. — [arXiv:1712.01208](https://arxiv.org/abs/1712.01208) [find](https://scholar.google.com/scholar?q=Tim+Kraska%2C+Alex+Beutel%2C+Ed+H%2E+Chi%2C+Jeffrey+Dean%2C+and+Neoklis+Polyzotis%2E+The+case+for+learned+index+structures%2E+CoRR%2C+abs%2F1712%2E01208%2C+2017%2E+%E2%80%94+%5BarXiv%3A1712%2E01208%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F1712%2E01208%29)
22. Andreas Krause and Daniel Golovin. Submodular function maximization. In Tractability: Practical Approaches to Hard Problems. Cambridge University Press, February 2014. [find](https://scholar.google.com/scholar?q=Andreas+Krause+and+Daniel+Golovin%2E+Submodular+function+maximization%2E+In+Tractability%3A+Practical+Approaches+to+Hard+Problems%2E+Cambridge+University+Press%2C+February+2014%2E)
23. Alex Krizhevsky, Ilya Sutskever, and Geoffrey E. Hinton. [Imagenet classification with deep convolutional neural networks](https://md.research.elamur.ai/free/en/papers/krizhevsky-2012-imagenet-classification-deep-convolutional/). In Advances in Neural Information Processing Systems 25, pages 1097–1105. 2012.
24. Andrew Lavin and Scott Gray. [Fast algorithms for convolutional neural networks](https://md.research.elamur.ai/free/en/papers/lavin-2015-fast-algorithms-convolutional-neural-networks/). In 2016 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2016, Las Vegas, NV, USA, June 27-30, 2016, pages 4013–4021, 2016.
25. Lisha Li, Kevin G. Jamieson, Giulia DeSalvo, Afshin Rostamizadeh, and Ameet Talwalkar. Efficient hyperparameter optimization and infinitely many armed bandits. CoRR, abs/1603.06560, 2016. — [arXiv:1603.06560](https://arxiv.org/abs/1603.06560) [find](https://scholar.google.com/scholar?q=Lisha+Li%2C+Kevin+G%2E+Jamieson%2C+Giulia+DeSalvo%2C+Afshin+Rostamizadeh%2C+and+Ameet+Talwalkar%2E+Efficient+hyperparameter+optimization+and+infinitely+many+armed+bandits%2E+CoRR%2C+abs%2F1603%2E06560%2C+2016%2E+%E2%80%94+%5BarXiv%3A160)
26. Azalia Mirhoseini, Hieu Pham, Quoc V. Le, Benoit Steiner, Rasmus Larsen, Yuefeng Zhou, Naveen Kumar, Mohammad Norouzi, Samy Bengio, and Jeff Dean. Device placement optimization with reinforcement learning. In Proceedings of the 34th International Conference on Machine Learning, ICML 2017, Sydney, NS… [find](https://scholar.google.com/scholar?q=Azalia+Mirhoseini%2C+Hieu+Pham%2C+Quoc+V%2E+Le%2C+Benoit+Steiner%2C+Rasmus+Larsen%2C+Yuefeng+Zhou%2C+Naveen+Kumar%2C+Mohammad+Norouzi%2C+Samy+Bengio%2C+and+Jeff+Dean%2E+Device+placement+optimization+with+reinforcement+lear)
27. Volodymyr Mnih, Koray Kavukcuoglu, David Silver, Andrei A Rusu, Joel Veness, Marc G Bellemare, Alex Graves, Martin Riedmiller, Andreas K Fidjeland, Georg Ostrovski, et al. Human-level control through deep reinforcement learning. Nature, 518(7540):529, 2015. [find](https://scholar.google.com/scholar?q=Volodymyr+Mnih%2C+Koray+Kavukcuoglu%2C+David+Silver%2C+Andrei+A+Rusu%2C+Joel+Veness%2C+Marc+G+Bellemare%2C+Alex+Graves%2C+Martin+Riedmiller%2C+Andreas+K+Fidjeland%2C+Georg+Ostrovski%2C+et+al%2E+Human%2Dlevel+control+through+)
28. Ravi Teja Mullapudi, Andrew Adams, Dillon Sharlet, Jonathan Ragan-Kelley, and Kayvon Fatahalian. Automatically scheduling halide image processing pipelines. ACM Trans. Graph., 35(4):83:1–83:11, July 2016. [find](https://scholar.google.com/scholar?q=Ravi+Teja+Mullapudi%2C+Andrew+Adams%2C+Dillon+Sharlet%2C+Jonathan+Ragan%2DKelley%2C+and+Kayvon+Fatahalian%2E+Automatically+scheduling+halide+image+processing+pipelines%2E+ACM+Trans%2E+Graph%2E%2C+35%284%29%3A83%3A1%E2%80%9383%3A11%2C+July+2)
29. George L Nemhauser, Laurence A Wolsey, and Marshall L Fisher. An analysis of approximations for maximizing submodular set functions—i. Mathematical Programming, 14(1):265–294, 1978. [find](https://scholar.google.com/scholar?q=George+L+Nemhauser%2C+Laurence+A+Wolsey%2C+and+Marshall+L+Fisher%2E+An+analysis+of+approximations+for+maximizing+submodular+set+functions%E2%80%94i%2E+Mathematical+Programming%2C+14%281%29%3A265%E2%80%93294%2C+1978%2E)
30. Shoumik Palkar, James J. Thomas, Deepak Narayanan, Anil Shanbhag, Rahul Palamuttam, Holger Pirk, Malte Schwarzkopf, Saman P. Amarasinghe, Samuel Madden, and Matei Zaharia. Weld: Rethinking the interface between data-intensive applications. CoRR, abs/1709.06416, 2017. — [arXiv:1709.06416](https://arxiv.org/abs/1709.06416) [find](https://scholar.google.com/scholar?q=Shoumik+Palkar%2C+James+J%2E+Thomas%2C+Deepak+Narayanan%2C+Anil+Shanbhag%2C+Rahul+Palamuttam%2C+Holger+Pirk%2C+Malte+Schwarzkopf%2C+Saman+P%2E+Amarasinghe%2C+Samuel+Madden%2C+and+Matei+Zaharia%2E+Weld%3A+Rethinking+the+interfa)
31. Alec Radford, Luke Metz, and Soumith Chintala. [Unsupervised representation learning with deep convolutional generative adversarial networks](https://md.research.elamur.ai/free/en/papers/radford-2015-unsupervised-representation-learning-deep-convolutional/). arXiv preprint arXiv:1511.06434, 2015.
32. Jonathan Ragan-Kelley, Connelly Barnes, Andrew Adams, Sylvain Paris, Frédo Durand, and Saman Amarasinghe. Halide: A language and compiler for optimizing parallelism, locality, and recomputation in image processing pipelines. In Proceedings of the 34th ACM SIGPLAN Conference on Programming Language D… [find](https://scholar.google.com/scholar?q=Jonathan+Ragan%2DKelley%2C+Connelly+Barnes%2C+Andrew+Adams%2C+Sylvain+Paris%2C+Fr%C3%A9do+Durand%2C+and+Saman+Amarasinghe%2E+Halide%3A+A+language+and+compiler+for+optimizing+parallelism%2C+locality%2C+and+recomputation+in+ima)
33. B. Shahriari, K. Swersky, Z. Wang, R. P. Adams, and N. de Freitas. Taking the human out of the loop: A review of bayesian optimization. Proceedings of the IEEE, 104(1):148–175, Jan 2016. [find](https://scholar.google.com/scholar?q=B%2E+Shahriari%2C+K%2E+Swersky%2C+Z%2E+Wang%2C+R%2E+P%2E+Adams%2C+and+N%2E+de+Freitas%2E+Taking+the+human+out+of+the+loop%3A+A+review+of+bayesian+optimization%2E+Proceedings+of+the+IEEE%2C+104%281%29%3A148%E2%80%93175%2C+Jan+2016%2E)
34. Jasper Snoek, Hugo Larochelle, and Ryan P. Adams. Practical bayesian optimization of machine learning algorithms. In Proceedings of the 25th International Conference on Neural Information Processing Systems - Volume 2, NIPS’12, pages 2951–2959, USA, 2012. [find](https://scholar.google.com/scholar?q=Jasper+Snoek%2C+Hugo+Larochelle%2C+and+Ryan+P%2E+Adams%2E+Practical+bayesian+optimization+of+machine+learning+algorithms%2E+In+Proceedings+of+the+25th+International+Conference+on+Neural+Information+Processing+S)
35. Jasper Snoek, Oren Rippel, Kevin Swersky, Ryan Kiros, Nadathur Satish, Narayanan Sundaram, Md. Mostofa Ali Patwary, Prabhat Prabhat, and Ryan P. Adams. Scalable bayesian optimization using deep neural networks. In Proceedings of the 32Nd International Conference on International Conference on Machin… [find](https://scholar.google.com/scholar?q=Jasper+Snoek%2C+Oren+Rippel%2C+Kevin+Swersky%2C+Ryan+Kiros%2C+Nadathur+Satish%2C+Narayanan+Sundaram%2C+Md%2E+Mostofa+Ali+Patwary%2C+Prabhat+Prabhat%2C+and+Ryan+P%2E+Adams%2E+Scalable+bayesian+optimization+using+deep+neural)
36. Michel Steuwer, Toomas Remmelg, and Christophe Dubach. Lift: A functional data-parallel ir for high-performance gpu code generation. In Proceedings of the 2017 International Symposium on Code Generation and Optimization, CGO ’17, pages 74–85, Piscataway, NJ, USA, 2017. IEEE Press. [find](https://scholar.google.com/scholar?q=Michel+Steuwer%2C+Toomas+Remmelg%2C+and+Christophe+Dubach%2E+Lift%3A+A+functional+data%2Dparallel+ir+for+high%2Dperformance+gpu+code+generation%2E+In+Proceedings+of+the+2017+International+Symposium+on+Code+Generati)
37. Arvind K. Sujeeth, HyoukJoong Lee, Kevin J. Brown, Hassan Chafi, Michael Wu, Anand R. Atreya, Kunle Olukotun, Tiark Rompf, and Martin Odersky. Optiml: An implicitly parallel domain-specific language for machine learning. In Proceedings of the 28th International Conference on International Conference… [find](https://scholar.google.com/scholar?q=Arvind+K%2E+Sujeeth%2C+HyoukJoong+Lee%2C+Kevin+J%2E+Brown%2C+Hassan+Chafi%2C+Michael+Wu%2C+Anand+R%2E+Atreya%2C+Kunle+Olukotun%2C+Tiark+Rompf%2C+and+Martin+Odersky%2E+Optiml%3A+An+implicitly+parallel+domain%2Dspecific+language+f)
38. Ilya Sutskever, Oriol Vinyals, and Quoc V. Le. [Sequence to sequence learning with neural networks](https://md.research.elamur.ai/free/en/papers/sutskever-2014-sequence-sequence-learning-neural-networks/). In Proceedings of the 27th International Conference on Neural Information Processing Systems - Volume 2, NIPS’14, pages 3104–3112, Cambridge, MA, USA, 2014. MIT Press.
39. Kai Sheng Tai, Richard Socher, and Christopher D Manning. Improved semantic representations from tree-structured long short-term memory networks. arXiv preprint arXiv:1503.00075, 2015. [find](https://scholar.google.com/scholar?q=Kai+Sheng+Tai%2C+Richard+Socher%2C+and+Christopher+D+Manning%2E+Improved+semantic+representations+from+tree%2Dstructured+long+short%2Dterm+memory+networks%2E+arXiv+preprint+arXiv%3A1503%2E00075%2C+2015%2E)
40. Nicolas Vasilache, Oleksandr Zinenko, Theodoros Theodoridis, Priya Goyal, Zachary DeVito, William S. Moses, Sven Verdoolaege, Andrew Adams, and Albert Cohen. [Tensor comprehensions: Framework-agnostic high-performance machine learning abstractions](https://md.research.elamur.ai/free/en/papers/vasilache-2018-tensor-comprehensions-framework-agnostic-high/). CoRR, abs/1802.04730, 2018.
41. Sven Verdoolaege, Juan Carlos Juega, Albert Cohen, José Ignacio Gómez, Christian Tenllado, and Francky Catthoor. Polyhedral parallel code generation for cuda. ACM Trans. Archit. Code Optim., 9(4):54:1–54:23, January 2013. [find](https://scholar.google.com/scholar?q=Sven+Verdoolaege%2C+Juan+Carlos+Juega%2C+Albert+Cohen%2C+Jos%C3%A9+Ignacio+G%C3%B3mez%2C+Christian+Tenllado%2C+and+Francky+Catthoor%2E+Polyhedral+parallel+code+generation+for+cuda%2E+ACM+Trans%2E+Archit%2E+Code+Optim%2E%2C+9%284%29%3A54%3A1)
42. R. Clint Whaley and Jack J. Dongarra. Automatically tuned linear algebra software. In Proceedings of the 1998 ACM/IEEE Conference on Supercomputing, SC ’98, pages 1–27, Washington, DC, USA, 1998. IEEE Computer Society. [find](https://scholar.google.com/scholar?q=R%2E+Clint+Whaley+and+Jack+J%2E+Dongarra%2E+Automatically+tuned+linear+algebra+software%2E+In+Proceedings+of+the+1998+ACM%2FIEEE+Conference+on+Supercomputing%2C+SC+%E2%80%9998%2C+pages+1%E2%80%9327%2C+Washington%2C+DC%2C+USA%2C+1998%2E+IEEE)
43. Wojciech Zaremba, Ilya Sutskever, and Oriol Vinyals. [Recurrent neural network regularization](https://md.research.elamur.ai/free/en/papers/zaremba-2014-recurrent-neural-network-regularization/). arXiv preprint arXiv:1409.2329, 2014.

*Updated 2 November 2026*