---
title: "EIE — paper by Han et al. (2016) — summary and ideas"
description: "The authors build EIE, a custom ASIC accelerator that runs directly on models already shrunk by… 45 references, cited by 145 papers here."
authors: ["Song Han", "Xingyu Liu", "Huizi Mao", "Jing Pu", "Ardavan Pedram", "Mark A. Horowitz", "William J. Dally"]
year: 2016
venue: "arXiv:1602.01528"
source: "https://arxiv.org/abs/1602.01528"
doi: "10.48550/arxiv.1602.01528"
arxiv: "1602.01528"
licence: "arXiv — non-exclusive license to distribute"
licence_url: "http://arxiv.org/licenses/nonexclusive-distrib/1.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/han-2016-eie-efficient-inference-engine-compressed/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/han-2016-eie-efficient-inference-engine-compressed/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2016](https://md.research.elamur.ai/free/en/papers/2016/) / EIE: Efficient Inference Engine on Compressed Deep Neural Network

# EIE: Efficient Inference Engine on Compressed Deep Neural Network

**Authors:** [Song Han](https://md.research.elamur.ai/free/en/authors/song-han/), [Xingyu Liu](https://md.research.elamur.ai/free/en/authors/xingyu-liu/), [Huizi Mao](https://md.research.elamur.ai/free/en/authors/huizi-mao/), Jing Pu, Ardavan Pedram, Mark A. Horowitz and William J. Dally · **Year:** 2016 · **Venue:** [arXiv:1602.01528](https://md.research.elamur.ai/free/en/venues/acm-ieee-international-symposium-on-computer-architecture-isca/) ([ISCA 2016](https://md.research.elamur.ai/free/en/venues/acm-ieee-international-symposium-on-computer-architecture-isca/2016/)) · **Licence:** [arXiv — non-exclusive license to distribute](http://arxiv.org/licenses/nonexclusive-distrib/1.0/)

**Links:** [Publisher's page](https://arxiv.org/abs/1602.01528) · [PDF](https://arxiv.org/pdf/1602.01528) · [DOI 10.48550/arxiv.1602.01528](https://doi.org/10.48550/arxiv.1602.01528) · [arXiv 1602.01528](https://arxiv.org/abs/1602.01528) · [OpenAlex](https://openalex.org/W2950656546)

**Citations:** 1,048 citations (OpenAlex, 2026-10-14) · [145](https://md.research.elamur.ai/free/en/papers/han-2016-eie-efficient-inference-engine-compressed/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
Large neural networks spend most of their energy fetching weights from off-chip DRAM rather than on arithmetic. A DRAM access costs about 128 times more than an on-chip SRAM read, which makes such models hard to run on battery-powered devices. Fully connected layers suffer most because their weights are not reused, and batching to amortise memory traffic adds unacceptable latency in real-time settings. The authors build EIE, a custom ASIC accelerator that runs directly on models already shrunk by Deep Compression (pruning plus weight sharing), so networks like AlexNet fit entirely in SRAM. EIE is an array of processing elements. Each element stores a slice of the sparse weight matrix, skips work for both pruned weights and zero input activations, and decodes 4-bit weight indices through a small table of shared values. On nine CNN and LSTM benchmarks, EIE beats uncompressed CPU and GPU baselines by 189× and 13× in speed and is thousands of times more energy-efficient. The work shows that model compression delivers its full benefit only when the hardware is designed around the compressed format.

## Key points
- The energy gains come from four sources, as the authors attribute them:
  - roughly 120× from keeping weights in SRAM instead of DRAM;
  - about 10× from weight sparsity;
  - 8× from weight sharing;
  - about 3× from skipping zero ReLU outputs.
- After pruning, the benchmark layers keep only 4–25% of their weights. Each surviving weight is a 4-bit pointer into a table of 16 shared values.
- Each PE in 45nm CMOS occupies 0.638 mm² and draws 9.16 mW at 800 MHz. It holds 131K compressed weights, which correspond to about 1.2M dense ones.
- Sixty-four PEs (40.8 mm²) handle the AlexNet FC layers at 1.88×10⁴ frames/s using under 600 mW. The design delivers 102 GOPS/s on the compressed model, equivalent to roughly 3 TOPS/s on a dense one.
- Against the baselines:
  - Intel i7-5930k CPU: 24,000× better energy efficiency.
  - GeForce TITAN X GPU: 3,400× better energy efficiency.
  - Tegra K1 mobile GPU: 2,700× better energy efficiency.
  - DaDianNao: 2.9× higher throughput, 19× better energy efficiency, 3× better area efficiency.

## Interesting ideas
- **Use both kinds of sparsity.** EIE skips a multiply only when the weight is nonzero and the incoming activation is also nonzero, rather than relying on fixed weight sparsity alone. About 70% of activations are zero in typical workloads, so this input-dependent zero-skipping yields large extra savings. The idea transfers to any sparse inference kernel.
- **Store weights as codebook indices.** Each weight is a tiny index into a short table of shared values rather than a full number. This trick could shrink storage for embedding tables or language models running on low-resource devices.
- **Interleave rows across processing units.** Matrix rows and output elements are distributed round-robin among the PEs, so each unit gets a mix of the irregular sparse work. This is a simple way to balance load and keep scaling good in any parallel sparse workload.
- **Do the energy arithmetic first.** The paper motivates its design with a quick estimate: a 1G-connection model at 20 Hz would need about 12.8 W for DRAM traffic alone. Such rough calculations are a cheap way to tell whether a deployment target is realistic before building anything.
- **Report savings per technique.** The paper credits each technique with its own multiplier and states throughput in both compressed and dense-equivalent terms. This makes it clear which design choice matters most and keeps comparisons with dense systems honest.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Stanford University](https://md.research.elamur.ai/free/en/universities/stanford-university/) — University, Stanford, United States · 506 works · 330,053 citations
- [Nvidia](https://md.research.elamur.ai/free/en/companies/nvidia/) — Company, Santa Clara, United States · 37 works · 12,656 citations
- [Nvidia (United Kingdom)](https://md.research.elamur.ai/free/en/companies/nvidia-united-kingdom/) — Company, Reading, United Kingdom · 27 works · 11,141 citations

📚 Shelves (5): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 18,614 works · [👁️ Computer Vision and Pattern Recognition](https://md.research.elamur.ai/free/en/shelves/computer-science/computer-vision-and-pattern-recognition/) · 3,486 works · [🔌 Hardware and Architecture](https://md.research.elamur.ai/free/en/shelves/computer-science/hardware-and-architecture/) · 167 works · [⚙️ Engineering](https://md.research.elamur.ai/free/en/shelves/engineering/) · 188 works · [⚙️ Electrical and Electronic Engineering](https://md.research.elamur.ai/free/en/shelves/engineering/electrical-and-electronic-engineering/) · 33 works

🧭 Topics (1): [🪶 Efficient models & compression](https://md.research.elamur.ai/free/en/topics/efficient-models/) · 297 works

## Cite

Cite as: Song Han, Xingyu Liu, Huizi Mao, Jing Pu, Ardavan Pedram, Mark A. Horowitz and William J. Dally. 2016. EIE: Efficient Inference Engine on Compressed Deep Neural Network. arXiv:1602.01528.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@misc{han2016eie,
  title = {{EIE: Efficient Inference Engine on Compressed Deep Neural Network}},
  author = {Song Han, Xingyu Liu, Huizi Mao, Jing Pu, Ardavan Pedram, Mark A. Horowitz and William J. Dally},
  year = {2016},
  doi = {10.48550/arxiv.1602.01528},
  eprint = {1602.01528},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.1602.01528}
}
```

- [← previous Matching Networks for One Shot Learning](https://md.research.elamur.ai/free/en/papers/vinyals-2016-matching-networks-one-shot-learning/)
- [next → MS MARCO: A Human Generated MAchine Reading COmprehension Dataset](https://md.research.elamur.ai/free/en/papers/bajaj-2016-ms-marco-human-generated/)
- [↑ back to papers of 2016](https://md.research.elamur.ai/free/en/papers/2016/)

## References

45 references, 5 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. A. Krizhevsky, I. Sutskever, and G. E. Hinton, “[Imagenet classification with deep convolutional neural networks](https://md.research.elamur.ai/free/en/papers/krizhevsky-2012-imagenet-classification-deep-convolutional/),” in NIPS, 2012.
2. C. Szegedy, W. Liu, Y. Jia, P. Sermanet, S. Reed, D. Anguelov, D. Erhan, V. Vanhoucke, and A. Rabinovich, “Going deeper with convolutions,” arXiv:1409.4842, 2014. [find](https://scholar.google.com/scholar?q=C%2E+Szegedy%2C+W%2E+Liu%2C+Y%2E+Jia%2C+P%2E+Sermanet%2C+S%2E+Reed%2C+D%2E+Anguelov%2C+D%2E+Erhan%2C+V%2E+Vanhoucke%2C+and+A%2E+Rabinovich%2C+%E2%80%9CGoing+deeper+with+convolutions%2C%E2%80%9D+arXiv%3A1409%2E4842%2C+2014%2E)
3. K. Simonyan and A. Zisserman, “[Very deep convolutional networks for large-scale image recognition](https://md.research.elamur.ai/free/en/papers/simonyan-2014-very-deep-convolutional-networks-large/),” arXiv:1409.1556, 2014.
4. T. Mikolov, M. Karafiát, L. Burget, J. Cernockỳ, and S. Khudanpur, “Recurrent neural network based language model.” in INTER-SPEECH, September 26-30, 2010, 2010, pp. 1045–1048. [find](https://scholar.google.com/scholar?q=T%2E+Mikolov%2C+M%2E+Karafia%CC%81t%2C+L%2E+Burget%2C+J%2E+Cernocky%CC%80%2C+and+S%2E+Khudanpur%2C+%E2%80%9CRecurrent+neural+network+based+language+model%2E%E2%80%9D+in+INTER%2DSPEECH%2C+September+26%2D30%2C+2010%2C+2010%2C+pp%2E+1045%E2%80%931048%2E)
5. Y. LeCun, L. Bottou, Y. Bengio, and P. Haffner, “Gradient-based learning applied to document recognition,” Proceedings of the IEEE, vol. 86, no. 11, pp. 2278–2324, 1998. [find](https://scholar.google.com/scholar?q=Y%2E+LeCun%2C+L%2E+Bottou%2C+Y%2E+Bengio%2C+and+P%2E+Haffner%2C+%E2%80%9CGradient%2Dbased+learning+applied+to+document+recognition%2C%E2%80%9D+Proceedings+of+the+IEEE%2C+vol%2E+86%2C+no%2E+11%2C+pp%2E+2278%E2%80%932324%2C+1998%2E)
6. Y. Taigman, M. Yang, M. Ranzato, and L. Wolf, “Deepface: Closing the gap to human-level performance in face verification,” in CVPR. IEEE, 2014, pp. 1701–1708. [find](https://scholar.google.com/scholar?q=Y%2E+Taigman%2C+M%2E+Yang%2C+M%2E+Ranzato%2C+and+L%2E+Wolf%2C+%E2%80%9CDeepface%3A+Closing+the+gap+to+human%2Dlevel+performance+in+face+verification%2C%E2%80%9D+in+CVPR%2E+IEEE%2C+2014%2C+pp%2E+1701%E2%80%931708%2E)
7. A. Karpathy and L. Fei-Fei, “[Deep visual-semantic alignments for generating image descriptions](https://md.research.elamur.ai/free/en/papers/karpathy-2014-deep-visual-semantic-alignments-generating/),” arXiv:1412.2306, 2014.
8. A. Coates, B. Huval, T. Wang, D. Wu, B. Catanzaro, and N. Andrew, “Deep learning with cots hpc systems,” in 30th ICML, 2013. [find](https://scholar.google.com/scholar?q=A%2E+Coates%2C+B%2E+Huval%2C+T%2E+Wang%2C+D%2E+Wu%2C+B%2E+Catanzaro%2C+and+N%2E+Andrew%2C+%E2%80%9CDeep+learning+with+cots+hpc+systems%2C%E2%80%9D+in+30th+ICML%2C+2013%2E)
9. M. Horowitz. Energy table for 45nm process, Stanford VLSI wiki. [Online]. Available: [https://sites.google.com/site/seecproject](https://sites.google.com/site/seecproject) [↗](https://sites.google.com/site/seecproject)
10. T. Chen, Z. Du, N. Sun, J. Wang, C. Wu, Y. Chen, and O. Temam, “Diannao: a small-footprint high-throughput accelerator for ubiquitous machine-learning,” in ASPLOS, 2014. [find](https://scholar.google.com/scholar?q=T%2E+Chen%2C+Z%2E+Du%2C+N%2E+Sun%2C+J%2E+Wang%2C+C%2E+Wu%2C+Y%2E+Chen%2C+and+O%2E+Temam%2C+%E2%80%9CDiannao%3A+a+small%2Dfootprint+high%2Dthroughput+accelerator+for+ubiquitous+machine%2Dlearning%2C%E2%80%9D+in+ASPLOS%2C+2014%2E)
11. Y. Chen, T. Luo, S. Liu, S. Zhang, L. He, J. Wang, L. Li, T. Chen, Z. Xu, N. Sun, and O. Temam, “Dadiannao: A machine-learning supercomputer,” in MICRO, December 2014. [find](https://scholar.google.com/scholar?q=Y%2E+Chen%2C+T%2E+Luo%2C+S%2E+Liu%2C+S%2E+Zhang%2C+L%2E+He%2C+J%2E+Wang%2C+L%2E+Li%2C+T%2E+Chen%2C+Z%2E+Xu%2C+N%2E+Sun%2C+and+O%2E+Temam%2C+%E2%80%9CDadiannao%3A+A+machine%2Dlearning+supercomputer%2C%E2%80%9D+in+MICRO%2C+December+2014%2E)
12. Z. Du, R. Fasthuber, T. Chen, P. Ienne, L. Li, T. Luo, X. Feng, Y. Chen, and O. Temam, “Shidiannao: shifting vision processing closer to the sensor,” in ISCA. ACM, 2015, pp. 92–104. [find](https://scholar.google.com/scholar?q=Z%2E+Du%2C+R%2E+Fasthuber%2C+T%2E+Chen%2C+P%2E+Ienne%2C+L%2E+Li%2C+T%2E+Luo%2C+X%2E+Feng%2C+Y%2E+Chen%2C+and+O%2E+Temam%2C+%E2%80%9CShidiannao%3A+shifting+vision+processing+closer+to+the+sensor%2C%E2%80%9D+in+ISCA%2E+ACM%2C+2015%2C+pp%2E+92%E2%80%93104%2E)
13. C. Farabet, C. Poulet, J. Y. Han, and Y. LeCun, “Cnp: An fpga-based processor for convolutional networks,” in FPL, 2009. [find](https://scholar.google.com/scholar?q=C%2E+Farabet%2C+C%2E+Poulet%2C+J%2E+Y%2E+Han%2C+and+Y%2E+LeCun%2C+%E2%80%9CCnp%3A+An+fpga%2Dbased+processor+for+convolutional+networks%2C%E2%80%9D+in+FPL%2C+2009%2E)
14. J. Qiu, J. Wang, S. Yao, K. Guo, B. Li, E. Zhou, J. Yu, T. Tang, N. Xu, S. Song, Y. Wang, and H. Yang, “Going deeper with embedded fpga platform for convolutional neural network,” in FPGA, 2016. [find](https://scholar.google.com/scholar?q=J%2E+Qiu%2C+J%2E+Wang%2C+S%2E+Yao%2C+K%2E+Guo%2C+B%2E+Li%2C+E%2E+Zhou%2C+J%2E+Yu%2C+T%2E+Tang%2C+N%2E+Xu%2C+S%2E+Song%2C+Y%2E+Wang%2C+and+H%2E+Yang%2C+%E2%80%9CGoing+deeper+with+embedded+fpga+platform+for+convolutional+neural+network%2C%E2%80%9D+in+FPGA%2C+2016%2E)
15. A. Shafiee and et al., “ISAAC: A convolutional neural network accelerator with in-situ analog arithmetic in crossbars,” ISCA, 2016. [find](https://scholar.google.com/scholar?q=A%2E+Shafiee+and+et+al%2E%2C+%E2%80%9CISAAC%3A+A+convolutional+neural+network+accelerator+with+in%2Dsitu+analog+arithmetic+in+crossbars%2C%E2%80%9D+ISCA%2C+2016%2E)
16. S. Han, J. Pool, J. Tran, and W. J. Dally, “Learning both weights and connections for efficient neural networks,” in Proceedings of Advances in Neural Information Processing Systems, 2015. [find](https://scholar.google.com/scholar?q=S%2E+Han%2C+J%2E+Pool%2C+J%2E+Tran%2C+and+W%2E+J%2E+Dally%2C+%E2%80%9CLearning+both+weights+and+connections+for+efficient+neural+networks%2C%E2%80%9D+in+Proceedings+of+Advances+in+Neural+Information+Processing+Systems%2C+2015%2E)
17. R. Girshick, “Fast R-CNN,” arXiv:1504.08083, 2015. [find](https://scholar.google.com/scholar?q=R%2E+Girshick%2C+%E2%80%9CFast+R%2DCNN%2C%E2%80%9D+arXiv%3A1504%2E08083%2C+2015%2E)
18. S. Hochreiter and J. Schmidhuber, “Long short-term memory,” Neural computation, 1997. [find](https://scholar.google.com/scholar?q=S%2E+Hochreiter+and+J%2E+Schmidhuber%2C+%E2%80%9CLong+short%2Dterm+memory%2C%E2%80%9D+Neural+computation%2C+1997%2E)
19. A. Graves and J. Schmidhuber, “Framewise phoneme classification with bidirectional lstm and other neural network architectures,” Neural Networks, 2005. [find](https://scholar.google.com/scholar?q=A%2E+Graves+and+J%2E+Schmidhuber%2C+%E2%80%9CFramewise+phoneme+classification+with+bidirectional+lstm+and+other+neural+network+architectures%2C%E2%80%9D+Neural+Networks%2C+2005%2E)
20. N. D. Lane and P. Georgiev, “Can deep learning revolutionize mobile sensing?” in International Workshop on Mobile Computing Systems and Applications. ACM, 2015, pp. 117–122. [find](https://scholar.google.com/scholar?q=N%2E+D%2E+Lane+and+P%2E+Georgiev%2C+%E2%80%9CCan+deep+learning+revolutionize+mobile+sensing%3F%E2%80%9D+in+International+Workshop+on+Mobile+Computing+Systems+and+Applications%2E+ACM%2C+2015%2C+pp%2E+117%E2%80%93122%2E)
21. Richard Dorrance and Fengbo Ren and Dejan Marković, “A Scalable Sparse Matrix-vector Multiplication Kernel for Energy-efficient Sparse-blas on FPGAs,” in FPGA, 2014. [find](https://scholar.google.com/scholar?q=Richard+Dorrance+and+Fengbo+Ren+and+Dejan+Markovic%CC%81%2C+%E2%80%9CA+Scalable+Sparse+Matrix%2Dvector+Multiplication+Kernel+for+Energy%2Defficient+Sparse%2Dblas+on+FPGAs%2C%E2%80%9D+in+FPGA%2C+2014%2E)
22. V. Nair and G. E. Hinton, “Rectified linear units improve restricted boltzmann machines,” in ICML, 2010. [find](https://scholar.google.com/scholar?q=V%2E+Nair+and+G%2E+E%2E+Hinton%2C+%E2%80%9CRectified+linear+units+improve+restricted+boltzmann+machines%2C%E2%80%9D+in+ICML%2C+2010%2E)
23. S. Han, H. Mao, and W. J. Dally, “Deep compression: Compressing deep neural networks with pruning, trained quantization and huffman coding,” International Conference on Learning Representations 2016. [find](https://scholar.google.com/scholar?q=S%2E+Han%2C+H%2E+Mao%2C+and+W%2E+J%2E+Dally%2C+%E2%80%9CDeep+compression%3A+Compressing+deep+neural+networks+with+pruning%2C+trained+quantization+and+huffman+coding%2C%E2%80%9D+International+Conference+on+Learning+Representations+2016%2E)
24. R. W. Vuduc, “Automatic performance tuning of sparse matrix kernels,” Ph.D. dissertation, UC Berkeley, 2003. [find](https://scholar.google.com/scholar?q=R%2E+W%2E+Vuduc%2C+%E2%80%9CAutomatic+performance+tuning+of+sparse+matrix+kernels%2C%E2%80%9D+Ph%2ED%2E+dissertation%2C+UC+Berkeley%2C+2003%2E)
25. N. Muralimanohar, R. Balasubramonian, and N. P. Jouppi, “Cacti 6.0: A tool to model large caches,” HP Laboratories, pp. 22–31, 2009. [find](https://scholar.google.com/scholar?q=N%2E+Muralimanohar%2C+R%2E+Balasubramonian%2C+and+N%2E+P%2E+Jouppi%2C+%E2%80%9CCacti+6%2E0%3A+A+tool+to+model+large+caches%2C%E2%80%9D+HP+Laboratories%2C+pp%2E+22%E2%80%9331%2C+2009%2E)
26. NVIDIA. Technical brief: NVIDIA jetson TK1 development kit bringing GPU-accelerated computing to embedded systems. [find](https://scholar.google.com/scholar?q=NVIDIA%2E+Technical+brief%3A+NVIDIA+jetson+TK1+development+kit+bringing+GPU%2Daccelerated+computing+to+embedded+systems%2E)
27. NVIDIA. Whitepaper: GPU-based deep learning inference: A performance and power analysis. [find](https://scholar.google.com/scholar?q=NVIDIA%2E+Whitepaper%3A+GPU%2Dbased+deep+learning+inference%3A+A+performance+and+power+analysis%2E)
28. Y. Jia, E. Shelhamer, J. Donahue, S. Karayev, J. Long, R. Girshick, S. Guadarrama, and T. Darrell, “[Caffe: Convolutional architecture for fast feature embedding](https://md.research.elamur.ai/free/en/papers/jia-2014-caffe-convolutional-architecture-fast-feature/),” arXiv:1408.5093, 2014.
29. J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, and L. Fei-Fei, “Imagenet: A large-scale hierarchical image database,” in Computer Vision and Pattern Recognition. 2009. [find](https://scholar.google.com/scholar?q=J%2E+Deng%2C+W%2E+Dong%2C+R%2E+Socher%2C+L%2E%2DJ%2E+Li%2C+K%2E+Li%2C+and+L%2E+Fei%2DFei%2C+%E2%80%9CImagenet%3A+A+large%2Dscale+hierarchical+image+database%2C%E2%80%9D+in+Computer+Vision+and+Pattern+Recognition%2E+2009%2E)
30. F. N. Iandola, S. Han, M. W. Moskewicz, K. Ashraf, W. J. Dally, and K. Keutzer, “Squeezenet: Alexnet-level accuracy with 50x fewer parameters and¡ 0.5mb model size,” arXiv:1602.07360, 2016. [find](https://scholar.google.com/scholar?q=F%2E+N%2E+Iandola%2C+S%2E+Han%2C+M%2E+W%2E+Moskewicz%2C+K%2E+Ashraf%2C+W%2E+J%2E+Dally%2C+and+K%2E+Keutzer%2C+%E2%80%9CSqueezenet%3A+Alexnet%2Dlevel+accuracy+with+50x+fewer+parameters+and%C2%A1+0%2E5mb+model+size%2C%E2%80%9D+arXiv%3A1602%2E07360%2C+2016%2E)
31. Ling Zhuo and Viktor K. Prasanna, “Sparse Matrix-Vector Multiplication on FPGAs,” in FPGA, 2005. [find](https://scholar.google.com/scholar?q=Ling+Zhuo+and+Viktor+K%2E+Prasanna%2C+%E2%80%9CSparse+Matrix%2DVector+Multiplication+on+FPGAs%2C%E2%80%9D+in+FPGA%2C+2005%2E)
32. V. Eijkhout, LAPACK working note 50: Distributed sparse data structures for linear algebra operations, 1992. [find](https://scholar.google.com/scholar?q=V%2E+Eijkhout%2C+LAPACK+working+note+50%3A+Distributed+sparse+data+structures+for+linear+algebra+operations%2C+1992%2E)
33. A. Lavin, “[Fast algorithms for convolutional neural networks](https://md.research.elamur.ai/free/en/papers/lavin-2015-fast-algorithms-convolutional-neural-networks/),” arXiv:1509.09308, 2015.
34. S. J. Hanson and L. Y. Pratt, “Comparing biases for minimal network construction with back-propagation,” in NIPS, 1989. [find](https://scholar.google.com/scholar?q=S%2E+J%2E+Hanson+and+L%2E+Y%2E+Pratt%2C+%E2%80%9CComparing+biases+for+minimal+network+construction+with+back%2Dpropagation%2C%E2%80%9D+in+NIPS%2C+1989%2E)
35. Y. LeCun, J. S. Denker, S. A. Solla, R. E. Howard, and L. D. Jackel, “Optimal brain damage.” in NIPs, vol. 89, 1989. [find](https://scholar.google.com/scholar?q=Y%2E+LeCun%2C+J%2E+S%2E+Denker%2C+S%2E+A%2E+Solla%2C+R%2E+E%2E+Howard%2C+and+L%2E+D%2E+Jackel%2C+%E2%80%9COptimal+brain+damage%2E%E2%80%9D+in+NIPs%2C+vol%2E+89%2C+1989%2E)
36. B. Hassibi, D. G. Stork et al., “Second order derivatives for network pruning: Optimal brain surgeon,” Advances in neural information processing systems, pp. 164–164, 1993. [find](https://scholar.google.com/scholar?q=B%2E+Hassibi%2C+D%2E+G%2E+Stork+et+al%2E%2C+%E2%80%9CSecond+order+derivatives+for+network+pruning%3A+Optimal+brain+surgeon%2C%E2%80%9D+Advances+in+neural+information+processing+systems%2C+pp%2E+164%E2%80%93164%2C+1993%2E)
37. E. L. Denton, W. Zaremba, J. Bruna, Y. LeCun, and R. Fergus, “Exploiting linear structure within convolutional networks for efficient evaluation,” in NIPS 2014. [find](https://scholar.google.com/scholar?q=E%2E+L%2E+Denton%2C+W%2E+Zaremba%2C+J%2E+Bruna%2C+Y%2E+LeCun%2C+and+R%2E+Fergus%2C+%E2%80%9CExploiting+linear+structure+within+convolutional+networks+for+efficient+evaluation%2C%E2%80%9D+in+NIPS+2014%2E)
38. X. Zhang, J. Zou, X. Ming, K. He, and J. Sun, “Efficient and accurate approximations of nonlinear convolutional networks,” arXiv:1411.4229, 2014. [find](https://scholar.google.com/scholar?q=X%2E+Zhang%2C+J%2E+Zou%2C+X%2E+Ming%2C+K%2E+He%2C+and+J%2E+Sun%2C+%E2%80%9CEfficient+and+accurate+approximations+of+nonlinear+convolutional+networks%2C%E2%80%9D+arXiv%3A1411%2E4229%2C+2014%2E)
39. B. Reagen, P. Whatmough, R. Adolf, S. Rama, H. Lee, S. K. Lee, J. M. Hernndez-Lobato, G.-Y. Wei, and D. Brooks, “Minerva: Enabling low-power, highly-accurate deep neural network accelerators,” ISCA, 2016. [find](https://scholar.google.com/scholar?q=B%2E+Reagen%2C+P%2E+Whatmough%2C+R%2E+Adolf%2C+S%2E+Rama%2C+H%2E+Lee%2C+S%2E+K%2E+Lee%2C+J%2E+M%2E+Hernndez%2DLobato%2C+G%2E%2DY%2E+Wei%2C+and+D%2E+Brooks%2C+%E2%80%9CMinerva%3A+Enabling+low%2Dpower%2C+highly%2Daccurate+deep+neural+network+accelerators%2C%E2%80%9D+ISCA%2C+2)
40. S. K. Esser and et al., “Convolutional networks for fast, energy-efficient neuromorphic computing,” arXiv:1603.08270, 2016. — [arXiv:1603.08270](https://arxiv.org/abs/1603.08270) [find](https://scholar.google.com/scholar?q=S%2E+K%2E+Esser+and+et+al%2E%2C+%E2%80%9CConvolutional+networks+for+fast%2C+energy%2Defficient+neuromorphic+computing%2C%E2%80%9D+arXiv%3A1603%2E08270%2C+2016%2E+%E2%80%94+%5BarXiv%3A1603%2E08270%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F1603%2E08270%29)
41. C. Zhang, P. Li, G. Sun, Y. Guan, B. Xiao, and J. Cong, “Optimizing fpga-based accelerator design for deep convolutional neural networks,” in FPGA, 2015. [find](https://scholar.google.com/scholar?q=C%2E+Zhang%2C+P%2E+Li%2C+G%2E+Sun%2C+Y%2E+Guan%2C+B%2E+Xiao%2C+and+J%2E+Cong%2C+%E2%80%9COptimizing+fpga%2Dbased+accelerator+design+for+deep+convolutional+neural+networks%2C%E2%80%9D+in+FPGA%2C+2015%2E)
42. Alexander Monakov and Anton Lokhmotov and Arutyun Avetisyan, “Automatically tuning sparse matrix-vector multiplication for GPU architectures,” in HiPEAC, 2010. [find](https://scholar.google.com/scholar?q=Alexander+Monakov+and+Anton+Lokhmotov+and+Arutyun+Avetisyan%2C+%E2%80%9CAutomatically+tuning+sparse+matrix%2Dvector+multiplication+for+GPU+architectures%2C%E2%80%9D+in+HiPEAC%2C+2010%2E)
43. N. Bell and M. Garland, “Efficient sparse matrix-vector multiplication on cuda,” Nvidia Technical Report NVR-2008-004, Tech. Rep., 2008. [find](https://scholar.google.com/scholar?q=N%2E+Bell+and+M%2E+Garland%2C+%E2%80%9CEfficient+sparse+matrix%2Dvector+multiplication+on+cuda%2C%E2%80%9D+Nvidia+Technical+Report+NVR%2D2008%2D004%2C+Tech%2E+Rep%2E%2C+2008%2E)
44. Bell, Nathan and Garland, Michael, “Implementing Sparse Matrix-vector Multiplication on Throughput-oriented Processors,” in High Performance Computing Networking, Storage and Analysis, 2009. [find](https://scholar.google.com/scholar?q=Bell%2C+Nathan+and+Garland%2C+Michael%2C+%E2%80%9CImplementing+Sparse+Matrix%2Dvector+Multiplication+on+Throughput%2Doriented+Processors%2C%E2%80%9D+in+High+Performance+Computing+Networking%2C+Storage+and+Analysis%2C+2009%2E)
45. J. Fowers and K. Ovtcharov and K. Strauss and E.S. Chung and G. Stitt, “A high memory bandwidth fpga accelerator for sparse matrix-vector multiplication,” in FCCM, 2014. [find](https://scholar.google.com/scholar?q=J%2E+Fowers+and+K%2E+Ovtcharov+and+K%2E+Strauss+and+E%2ES%2E+Chung+and+G%2E+Stitt%2C+%E2%80%9CA+high+memory+bandwidth+fpga+accelerator+for+sparse+matrix%2Dvector+multiplication%2C%E2%80%9D+in+FCCM%2C+2014%2E)

*Updated 2 November 2026*