---
title: "Dissecting the NVIDIA Volta GPU Architecture via"
description: "Much of the hardware's low-level behaviour is not publicly documented, so the authors use… 19 references, cited by 109 papers here."
authors: ["Zhe Jia", "Marco Maggioni", "Benjamin Staiger", "Daniele P. Scarpazza"]
year: 2018
venue: "arXiv technical report"
source: "https://arxiv.org/abs/1804.06826"
arxiv: "1804.06826"
licence: "arXiv — non-exclusive distribution license (arXiv OAI)"
licence_url: "http://arxiv.org/licenses/nonexclusive-distrib/1.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/jia-2018-dissecting-nvidia-volta-gpu-architecture/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/jia-2018-dissecting-nvidia-volta-gpu-architecture/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2018](https://md.research.elamur.ai/free/en/papers/2018/page/2/) / Dissecting the NVIDIA Volta GPU Architecture via Microbenchmarking

# Dissecting the NVIDIA Volta GPU Architecture via Microbenchmarking

**Authors:** Zhe Jia, Marco Maggioni, Benjamin Staiger and Daniele P. Scarpazza · **Year:** 2018 · **Venue:** arXiv technical report · **Licence:** [arXiv — non-exclusive distribution license (arXiv OAI)](http://arxiv.org/licenses/nonexclusive-distrib/1.0/)

**Links:** [Publisher's page](https://doi.org/10.4230/oasics.wcet.2024.3) · [PDF](https://arxiv.org/pdf/1804.06826) · [arXiv 1804.06826](https://arxiv.org/abs/1804.06826) · [OpenAlex](https://openalex.org/W2796649226)

**Citations:** 212 citations (OpenAlex, 2026-10-14) · [109](https://md.research.elamur.ai/free/en/papers/jia-2018-dissecting-nvidia-volta-gpu-architecture/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
This technical report from the High Performance Computing R&D Team at Citadel (Chicago) examines NVIDIA's Volta GPU architecture. Much of the hardware's low-level behaviour is not publicly documented, so the authors use microbenchmarking: small, targeted test programs that expose hidden properties of the chip. Based on its table of contents, the study covers four areas. The first is instruction-level details: control information, the scheduler and instruction encoding. The second is the full memory hierarchy, from register banks to TLBs. The third is the latency and throughput of native instructions, atomics and tensor cores. The fourth is Volta boards with NVLink. An opening chapter explains why such details matter to performance-minded programmers. The supplied extract contains only the front matter and table of contents, so measured values and final conclusions cannot be summarised here. The report matters because it gives developers of highly tuned GPU code an empirical map of Volta's internals that vendor documentation does not provide.

## Key points
- The authors are Zhe Jia, Marco Maggioni, Benjamin Staiger and Daniele P. Scarpazza (Citadel). The report is arXiv:1804.06826 (cs.DC), first edition, dated April 18, 2018. Zhe Jia previewed parts of it at the 2018 GPU Technology Conference in San Jose.
- The instruction chapter covers control information embedded in instructions, scheduler behaviour and the encoding of instructions.
- The memory-hierarchy chapter has eight parts:
  - L1 data cache
  - L2 cache
  - instruction caches
  - constant memory and cache
  - register banks
  - shared memory
  - global memory
  - TLBs
- The performance chapter measures native instructions, atomic operations, tensor cores and overall floating-point performance.
- A separate chapter studies NVLink-enabled Volta boards. It covers peer-to-peer and host communication, arithmetic throughput, memory bandwidth and latency.

## Interesting ideas
- **Reverse-engineering through microbenchmarks.** Narrowly scoped probes can reveal properties a vendor does not document. The same approach could characterise any opaque platform, such as cloud instances, accelerators or storage systems, before tuning workloads for it.
- **Layer-by-layer memory study.** Measuring each cache level, register banks and TLBs separately shows where data-access bottlenecks occur. This is useful for anyone optimising memory-bound code, including large NLP model inference.
- **Separating latency from throughput.** Reporting both for instructions, atomics and tensor cores helps decide whether code is limited by dependency chains or by issue capacity. This distinction applies broadly in performance engineering.
- **Starting with "why it matters".** Opening with motivation before the low-level measurements makes dense hardware findings easier to act on. The same structure works well for internal technical reports.
- **Versioned, living report.** Releasing numbered editions on arXiv lets findings grow as new hardware or insights appear. A shared research library could adopt this pattern for evolving internal documentation.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Institut de Recherche en Informatique de Toulouse](https://md.research.elamur.ai/free/en/labs/institut-de-recherche-en-informatique-de-toulouse/) — Research institute, Toulouse, France · 36 works · 2,193 citations

Also on the byline: [Office National d'Études et de Recherches Aérospatiales](https://ror.org/005y2ap84).

📚 Shelves (3): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 18,614 works · [🔌 Hardware and Architecture](https://md.research.elamur.ai/free/en/shelves/computer-science/hardware-and-architecture/) · 167 works · [🖥️ Computer Graphics and Computer-Aided Design](https://md.research.elamur.ai/free/en/shelves/computer-science/computer-graphics-and-computer-aided-design/) · 16 works

🧭 Topics (1): [⚡ GPU & parallel computing](https://md.research.elamur.ai/free/en/topics/gpu-parallel-computing/) · 41 works

## Cite

Cite as: Zhe Jia, Marco Maggioni, Benjamin Staiger and Daniele P. Scarpazza. 2018. Dissecting the NVIDIA Volta GPU Architecture via Microbenchmarking. arXiv technical report.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@misc{jia2018dissecting,
  title = {{Dissecting the NVIDIA Volta GPU Architecture via Microbenchmarking}},
  author = {Zhe Jia, Marco Maggioni, Benjamin Staiger and Daniele P. Scarpazza},
  year = {2018},
  doi = {10.4230/oasics.wcet.2024.3},
  eprint = {1804.06826},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.4230/oasics.wcet.2024.3}
}
```

- [← previous Attaining the Unattainable? Reassessing Claims of Human Parity in Neural Machine Translation](https://md.research.elamur.ai/free/en/papers/toral-2018-attaining-unattainable-reassessing-claims-human/)
- [next → The Adverse Effects of Code Duplication in Machine Learning Models of Code](https://md.research.elamur.ai/free/en/papers/allamanis-2018-adverse-effects-code-duplication-machine/)
- [↑ back to papers of 2018](https://md.research.elamur.ai/free/en/papers/2018/page/2/)

## References

19 references, 1 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. E. Lindholm, J. Nickolls, S. Oberman, and J. Montrym, “NVIDIA Tesla: A unified graphics and computing architecture,” IEEE Micro, vol. 28, no. 2, pp. 39–55, Mar. 2008. [Online]. Available: http://dx.doi.org/10.1109/MM.2008.31 [↗](http://dx.doi.org/10.1109/MM.2008.31)
2. NVIDIA’s Next Generation CUDA Compute Architecture: Fermi. Corporation, 2009. [find](https://scholar.google.com/scholar?q=NVIDIA%E2%80%99s+Next+Generation+CUDA+Compute+Architecture%3A+Fermi%2E+Corporation%2C+2009%2E)
3. Kepler GK110 Whitepaper. NVIDIA Corporation, 2012. [find](https://scholar.google.com/scholar?q=Kepler+GK110+Whitepaper%2E+NVIDIA+Corporation%2C+2012%2E)
4. GeForce GTX 980 - Featuring Maxwell, The Most Advanced GPU Ever Made, Whitepaper. NVIDIA Corporation, 2014. [find](https://scholar.google.com/scholar?q=GeForce+GTX+980+%2D+Featuring+Maxwell%2C+The+Most+Advanced+GPU+Ever+Made%2C+Whitepaper%2E+NVIDIA+Corporation%2C+2014%2E)
5. Pascal GP100 Whitepaper. NVIDIA Corporation, 2016. [find](https://scholar.google.com/scholar?q=Pascal+GP100+Whitepaper%2E+NVIDIA+Corporation%2C+2016%2E)
6. [Nvidia Tesla V100 GPU Architecture](https://md.research.elamur.ai/free/en/papers/nvidia-2017-nvidia-tesla-v100-gpu-architecture/), The World’s Most Advanced Data Center GPU. NVIDIA Corporation, 2017.
7. Y. Hou, “Asfermi,” 2011. [Online]. Available: [https://github.com/hyqneuron/asfermi](https://github.com/hyqneuron/asfermi) [↗](https://github.com/hyqneuron/asfermi)
8. C. Wang, Z. Jia, and K. Chen, “Tuning performance on Kepler GPUs: An introduction to Kepler assembler and its usage in CNN optimization,” in GPU Technology Conference Presentation, vol. 6173, 2015. [find](https://scholar.google.com/scholar?q=C%2E+Wang%2C+Z%2E+Jia%2C+and+K%2E+Chen%2C+%E2%80%9CTuning+performance+on+Kepler+GPUs%3A+An+introduction+to+Kepler+assembler+and+its+usage+in+CNN+optimization%2C%E2%80%9D+in+GPU+Technology+Conference+Presentation%2C+vol%2E+6173%2C+2015%2E)
9. S. Gray, “maxas,” 2016. [Online]. Available: [https://github.com/NervanaSystems/maxas/wiki/Control-Codes](https://github.com/NervanaSystems/maxas/wiki/Control-Codes) [↗](https://github.com/NervanaSystems/maxas/wiki/Control-Codes)
10. H. Wong, M. M. Papadopoulou, M. Sadooghi-Alvandi, and A. Moshovos, “Demystifying GPU microarchitecture through microbenchmarking,” in 2010 IEEE International Symposium on Performance Analysis of Systems Software (ISPASS), March 2010, pp. 235–246. [find](https://scholar.google.com/scholar?q=H%2E+Wong%2C+M%2E+M%2E+Papadopoulou%2C+M%2E+Sadooghi%2DAlvandi%2C+and+A%2E+Moshovos%2C+%E2%80%9CDemystifying+GPU+microarchitecture+through+microbenchmarking%2C%E2%80%9D+in+2010+IEEE+International+Symposium+on+Performance+Analysis+of+Syste)
11. X. Zhang, G. Tan, S. Xue, J. Li, K. Zhou, and M. Chen, “Understanding the GPU microarchitecture to achieve bare-metal performance tuning,” in Proceedings of the 22nd ACM SIGPLAN Symposium on Principles and NVIDIA Practice of Parallel Programming, ser. PPoPP ’17. ACM, 2017, pp. 31–43. [find](https://scholar.google.com/scholar?q=X%2E+Zhang%2C+G%2E+Tan%2C+S%2E+Xue%2C+J%2E+Li%2C+K%2E+Zhou%2C+and+M%2E+Chen%2C+%E2%80%9CUnderstanding+the+GPU+microarchitecture+to+achieve+bare%2Dmetal+performance+tuning%2C%E2%80%9D+in+Proceedings+of+the+22nd+ACM+SIGPLAN+Symposium+on+Principle)
12. X. Mei and X. Chu, “Dissecting GPU memory hierarchy through microbenchmarking,” IEEE Transactions on Parallel and Distributed Systems, vol. 28, no. 1, pp. 72–86, Jan 2017. [find](https://scholar.google.com/scholar?q=X%2E+Mei+and+X%2E+Chu%2C+%E2%80%9CDissecting+GPU+memory+hierarchy+through+microbenchmarking%2C%E2%80%9D+IEEE+Transactions+on+Parallel+and+Distributed+Systems%2C+vol%2E+28%2C+no%2E+1%2C+pp%2E+72%E2%80%9386%2C+Jan+2017%2E)
13. A. Lavin, “maxDNN: an efficient convolution kernel for deep learning with Maxwell GPUs,” arXiv preprint arXiv:1501.06633, 2015. [find](https://scholar.google.com/scholar?q=A%2E+Lavin%2C+%E2%80%9CmaxDNN%3A+an+efficient+convolution+kernel+for+deep+learning+with+Maxwell+GPUs%2C%E2%80%9D+arXiv+preprint+arXiv%3A1501%2E06633%2C+2015%2E)
14. G. Tan, L. Li, S. Triechle, E. Phillips, Y. Bao, and N. Sun, “Fast implementation of DGEMM on Fermi GPU,” in Proceedings of 2011 International Conference for High Performance Computing, Networking, Storage and Analysis. ACM, 2011, p. 35. [find](https://scholar.google.com/scholar?q=G%2E+Tan%2C+L%2E+Li%2C+S%2E+Triechle%2C+E%2E+Phillips%2C+Y%2E+Bao%2C+and+N%2E+Sun%2C+%E2%80%9CFast+implementation+of+DGEMM+on+Fermi+GPU%2C%E2%80%9D+in+Proceedings+of+2011+International+Conference+for+High+Performance+Computing%2C+Networking%2C+St)
15. J. Kurzak, S. Tomov, and J. Dongarra, “Autotuning GEMM kernels for the Fermi GPU,” IEEE Transactions on Parallel and Distributed Systems, vol. 23, no. 11, pp. 2045–2057, 2012. [find](https://scholar.google.com/scholar?q=J%2E+Kurzak%2C+S%2E+Tomov%2C+and+J%2E+Dongarra%2C+%E2%80%9CAutotuning+GEMM+kernels+for+the+Fermi+GPU%2C%E2%80%9D+IEEE+Transactions+on+Parallel+and+Distributed+Systems%2C+vol%2E+23%2C+no%2E+11%2C+pp%2E+2045%E2%80%932057%2C+2012%2E)
16. R. Nath, S. Tomov, and J. Dongarra, “An improved MAGMA GEMM for Fermi graphics processing units,” The International Journal of High Performance Computing Applications, vol. 24, no. 4, pp. 511–515, 2010. [find](https://scholar.google.com/scholar?q=R%2E+Nath%2C+S%2E+Tomov%2C+and+J%2E+Dongarra%2C+%E2%80%9CAn+improved+MAGMA+GEMM+for+Fermi+graphics+processing+units%2C%E2%80%9D+The+International+Journal+of+High+Performance+Computing+Applications%2C+vol%2E+24%2C+no%2E+4%2C+pp%2E+511%E2%80%93515%2C+201)
17. J. Lai and A. Seznec, “Performance upper bound analysis and optimization of SGEMM on Fermi and Kepler GPUs,” in Code Generation and Optimization (CGO), 2013 IEEE/ACM International Symposium on. IEEE, 2013, pp. 1–10. [find](https://scholar.google.com/scholar?q=J%2E+Lai+and+A%2E+Seznec%2C+%E2%80%9CPerformance+upper+bound+analysis+and+optimization+of+SGEMM+on+Fermi+and+Kepler+GPUs%2C%E2%80%9D+in+Code+Generation+and+Optimization+%28CGO%29%2C+2013+IEEE%2FACM+International+Symposium+on%2E+IEEE%2C+)
18. “Nvidia cuobjdump and nvdisasm,” 2016. [Online]. Available: [https://docs.nvidia.com/cuda/cuda-binary-utilities/](https://docs.nvidia.com/cuda/cuda-binary-utilities/) [↗](https://docs.nvidia.com/cuda/cuda-binary-utilities/)
19. Nvidia Tesla V100 GPU Accelerator. BIBLIOGRAPHY New York, NY, USA: NVIDIA Corporation, 2017. [find](https://scholar.google.com/scholar?q=Nvidia+Tesla+V100+GPU+Accelerator%2E+BIBLIOGRAPHY+New+York%2C+NY%2C+USA%3A+NVIDIA+Corporation%2C+2017%2E)

*Updated 2 November 2026*