---
title: "Deepak Narayanan · Nvidia — author profile and papers"
description: "Deepak Narayanan, Nvidia: 5 papers on Elamur Research (2018–2024), h-index 19 — affiliations, topics, co-authors and who cites them in…"
canonical: "https://research.elamur.ai/free/en/authors/deepak-narayanan/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/authors/deepak-narayanan/)

[home](https://md.research.elamur.ai/free/en/) / [authors](https://md.research.elamur.ai/free/en/authors/) / Deepak Narayanan

# Deepak Narayanan

Deepak Narayanan has 5 papers in our library, in 2018–2024, most of them on [Language models & pretraining](https://md.research.elamur.ai/free/en/topics/language-models/), [Efficient models & compression](https://md.research.elamur.ai/free/en/topics/efficient-models/); often with [Amar Phanishayee](https://md.research.elamur.ai/free/en/authors/amar-phanishayee/), [Matei Zaharia](https://md.research.elamur.ai/free/en/authors/matei-zaharia/), [Albert Gu](https://md.research.elamur.ai/free/en/authors/albert-gu/).

5 papers in our library · 2018–2024

[OpenAlex](https://openalex.org/A5079514101) · [ORCID](https://orcid.org/0000-0002-3020-2848) · [Wikidata](https://www.wikidata.org/wiki/Q131129545) · [**h-index 19**](https://openalex.org/authors/A5079514101) · [50 works](https://openalex.org/works?filter=author.id:A5079514101) · [5,580 citations](https://openalex.org/works?filter=author.id:A5079514101&sort=cited_by_count:desc) OpenAlex, 2026-10-14

[📄 5 works in our library](https://md.research.elamur.ai/free/en/authors/deepak-narayanan/#papers) · [🔗 Cited 3,829 times in our library](https://md.research.elamur.ai/free/en/authors/deepak-narayanan/cited-by/) · [5 more in our library without a page yet](#also) · [⭐ Paper Stars ranking: #4012](https://md.research.elamur.ai/free/en/paper-stars/page/41/#star-4012)

The profile is the library's match of the name; two people with one name are not always told apart.

Elsewhere: [Google Scholar](https://scholar.google.com/citations?view_op=search_authors&mauthors=Deepak+Narayanan) · [Semantic Scholar](https://www.semanticscholar.org/search?q=Deepak+Narayanan) · [dblp](https://dblp.org/search?q=Deepak+Narayanan) · [ACL Anthology](https://aclanthology.org/search/?q=Deepak+Narayanan)

## Affiliations

[🏢**Nvidia** Santa Clara · United States 2023–2026 · 5 papers*current*](https://md.research.elamur.ai/free/en/companies/nvidia/) · [🏢**Nvidia** Reading · United Kingdom 2023–2026 · 4 papers*current*](https://md.research.elamur.ai/free/en/companies/nvidia-united-kingdom/) · [🎓**Stanford University** Stanford · United States 2017–2022 · 13 papers](https://md.research.elamur.ai/free/en/universities/stanford-university/) · [🏢**Microsoft** Redmond · United States 2020–2021 · 3 papers](https://md.research.elamur.ai/free/en/companies/microsoft/)

From the bylines of their papers (OpenAlex, 2026-10-14), ORCID and Wikidata; institutions on papers are matched by machine. 3 more institutions on single bylines are not listed.

## Fields, topics and concepts

Of its works in our library, with the number of works on each.

📚 Shelves 8 The field of science this work belongs to (OpenAlex fields).[Computer Science · 5 works](https://md.research.elamur.ai/free/en/shelves/computer-science/) · [Artificial Intelligence · 4 works](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · [Computer Vision and Pattern Recognition · 4 works](https://md.research.elamur.ai/free/en/shelves/computer-science/computer-vision-and-pattern-recognition/) · [Hardware and Architecture · 3 works](https://md.research.elamur.ai/free/en/shelves/computer-science/hardware-and-architecture/) · [Arts and Humanities · 2 works](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · [Language and Linguistics · 2 works](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · [Computer Networks and Communications · 1 work](https://md.research.elamur.ai/free/en/shelves/computer-science/computer-networks-and-communications/) · [Information Systems · 1 work](https://md.research.elamur.ai/free/en/shelves/computer-science/information-systems/)

🧭 Topics 2 Research subjects and languages the work is about.[Efficient models & compression · 1 work](https://md.research.elamur.ai/free/en/topics/efficient-models/) · [Language models & pretraining · 1 work](https://md.research.elamur.ai/free/en/topics/language-models/)

## Works by year

### 2020s

[**2024** 2 papers](https://md.research.elamur.ai/free/en/authors/deepak-narayanan/year/2024/) · [**2020** 2 papers](https://md.research.elamur.ai/free/en/authors/deepak-narayanan/year/2020/)

### 2010s

[**2018** 1 paper](https://md.research.elamur.ai/free/en/authors/deepak-narayanan/year/2018/)

## Papers

The most cited first (OpenAlex and our library).

1. **[PipeDream: Fast and Efficient Pipeline Parallel DNN Training](https://md.research.elamur.ai/free/en/papers/harlap-2018-pipedream-fast-efficient-pipeline-parallel/)** *(2018 · 162 citations)* Data-parallel DNN training gives every GPU a full model copy and synchronizes weights after each step. As models grow and GPUs get faster, network traffic increasingly dominates runtime. For VGG16, it can take up to 85% of training time. Classic model parallelism avoids exchanging all weights but leaves most workers…
2. **[Memory-Efficient Pipeline-Parallel DNN Training](https://md.research.elamur.ai/free/en/papers/narayanan-2020-memory-efficient-pipeline-parallel-dnn/)** *(2020 · 153 citations)* Very large language models often cannot fit their weights and activations on one accelerator, so training must be spread across devices.
3. **[Heterogeneity-Aware Cluster Scheduling Policies for Deep Learning Workloads](https://md.research.elamur.ai/free/en/papers/narayanan-2020-heterogeneity-aware-cluster-scheduling-policies/)** *(2020 · 141 citations)* Deep learning training clusters now mix accelerators of different generations, and models benefit unequally from newer hardware.
4. **[An Empirical Study of Mamba-based Language Models](https://md.research.elamur.ai/free/en/papers/waleffe-2024-empirical-study-mamba-based-language/)** *(2024 · 67 citations)* Selective state-space models (SSMs) such as Mamba could make inference cheaper than in Transformers. They avoid attention's quadratic cost and the ever-growing key-value cache. However, earlier comparisons with identical training data covered only small models. This NVIDIA-led report addresses that gap. It trains…
5. **[Nemotron-4 15B Technical Report](https://md.research.elamur.ai/free/en/papers/parmar-2024-nemotron-4-15b-technical-report/)** *(2024 · 30 citations)* NVIDIA set out to build the strongest general-purpose LLM that fits on a single A100 or H100 GPU. Following the Chinchilla-inspired idea of spending compute on more data rather than more parameters, the authors trained a 15B-parameter decoder-only Transformer on 8 trillion tokens. The mix was about 70% English, 15%…

## All works

10 works of theirs in our library, the most cited first; 5 with a page here, the others link their original until we summarise them.

1. **[BLOOM: A 176B-Parameter Open-Access Multilingual Language Model ↗](https://arxiv.org/abs/2211.05100) arXiv:2211.05100 · no summary yet**BigScience Workshop et al.*2022 · 1,606 citations*
2. **[Holistic Evaluation of Language Models ↗](https://arxiv.org/abs/2211.09110) arXiv:2211.09110 · Published in Transactions on Machine Learning Research (TMLR), 2023 · no summary yet**Percy Liang et al.*2022 · 1,001 citations*
3. [**PipeDream: Fast and Efficient Pipeline Parallel DNN Training arXiv:1806.03377**Aaron Harlap, Deepak Narayanan, Amar Phanishayee, et al.*2018 · 162 citations*](https://md.research.elamur.ai/free/en/papers/harlap-2018-pipedream-fast-efficient-pipeline-parallel/)
4. **[Memory-Efficient Pipeline-Parallel DNN Training](https://md.research.elamur.ai/free/en/papers/narayanan-2020-memory-efficient-pipeline-parallel-dnn/) [International Conference on Machine Learning (ICML)](https://md.research.elamur.ai/free/en/venues/international-conference-on-machine-learning/)**Deepak Narayanan, Amar Phanishayee, Kaiyu Shi, et al.*2020 · 153 citations*
5. [**Heterogeneity-Aware Cluster Scheduling Policies for Deep Learning Workloads arXiv:2008.09213**Deepak Narayanan, Keshav Santhanam, Fiodar Kazhamiaka, et al.*2020 · 141 citations*](https://md.research.elamur.ai/free/en/papers/narayanan-2020-heterogeneity-aware-cluster-scheduling-policies/)
6. **[Holistic Evaluation of Text-to-Image Models ↗](https://doi.org/10.52202/075280-3067) neural information processing systems · no summary yet**Tony Lee et al.*2023 · 77 citations*
7. [**An Empirical Study of Mamba-based Language Models arXiv:2406.07887**Roger Waleffe, Wonmin Byeon, Duncan Riach, et al.*2024 · 67 citations*](https://md.research.elamur.ai/free/en/papers/waleffe-2024-empirical-study-mamba-based-language/)
8. [**Nemotron-4 15B Technical Report arXiv:2402.16819**Jupinder Parmar, Shrimai Prabhumoye, Joseph Jennings, et al.*2024 · 30 citations*](https://md.research.elamur.ai/free/en/papers/parmar-2024-nemotron-4-15b-technical-report/)
9. **[Cheaply Evaluating Inference Efficiency Metrics for Autoregressive Transformer APIs ↗](https://arxiv.org/abs/2305.02440) arXiv (Cornell University) · no summary yet**Deepak Narayanan et al.*2023 · 6 citations*
10. **[Cheaply Estimating Inference Efficiency Metrics for Autoregressive Transformer Models ↗](https://doi.org/10.52202/075280-2904) neural information processing systems · no summary yet**Deepak Narayanan et al.*2023 · 3 citations*

## Co-authors here

12 of 18 co-authors with a page, the most papers together first.

- [Amar Phanishayee](https://md.research.elamur.ai/free/en/authors/amar-phanishayee/) ×3
- [Matei Zaharia](https://md.research.elamur.ai/free/en/authors/matei-zaharia/) ×2
- [Albert Gu](https://md.research.elamur.ai/free/en/authors/albert-gu/)
- [Ayush Dattagupta](https://md.research.elamur.ai/free/en/authors/ayush-dattagupta/)
- [Brandon Norick](https://md.research.elamur.ai/free/en/authors/brandon-norick/)
- [Chen Zhu](https://md.research.elamur.ai/free/en/authors/chen-zhu/)
- [Dan Su](https://md.research.elamur.ai/free/en/authors/dan-su/)
- [Joseph Jennings](https://md.research.elamur.ai/free/en/authors/joseph-jennings/)
- [Jupinder Parmar](https://md.research.elamur.ai/free/en/authors/jupinder-parmar/)
- [Keshav Santhanam](https://md.research.elamur.ai/free/en/authors/keshav-santhanam/)
- [Mostofa Patwary](https://md.research.elamur.ai/free/en/authors/mostofa-patwary/)
- [Sandeep Subramanian](https://md.research.elamur.ai/free/en/authors/sandeep-subramanian/)

## Venues

Where their works here were published, the most works first, with the editions of those that have a page.

- neural information processing systems ×2
- [International Conference on Machine Learning (ICML)](https://md.research.elamur.ai/free/en/venues/international-conference-on-machine-learning/): [ICML 2020](https://md.research.elamur.ai/free/en/venues/international-conference-on-machine-learning/2020/)

## Cited by

Papers of our library cite their works 3,829 times (of the 149,439 papers whose bibliographies we read); [802 citing papers with a page here →](https://md.research.elamur.ai/free/en/authors/deepak-narayanan/cited-by/).

Updated 2 November 2026