--- title: "Rewon Child — author profile and papers" description: "Rewon Child, Google: 2 papers on Elamur Research (2020–2022), h-index 11 — affiliations, topics, co-authors and who cites them in our library." canonical: "https://research.elamur.ai/free/en/authors/rewon-child/" --- [Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/authors/rewon-child/) [home](https://md.research.elamur.ai/free/en/) / [authors](https://md.research.elamur.ai/free/en/authors/) / Rewon Child # Rewon Child Rewon Child works on large neural [language models](https://md.research.elamur.ai/free/en/topics/language-models/), asking how such models behave as they grow and how they can be trained at very large scale. The papers here concern autoregressive Transformer models for [text generation](https://md.research.elamur.ai/free/en/topics/text-generation/). They pair empirical study of model quality with the engineering of distributed systems that split one model across many GPUs. The most cited of these works is [Scaling Laws for Neural Language Models](https://md.research.elamur.ai/free/en/papers/kaplan-2020-scaling-laws-neural-language-models/) (2020). It trained a wide range of Transformers on the WebText2 corpus, changing the size of the model and of the training set, how long training lasted, the batch size and the architecture's shape, and compared them against LSTMs and recurrent variants. Test loss turned out to drop along simple power-law curves in parameter count, data and compute, each taken on its own, provided neither of the remaining factors held progress back. The second paper, [Using DeepSpeed and Megatron to Train Megatron-Turing NLG 530B](https://md.research.elamur.ai/free/en/papers/smith-2022-using-deepspeed-megatron-train-megatron/) (2022), reports a combined Microsoft and NVIDIA project to train a very large dense left-to-right Transformer generator. It joins Megatron-LM with DeepSpeed in a three-way parallel design: copies of the model process different data with optimizer state sharded through ZeRO, layers are staged across devices in a pipeline, and individual tensors are split among GPUs that share one DGX node. The more recent work shifts from measuring how loss scales toward the systems side of training models too large for a single accelerator's memory, where software and hardware must be planned together to keep training time within reach. Child's most recent affiliation is [Google](https://md.research.elamur.ai/free/en/companies/google/), with earlier bylines at [OpenAI](https://md.research.elamur.ai/free/en/companies/openai/) around 2020 and at [Baidu](https://md.research.elamur.ai/free/en/companies/baidu/) in 2017. Co-authors on the scaling study include [Jared Kaplan](https://md.research.elamur.ai/free/en/authors/jared-kaplan/), [Sam McCandlish](https://md.research.elamur.ai/free/en/authors/sam-mccandlish/) and Dario Amodei. The Megatron-Turing paper was written with [Mostofa Patwary](https://md.research.elamur.ai/free/en/authors/mostofa-patwary/), Jared Casper and Samyam Rajbhandari, among others. Rewon Child has 2 papers in our library, in 2020–2022, most of them on [Language models & pretraining](https://md.research.elamur.ai/free/en/topics/language-models/), [Text generation](https://md.research.elamur.ai/free/en/topics/text-generation/); often with [Alec Radford](https://md.research.elamur.ai/free/en/authors/alec-radford/), [Dario Amodei](https://md.research.elamur.ai/free/en/authors/dario-amodei/), [Jared Casper](https://md.research.elamur.ai/free/en/authors/jared-casper/). 2 papers in our library · 2020–2022 current affiliation [Google](https://md.research.elamur.ai/free/en/companies/google/) · Mountain View United States [OpenAlex](https://openalex.org/A5084512458) · [**h-index 11**](https://openalex.org/authors/A5084512458) · [14 works](https://openalex.org/works?filter=author.id:A5084512458) · [8,090 citations](https://openalex.org/works?filter=author.id:A5084512458&sort=cited_by_count:desc) OpenAlex, 2026-10-14 [2 works in our library](https://md.research.elamur.ai/free/en/authors/rewon-child/#papers) · [2,123 citations in our library](https://md.research.elamur.ai/free/en/authors/rewon-child/cited-by/) · [⭐ Paper Star rank #1019](https://md.research.elamur.ai/free/en/paper-stars/page/11/#star-1019) The profile is the library's match of the name; two people with one name are not always told apart. Elsewhere: [Google Scholar](https://scholar.google.com/citations?view_op=search_authors&mauthors=Rewon+Child) · [Semantic Scholar](https://www.semanticscholar.org/search?q=Rewon+Child) · [dblp](https://dblp.org/search?q=Rewon+Child) · [ACL Anthology](https://aclanthology.org/search/?q=Rewon+Child) ## Affiliations [🏢**Google** Mountain View · United States 2022 · 1 paper*current*](https://md.research.elamur.ai/free/en/companies/google/) · [🏢**OpenAI** San Francisco · United States 2020 · 4 papers](https://md.research.elamur.ai/free/en/companies/openai/) · [🏢**Baidu** Beijing · China 2017 · 3 papers](https://md.research.elamur.ai/free/en/companies/baidu/) From the bylines of their papers (OpenAlex, 2026-10-14), ORCID and Wikidata; institutions on papers are matched by machine. ## Topics [Language models & pretraining 2](https://md.research.elamur.ai/free/en/topics/language-models/) · [Text generation 1](https://md.research.elamur.ai/free/en/topics/text-generation/) ## Works by year ### 2020s [**2022** 1 paper](https://md.research.elamur.ai/free/en/papers/smith-2022-using-deepspeed-megatron-train-megatron/) · [**2020** 1 paper](https://md.research.elamur.ai/free/en/papers/kaplan-2020-scaling-laws-neural-language-models/) ## Papers The most cited first (OpenAlex and our library). 1. **[Scaling Laws for Neural Language Models](https://md.research.elamur.ai/free/en/papers/kaplan-2020-scaling-laws-neural-language-models/)** *(2020 · 1,814 citations)* The authors from OpenAI study how the quality of a language model depends on its scale. They measure quality by the cross-entropy loss on test data. To this end, they trained a large series of Transformer models on the WebText2 corpus, ranging from thousands to roughly a billion parameters, excluding embeddings. In… 2. **[Using DeepSpeed and Megatron to Train Megatron-Turing NLG 530B, A Large-Scale Generative Language Model](https://md.research.elamur.ai/free/en/papers/smith-2022-using-deepspeed-megatron-train-megatron/)** *(2022 · 595 citations)* Large language models keep growing, and training them hits two limits. Their parameters no longer fit in one GPU's memory, and without careful co-design of software and hardware the training time becomes impractical. In a joint Microsoft–NVIDIA effort, the authors describe how they trained Megatron-Turing NLG 530B… ## Co-authors here 12 of 13 co-authors with a page, the most papers together first. - [Alec Radford](https://md.research.elamur.ai/free/en/authors/alec-radford/) - [Dario Amodei](https://md.research.elamur.ai/free/en/authors/dario-amodei/) - [Jared Casper](https://md.research.elamur.ai/free/en/authors/jared-casper/) - [Jared Kaplan](https://md.research.elamur.ai/free/en/authors/jared-kaplan/) - [Mostofa Patwary](https://md.research.elamur.ai/free/en/authors/mostofa-patwary/) - [Sam McCandlish](https://md.research.elamur.ai/free/en/authors/sam-mccandlish/) - [Samyam Rajbhandari](https://md.research.elamur.ai/free/en/authors/samyam-rajbhandari/) - [Scott Gray](https://md.research.elamur.ai/free/en/authors/scott-gray/) - [Shrimai Prabhumoye](https://md.research.elamur.ai/free/en/authors/shrimai-prabhumoye/) - [Tom B. Brown](https://md.research.elamur.ai/free/en/authors/tom-b-brown/) - [Tom Henighan](https://md.research.elamur.ai/free/en/authors/tom-henighan/) - [Vijay Korthikanti](https://md.research.elamur.ai/free/en/authors/vijay-korthikanti/) ## Cited by Papers of our library cite their works 2,123 times (of the 140,884 papers whose bibliographies we read); [207 citing papers with a page here →](https://md.research.elamur.ai/free/en/authors/rewon-child/cited-by/).