--- title: "Tom B. Brown — author profile and papers" description: "Tom B. Brown: 3 papers on Elamur Research (2020) — affiliations, topics, co-authors and who cites them in our library." canonical: "https://research.elamur.ai/free/en/authors/tom-b-brown/" --- [Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/authors/tom-b-brown/) [home](https://md.research.elamur.ai/free/en/) / [authors](https://md.research.elamur.ai/free/en/authors/) / Tom B. Brown # Tom B. Brown Tom B. Brown works on large neural language models and, more broadly, on how autoregressive generative models change as they grow. The papers here, all from 2020, ask what happens when Transformer models are made larger, trained on more data, and given more compute. They answer through extensive series of training runs, judged by loss on held-out data and by evaluation across many benchmark tasks. The work sits within [language models & pretraining](https://md.research.elamur.ai/free/en/topics/language-models/). The most cited is [Language Models are Few-Shot Learners](https://md.research.elamur.ai/free/en/papers/brown-2020-language-models-few-shot-learners/), which introduced GPT-3, a very large autoregressive transformer trained on filtered web text. It was tested on a wide range of tasks with no fine-tuning: an instruction and a handful of examples are placed in the prompt, a behaviour the authors call in-context learning. [Scaling Laws for Neural Language Models](https://md.research.elamur.ai/free/en/papers/kaplan-2020-scaling-laws-neural-language-models/) trained families of Transformers on WebText2 while varying model size, dataset size, training time, batch size and architecture shape. It found that loss falls along simple power laws in parameters, data and compute, as long as neither of the other two factors holds it back. The newest work carries the scaling question beyond text. [Scaling Laws for Autoregressive Generative Modeling](https://md.research.elamur.ai/free/en/papers/henighan-2020-scaling-laws-autoregressive-generative-modeling/) fits the same kind of curves to images, video, paired captions and images, and procedurally generated mathematics problems. In each domain the loss follows a smooth power law plus a constant, and the authors read that constant as an estimate of the inherent entropy of the data. Brown's papers here come from OpenAI research teams. Frequent co-authors include [Jared Kaplan](https://md.research.elamur.ai/free/en/authors/jared-kaplan/), [Tom Henighan](https://md.research.elamur.ai/free/en/authors/tom-henighan/), [Benjamin Mann](https://md.research.elamur.ai/free/en/authors/benjamin-mann/), [Prafulla Dhariwal](https://md.research.elamur.ai/free/en/authors/prafulla-dhariwal/) and [Scott Gray](https://md.research.elamur.ai/free/en/authors/scott-gray/). Tom B. Brown has 3 papers in our library, in 2020, most of them on [Language models & pretraining](https://md.research.elamur.ai/free/en/topics/language-models/); often with [Jared Kaplan](https://md.research.elamur.ai/free/en/authors/jared-kaplan/), [Benjamin Mann](https://md.research.elamur.ai/free/en/authors/benjamin-mann/), [Prafulla Dhariwal](https://md.research.elamur.ai/free/en/authors/prafulla-dhariwal/). 3 papers in our library · 2020 No OpenAlex profile matched yet. [📄 3 works in our library](https://md.research.elamur.ai/free/en/authors/tom-b-brown/#papers) · [🔗 12,711 citations in our library](https://md.research.elamur.ai/free/en/authors/tom-b-brown/cited-by/) · [⭐ Paper Star rank #135](https://md.research.elamur.ai/free/en/paper-stars/page/2/#star-135) The profile is the library's match of the name; two people with one name are not always told apart. Elsewhere: [Google Scholar](https://scholar.google.com/citations?view_op=search_authors&mauthors=Tom+B.+Brown) · [Semantic Scholar](https://www.semanticscholar.org/search?q=Tom+B.+Brown) · [dblp](https://dblp.org/search?q=Tom+B.+Brown) · [ACL Anthology](https://aclanthology.org/search/?q=Tom+B.+Brown) ## Topics [Language models & pretraining (2)](https://md.research.elamur.ai/free/en/topics/language-models/) ## Works by year ### 2020s [**2020** 3 papers](https://md.research.elamur.ai/free/en/authors/tom-b-brown/year/2020/) ## Papers The most cited first (OpenAlex and our library). 1. **[Language Models are Few-Shot Learners](https://md.research.elamur.ai/free/en/papers/brown-2020-language-models-few-shot-learners/)** *(2020 · 10,782 citations)* The standard approach in NLP has a significant drawback: after pre-training, a model is usually fine-tuned on thousands of labeled examples for each individual task. 2. **[Scaling Laws for Neural Language Models](https://md.research.elamur.ai/free/en/papers/kaplan-2020-scaling-laws-neural-language-models/)** *(2020 · 1,814 citations)* The authors from OpenAI study how the quality of a language model depends on its scale. They measure quality by the cross-entropy loss on test data. To this end, they trained a large series of Transformer models on the WebText2 corpus, ranging from thousands to roughly a billion parameters, excluding embeddings. In… 3. **[Scaling Laws for Autoregressive Generative Modeling](https://md.research.elamur.ai/free/en/papers/henighan-2020-scaling-laws-autoregressive-generative-modeling/)** *(2020 · 115 citations)* The authors from OpenAI examine whether the scaling laws previously found for language models also hold for other data types. ## Co-authors here 12 of 19 co-authors with a page, the most papers together first. - [Jared Kaplan](https://md.research.elamur.ai/free/en/authors/jared-kaplan/) ×3 - [Benjamin Mann](https://md.research.elamur.ai/free/en/authors/benjamin-mann/) ×2 - [Prafulla Dhariwal](https://md.research.elamur.ai/free/en/authors/prafulla-dhariwal/) ×2 - [Scott Gray](https://md.research.elamur.ai/free/en/authors/scott-gray/) ×2 - [Tom Henighan](https://md.research.elamur.ai/free/en/authors/tom-henighan/) ×2 - [Alec Radford](https://md.research.elamur.ai/free/en/authors/alec-radford/) - [Amanda Askell](https://md.research.elamur.ai/free/en/authors/amanda-askell/) - [Ariel Herbert-Voss](https://md.research.elamur.ai/free/en/authors/ariel-herbert-voss/) - [Arvind Neelakantan](https://md.research.elamur.ai/free/en/authors/arvind-neelakantan/) - [Chris Hallacy](https://md.research.elamur.ai/free/en/authors/chris-hallacy/) - [Christopher Hesse](https://md.research.elamur.ai/free/en/authors/christopher-hesse/) - [Dario Amodei](https://md.research.elamur.ai/free/en/authors/dario-amodei/) ## Cited by Papers of our library cite their works 12,711 times (of the 140,884 papers whose bibliographies we read); [871 citing papers with a page here →](https://md.research.elamur.ai/free/en/authors/tom-b-brown/cited-by/).