--- title: "Tom Henighan — author profile and papers" description: "Tom Henighan, SLAC National Accelerator Laboratory: 4 papers on Elamur Research (2020–2022), h-index 16 — affiliations, topics, co-authors and who cites…" canonical: "https://research.elamur.ai/free/en/authors/tom-henighan/" --- [Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/authors/tom-henighan/) [home](https://md.research.elamur.ai/free/en/) / [authors](https://md.research.elamur.ai/free/en/authors/) / Tom Henighan # Tom Henighan Tom Henighan studies how the behaviour of neural networks changes as they grow. Henighan's papers here centre on scaling laws: empirical relationships between test loss and model size, dataset size and compute budget. The usual method is to train families of transformer models across a wide span of scales and fit simple curves to the results. Recurring questions include how far these regularities extend beyond text, what pre-training is worth for a downstream task, and how data quality shapes learning. The most cited of these papers is [Scaling Laws for Neural Language Models](https://md.research.elamur.ai/free/en/papers/kaplan-2020-scaling-laws-neural-language-models/). It trained a broad series of Transformers on WebText2, varying size, data, training length, batch size and shape. It found that cross-entropy loss falls as a smooth power law in each of parameters, data and compute, as long as the other factors are not limiting. [Scaling Laws for Autoregressive Generative Modeling](https://md.research.elamur.ai/free/en/papers/henighan-2020-scaling-laws-autoregressive-generative-modeling/) carried this question over to images, video, caption–image pairs and generated mathematics. It reported power-law-plus-constant trends in every domain and read the constant as an estimate of the data's own entropy. The later work moves from describing scaling to explaining specific effects. [Scaling Laws for Transfer](https://md.research.elamur.ai/free/en/papers/hernandez-2021-scaling-laws-transfer/) measures the benefit of language-model pre-training on low-data tasks and expresses it as an equivalent amount of extra data. [Scaling Laws and Interpretability of Learning from Repeated Data](https://md.research.elamur.ai/free/en/papers/hernandez-2022-scaling-laws-interpretability-learning-repeated/) asks why repeating even a small share of training tokens damages models so much. It combines scaling analysis with mechanistic [interpretability](https://md.research.elamur.ai/free/en/topics/interpretability/) and finds a marked double-descent pattern. The listed current affiliations are SLAC National Accelerator Laboratory and [Stanford University](https://md.research.elamur.ai/free/en/universities/stanford-university/). The papers here came out of research groups at OpenAI and later Anthropic, within the area of [language models and pretraining](https://md.research.elamur.ai/free/en/topics/language-models/). Frequent co-authors include [Jared Kaplan](https://md.research.elamur.ai/free/en/authors/jared-kaplan/), [Danny Hernandez](https://md.research.elamur.ai/free/en/authors/danny-hernandez/), [Sam McCandlish](https://md.research.elamur.ai/free/en/authors/sam-mccandlish/) and [Tom B. Brown](https://md.research.elamur.ai/free/en/authors/tom-b-brown/). Tom Henighan has 4 papers in our library, in 2020–2022, most of them on [Language models & pretraining](https://md.research.elamur.ai/free/en/topics/language-models/), [Interpretability](https://md.research.elamur.ai/free/en/topics/interpretability/); often with [Jared Kaplan](https://md.research.elamur.ai/free/en/authors/jared-kaplan/), [Danny Hernandez](https://md.research.elamur.ai/free/en/authors/danny-hernandez/), [Sam McCandlish](https://md.research.elamur.ai/free/en/authors/sam-mccandlish/). 4 papers in our library · 2020–2022 [OpenAlex](https://openalex.org/A5036481974) · [ORCID](https://orcid.org/0000-0003-0745-370X) · [**h-index 16**](https://openalex.org/authors/A5036481974) · [36 works](https://openalex.org/works?filter=author.id:A5036481974) · [1,277 citations](https://openalex.org/works?filter=author.id:A5036481974&sort=cited_by_count:desc) OpenAlex, 2026-10-14 [📄 4 works in our library](https://md.research.elamur.ai/free/en/authors/tom-henighan/#papers) · [🔗 4,429 citations in our library](https://md.research.elamur.ai/free/en/authors/tom-henighan/cited-by/) · [3 more in our library without a page yet](#also) · [⭐ Paper Star rank #1146](https://md.research.elamur.ai/free/en/paper-stars/page/12/#star-1146) The profile is the library's match of the name; two people with one name are not always told apart. Elsewhere: [Google Scholar](https://scholar.google.com/citations?view_op=search_authors&mauthors=Tom+Henighan) · [Semantic Scholar](https://www.semanticscholar.org/search?q=Tom+Henighan) · [dblp](https://dblp.org/search?q=Tom+Henighan) · [ACL Anthology](https://aclanthology.org/search/?q=Tom+Henighan) ## Affiliations [🔬**SLAC National Accelerator Laboratory** Menlo Park · United States 2015–2023 · 14 papers*current*](https://ror.org/05gzmn429) · [🎓**Stanford University** Stanford · United States 2016–2023 · 9 papers*current*](https://md.research.elamur.ai/free/en/universities/stanford-university/) · [🎓**The Ohio State University** Columbus · United States 2009–2012 · 6 papers](https://md.research.elamur.ai/free/en/universities/the-ohio-state-university/) · [🎓**Delft University of Technology** Delft · Netherlands 2011–2014 · 4 papers](https://md.research.elamur.ai/free/en/universities/delft-university-of-technology/) From the bylines of their papers (OpenAlex, 2026-10-14), ORCID and Wikidata; institutions on papers are matched by machine. 2 more institutions on single bylines are not listed. ## Topics [Language models & pretraining (2)](https://md.research.elamur.ai/free/en/topics/language-models/) · [Interpretability (1)](https://md.research.elamur.ai/free/en/topics/interpretability/) ## Works by year ### 2020s [**2022** 1 paper](https://md.research.elamur.ai/free/en/papers/hernandez-2022-scaling-laws-interpretability-learning-repeated/) · [**2021** 1 paper](https://md.research.elamur.ai/free/en/papers/hernandez-2021-scaling-laws-transfer/) · [**2020** 2 papers](https://md.research.elamur.ai/free/en/authors/tom-henighan/year/2020/) ## Papers The most cited first (OpenAlex and our library). 1. **[Scaling Laws for Neural Language Models](https://md.research.elamur.ai/free/en/papers/kaplan-2020-scaling-laws-neural-language-models/)** *(2020 · 1,814 citations)* The authors from OpenAI study how the quality of a language model depends on its scale. They measure quality by the cross-entropy loss on test data. To this end, they trained a large series of Transformer models on the WebText2 corpus, ranging from thousands to roughly a billion parameters, excluding embeddings. In… 2. **[Scaling Laws for Autoregressive Generative Modeling](https://md.research.elamur.ai/free/en/papers/henighan-2020-scaling-laws-autoregressive-generative-modeling/)** *(2020 · 115 citations)* The authors from OpenAI examine whether the scaling laws previously found for language models also hold for other data types. 3. **[Scaling Laws for Transfer](https://md.research.elamur.ai/free/en/papers/hernandez-2021-scaling-laws-transfer/)** *(2021 · 102 citations)* This paper asks how much language-model pre-training is worth when the target task has little data. It answers in units of data. The authors trained transformers across roughly four orders of magnitude in both non-embedding parameters and dataset size, using three curricula: 4. **[Scaling Laws and Interpretability of Learning from Repeated Data](https://md.research.elamur.ai/free/en/papers/hernandez-2022-scaling-laws-interpretability-learning-repeated/)** *(2022 · 62 citations)* Large language models often see the same data several times. Sometimes this is done deliberately, to upweight high-quality sources, and sometimes it results from incomplete deduplication. The authors from Anthropic investigate why even a small fraction of repeated data harms models so severely. They train a family of… ## Also in our library 3 more works of theirs are in our library without a page here yet; they will appear as we publish them. ## Co-authors here 12 of 19 co-authors with a page, the most papers together first. - [Jared Kaplan](https://md.research.elamur.ai/free/en/authors/jared-kaplan/) ×3 - [Danny Hernandez](https://md.research.elamur.ai/free/en/authors/danny-hernandez/) ×2 - [Sam McCandlish](https://md.research.elamur.ai/free/en/authors/sam-mccandlish/) ×2 - [Scott Gray](https://md.research.elamur.ai/free/en/authors/scott-gray/) ×2 - [Tom B. Brown](https://md.research.elamur.ai/free/en/authors/tom-b-brown/) ×2 - [Alec Radford](https://md.research.elamur.ai/free/en/authors/alec-radford/) - [Ben Mann](https://md.research.elamur.ai/free/en/authors/ben-mann/) - [Benjamin Mann](https://md.research.elamur.ai/free/en/authors/benjamin-mann/) - [Chris Hallacy](https://md.research.elamur.ai/free/en/authors/chris-hallacy/) - [Christopher Hesse](https://md.research.elamur.ai/free/en/authors/christopher-hesse/) - [Dario Amodei](https://md.research.elamur.ai/free/en/authors/dario-amodei/) - [Dawn Drain](https://md.research.elamur.ai/free/en/authors/dawn-drain/) ## Cited by Papers of our library cite their works 4,429 times (of the 140,884 papers whose bibliographies we read); [321 citing papers with a page here →](https://md.research.elamur.ai/free/en/authors/tom-henighan/cited-by/).