---
title: "Understanding the Capabilities, Limitations, and Societal Impact"
description: "The report introduces no new data, model or benchmark. Cited by 57 papers here. This document is not an experimental study."
authors: ["Alex Tamkin", "Miles Brundage", "Jack Clark", "Deep Ganguli"]
year: 2021
venue: "arXiv:2102.02503"
source: "https://arxiv.org/abs/2102.02503"
doi: "10.48550/arxiv.2102.02503"
arxiv: "2102.02503"
licence: "arXiv — non-exclusive license to distribute"
licence_url: "http://arxiv.org/licenses/nonexclusive-distrib/1.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/tamkin-2021-understanding-capabilities-limitations-societal-impact/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/tamkin-2021-understanding-capabilities-limitations-societal-impact/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2021](https://md.research.elamur.ai/free/en/papers/2021/page/3/) / Understanding the Capabilities, Limitations, and Societal Impact of Large Language Models

# Understanding the Capabilities, Limitations, and Societal Impact of Large Language Models

**Authors:** [Alex Tamkin](https://md.research.elamur.ai/free/en/authors/alex-tamkin/), [Miles Brundage](https://md.research.elamur.ai/free/en/authors/miles-brundage/), [Jack Clark](https://md.research.elamur.ai/free/en/authors/jack-clark/) and [Deep Ganguli](https://md.research.elamur.ai/free/en/authors/deep-ganguli/) · **Year:** 2021 · **Venue:** arXiv:2102.02503 · **Licence:** [arXiv — non-exclusive license to distribute](http://arxiv.org/licenses/nonexclusive-distrib/1.0/)

**Links:** [Publisher's page](https://arxiv.org/abs/2102.02503) · [PDF](https://arxiv.org/pdf/2102.02503) · [DOI 10.48550/arxiv.2102.02503](https://doi.org/10.48550/arxiv.2102.02503) · [arXiv 2102.02503](https://arxiv.org/abs/2102.02503) · [OpenAlex](https://openalex.org/W3128912454)

**Citations:** 128 citations (OpenAlex, 2026-10-14) · [57](https://md.research.elamur.ai/free/en/papers/tamkin-2021-understanding-capabilities-limitations-societal-impact/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
This document is not an experimental study. It is a structured report of a closed meeting on 14 October 2020, where researchers from OpenAI, Stanford HAI and other universities discussed open questions raised by GPT-3. Attendees came from computer science, linguistics, philosophy, political science, cyber policy and related fields, and spoke under the Chatham House Rule, so views are not attributed. The authors organise the debate into two strands. The technical strand covers:
- how steadily abilities grow with model and data size,
- rival definitions of language "understanding",
- the value of multimodal training,
- aligning model objectives with human values.

The societal strand covers:
- the near-impossibility of predicting every use of a general-purpose model,
- access and deployment norms,
- automated disinformation,
- bias mitigation,
- effects on labour and education.

The report introduces no new data, model or benchmark. Its contribution is a map of agreements and disputes, closing with a list of research questions. It matters because it set out an early agenda for the responsible development of large language models.

## Key points
- GPT-3 has 175 billion parameters trained on 570 GB of text. Its predecessor, GPT-2, had 1.5 billion parameters and 40 GB of text. Few-shot learning from in-context examples appeared simply through scaling.
- Participants estimated that frontier developers keep only a six- to nine-month lead before others replicate their results. They saw this as a reason to set norms quickly while few actors have such models.
- Proposed bias remedies included:
  - curating the training data,
  - separate output filters,
  - fine-tuning,
  - CTRL-style data tagging,
  - reinforcement learning from human feedback,
  - prompt design,
  - pre-deployment bias test suites,
  - large-scale red-teaming.
  
  None was considered sufficient on its own.
- Participants disagreed on whether machine-generated disinformation is cheaper than human-produced content and called for empirical study of the question. Suggested responses included metadata-based detection and cryptographic authentication of media.
- The report ends with about ten open research directions, covering scaling limits, multimodal architectures, model access policy, academic compute, and the threat landscape. It also notes DALL-E as a multimodal follow-up released after the meeting.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

📚 Shelves (6): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 18,614 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 17,965 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 17,450 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 17,296 works · [🤝 Social Sciences](https://md.research.elamur.ai/free/en/shelves/social-sciences/) · 1,954 works · [🤝 Safety Research](https://md.research.elamur.ai/free/en/shelves/social-sciences/safety-research/) · 626 works

🧭 Topics (1): [🧠 Language models & pretraining](https://md.research.elamur.ai/free/en/topics/language-models/) · 3,894 works

## Cite

Cite as: Alex Tamkin, Miles Brundage, Jack Clark and Deep Ganguli. 2021. Understanding the Capabilities, Limitations, and Societal Impact of Large Language Models. arXiv:2102.02503.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@misc{tamkin2021understanding,
  title = {{Understanding the Capabilities, Limitations, and Societal Impact of Large Language Models}},
  author = {Alex Tamkin, Miles Brundage, Jack Clark and Deep Ganguli},
  year = {2021},
  doi = {10.48550/arxiv.2102.02503},
  eprint = {2102.02503},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2102.02503}
}
```

- [← previous We Need to Consider Disagreement in Evaluation](https://md.research.elamur.ai/free/en/papers/basile-2021-we-need-consider-disagreement-evaluation/)
- [next → GeoQA: A Geometric Question Answering Benchmark Towards Multimodal Numerical Reasoning](https://md.research.elamur.ai/free/en/papers/chen-2021-geoqa-geometric-question-answering-benchmark/)
- [↑ back to papers of 2021](https://md.research.elamur.ai/free/en/papers/2021/page/3/)

*Updated 2 November 2026*