---
title: "A Rate–Distortion view of human pragmatic reasoning? (2021)"
description: "They then propose RD-RSA. 6 references, cited by 10 papers here. Among them: 59."
authors: ["Noga Zaslavsky", "Jennifer Hu", "Roger P. Levy"]
year: 2021
venue: "Proceedings of the Society for Computation in Linguistics 2021"
source: "https://aclanthology.org/2021.scil-1.32/"
arxiv: "2005.06641"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/zaslavsky-2021-rate-distortion-view-human-pragmatic/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/zaslavsky-2021-rate-distortion-view-human-pragmatic/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2021](https://md.research.elamur.ai/free/en/papers/2021/page/4/) / A Rate–Distortion view of human pragmatic reasoning?

# A Rate–Distortion view of human pragmatic reasoning?

**Authors:** Noga Zaslavsky, [Jennifer Hu](https://md.research.elamur.ai/free/en/authors/jennifer-hu/) and [Roger P. Levy](https://md.research.elamur.ai/free/en/authors/roger-levy/) · **Year:** 2021 · **Venue:** [Proceedings of the Society for Computation in Linguistics 2021](https://md.research.elamur.ai/free/en/venues/scil/) ([SCiL 2021](https://md.research.elamur.ai/free/en/venues/scil/2021/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2021.scil-1.32/) · [PDF](https://aclanthology.org/2021.scil-1.32.pdf) · [arXiv 2005.06641](https://arxiv.org/abs/2005.06641) · [OpenAlex](https://openalex.org/W3025524570)

**Citations:** 59 citations (OpenAlex, 2026-10-14) · [10](https://md.research.elamur.ai/free/en/papers/zaslavsky-2021-rate-distortion-view-human-pragmatic/cited-by/) in our library, of the 141,840 papers whose bibliographies we read

## Summary
The Rational Speech Act (RSA) framework models pragmatics as a speaker and a listener who recursively reason about each other. Earlier work, based only on numeric simulations, conjectured that each round of this recursion raises expected communicative utility. This short paper tests that conjecture analytically and ties RSA to Shannon's Rate–Distortion (RD) theory.

The authors prove that RSA updates form an alternating maximization (AM) procedure. What RSA optimizes is not expected utility alone. It is a combined objective: expected utility plus the conditional entropy of the speaker's productions, where higher entropy stands for lower effort. As a result, expected utility can actually fall at deeper recursion levels.

They then propose RD-RSA. In this variant, the speaker trades utility against the mutual information between meanings and utterances, which makes the speaker's soft-max depend on the marginal utterance probability. On the human reference-game data of Vogel et al. (2014), RD-RSA fits as well as RSA. Unlike RSA, it has no built-in bias toward uninformative, random utterances. The result gives pragmatic reasoning a principled information-theoretic basis.

## Key points
- Standard RSA setup: the listener is Bayesian, L(m|u) ∝ S(u|m)P(m), with a shared prior P(m). The speaker is a soft-max, S(u|m) ∝ exp(αV), where the utility is V = log L(m|u) − C(u).
- Theorem: for every α ≥ 0, each speaker or listener step increases G_α = H_S(U|M) + α·E_S[V_L]. This makes RSA an AM algorithm in the sense of Csiszár and Shields (2004).
- The authors give counter-examples in which expected utility falls with recursion depth while conditional entropy rises. These cases disprove the assertion, put forward by Peloquin et al. (2019) and Yuan et al. (2018), that RSA's recursion always yields higher utility.
- RD-RSA minimizes F_α = I_S(M;U) − α·E_S[V_L]. Its speaker is S(u|m) ∝ S(u)·exp(αV), and the marginal S(u) is re-estimated at every iteration. The listener stays Bayesian.
- Evaluation used simulated reasoning trajectories against human reference-game behavior from Vogel et al. (2014). RD-RSA matched RSA's fit, and the analysis showed that only RSA leans toward non-informative production.

## Interesting ideas
- The paper recasts an iterative heuristic procedure as coordinate ascent on one explicit objective. This shows what each step actually improves and why "more iterations" need not mean "better" on the metric you care about.
- Weighting a speaker's choice by the running marginal frequency of utterances removes a drift toward uniform, uninformative choices. The fix is cheap and applies to any soft-max chooser among alternatives.
- The authors treat speaker and listener as encoder and decoder, so communication cost is measured in bits as mutual information between meanings and utterances.
- The authors refute a popular conjecture by finding explicit counter-examples, instead of relying on simulations that seemed to support it.
- Two models with different theoretical motivations are compared on the same human behavioral data and also on their built-in biases, not just on fit.

## For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

- **I1** (frames): The implicature reasoner in prag could log an explicit objective value after each speaker/listener step, so every inference round is explainable and convergence can be checked.
- **I2** (back): When generation picks among paraphrases of a meaning graph, the soft-max could be multiplied by an iteratively updated utterance marginal, which favors consistent, informative wordings over random variety.
- **I3** (back): The text → graph → text compressor could report I(M;U)-style rate alongside reconstruction quality, giving a principled rate–distortion curve for comparing settings.
- **I4** (learn): Evaluation could include targeted searches for inputs where a supposedly monotone repair or self-play loop gets worse, serving as negative controls.
- **I5** (answer): When choosing between alternative pragmatic readers, check for systematic biases (e.g., toward vague answers) as well as test accuracy.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Massachusetts Institute of Technology](https://md.research.elamur.ai/free/en/universities/massachusetts-institute-of-technology/) — University, Cambridge, United States · 159 works · 65,157 citations

📚 Shelves (6): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 9,085 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 8,888 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 9,296 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 9,206 works · [💭 Psychology](https://md.research.elamur.ai/free/en/shelves/psychology/) · 798 works · [💭 Experimental and Cognitive Psychology](https://md.research.elamur.ai/free/en/shelves/psychology/experimental-and-cognitive-psychology/) · 463 works

🧭 Topics (1): [🧮 Reasoning](https://md.research.elamur.ai/free/en/topics/reasoning/) · 510 works

## Cite

Cite as: Noga Zaslavsky, Jennifer Hu and Roger P. Levy. 2021. A Rate–Distortion view of human pragmatic reasoning?. Proceedings of the Society for Computation in Linguistics 2021.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{zaslavsky2021rate,
  title = {{A Rate–Distortion view of human pragmatic reasoning?}},
  author = {Noga Zaslavsky, Jennifer Hu and Roger P. Levy},
  booktitle = {Proceedings of the Society for Computation in Linguistics 2021},
  year = {2021},
  eprint = {2005.06641},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2005.06641}
}
```

- [← previous Toward Gender-Inclusive Coreference Resolution: An Analysis of Gender and Bias Throughout the Machine Learning Lifecycle*](https://md.research.elamur.ai/free/en/papers/cao-2021-gender-inclusive-coreference-resolution-analysis/)
- [next → It's All in the Heads: Using Attention Heads as a Baseline for Cross-Lingual Transfer in Commonsense Reasoning](https://md.research.elamur.ai/free/en/papers/tikhonov-2021-it-s-all-heads-attention/)
- [↑ back to papers of 2021](https://md.research.elamur.ai/free/en/papers/2021/page/4/)

## References

6 references, 1 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. I. Csiszár and P.C. Shields. 2004. Information theory and statistics: A tutorial. Foundations and Trends in Communications and Information Theory, 1(4):417–528. [↗](https://doi.org/10.1561/9781933019543)
2. Michael C Frank and Noah D Goodman. 2012. [Predicting pragmatic reasoning in language games](https://md.research.elamur.ai/free/en/papers/frank-2012-predicting-pragmatic-reasoning-language-games/). Science, 336(6084):998–998. [↗](https://doi.org/10.1126/science.1218633)
3. Noah D Goodman and Michael C Frank. 2016. Pragmatic language interpretation as probabilistic inference. Trends Cogn. Sci, 20(11):818–829. [↗](https://doi.org/10.1016/j.tics.2016.08.005)
4. Benjamin N. Peloquin, Noah D. Goodman, and Michael C. Frank. 2019. The interactions of rational, pragmatic agents lead to efficient language structure and use. In Proceedings of the 41st Annual Meeting of the Cognitive Science Society, pages 912–917. [↗](https://doi.org/10.1111/tops.12489)
5. Claude E. Shannon. 1948. A mathematical theory of communication. Bell System Technical Journal, 27. [↗](https://doi.org/10.1002/j.1538-7305.1948.tb01338.x)
6. Adam Vogel, Andrés Goméz Emilsson, Michael C. Frank, Daniel Jurafsky, and Christopher Potts. 2014. Learning to reason pragmatically with cognitive limitations. In Proceedings of the 36th Annual Meeting of the Cognitive Science Society, pages 3055–3060. *Figure 2: Top: Simulated model trajectories in RSA (left) and RD-RSA (right). Bottom: Pearson correlation between model predictions and human behavioral results from Vogel et al. (2014).* Arianna Yuan, Will Monroe, Yu Bai, and Nate Kush-man. 2018. Understanding the Rational Speech Act model. In Proceedings of the 40th Annual Conference of the Cognitive Science Society, pages 2759– 2764. [find](https://scholar.google.com/scholar?q=Adam+Vogel%2C+Andre%CC%81s+Gome%CC%81z+Emilsson%2C+Michael+C%2E+Frank%2C+Daniel+Jurafsky%2C+and+Christopher+Potts%2E+2014%2E+Learning+to+reason+pragmatically+with+cognitive+limitations%2E+In+Proceedings+of+the+36th+Annual+Meet)

*Updated 24 October 2026*