--- title: "Reasoning about Pragmatics with Neural — Andreas 2016" description: "The paper addresses contrastive scene description in a two-player reference game." authors: ["Jacob Andreas", "Dan Klein"] year: 2016 venue: "Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing" source: "https://aclanthology.org/D16-1125/" doi: "10.18653/v1/d16-1125" arxiv: "1604.00562" licence: "CC BY 4.0 (ACL Anthology)" licence_url: "https://creativecommons.org/licenses/by/4.0/" summary_licence: "Elamur Cosmic License" canonical: "https://research.elamur.ai/free/en/papers/andreas-2016-reasoning-about-pragmatics-neural-listeners/" --- [Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/andreas-2016-reasoning-about-pragmatics-neural-listeners/) [home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2016](https://md.research.elamur.ai/free/en/papers/2016/page/3/) / Reasoning about Pragmatics with Neural Listeners and Speakers # Reasoning about Pragmatics with Neural Listeners and Speakers **Authors:** [Jacob Andreas](https://md.research.elamur.ai/free/en/authors/jacob-andreas/), [Dan Klein](https://md.research.elamur.ai/free/en/authors/dan-klein/) · **Year:** 2016 · **Venue:** Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/) **Links:** [Publisher's page](https://aclanthology.org/D16-1125/) · [PDF](https://aclanthology.org/D16-1125.pdf) · [DOI 10.18653/v1/d16-1125](https://doi.org/10.18653/v1/d16-1125) · [arXiv 1604.00562](https://arxiv.org/abs/1604.00562) · [OpenAlex](https://openalex.org/W2964183327) **Citations:** 116 citations (openalex-snapshot · 2026-10-14) · [82](#cited-by) in our library, of the 140673 papers whose bibliographies we read ## Summary The paper addresses contrastive scene description in a two-player reference game. A speaker must produce an utterance that lets a listener pick a target scene over a distractor. Earlier "direct" approaches learn pragmatic behavior but need training data annotated specifically for pragmatics. Earlier "derived" (rational speech acts) approaches reason about a listener but depend on hand-built grammars and small domains. Andreas and Klein combine the two. They learn a literal speaker S0 and a literal listener L0 from ordinary, non-contrastive captions of the abstract scenes dataset (Zitnick et al., 2014). Both are built from small neural modules: referent and description encoders, a choice ranker, and a feedforward conditional language model. A reasoning speaker S1 then samples candidate captions from S0 and rescores them with a weighted mix of listener and speaker probabilities. In human-played games on the test set, S1 succeeds 81% of the time, against 69% for a reimplementation of Mao et al. (2015). This shows that learned pragmatics can scale to realistic data without pragmatic supervision or domain engineering. ## Key points - Test accuracy on all pairs: S1 81%, Contrastive baseline (Mao et al., 2015) 69%, Literal S0 64%. On hard pairs the scores are 68%, 58% and 53%. All differences are significant (p \< 0.05, Binomial). - The interpolation weight λ trades accuracy (pure L0) against fluency (pure S0). With λ = 0.02, fluency stays at 4.8, the same as the literal speaker, with little loss of accuracy. - Accuracy keeps improving as the number of sampled candidates grows up to 100, so the final system uses 100 samples. - A "compiled" direct speaker was trained on S1's own outputs. It reaches a mean of 69% versus 66% for S0, a gain that is not significant, and stays well below S1's 83%. This suggests explicit reasoning is hard to distill into a shallow network. - Models, human annotations and code are released at github.com/jacobandreas/pragma. *Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.* Fields: [Linguistics & languages](https://md.research.elamur.ai/free/en/shelves/linguistics/) ## Cited by Cited by 82 papers of our library cite it (of the 140673 whose bibliographies we read); 17 of them have a page here, 65 not yet. - [Experience Grounds Language](https://md.research.elamur.ai/free/en/papers/bisk-2020-experience-grounds-language/) - [Unified Pragmatic Models for Generating and Following Instructions](https://md.research.elamur.ai/free/en/papers/fried-2018-unified-pragmatic-models-generating-following/) - [Pragmatics in Language Grounding: Phenomena, Tasks, and Modeling Approaches](https://md.research.elamur.ai/free/en/papers/fried-2023-pragmatics-language-grounding-phenomena-tasks/) - [Pragmatically Informative Image Captioning with Character-Level Inference](https://md.research.elamur.ai/free/en/papers/gordon-2018-pragmatically-informative-image-captioning-character/) - [An Incremental Iterated Response Model of Pragmatics](https://md.research.elamur.ai/free/en/papers/gordon-2019-incremental-iterated-response-model-pragmatics/) - [Learning to Write with Cooperative Discriminators](https://md.research.elamur.ai/free/en/papers/holtzman-2018-learning-write-cooperative-discriminators/) - [Will I Sound Like Me? Improving Persona Consistency in Dialogues through Pragmatic Self-Consciousness](https://md.research.elamur.ai/free/en/papers/kim-2020-will-i-sound-like-me/) - [Perspective-taking and Pragmatics for Generating Empathetic Responses Focused on Emotion Causes](https://md.research.elamur.ai/free/en/papers/kim-2021-perspective-taking-pragmatics-generating-empathetic/) - [GeDi: Generative Discriminator Guided Sequence Generation](https://md.research.elamur.ai/free/en/papers/krause-2021-gedi-generative-discriminator-guided-sequence/) - [Referring Expressions with Rational Speech Act Framework: A Probabilistic Approach](https://md.research.elamur.ai/free/en/papers/le-2022-referring-expressions-rational-speech-act/) - [Colors in Context: A Pragmatic Neural Model for Grounded Language Understanding](https://md.research.elamur.ai/free/en/papers/monroe-2017-colors-context-pragmatic-neural-model/) - [Harnessing the linguistic signal to predict scalar inferences](https://md.research.elamur.ai/free/en/papers/schuster-2020-harnessing-linguistic-signal-predict-scalar/) - [A practical introduction to the Rational Speech Act modeling framework](https://md.research.elamur.ai/free/en/papers/scontras-2021-practical-introduction-rational-speech-act/) - [Modeling Cross-Cultural Pragmatic Inference with Codenames Duet](https://md.research.elamur.ai/free/en/papers/shaikh-2023-modeling-cross-cultural-pragmatic-inference/) - [Pragmatically Informative Text Generation](https://md.research.elamur.ai/free/en/papers/shen-2019-pragmatically-informative-text-generation/) - [RSA-Control: A Pragmatics-Grounded Lightweight Controllable Text Generation Framework](https://md.research.elamur.ai/free/en/papers/wang-2024-rsa-control-pragmatics-grounded-lightweight/) - [Know What You Don't Know: Modeling a Pragmatic Speaker that Refers to Objects of Unknown Categories](https://md.research.elamur.ai/free/en/papers/zarrie-2019-know-what-you-don-t/) ## Cite The paper itself, not this page. BibTeX from the library's record of the work. ```bibtex @inproceedings{andreas2016reasoning, title = {{Reasoning about Pragmatics with Neural Listeners and Speakers}}, author = {Andreas, Jacob and Klein, Dan}, booktitle = {Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing}, year = {2016}, doi = {10.18653/v1/d16-1125}, eprint = {1604.00562}, archivePrefix = {arXiv}, url = {https://doi.org/10.18653/v1/d16-1125} } ``` - [← previous Putting Sarcasm Detection into Context: The Effects of Class Imbalance and Manual Labelling on Supervised Machine Classification of Twitter Conversations](https://md.research.elamur.ai/free/en/papers/abercrombie-2016-putting-sarcasm-detection-into-context/) - [next → Recognizing Implicit Discourse Relations via Repeated Reading: Neural Networks with Multi-Level Attention](https://md.research.elamur.ai/free/en/papers/liu-2016-recognizing-implicit-discourse-relations-repeated/) - [↑ back to papers of 2016](https://md.research.elamur.ai/free/en/papers/2016/page/3/)