---
title: "Leakage-Adjusted Simulatability — paper by Hase et al. (2020)"
description: "They propose leakage-adjusted simulatability (LAS). 39 references, cited by 59 papers here."
authors: ["Peter Hase", "Shiyue Zhang", "Harry Xie", "Mohit Bansal"]
year: 2020
venue: "Findings of the Association for Computational Linguistics: EMNLP 2020"
source: "https://aclanthology.org/2020.findings-emnlp.390/"
doi: "10.18653/v1/2020.findings-emnlp.390"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/hase-2020-leakage-adjusted-simulatability-can-models/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/hase-2020-leakage-adjusted-simulatability-can-models/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2020](https://md.research.elamur.ai/free/en/papers/2020/page/5/) / Leakage-Adjusted Simulatability: Can Models Generate Non-Trivial Explanations of Their Behavior in Natural Language?

# Leakage-Adjusted Simulatability: Can Models Generate Non-Trivial Explanations of Their Behavior in Natural Language?

**Authors:** [Peter Hase](https://md.research.elamur.ai/free/en/authors/peter-hase/), [Shiyue Zhang](https://md.research.elamur.ai/free/en/authors/shiyue-zhang/), Harry Xie and [Mohit Bansal](https://md.research.elamur.ai/free/en/authors/mohit-bansal/) · **Year:** 2020 · **Venue:** [Findings of the Association for Computational Linguistics: EMNLP 2020](https://md.research.elamur.ai/free/en/venues/findings/) ([Findings of the Association for Computational Linguistics 2020](https://md.research.elamur.ai/free/en/venues/findings/2020/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2020.findings-emnlp.390/) · [PDF](https://aclanthology.org/2020.findings-emnlp.390.pdf) · [DOI 10.18653/v1/2020.findings-emnlp.390](https://doi.org/10.18653/v1/2020.findings-emnlp.390) · [OpenAlex](https://openalex.org/W3102035862)

**Citations:** 61 citations (OpenAlex, 2026-10-14) · [59](https://md.research.elamur.ai/free/en/papers/hase-2020-leakage-adjusted-simulatability-can-models/cited-by/) in our library, of the 141,840 papers whose bibliographies we read

## Summary
Models trained on datasets that pair labels with free-text human explanations can write their own natural language justifications. So far, these outputs have mostly been judged by how closely they resemble human-written text, using BLEU or human similarity judgments. The authors argue that such checks capture plausibility rather than faithfulness to the model's actual decision.

They propose leakage-adjusted simulatability (LAS). An observer, here a pretrained model acting as a stand-in for a person, tries to predict the task model's output with and without the explanation. The score accounts for cases where the explanation simply gives away the label.

Using T5, they compare four graphical models (reasoning vs. rationalizing, single-task vs. multi-task) on CoS-E/CQA and e-SNLI. They validate LAS with an expert simulation study and Mechanical Turk ratings, and they also tune explanations through a multi-agent game that penalizes leakage. Their new ST-RA rationalizing model reaches close to human-level LAS. The work matters because it offers an automatic, faithfulness-oriented alternative to surface-similarity metrics.

## Key points
- Four setups are compared, all built on T5:
  - MT-RA (from Camburu et al.)
  - ST-RE/CAGE (from Rajani et al.)
  - the new ST-RA
  - MT-RE
- Rationalizing models, which condition explanations on the predicted label, beat reasoning models on LAS. Adding explanations caused no significant accuracy loss: −0.47 on SNLI and −2.10 on CQA, both non-significant.
- Crowdsourced ratings covered 200 test examples per dataset, yielding 902 ratings after filtering. Ratings tracked label leakage most strongly, not LAS. Expert simulators' own task accuracy was about 70% (SNLI) and 72% (CQA).
- In the multi-agent game, SGD optimization cut label leakage for CQA MT-RA from 85.58% to 75.21%. Only one LAS gain (MT-RE on SNLI) was statistically significant, and BLEU dropped.
- Code is released at github.com/peterbhase/LAS-NL-Explanations.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [University of North Carolina at Chapel Hill](https://md.research.elamur.ai/free/en/universities/university-of-north-carolina-at-chapel-hill/) — University, Chapel Hill, United States · 37 works · 14,332 citations

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 9,086 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 8,889 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 9,296 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 9,206 works

## Cite

Cite as: Peter Hase, Shiyue Zhang, Harry Xie and Mohit Bansal. 2020. Leakage-Adjusted Simulatability: Can Models Generate Non-Trivial Explanations of Their Behavior in Natural Language?. Findings of the Association for Computational Linguistics: EMNLP 2020.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{hase2020leakage,
  title = {{Leakage-Adjusted Simulatability: Can Models Generate Non-Trivial Explanations of Their Behavior in Natural Language?}},
  author = {Peter Hase, Shiyue Zhang, Harry Xie and Mohit Bansal},
  booktitle = {Findings of the Association for Computational Linguistics: EMNLP 2020},
  year = {2020},
  doi = {10.18653/v1/2020.findings-emnlp.390},
  url = {https://doi.org/10.18653/v1/2020.findings-emnlp.390}
}
```

- [← previous UniMorph 3.0: Universal Morphology](https://md.research.elamur.ai/free/en/papers/mccarthy-2020-unimorph-3-0-universal-morphology/)
- [next → Task-Oriented Dialogue as Dataflow Synthesis](https://md.research.elamur.ai/free/en/papers/andreas-2020-task-oriented-dialogue-as-dataflow/)
- [↑ back to papers of 2020](https://md.research.elamur.ai/free/en/papers/2020/page/5/)

## References

39 references, 20 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Jasmijn Bastings, Wilker Aziz, and Ivan Titov. 2019. [Interpretable neural predictions with differentiable binary variables](https://md.research.elamur.ai/free/en/papers/bastings-2019-interpretable-neural-predictions-differentiable-binary/). In ACL 2019, pages 2963–2977, Florence, Italy. Association for Computational Linguistics.
2. Samuel R. Bowman, Gabor Angeli, Christopher Potts, and Christopher D. Manning. 2015. A large annotated corpus for learning natural language inference. In EMNLP 2015. [find](https://scholar.google.com/scholar?q=Samuel+R%2E+Bowman%2C+Gabor+Angeli%2C+Christopher+Potts%2C+and+Christopher+D%2E+Manning%2E+2015%2E+A+large+annotated+corpus+for+learning+natural+language+inference%2E+In+EMNLP+2015%2E)
3. Oana-Maria Camburu, Tim Rocktäschel, Thomas Lukasiewicz, and Phil Blunsom. 2018. [e-snli: Natural language inference with natural language explanations](https://md.research.elamur.ai/free/en/papers/camburu-2018-e-snli-natural-language-inference/). In NeurIPS 2018.
4. Jacob Cohen. 1988. Statistical Power Analysis for the Behavioral Sciences. Routledge. [find](https://scholar.google.com/scholar?q=Jacob+Cohen%2E+1988%2E+Statistical+Power+Analysis+for+the+Behavioral+Sciences%2E+Routledge%2E)
5. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. [Bert: Pre-training of deep bidirectional transformers for language understanding](https://md.research.elamur.ai/free/en/papers/devlin-2019-bert-pre-training-deep-bidirectional/). In ACL 2019.
6. Jay DeYoung, Sarthak Jain, Nazneen Fatema Rajani, Eric Lehman, Caiming Xiong, Richard Socher, and Byron C. Wallace. 2020. [Eraser: A benchmark to evaluate rationalized nlp models](https://md.research.elamur.ai/free/en/papers/deyoung-2020-eraser-benchmark-evaluate-rationalized-nlp/). In ACL 2020, volume abs/1911.03429.
7. Jesse Dodge, Gabriel Ilharco, Roy Schwartz, Ali Farhadi, Hannaneh Hajishirzi, and Noah A. Smith. 2020. Fine-tuning pretrained language models: Weight initializations, data orders, and early stopping. CoRR, abs/2002.06305. [find](https://scholar.google.com/scholar?q=Jesse+Dodge%2C+Gabriel+Ilharco%2C+Roy+Schwartz%2C+Ali+Farhadi%2C+Hannaneh+Hajishirzi%2C+and+Noah+A%2E+Smith%2E+2020%2E+Fine%2Dtuning+pretrained+language+models%3A+Weight+initializations%2C+data+orders%2C+and+early+stopping%2E+)
8. Finale Doshi-Velez and Been Kim. 2017. [Towards a rigorous science of interpretable machine learning](https://md.research.elamur.ai/free/en/papers/doshi-velez-2017-towards-rigorous-science-interpretable-machine/). arXiv: Machine Learning.
9. Bradley Efron and Robert J Tibshirani. 1994. An Introduction to the Bootstrap. CRC press. [find](https://scholar.google.com/scholar?q=Bradley+Efron+and+Robert+J+Tibshirani%2E+1994%2E+An+Introduction+to+the+Bootstrap%2E+CRC+press%2E)
10. Peter Hase and Mohit Bansal. 2020. [Evaluating explainable ai: Which algorithmic explanations help users predict model behavior?](https://md.research.elamur.ai/free/en/papers/hase-2020-evaluating-explainable-ai-which-algorithmic/) In ACL 2020.
11. Lisa Anne Hendricks, Zeynep Akata, Marcus Rohrbach, Jeff Donahue, Bernt Schiele, and Trevor Darrell. 2016. Generating visual explanations. In ECCV 2016. [find](https://scholar.google.com/scholar?q=Lisa+Anne+Hendricks%2C+Zeynep+Akata%2C+Marcus+Rohrbach%2C+Jeff+Donahue%2C+Bernt+Schiele%2C+and+Trevor+Darrell%2E+2016%2E+Generating+visual+explanations%2E+In+ECCV+2016%2E)
12. Lisa Anne Hendricks, Ronghang Hu, Trevor Darrell, and Zeynep Akata. 2018. Grounding visual explanations. In ECCV 2018. [find](https://scholar.google.com/scholar?q=Lisa+Anne+Hendricks%2C+Ronghang+Hu%2C+Trevor+Darrell%2C+and+Zeynep+Akata%2E+2018%2E+Grounding+visual+explanations%2E+In+ECCV+2018%2E)
13. Sara Hooker, Dumitru Erhan, Pieter-Jan Kindermans, and Been Kim. 2019. A benchmark for interpretability methods in deep neural networks. In Proceedings of the 33rd International Conference on Neural Information Processing Systems. [find](https://scholar.google.com/scholar?q=Sara+Hooker%2C+Dumitru+Erhan%2C+Pieter%2DJan+Kindermans%2C+and+Been+Kim%2E+2019%2E+A+benchmark+for+interpretability+methods+in+deep+neural+networks%2E+In+Proceedings+of+the+33rd+International+Conference+on+Neural+I)
14. Alon Jacovi and Yoav Goldberg. 2020a. [Aligning faithful interpretations with their social attribution](https://md.research.elamur.ai/free/en/papers/jacovi-2021-aligning-faithful-interpretations-their-social/). arXiv preprint arXiv:2006.01067.
15. Alon Jacovi and Yoav Goldberg. 2020b. [Towards faithfully interpretable nlp systems: How should we define and evaluate faithfulness?](https://md.research.elamur.ai/free/en/papers/jacovi-2020-faithfully-interpretable-nlp-systems-how/) In ACL 2020.
16. Sarthak Jain, Sarah Wiegreffe, Yuval Pinter, and Byron C. Wallace. 2020. [Learning to faithfully rationalize by construction](https://md.research.elamur.ai/free/en/papers/jain-2020-learning-faithfully-rationalize-construction/). In ACL 2020, pages 4459–4473, Online. Association for Computational Linguistics.
17. Jinkyu Kim, Anna Rohrbach, Trevor Darrell, John F. Canny, and Zeynep Akata. 2018. Textual explanations for self-driving vehicles. In ECCV 2018. [find](https://scholar.google.com/scholar?q=Jinkyu+Kim%2C+Anna+Rohrbach%2C+Trevor+Darrell%2C+John+F%2E+Canny%2C+and+Zeynep+Akata%2E+2018%2E+Textual+explanations+for+self%2Ddriving+vehicles%2E+In+ECCV+2018%2E)
18. Sawan Kumar and Partha Talukdar. 2020. [Nile : Natural language inference with faithful natural language explanations](https://md.research.elamur.ai/free/en/papers/kumar-2020-nile-natural-language-inference-faithful/). In ACL 2020.
19. Angeliki Lazaridou, Alexander Peysakhovich, and Marco Baroni. 2017. Multi-agent cooperation and the emergence of (natural) language. In ICLR 2017. [find](https://scholar.google.com/scholar?q=Angeliki+Lazaridou%2C+Alexander+Peysakhovich%2C+and+Marco+Baroni%2E+2017%2E+Multi%2Dagent+cooperation+and+the+emergence+of+%28natural%29+language%2E+In+ICLR+2017%2E)
20. Angeliki Lazaridou, Anna Potapenko, and Olivier Tieleman. 2020. Multi-agent communication meets natural language: Synergies between functional and structural language learning. In ACL 2020, pages 7663–7674. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Angeliki+Lazaridou%2C+Anna+Potapenko%2C+and+Olivier+Tieleman%2E+2020%2E+Multi%2Dagent+communication+meets+natural+language%3A+Synergies+between+functional+and+structural+language+learning%2E+In+ACL+2020%2C+pages+7663)
21. Wang Ling, Dani Yogatama, Chris Dyer, and Phil Blunsom. 2017. Program induction by rationale generation: Learning to solve and explain algebraic word problems. In ACL 2017. [find](https://scholar.google.com/scholar?q=Wang+Ling%2C+Dani+Yogatama%2C+Chris+Dyer%2C+and+Phil+Blunsom%2E+2017%2E+Program+induction+by+rationale+generation%3A+Learning+to+solve+and+explain+algebraic+word+problems%2E+In+ACL+2017%2E)
22. Chris J. Maddison, Andriy Mnih, and Yee Whye Teh. 2017. [The concrete distribution: A continuous relaxation of discrete random variables](https://md.research.elamur.ai/free/en/papers/maddison-2016-concrete-distribution-continuous-relaxation-discrete/). In ICLR 2017.
23. Sharan Narang, Colin Raffel, Katherine J. Lee, Adam Roberts, Noah Fiedel, and Karishma Malkan. 2020. WT5?! training text-to-text models to explain their predictions. ArXiv, abs/2004.14546. [find](https://scholar.google.com/scholar?q=Sharan+Narang%2C+Colin+Raffel%2C+Katherine+J%2E+Lee%2C+Adam+Roberts%2C+Noah+Fiedel%2C+and+Karishma+Malkan%2E+2020%2E+WT5%3F%21+training+text%2Dto%2Dtext+models+to+explain+their+predictions%2E+ArXiv%2C+abs%2F2004%2E14546%2E)
24. Dong Nguyen. 2018. [Comparing automatic and human evaluation of local explanations for text classification](https://md.research.elamur.ai/free/en/papers/nguyen-2018-comparing-automatic-human-evaluation-local/). In NAACL-HLT 2018.
25. Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. [Bleu: a method for automatic evaluation of machine translation](https://md.research.elamur.ai/free/en/papers/papineni-2002-bleu-method-automatic-evaluation-machine/). In ACL 2002.
26. Dong Huk Park, Lisa Anne Hendricks, Zeynep Akata, Anna Rohrbach, Bernt Schiele, Trevor Darrell, and Marcus Rohrbach. 2018. Multimodal explanations: Justifying decisions and pointing to the evidence. 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 8779–8788. [find](https://scholar.google.com/scholar?q=Dong+Huk+Park%2C+Lisa+Anne+Hendricks%2C+Zeynep+Akata%2C+Anna+Rohrbach%2C+Bernt+Schiele%2C+Trevor+Darrell%2C+and+Marcus+Rohrbach%2E+2018%2E+Multimodal+explanations%3A+Justifying+decisions+and+pointing+to+the+evidence%2E+2)
27. Ellie Pavlick and Tom Kwiatkowski. 2019. [Inherent disagreements in human textual inferences](https://md.research.elamur.ai/free/en/papers/pavlick-2019-inherent-disagreements-human-textual-inferences/). Transactions of the Association for Computational Linguistics, 7:677–694.
28. Judea Pearl. 2009. Causal inference in statistics: An overview. Statistics Surveys, 3(0):96–146. [find](https://scholar.google.com/scholar?q=Judea+Pearl%2E+2009%2E+Causal+inference+in+statistics%3A+An+overview%2E+Statistics+Surveys%2C+3%280%29%3A96%E2%80%93146%2E)
29. John Platt. 2000. Probabilistic outputs for support vector machines and comparisons to regularized likelihood methods. Adv. Large Margin Classif., 10. [find](https://scholar.google.com/scholar?q=John+Platt%2E+2000%2E+Probabilistic+outputs+for+support+vector+machines+and+comparisons+to+regularized+likelihood+methods%2E+Adv%2E+Large+Margin+Classif%2E%2C+10%2E)
30. Matt Post. 2018. [A call for clarity in reporting bleu scores](https://md.research.elamur.ai/free/en/papers/post-2018-call-clarity-reporting-bleu-scores/). In Proceedings of the Third Conference on Machine Translation: Research Papers, pages 186– 191.
31. Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. 2019. Language models are unsupervised multitask learners. In OpenAI Technical Report. [find](https://scholar.google.com/scholar?q=Alec+Radford%2C+Jeffrey+Wu%2C+Rewon+Child%2C+David+Luan%2C+Dario+Amodei%2C+and+Ilya+Sutskever%2E+2019%2E+Language+models+are+unsupervised+multitask+learners%2E+In+OpenAI+Technical+Report%2E)
32. Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. 2019. [Exploring the limits of transfer learning with a unified text-to-text transformer](https://md.research.elamur.ai/free/en/papers/raffel-2019-exploring-limits-transfer-learning-unified/). ArXiv, abs/1910.10683.
33. Nazneen Fatema Rajani, Bryan McCann, Caiming Xiong, and Richard Socher. 2019. [Explain yourself! leveraging language models for commonsense reasoning](https://md.research.elamur.ai/free/en/papers/rajani-2019-explain-yourself-leveraging-language-models/). In ACL 2019.
34. Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin. 2016. [”Why Should I Trust You?”: Explaining the predictions of any classifier](https://md.research.elamur.ai/free/en/papers/ribeiro-2016-why-should-i-trust-explaining/). Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining.
35. Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin. 2018. Anchors: High-precision model-agnostic explanations. In AAAI 2018. [find](https://scholar.google.com/scholar?q=Marco+Tulio+Ribeiro%2C+Sameer+Singh%2C+and+Carlos+Guestrin%2E+2018%2E+Anchors%3A+High%2Dprecision+model%2Dagnostic+explanations%2E+In+AAAI+2018%2E)
36. Cynthia Rudin. 2018. [Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead](https://md.research.elamur.ai/free/en/papers/rudin-2018-stop-explaining-black-box/). Nature Machine Intelligence, 1:206–215.
37. Cunxiang Wang, Shuailong Liang, Yue Zhang, Xiaonan Li, and Tian Gao. 2019. [Does it make sense? and why? a pilot study for sense making and explanation](https://md.research.elamur.ai/free/en/papers/wang-2019-does-it-make-sense-why/). In ACL 2019.
38. Ronald J. Williams. 1992. Simple statistical gradient-following algorithms for connectionist reinforcement learning. Machine Learning, 8:229–256. [find](https://scholar.google.com/scholar?q=Ronald+J%2E+Williams%2E+1992%2E+Simple+statistical+gradient%2Dfollowing+algorithms+for+connectionist+reinforcement+learning%2E+Machine+Learning%2C+8%3A229%E2%80%93256%2E)
39. Rowan Zellers, Yonatan Bisk, Ali Farhadi, and Yejin Choi. 2019. From recognition to cognition: Visual commonsense reasoning. In IEEE/CVF 2019. [find](https://scholar.google.com/scholar?q=Rowan+Zellers%2C+Yonatan+Bisk%2C+Ali+Farhadi%2C+and+Yejin+Choi%2E+2019%2E+From+recognition+to+cognition%3A+Visual+commonsense+reasoning%2E+In+IEEE%2FCVF+2019%2E)

*Updated 24 October 2026*