---
title: "Comparing Automatic and Human Evaluation of Local"
description: "Nguyen tests whether cheap automatic metrics agree with human judgement. 0.797 accuracy. 30 references, cited by 67 papers here."
authors: ["Dong Nguyen"]
year: 2018
venue: "Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers)"
source: "https://aclanthology.org/N18-1097/"
doi: "10.18653/v1/n18-1097"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/nguyen-2018-comparing-automatic-human-evaluation-local/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/nguyen-2018-comparing-automatic-human-evaluation-local/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2018](https://md.research.elamur.ai/free/en/papers/2018/page/3/) / Comparing Automatic and Human Evaluation of Local Explanations for Text Classification

# Comparing Automatic and Human Evaluation of Local Explanations for Text Classification

**Authors:** [Dong Nguyen](https://md.research.elamur.ai/free/en/authors/dong-nguyen/) · **Year:** 2018 · **Venue:** [Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers)](https://md.research.elamur.ai/free/en/venues/naacl/) ([NAACL 2018](https://md.research.elamur.ai/free/en/venues/naacl/2018/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/N18-1097/) · [PDF](https://aclanthology.org/N18-1097.pdf) · [DOI 10.18653/v1/n18-1097](https://doi.org/10.18653/v1/n18-1097) · [OpenAlex](https://openalex.org/W2804878416)

**Citations:** 173 citations (OpenAlex, 2026-10-14) · [67](https://md.research.elamur.ai/free/en/papers/nguyen-2018-comparing-automatic-human-evaluation-local/cited-by/) in our library, of the 141,840 papers whose bibliographies we read

## Summary
Explanations that point to the words driving a single prediction are a popular way to make text classifiers more transparent, but it is unclear how to judge their quality. Nguyen tests whether cheap automatic metrics agree with human judgement. Four word-level explanation methods (a random baseline, LIME, word omission, and first-derivative saliency) explain the outputs of a Logistic Regression model and an MLP. The models are trained on two binary tasks: 20news (Christianity vs. Atheism) and movie review sentiment.

Fidelity is scored automatically by deleting the highlighted words, using the switching point and AOPC. Human quality is measured in a crowdsourced forward-prediction task, where workers see an explanation and guess what the classifier predicted.

The automatic scores correlate only moderately with how often workers guess correctly. Classifier confidence and the type of prediction outcome (true or false positive or negative) also shape human accuracy. This matters because deletion-based metrics are useful but cannot stand in for user studies.

## Key points
- **Data:**
  - Movie reviews come from Pang and Lee (2004), using the Zaidan et al. (2007) version, split 60/20/20 into train, development and test.
  - 20news uses the 20news-bydate version, restricted to two classes.
- **Classifiers:**
  - Logistic Regression (Scikit-learn, TF-IDF unigrams) reaches 0.797 accuracy on movies and 0.921 on 20news.
  - The Keras MLP (512 hidden units) reaches 0.832 and 0.939.
- **Omission as reference:** for LR, word omission matches the known feature contributions, so it provides a ground-truth ranking.
- **Regression analysis:** an OLS model on the movie data (n = 1,620, R² = 0.177) shows that a higher switching point significantly lowers crowd accuracy. Classifier confidence and prediction outcome also have significant effects.
- **Noise experiment:** randomly replacing 0–80% of the top-10 saliency words lowers mean AOPC from 0.2627 to 0.0411. Crowd accuracy on true positives falls from 0.940 to 0.736.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [University of Edinburgh](https://md.research.elamur.ai/free/en/universities/university-of-edinburgh/) — University, Edinburgh, United Kingdom · 255 works · 68,771 citations
- [The Alan Turing Institute](https://md.research.elamur.ai/free/en/labs/the-alan-turing-institute/) — Research institute, London, United Kingdom · 19 works · 3,608 citations

📚 Shelves (5): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 9,086 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 8,889 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 9,296 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 9,206 works · [💻 Software](https://md.research.elamur.ai/free/en/shelves/computer-science/software/) · 25 works

🧭 Topics (1): [📏 Evaluation & benchmarks](https://md.research.elamur.ai/free/en/topics/evaluation-benchmarks/) · 685 works

## Cite

Cite as: Dong Nguyen. 2018. Comparing Automatic and Human Evaluation of Local Explanations for Text Classification. Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers).

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{nguyen2018comparing,
  title = {{Comparing Automatic and Human Evaluation of Local Explanations for Text Classification}},
  author = {Dong Nguyen},
  booktitle = {Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers)},
  year = {2018},
  doi = {10.18653/v1/n18-1097},
  url = {https://doi.org/10.18653/v1/n18-1097}
}
```

- [← previous Towards Exploiting Background Knowledge for Building Conversation Systems](https://md.research.elamur.ai/free/en/papers/moghe-2018-exploiting-background-knowledge-building-conversation/)
- [next → Towards Robust Neural Machine Translation](https://md.research.elamur.ai/free/en/papers/cheng-2018-robust-neural-machine-translation/)
- [↑ back to papers of 2018](https://md.research.elamur.ai/free/en/papers/2018/page/3/)

## References

30 references, 9 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Leila Arras, Franziska Horn, Grégoire Montavon, Klaus-Robert Muller, and Wojciech Samek. 2016. Explaining predictions of non-linear classifiers in NLP. In Proceedings of the 1st Workshop on Representation Learning for NLP. pages 1–7. [find](https://scholar.google.com/scholar?q=Leila+Arras%2C+Franziska+Horn%2C+Gre%CC%81goire+Montavon%2C+Klaus%2DRobert+Muller%2C+and+Wojciech+Samek%2E+2016%2E+Explaining+predictions+of+non%2Dlinear+classifiers+in+NLP%2E+In+Proceedings+of+the+1st+Workshop+on+Represent)
2. Leila Arras, Franziska Horn, Grégoire Montavon, Klaus-Robert Müller, and Wojciech Samek. 2017. ”What is relevant in a text document?”: An interpretable machine learning approach. PLOS ONE 12(8):e0181142. [find](https://scholar.google.com/scholar?q=Leila+Arras%2C+Franziska+Horn%2C+Gre%CC%81goire+Montavon%2C+Klaus%2DRobert+Mu%CC%88ller%2C+and+Wojciech+Samek%2E+2017%2E+%E2%80%9DWhat+is+relevant+in+a+text+document%3F%E2%80%9D%3A+An+interpretable+machine+learning+approach%2E+PLOS+ONE+12%288%29%3Ae018)
3. Malika Aubakirova and Mohit Bansal. 2016. [Interpreting neural networks to improve politeness comprehension](https://md.research.elamur.ai/free/en/papers/aubakirova-2016-interpreting-neural-networks-improve-politeness/). In Proceedings of EMNLP 2016. pages 2035–2041.
4. Chollet et al. 2015. Keras. [https://github.com/fchollet/keras](https://github.com/fchollet/keras). [↗](https://github.com/fchollet/keras)
5. Mark W. Craven. 1996. Extracting comprehensible models from trained neural networks. Ph.D. thesis, University of Wisconsin–Madison. [find](https://scholar.google.com/scholar?q=Mark+W%2E+Craven%2E+1996%2E+Extracting+comprehensible+models+from+trained+neural+networks%2E+Ph%2ED%2E+thesis%2C+University+of+Wisconsin%E2%80%93Madison%2E)
6. Yanzhuo Ding, Yang Liu, Huanbo Luan, and Maosong Sun. 2017. Visualizing and understanding neural machine translation. In Proceedings of ACL 2017. pages 1150–1159. [find](https://scholar.google.com/scholar?q=Yanzhuo+Ding%2C+Yang+Liu%2C+Huanbo+Luan%2C+and+Maosong+Sun%2E+2017%2E+Visualizing+and+understanding+neural+machine+translation%2E+In+Proceedings+of+ACL+2017%2E+pages+1150%E2%80%931159%2E)
7. Finale Doshi-Velez and Been Kim. 2017. [Towards a rigorous science of interpretable machine learning](https://md.research.elamur.ai/free/en/papers/doshi-velez-2017-towards-rigorous-science-interpretable-machine/). In arXiv preprint arXiv:1702.08608.
8. Alex A. Freitas. 2014. Comprehensible classification models: A position paper. SIGKDD Explorations Newsletter 15(1):1–10. [find](https://scholar.google.com/scholar?q=Alex+A%2E+Freitas%2E+2014%2E+Comprehensible+classification+models%3A+A+position+paper%2E+SIGKDD+Explorations+Newsletter+15%281%29%3A1%E2%80%9310%2E)
9. Yash Goyal, Akrit Mohapatra, Devi Parikh, and Dhruv Batra. 2016. Towards transparent AI systems: Interpreting visual question answering models. In International Conference on Machine Learning (ICML) Workshop on Visualization for Deep Learning. [find](https://scholar.google.com/scholar?q=Yash+Goyal%2C+Akrit+Mohapatra%2C+Devi+Parikh%2C+and+Dhruv+Batra%2E+2016%2E+Towards+transparent+AI+systems%3A+Interpreting+visual+question+answering+models%2E+In+International+Conference+on+Machine+Learning+%28ICML%29+W)
10. Abram Handler, Su Lin Blodgett, and Brendan O’Connor. 2016. Visualizing textual models with in-text and word-as-pixel highlighting. In Proceedings of the 2016 Workshop on Human Interpretability in Machine Learning. [find](https://scholar.google.com/scholar?q=Abram+Handler%2C+Su+Lin+Blodgett%2C+and+Brendan+O%E2%80%99Connor%2E+2016%2E+Visualizing+textual+models+with+in%2Dtext+and+word%2Das%2Dpixel+highlighting%2E+In+Proceedings+of+the+2016+Workshop+on+Human+Interpretability+in+Mac)
11. Jonathan L. Herlocker, Joseph A. Konstan, and John Riedl. 2000. Explaining collaborative filtering recommendations. In Proceedings of CSCW ’00. pages 241–250. [find](https://scholar.google.com/scholar?q=Jonathan+L%2E+Herlocker%2C+Joseph+A%2E+Konstan%2C+and+John+Riedl%2E+2000%2E+Explaining+collaborative+filtering+recommendations%2E+In+Proceedings+of+CSCW+%E2%80%9900%2E+pages+241%E2%80%93250%2E)
12. Ákos Kádár, Grzegorz Chrupala, and Afra Alishahi. 2016. [Representation of linguistic form and function in recurrent neural networks](https://md.research.elamur.ai/free/en/papers/kadar-2017-representation-linguistic-form-function-recurrent/). CoRR abs/1602.08952.
13. Ashish Kapoor, Bongshin Lee, Desney Tan, and Eric Horvitz. 2010. Interactive optimization for steering machine classification. In Proceedings of CHI ’10. pages 1343–1352. [find](https://scholar.google.com/scholar?q=Ashish+Kapoor%2C+Bongshin+Lee%2C+Desney+Tan%2C+and+Eric+Horvitz%2E+2010%2E+Interactive+optimization+for+steering+machine+classification%2E+In+Proceedings+of+CHI+%E2%80%9910%2E+pages+1343%E2%80%931352%2E)
14. Pang Wei Koh and Percy Liang. 2017. [Understanding black-box predictions via influence functions](https://md.research.elamur.ai/free/en/papers/koh-2017-understanding-black-box-predictions-influence/). In Proceedings of ICML 2017. pages 1885–1894. a deep neural network has learned. IEEE Transactions on Neural Networks and Learning Systems 28(11):2660 – 2673.
15. Todd Kulesza, Margaret Burnett, Weng-Keen Wong, and Simone Stumpf. 2015. Principles of explanatory debugging to personalize interactive machine learning. In Proceedings of IUI ’15. pages 126–137. [find](https://scholar.google.com/scholar?q=Todd+Kulesza%2C+Margaret+Burnett%2C+Weng%2DKeen+Wong%2C+and+Simone+Stumpf%2E+2015%2E+Principles+of+explanatory+debugging+to+personalize+interactive+machine+learning%2E+In+Proceedings+of+IUI+%E2%80%9915%2E+pages+126%E2%80%93137%2E)
16. Karen Simonyan, Andrea Vedaldi, and Andrew Zisserman. 2013. [Deep inside convolutional networks: Visualising image classification models and saliency maps](https://md.research.elamur.ai/free/en/papers/simonyan-2013-deep-inside-convolutional-networks-visualising/). CoRR abs/1312.6034.
17. Jiwei Li, Xinlei Chen, Eduard Hovy, and Dan Jurafsky. 2016a. [Visualizing and understanding neural models in NLP](https://md.research.elamur.ai/free/en/papers/li-2016-visualizing-understanding-neural-models-nlp/). In Proceedings of NAACL 2016. pages 681–691.
18. Omar Zaidan, Jason Eisner, and Christine Piatko. 2007. [Using “annotator rationales” to improve machine learning for text categorization](https://md.research.elamur.ai/free/en/papers/zaidan-2007-annotator-rationales-improve-machine-learning/). In Proceedings of NAACL 2007. pages 260–267.
19. Jiwei Li, Will Monroe, and Dan Jurafsky. 2016b. Understanding neural networks through representation erasure. arXiv preprint arXiv:1612.08220 . [find](https://scholar.google.com/scholar?q=Jiwei+Li%2C+Will+Monroe%2C+and+Dan+Jurafsky%2E+2016b%2E+Understanding+neural+networks+through+representation+erasure%2E+arXiv+preprint+arXiv%3A1612%2E08220+%2E)
20. Matthew D. Zeiler and Rob Fergus. 2014. [Visualizing and understanding convolutional networks](https://md.research.elamur.ai/free/en/papers/zeiler-2013-visualizing-understanding-convolutional-networks/). In ECCV 2014. pages 818–833.
21. Brian Y. Lim, Anind K. Dey, and Daniel Avrahami. 2009. Why and why not explanations improve the intelligibility of context-aware intelligent systems. In Proceedings of CHI ’09. pages 2119–2128. [find](https://scholar.google.com/scholar?q=Brian+Y%2E+Lim%2C+Anind+K%2E+Dey%2C+and+Daniel+Avrahami%2E+2009%2E+Why+and+why+not+explanations+improve+the+intelligibility+of+context%2Daware+intelligent+systems%2E+In+Proceedings+of+CHI+%E2%80%9909%2E+pages+2119%E2%80%932128%2E)
22. A Zachary C. Lipton. 2016. The mythos of model interpretability. In Proceedings of the 2016 ICML Workshop on Human Interpretability in Machine Learning (WHI 2016). pages 96–100. [find](https://scholar.google.com/scholar?q=A+Zachary+C%2E+Lipton%2E+2016%2E+The+mythos+of+model+interpretability%2E+In+Proceedings+of+the+2016+ICML+Workshop+on+Human+Interpretability+in+Machine+Learning+%28WHI+2016%29%2E+pages+96%E2%80%93100%2E)
23. Christopher D. Manning. 2015. Computational linguistics and deep learning. Computational Linguistics 41(4):701–707. [find](https://scholar.google.com/scholar?q=Christopher+D%2E+Manning%2E+2015%2E+Computational+linguistics+and+deep+learning%2E+Computational+Linguistics+41%284%29%3A701%E2%80%93707%2E)
24. David Martens and Foster Provost. 2014. Explaining data-driven document classifications. MIS Quarterly 38(1):73–100. [find](https://scholar.google.com/scholar?q=David+Martens+and+Foster+Provost%2E+2014%2E+Explaining+data%2Ddriven+document+classifications%2E+MIS+Quarterly+38%281%29%3A73%E2%80%93100%2E)
25. Bo Pang and Lillian Lee. 2004. A sentimental education: Sentiment analysis using subjectivity. In Proceedings of ACL. pages 271–278. [find](https://scholar.google.com/scholar?q=Bo+Pang+and+Lillian+Lee%2E+2004%2E+A+sentimental+education%3A+Sentiment+analysis+using+subjectivity%2E+In+Proceedings+of+ACL%2E+pages+271%E2%80%93278%2E)
26. Paul. 2016. Interpretable machine learning: Lessons from topic modeling. In Proceedings of the CHI Workshop on Human-Centered Machine Learning. [find](https://scholar.google.com/scholar?q=Paul%2E+2016%2E+Interpretable+machine+learning%3A+Lessons+from+topic+modeling%2E+In+Proceedings+of+the+CHI+Workshop+on+Human%2DCentered+Machine+Learning%2E)
27. Fabian Pedregosa, Gaël Varoquaux, Alexandre Gramfort, Vincent Michel, Bertrand Thirion, Olivier Grisel, Mathieu Blondel, Peter Prettenhofer, Ron Weiss, Vincent Dubourg, Jake Vanderplas, Alexandre Passos, David Cournapeau, Matthieu Brucher, Matthieu Perrot, and Édouard Duchesnay. 2011. Scikit-learn… [find](https://scholar.google.com/scholar?q=Fabian+Pedregosa%2C+Gae%CC%88l+Varoquaux%2C+Alexandre+Gramfort%2C+Vincent+Michel%2C+Bertrand+Thirion%2C+Olivier+Grisel%2C+Mathieu+Blondel%2C+Peter+Prettenhofer%2C+Ron+Weiss%2C+Vincent+Dubourg%2C+Jake+Vanderplas%2C+Alexandre+Pas)
28. Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin. 2016. [“Why should I trust you?”: Explaining the predictions of any classifier](https://md.research.elamur.ai/free/en/papers/ribeiro-2016-why-should-i-trust-explaining/). In Proceedings of KDD ’16. pages 1135–1144.
29. Marko Robnik-Šikonja and Igor Kononenko. 2008. Explaining classifications for individual instances. IEEE Transactions on Knowledge and Data Engineering 20(5):589–600. [find](https://scholar.google.com/scholar?q=Marko+Robnik%2DS%CC%8Cikonja+and+Igor+Kononenko%2E+2008%2E+Explaining+classifications+for+individual+instances%2E+IEEE+Transactions+on+Knowledge+and+Data+Engineering+20%285%29%3A589%E2%80%93600%2E)
30. Wojciech Samek, Alexander Binder, Gregoire Montavon, Sebastian Lapuschkin, and Klaus-Robert Müller. 2017. Evaluating the visualization of what [find](https://scholar.google.com/scholar?q=Wojciech+Samek%2C+Alexander+Binder%2C+Gregoire+Montavon%2C+Sebastian+Lapuschkin%2C+and+Klaus%2DRobert+Mu%CC%88ller%2E+2017%2E+Evaluating+the+visualization+of+what)

*Updated 24 October 2026*