---
title: "A Diagnostic Study of Explainability — Atanasova 2020"
description: "Post-hoc explainability methods assign saliency scores to input words, but they often disagree with one another, and practitioners have little guidance…"
authors: ["Pepa Atanasova", "Jakob Grue Simonsen", "Christina Lioma", "Isabelle Augenstein"]
year: 2020
venue: "Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)"
source: "https://aclanthology.org/2020.emnlp-main.263/"
doi: "10.18653/v1/2020.emnlp-main.263"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/atanasova-2020-diagnostic-study-explainability-techniques-text/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/atanasova-2020-diagnostic-study-explainability-techniques-text/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2020](https://md.research.elamur.ai/free/en/papers/2020/) / A Diagnostic Study of Explainability Techniques for Text Classification

# A Diagnostic Study of Explainability Techniques for Text Classification

**Authors:** [Pepa Atanasova](https://md.research.elamur.ai/free/en/authors/pepa-atanasova/), [Jakob Grue Simonsen](https://md.research.elamur.ai/free/en/authors/jakob-grue-simonsen/), [Christina Lioma](https://md.research.elamur.ai/free/en/authors/christina-lioma/), [Isabelle Augenstein](https://md.research.elamur.ai/free/en/authors/isabelle-augenstein/) · **Year:** 2020 · **Venue:** Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2020.emnlp-main.263/) · [PDF](https://aclanthology.org/2020.emnlp-main.263.pdf) · [DOI 10.18653/v1/2020.emnlp-main.263](https://doi.org/10.18653/v1/2020.emnlp-main.263) · [OpenAlex](https://openalex.org/W3101155149)

**Citations:** 177 citations (openalex-snapshot · 2026-10-14) · [96](#cited-by) in our library, of the 140884 papers whose bibliographies we read

## Summary
Post-hoc explainability methods assign saliency scores to input words, but they often disagree with one another, and practitioners have little guidance on which method suits a given task and model. Atanasova et al. address this by defining a set of diagnostic properties, each paired with an automatic measure. The properties are agreement with human rationales (HA, scored with MAP), Faithfulness, Confidence Indication, Rationale Consistency and Dataset Consistency. They extend earlier property lists from Robnik-Šikonja and Bohanec and from DeYoung et al. The authors apply these properties to gradient-based, perturbation-based and simplification-based techniques. The evaluation spans three text classification datasets with word-level human annotations (e-SNLI, TSE, IMDB) and three architectures (CNN, LSTM, Transformer). Gradient-based explanations, particularly those aggregated with the L2 norm, came out ahead on almost every property. The perturbation and simplification methods were slower and mostly weaker. The study matters because it offers a reusable, task-agnostic benchmark for selecting an explanation method, and the authors argue it also transfers to ML fields beyond NLP.

## Key points
- Compared techniques include gradient-based variants such as InputXGrad (with L2 and mean aggregation), plus ShapSampl, LIME and Occlusion. The CNN obtained the top Faithfulness score for seven of the nine methods.
- The Transformer showed the highest agreement with human rationales, with InputXGrad ℓ2 performing best. The CNN agreed with humans more than the LSTM, even though the LSTM had higher F1 on e-SNLI. Mean-aggregated gradient methods scored close to randomly initialized models.
- ShapSampl, LIME and Occlusion were the strongest at Confidence Indication. However, they are computationally expensive and weaker on the other properties.
- No single method won on Rationale Consistency. Occlusion did best for the LSTM, and the authors suggest that simple perturbation suits recurrent models.
- High Faithfulness did not imply high human agreement, which indicates the two are distinct properties. The code is available at github.com/copenlu/xai-benchmark.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

Fields: [Natural language processing](https://md.research.elamur.ai/free/en/shelves/nlp/)

## Cited by

Cited by

96 papers of our library cite it (of the 140884 whose bibliographies we read); 3 of them have a page here, 93 not yet.

- [A Survey on Automated Fact-Checking](https://md.research.elamur.ai/free/en/papers/guo-2022-survey-automated-fact-checking/)
- [Human Rationales as Attribution Priors for Explainable Stance Detection](https://md.research.elamur.ai/free/en/papers/jayaram-2021-human-rationales-as-attribution-priors/)
- [Measuring Association Between Labels and Free-Text Rationales](https://md.research.elamur.ai/free/en/papers/wiegreffe-2021-measuring-association-between-labels-free/)

## Cite

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{atanasova2020diagnostic,
  title = {{A Diagnostic Study of Explainability Techniques for Text Classification}},
  author = {Atanasova, Pepa and Simonsen, Jakob Grue and Lioma, Christina and Augenstein, Isabelle},
  booktitle = {Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)},
  year = {2020},
  doi = {10.18653/v1/2020.emnlp-main.263},
  url = {https://doi.org/10.18653/v1/2020.emnlp-main.263}
}
```

- [← previous A Corpus of Turkish Offensive Language on Social Media](https://md.research.elamur.ai/free/en/papers/coltekin-2020-corpus-turkish-offensive-language-social/)
- [next → A Formal Analysis of Multimodal Referring Strategies Under Common Ground](https://md.research.elamur.ai/free/en/papers/krishnaswamy-2020-formal-analysis-multimodal-referring-strategies/)
- [↑ back to papers of 2020](https://md.research.elamur.ai/free/en/papers/2020/)

## References

45 references, 12 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Julius Adebayo, Justin Gilmer, Michael Muelly, Ian Goodfellow, Moritz Hardt, and Been Kim. 2018. Sanity Checks for Saliency Maps. In Proceedings of the 32Nd International Conference on Neural Information Processing Systems, NIPS’18, pages 9525– 9536, USA. Curran Associates Inc. [find](https://scholar.google.com/scholar?q=Julius+Adebayo%2C+Justin+Gilmer%2C+Michael+Muelly%2C+Ian+Goodfellow%2C+Moritz+Hardt%2C+and+Been+Kim%2E+2018%2E+Sanity+Checks+for+Saliency+Maps%2E+In+Proceedings+of+the+32Nd+International+Conference+on+Neural+Informat)
2. David Alvarez-Melis and Tommi S Jaakkola. 2018. On the robustness of interpretability methods. arXiv preprint arXiv:1806.08049. — [arXiv:1806.08049](https://arxiv.org/abs/1806.08049) [find](https://scholar.google.com/scholar?q=David+Alvarez%2DMelis+and+Tommi+S+Jaakkola%2E+2018%2E+On+the+robustness+of+interpretability+methods%2E+arXiv+preprint+arXiv%3A1806%2E08049%2E+%E2%80%94+%5BarXiv%3A1806%2E08049%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F1806%2E08049%29)
3. Leila Arras, Ahmed Osman, Klaus-Robert Müller, and Wojciech Samek. 2019. Evaluating Recurrent Neural Network Explanations. In Proceedings of the 2019 ACL Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP, pages 113– 126, Florence, Italy. Association for Computational Linguist… [find](https://scholar.google.com/scholar?q=Leila+Arras%2C+Ahmed+Osman%2C+Klaus%2DRobert+Mu%CC%88ller%2C+and+Wojciech+Samek%2E+2019%2E+Evaluating+Recurrent+Neural+Network+Explanations%2E+In+Proceedings+of+the+2019+ACL+Workshop+BlackboxNLP%3A+Analyzing+and+Interpret)
4. Alejandro Barredo Arrieta, Natalia Daz-Rodrguez, Javier Del Ser, Adrien Bennetot, Siham Tabik, Alberto Barbado, Salvador Garcia, Sergio Gil-Lopez, Daniel Molina, Richard Benjamins, Raja Chatila, and Francisco Herrera. 2020. Explainable Artificial Intelligence (XAI): Concepts, taxonomies, opportuniti… [find](https://scholar.google.com/scholar?q=Alejandro+Barredo+Arrieta%2C+Natalia+Daz%2DRodrguez%2C+Javier+Del+Ser%2C+Adrien+Bennetot%2C+Siham+Tabik%2C+Alberto+Barbado%2C+Salvador+Garcia%2C+Sergio+Gil%2DLopez%2C+Daniel+Molina%2C+Richard+Benjamins%2C+Raja+Chatila%2C+and+F)
5. Pepa Atanasova, Jakob Grue Simonsen, Christina Lioma, and Isabelle Augenstein. 2020a. Generating Fact Checking Explanations. In ACL, pages 7352– 7364. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Pepa+Atanasova%2C+Jakob+Grue+Simonsen%2C+Christina+Lioma%2C+and+Isabelle+Augenstein%2E+2020a%2E+Generating+Fact+Checking+Explanations%2E+In+ACL%2C+pages+7352%E2%80%93+7364%2E+Association+for+Computational+Linguistics%2E) [our card](https://md.research.elamur.ai/free/en/papers/atanasova-2020-generating-fact-checking-explanations/)
6. Pepa Atanasova, Dustin Wright, and Isabelle Augenstein. 2020b. Generating Label Cohesive and Well-Formed Adversarial Claims. In Proceedings of EMNLP. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Pepa+Atanasova%2C+Dustin+Wright%2C+and+Isabelle+Augenstein%2E+2020b%2E+Generating+Label+Cohesive+and+Well%2DFormed+Adversarial+Claims%2E+In+Proceedings+of+EMNLP%2E+Association+for+Computational+Linguistics%2E)
7. Trapit Bansal, David Belanger, and Andrew McCallum. 2016. Ask the GRU: Multi-Task Learning for Deep Text Recommendations. In Proceedings of the 10th ACM Conference on Recommender Systems, RecSys 16, page 107114, New York, NY, USA. Association for Computing Machinery. [find](https://scholar.google.com/scholar?q=Trapit+Bansal%2C+David+Belanger%2C+and+Andrew+McCallum%2E+2016%2E+Ask+the+GRU%3A+Multi%2DTask+Learning+for+Deep+Text+Recommendations%2E+In+Proceedings+of+the+10th+ACM+Conference+on+Recommender+Systems%2C+RecSys+16%2C+p)
8. Oana-Maria Camburu, Tim Rocktäschel, Thomas Lukasiewicz, and Phil Blunsom. 2018. e-SNLI: Natural Language Inference with Natural Language Explanations. In S. Bengio, H. Wallach, H. Larochelle, K. Grauman, N. Cesa-Bianchi, and R. Garnett, editors, Advances in Neural Information Processing Systems 31… [find](https://scholar.google.com/scholar?q=Oana%2DMaria+Camburu%2C+Tim+Rockta%CC%88schel%2C+Thomas+Lukasiewicz%2C+and+Phil+Blunsom%2E+2018%2E+e%2DSNLI%3A+Natural+Language+Inference+with+Natural+Language+Explanations%2E+In+S%2E+Bengio%2C+H%2E+Wallach%2C+H%2E+Larochelle%2C+K%2E+Gra) [our card](https://md.research.elamur.ai/free/en/papers/camburu-2018-e-snli-natural-language-inference/)
9. Rich Caruana, Yin Lou, Johannes Gehrke, Paul Koch, Marc Sturm, and Noemie Elhadad. 2015. Intelligible Models for HealthCare: Predicting Pneumonia Risk and Hospital 30-Day Readmission. In Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD 15, page… [find](https://scholar.google.com/scholar?q=Rich+Caruana%2C+Yin+Lou%2C+Johannes+Gehrke%2C+Paul+Koch%2C+Marc+Sturm%2C+and+Noemie+Elhadad%2E+2015%2E+Intelligible+Models+for+HealthCare%3A+Predicting+Pneumonia+Risk+and+Hospital+30%2DDay+Readmission%2E+In+Proceedings+o)
10. Javier Castro, Daniel Gómez, and Juan Tejada. 2009. Polynomial Calculation of the Shapley Value Based on Sampling. Comput. Oper. Res., 36(5):17261730. [find](https://scholar.google.com/scholar?q=Javier+Castro%2C+Daniel+Go%CC%81mez%2C+and+Juan+Tejada%2E+2009%2E+Polynomial+Calculation+of+the+Shapley+Value+Based+on+Sampling%2E+Comput%2E+Oper%2E+Res%2E%2C+36%285%29%3A17261730%2E)
11. Christopher Clark, Kenton Lee, Ming-Wei Chang, Tom Kwiatkowski, Michael Collins, and Kristina Toutanova. 2019. BoolQ: Exploring the Surprising Difficulty of Natural Yes/No Questions. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics:… [find](https://scholar.google.com/scholar?q=Christopher+Clark%2C+Kenton+Lee%2C+Ming%2DWei+Chang%2C+Tom+Kwiatkowski%2C+Michael+Collins%2C+and+Kristina+Toutanova%2E+2019%2E+BoolQ%3A+Exploring+the+Surprising+Difficulty+of+Natural+Yes%2FNo+Questions%2E+In+Proceedings+of) [our card](https://md.research.elamur.ai/free/en/papers/clark-2019-boolq-exploring-surprising-difficulty-natural/)
12. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies,… [find](https://scholar.google.com/scholar?q=Jacob+Devlin%2C+Ming%2DWei+Chang%2C+Kenton+Lee%2C+and+Kristina+Toutanova%2E+2019%2E+BERT%3A+Pre%2Dtraining+of+Deep+Bidirectional+Transformers+for+Language+Understanding%2E+In+Proceedings+of+the+2019+Conference+of+the+N)
13. Jay DeYoung, Sarthak Jain, Nazneen Fatema Rajani, Eric Lehman, Caiming Xiong, Richard Socher, and Byron C Wallace. 2020. ERASER: A Benchmark to Evaluate Rationalized NLP Models. 2020. [find](https://scholar.google.com/scholar?q=Jay+DeYoung%2C+Sarthak+Jain%2C+Nazneen+Fatema+Rajani%2C+Eric+Lehman%2C+Caiming+Xiong%2C+Richard+Socher%2C+and+Byron+C+Wallace%2E+2020%2E+ERASER%3A+A+Benchmark+to+Evaluate+Rationalized+NLP+Models%2E+2020%2E) [our card](https://md.research.elamur.ai/free/en/papers/deyoung-2020-eraser-benchmark-evaluate-rationalized-nlp/)
14. Shi Feng, Eric Wallace, Alvin Grissom II, Mohit Iyyer, Pedro Rodriguez, and Jordan Boyd-Graber. 2018. Pathologies of Neural Models Make Interpretations Difficult. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 3719–3728, Brussels, Belgium. Associatio… [find](https://scholar.google.com/scholar?q=Shi+Feng%2C+Eric+Wallace%2C+Alvin+Grissom+II%2C+Mohit+Iyyer%2C+Pedro+Rodriguez%2C+and+Jordan+Boyd%2DGraber%2E+2018%2E+Pathologies+of+Neural+Models+Make+Interpretations+Difficult%2E+In+Proceedings+of+the+2018+Conference)
15. Kunihiko Fukushima. 1980. Neocognitron: A self-organizing neural network model for a mechanism of pattern recognition unaffected by shift in position. Biological cybernetics, 36(4):193–202. [find](https://scholar.google.com/scholar?q=Kunihiko+Fukushima%2E+1980%2E+Neocognitron%3A+A+self%2Dorganizing+neural+network+model+for+a+mechanism+of+pattern+recognition+unaffected+by+shift+in+position%2E+Biological+cybernetics%2C+36%284%29%3A193%E2%80%93202%2E)
16. Chaoyu Guan, Xiting Wang, Quanshi Zhang, Runjin Chen, Di He, and Xing Xie. 2019. Towards a Deep and Unified Understanding of Deep Neural Models in NLP. In Proceedings of the 36th International Conference on Machine Learning, volume 97 of Proceedings of Machine Learning Research, pages 2454–2463, Lon… [find](https://scholar.google.com/scholar?q=Chaoyu+Guan%2C+Xiting+Wang%2C+Quanshi+Zhang%2C+Runjin+Chen%2C+Di+He%2C+and+Xing+Xie%2E+2019%2E+Towards+a+Deep+and+Unified+Understanding+of+Deep+Neural+Models+in+NLP%2E+In+Proceedings+of+the+36th+International+Confere)
17. Richard HR Hahnloser, Rahul Sarpeshkar, Misha A Mahowald, Rodney J Douglas, and H Sebastian Seung. 2000. Digital selection and analogue amplification coexist in a cortex-inspired silicon circuit. Nature, 405(6789):947–951. [find](https://scholar.google.com/scholar?q=Richard+HR+Hahnloser%2C+Rahul+Sarpeshkar%2C+Misha+A+Mahowald%2C+Rodney+J+Douglas%2C+and+H+Sebastian+Seung%2E+2000%2E+Digital+selection+and+analogue+amplification+coexist+in+a+cortex%2Dinspired+silicon+circuit%2E+Natu)
18. Sepp Hochreiter and Jürgen Schmidhuber. 1997. Long short-term memory. Neural computation, 9(8):1735–1780. [find](https://scholar.google.com/scholar?q=Sepp+Hochreiter+and+Ju%CC%88rgen+Schmidhuber%2E+1997%2E+Long+short%2Dterm+memory%2E+Neural+computation%2C+9%288%29%3A1735%E2%80%931780%2E)
19. Alon Jacovi and Yoav Goldberg. 2020. Towards faithfully interpretable nlp systems: How should we define and evaluate faithfulness? arXiv preprint arXiv:2004.03685. [find](https://scholar.google.com/scholar?q=Alon+Jacovi+and+Yoav+Goldberg%2E+2020%2E+Towards+faithfully+interpretable+nlp+systems%3A+How+should+we+define+and+evaluate+faithfulness%3F+arXiv+preprint+arXiv%3A2004%2E03685%2E) [our card](https://md.research.elamur.ai/free/en/papers/jacovi-2020-faithfully-interpretable-nlp-systems-how/)
20. Ulf Johansson, Rikard König, and Lars Niklasson. 2004. The Truth is in There Rule Extraction from Opaque Models Using Genetic Programming. In Proceedings of the Seventeenth International Florida Artificial Intelligence Research Society Conference, FLAIRS 2004. AAAI Press. [find](https://scholar.google.com/scholar?q=Ulf+Johansson%2C+Rikard+Ko%CC%88nig%2C+and+Lars+Niklasson%2E+2004%2E+The+Truth+is+in+There+Rule+Extraction+from+Opaque+Models+Using+Genetic+Programming%2E+In+Proceedings+of+the+Seventeenth+International+Florida+Arti)
21. Daniel Khashabi, Snigdha Chaturvedi, Michael Roth, Shyam Upadhyay, and Dan Roth. 2018. Looking Beyond the Surface: A Challenge Set for Reading Comprehension over Multiple Sentences. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics:… [find](https://scholar.google.com/scholar?q=Daniel+Khashabi%2C+Snigdha+Chaturvedi%2C+Michael+Roth%2C+Shyam+Upadhyay%2C+and+Dan+Roth%2E+2018%2E+Looking+Beyond+the+Surface%3A+A+Challenge+Set+for+Reading+Comprehension+over+Multiple+Sentences%2E+In+Proceedings+of+) [our card](https://md.research.elamur.ai/free/en/papers/khashabi-2018-looking-beyond-surface-challenge-set/)
22. Pieter-Jan Kindermans, Sara Hooker, Julius Adebayo, Maximilian Alber, Kristof T Schütt, Sven Dähne, Dumitru Erhan, and Been Kim. 2019. The (un) reliability of saliency methods. In Explainable AI: Interpreting, Explaining and Visualizing Deep Learning, pages 267–280. Springer. [find](https://scholar.google.com/scholar?q=Pieter%2DJan+Kindermans%2C+Sara+Hooker%2C+Julius+Adebayo%2C+Maximilian+Alber%2C+Kristof+T+Schu%CC%88tt%2C+Sven+Da%CC%88hne%2C+Dumitru+Erhan%2C+and+Been+Kim%2E+2019%2E+The+%28un%29+reliability+of+saliency+methods%2E+In+Explainable+AI%3A+In)
23. Pieter-Jan Kindermans, Kristof Schütt, Klaus-Robert Müller, and Sven Dähne. 2016. Investigating the influence of noise and distractors on the interpretation of neural networks. ArXiv, abs/1611.07270. — [arXiv:1611.07270](https://arxiv.org/abs/1611.07270) [find](https://scholar.google.com/scholar?q=Pieter%2DJan+Kindermans%2C+Kristof+Schu%CC%88tt%2C+Klaus%2DRobert+Mu%CC%88ller%2C+and+Sven+Da%CC%88hne%2E+2016%2E+Investigating+the+influence+of+noise+and+distractors+on+the+interpretation+of+neural+networks%2E+ArXiv%2C+abs%2F1611%2E0727)
24. Tao Lei, Regina Barzilay, and Tommi Jaakkola. 2016. Rationalizing Neural Predictions. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 107–117, Austin, Texas. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Tao+Lei%2C+Regina+Barzilay%2C+and+Tommi+Jaakkola%2E+2016%2E+Rationalizing+Neural+Predictions%2E+In+Proceedings+of+the+2016+Conference+on+Empirical+Methods+in+Natural+Language+Processing%2C+pages+107%E2%80%93117%2C+Austin%2C+) [our card](https://md.research.elamur.ai/free/en/papers/lei-2016-rationalizing-neural-predictions/)
25. Piyawat Lertvittayakumjorn and Francesca Toni. 2019. Human-grounded Evaluations of Explanation Methods for Text Classification. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-I… [find](https://scholar.google.com/scholar?q=Piyawat+Lertvittayakumjorn+and+Francesca+Toni%2E+2019%2E+Human%2Dgrounded+Evaluations+of+Explanation+Methods+for+Text+Classification%2E+In+Proceedings+of+the+2019+Conference+on+Empirical+Methods+in+Natural+La)
26. Scott M Lundberg and Su-In Lee. 2017. A unified approach to interpreting model predictions. In Advances in neural information processing systems, pages 4765–4774. [find](https://scholar.google.com/scholar?q=Scott+M+Lundberg+and+Su%2DIn+Lee%2E+2017%2E+A+unified+approach+to+interpreting+model+predictions%2E+In+Advances+in+neural+information+processing+systems%2C+pages+4765%E2%80%934774%2E) [our card](https://md.research.elamur.ai/free/en/papers/lundberg-2017-unified-approach-interpreting-model-predictions/)
27. David Martens, Johan Huysmans, Rudy Setiono, Jan Vanthienen, and Bart Baesens. 2008. Rule Extraction from Support Vector Machines: An Overview of Issues and Application in Credit Scoring, pages 33– 63. Springer Berlin Heidelberg, Berlin, Heidelberg. [find](https://scholar.google.com/scholar?q=David+Martens%2C+Johan+Huysmans%2C+Rudy+Setiono%2C+Jan+Vanthienen%2C+and+Bart+Baesens%2E+2008%2E+Rule+Extraction+from+Support+Vector+Machines%3A+An+Overview+of+Issues+and+Application+in+Credit+Scoring%2C+pages+33%E2%80%93+63)
28. Menaka Narayanan, Emily Chen, Jeffrey He, Been Kim, Sam Gershman, and Finale Doshi-Velez. 2018. How do humans understand explanations from machine learning systems? an evaluation of the human-interpretability of explanation. arXiv preprint arXiv:1802.00682. — [arXiv:1802.00682](https://arxiv.org/abs/1802.00682) [find](https://scholar.google.com/scholar?q=Menaka+Narayanan%2C+Emily+Chen%2C+Jeffrey+He%2C+Been+Kim%2C+Sam+Gershman%2C+and+Finale+Doshi%2DVelez%2E+2018%2E+How+do+humans+understand+explanations+from+machine+learning+systems%3F+an+evaluation+of+the+human%2Dinterpre)
29. Jeffrey Pennington, Richard Socher, and Christopher D. Manning. 2014. GloVe: Global Vectors for Word Representation. In Empirical Methods in Natural Language Processing (EMNLP), pages 1532–1543. [find](https://scholar.google.com/scholar?q=Jeffrey+Pennington%2C+Richard+Socher%2C+and+Christopher+D%2E+Manning%2E+2014%2E+GloVe%3A+Global+Vectors+for+Word+Representation%2E+In+Empirical+Methods+in+Natural+Language+Processing+%28EMNLP%29%2C+pages+1532%E2%80%931543%2E)
30. Nina Poerner, Hinrich Schütze, and Benjamin Roth. 2018. Evaluating neural network explanation methods using hybrid documents and morphosyntactic agreement. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 340–350, Melbourne, A… [find](https://scholar.google.com/scholar?q=Nina+Poerner%2C+Hinrich+Schu%CC%88tze%2C+and+Benjamin+Roth%2E+2018%2E+Evaluating+neural+network+explanation+methods+using+hybrid+documents+and+morphosyntactic+agreement%2E+In+Proceedings+of+the+56th+Annual+Meeting+o)
31. General Data Protection Regulation. 2016. Regulation (eu) 2016/679 of the european parliament and of the council of 27 april 2016 on the protection of natural persons with regard to the processing of personal data and on the free movement of such data, and repealing directive 95/46. Official Journal… [find](https://scholar.google.com/scholar?q=General+Data+Protection+Regulation%2E+2016%2E+Regulation+%28eu%29+2016%2F679+of+the+european+parliament+and+of+the+council+of+27+april+2016+on+the+protection+of+natural+persons+with+regard+to+the+processing+of+)
32. Marco Tulio Ribeiro, UW EDU, Sameer Singh, and Carlos Guestrin. 2016. Model-Agnostic Interpretability of Machine Learning. In ICML Workshop on Human Interpretability in Machine Learning. [find](https://scholar.google.com/scholar?q=Marco+Tulio+Ribeiro%2C+UW+EDU%2C+Sameer+Singh%2C+and+Carlos+Guestrin%2E+2016%2E+Model%2DAgnostic+Interpretability+of+Machine+Learning%2E+In+ICML+Workshop+on+Human+Interpretability+in+Machine+Learning%2E)
33. Michael L Rich. 2016. Machine learning, automated suspicion algorithms, and the fourth amendment. University of Pennsylvania Law Review, pages 871– 929. [find](https://scholar.google.com/scholar?q=Michael+L+Rich%2E+2016%2E+Machine+learning%2C+automated+suspicion+algorithms%2C+and+the+fourth+amendment%2E+University+of+Pennsylvania+Law+Review%2C+pages+871%E2%80%93+929%2E)
34. Marko Robnik-Šikonja and Marko Bohanec. 2018. Perturbation-based explanations of prediction models. In Human and machine learning, pages 159– 175. Springer. [find](https://scholar.google.com/scholar?q=Marko+Robnik%2DS%CC%8Cikonja+and+Marko+Bohanec%2E+2018%2E+Perturbation%2Dbased+explanations+of+prediction+models%2E+In+Human+and+machine+learning%2C+pages+159%E2%80%93+175%2E+Springer%2E)
35. Lloyd S Shapley. 1953. A value for n-person games. Contributions to the Theory of Games, 2(28):307– 317. [find](https://scholar.google.com/scholar?q=Lloyd+S+Shapley%2E+1953%2E+A+value+for+n%2Dperson+games%2E+Contributions+to+the+Theory+of+Games%2C+2%2828%29%3A307%E2%80%93+317%2E)
36. Karen Simonyan, Andrea Vedaldi, and Andrew Zisserman. 2013. Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps. CoRR, abs/1312.6034. [find](https://scholar.google.com/scholar?q=Karen+Simonyan%2C+Andrea+Vedaldi%2C+and+Andrew+Zisserman%2E+2013%2E+Deep+Inside+Convolutional+Networks%3A+Visualising+Image+Classification+Models+and+Saliency+Maps%2E+CoRR%2C+abs%2F1312%2E6034%2E) [our card](https://md.research.elamur.ai/free/en/papers/simonyan-2013-deep-inside-convolutional-networks-visualising/)
37. Jost Tobias Springenberg, Alexey Dosovitskiy, Thomas Brox, and Martin Riedmiller. 2014. Striving for simplicity: The all convolutional net. arXiv preprint arXiv:1412.6806. — [arXiv:1412.6806](https://arxiv.org/abs/1412.6806) [find](https://scholar.google.com/scholar?q=Jost+Tobias+Springenberg%2C+Alexey+Dosovitskiy%2C+Thomas+Brox%2C+and+Martin+Riedmiller%2E+2014%2E+Striving+for+simplicity%3A+The+all+convolutional+net%2E+arXiv+preprint+arXiv%3A1412%2E6806%2E+%E2%80%94+%5BarXiv%3A1412%2E6806%5D%28https%3A%2F%2F)
38. Sanjay Subramanian, Ben Bogin, Nitish Gupta, Tomer Wolfson, Sameer Singh, Jonathan Berant, and Matt Gardner. 2020. Obtaining faithful interpretations from compositional neural networks. arXiv preprint arXiv:2005.00724. [find](https://scholar.google.com/scholar?q=Sanjay+Subramanian%2C+Ben+Bogin%2C+Nitish+Gupta%2C+Tomer+Wolfson%2C+Sameer+Singh%2C+Jonathan+Berant%2C+and+Matt+Gardner%2E+2020%2E+Obtaining+faithful+interpretations+from+compositional+neural+networks%2E+arXiv+preprint)
39. Mukund Sundararajan, Ankur Taly, and Qiqi Yan. 2017. Axiomatic attribution for deep networks. In Proceedings of the 34th International Conference on Machine Learning-Volume 70, pages 3319–3328. JMLR. org. [find](https://scholar.google.com/scholar?q=Mukund+Sundararajan%2C+Ankur+Taly%2C+and+Qiqi+Yan%2E+2017%2E+Axiomatic+attribution+for+deep+networks%2E+In+Proceedings+of+the+34th+International+Conference+on+Machine+Learning%2DVolume+70%2C+pages+3319%E2%80%933328%2E+JMLR%2E+)
40. Shikhar Vashishth, Shyam Upadhyay, Gaurav Singh Tomar, and Manaal Faruqui. 2019. Attention interpretability across nlp tasks. arXiv preprint arXiv:1909.11218. — [arXiv:1909.11218](https://arxiv.org/abs/1909.11218) [find](https://scholar.google.com/scholar?q=Shikhar+Vashishth%2C+Shyam+Upadhyay%2C+Gaurav+Singh+Tomar%2C+and+Manaal+Faruqui%2E+2019%2E+Attention+interpretability+across+nlp+tasks%2E+arXiv+preprint+arXiv%3A1909%2E11218%2E+%E2%80%94+%5BarXiv%3A1909%2E11218%5D%28https%3A%2F%2Farxiv%2Eorg%2Fab)
41. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In Advances in neural information processing systems, pages 5998–6008. [find](https://scholar.google.com/scholar?q=Ashish+Vaswani%2C+Noam+Shazeer%2C+Niki+Parmar%2C+Jakob+Uszkoreit%2C+Llion+Jones%2C+Aidan+N+Gomez%2C+%C5%81ukasz+Kaiser%2C+and+Illia+Polosukhin%2E+2017%2E+Attention+is+all+you+need%2E+In+Advances+in+neural+information+processi)
42. Jörg Wagner, Jan Mathias Köhler, Tobias Gindele, Leon Hetzel, Jakob Thaddäus Wiedemer, and Sven Behnke. 2019. Interpretable and Fine-Grained Visual Explanations for Convolutional Neural Networks. In 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 9089–9099. IEEE. [find](https://scholar.google.com/scholar?q=Jo%CC%88rg+Wagner%2C+Jan+Mathias+Ko%CC%88hler%2C+Tobias+Gindele%2C+Leon+Hetzel%2C+Jakob+Thadda%CC%88us+Wiedemer%2C+and+Sven+Behnke%2E+2019%2E+Interpretable+and+Fine%2DGrained+Visual+Explanations+for+Convolutional+Neural+Networks%2E+I)
43. Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pier-ric Cistac, Tim Rault, R’emi Louf, Morgan Funtowicz, and Jamie Brew. 2019. HuggingFace’s Transformers: State-of-the-art Natural Language Processing. ArXiv, abs/1910.03771. [find](https://scholar.google.com/scholar?q=Thomas+Wolf%2C+Lysandre+Debut%2C+Victor+Sanh%2C+Julien+Chaumond%2C+Clement+Delangue%2C+Anthony+Moi%2C+Pier%2Dric+Cistac%2C+Tim+Rault%2C+R%E2%80%99emi+Louf%2C+Morgan+Funtowicz%2C+and+Jamie+Brew%2E+2019%2E+HuggingFace%E2%80%99s+Transformers%3A+St) [our card](https://md.research.elamur.ai/free/en/papers/wolf-2020-transformers-state-art-natural-language/)
44. Omar Zaidan, Jason Eisner, and Christine Piatko. 2007. Using “Annotator Rationales” to Improve Machine Learning for Text Categorization. In Human Language Technologies 2007: The Conference of the North American Chapter of the Association for Computational Linguistics; Proceedings of the Main Confere… [find](https://scholar.google.com/scholar?q=Omar+Zaidan%2C+Jason+Eisner%2C+and+Christine+Piatko%2E+2007%2E+Using+%E2%80%9CAnnotator+Rationales%E2%80%9D+to+Improve+Machine+Learning+for+Text+Categorization%2E+In+Human+Language+Technologies+2007%3A+The+Conference+of+the+Nort) [our card](https://md.research.elamur.ai/free/en/papers/zaidan-2007-annotator-rationales-improve-machine-learning/)
45. Matthew D Zeiler and Rob Fergus. 2014. Visualizing and understanding convolutional networks. In European conference on computer vision, pages 818–833. Springer. [find](https://scholar.google.com/scholar?q=Matthew+D+Zeiler+and+Rob+Fergus%2E+2014%2E+Visualizing+and+understanding+convolutional+networks%2E+In+European+conference+on+computer+vision%2C+pages+818%E2%80%93833%2E+Springer%2E) [our card](https://md.research.elamur.ai/free/en/papers/zeiler-2013-visualizing-understanding-convolutional-networks/)