---
title: "Interpreting Neural Networks to Improve — Aubakirova 2016"
description: "The paper asks whether a neural model can predict politeness in online requests without hand-crafted features, and what such a model actually learns."
authors: ["Malika Aubakirova", "Mohit Bansal"]
year: 2016
venue: "Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing"
source: "https://aclanthology.org/D16-1216/"
doi: "10.18653/v1/d16-1216"
arxiv: "1610.02683"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/papers/aubakirova-2016-interpreting-neural-networks-improve-politeness/"
---
[Elamur Research](https://md.research.elamur.ai/) · [this page in HTML](https://research.elamur.ai/free/papers/aubakirova-2016-interpreting-neural-networks-improve-politeness/)

[home](https://md.research.elamur.ai/) / [papers](https://md.research.elamur.ai/free/papers/) / [2016](https://md.research.elamur.ai/free/papers/2016/page/2/) / Interpreting Neural Networks to Improve Politeness Comprehension

# Interpreting Neural Networks to Improve Politeness Comprehension

**Authors:** Malika Aubakirova, [Mohit Bansal](https://md.research.elamur.ai/free/authors/mohit-bansal/) · **Year:** 2016 · **Venue:** Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/D16-1216/) · [PDF](https://aclanthology.org/D16-1216.pdf) · [DOI 10.18653/v1/d16-1216](https://doi.org/10.18653/v1/d16-1216) · [arXiv 1610.02683](https://arxiv.org/abs/1610.02683) · [OpenAlex](https://openalex.org/W2530299326)

**Citations:** 58 citations (openalex-snapshot · 2026-10-14) · [28](#cited-by) in our library, of the 140673 papers whose bibliographies we read

## Summary
The paper asks whether a neural model can predict politeness in online requests without hand-crafted features, and what such a model actually learns. Using the Wikipedia (Wiki) and Stack Exchange (SE) request corpora of Danescu-Niculescu-Mizil et al. (2013), the authors train a single-layer CNN with max-pooling on raw text. They use a train/dev/test split instead of leave-one-out cross-validation. SE serves purely as a transfer domain with no retuning. The CNN outperforms both the bag-of-words baseline and the theory-driven linguistic-feature baseline.

The main contribution is interpretive. Three probing techniques are applied:
- activation clustering of individual neurons,
- first-derivative saliency heatmaps,
- PCA plots of word embeddings before and after training.

These analyses recover known politeness strategies such as Deference, Direct Question, Gratitude and Counterfactual Modal. They also refine some of them and expose two new negative-politeness cues: indefinite pronouns and punctuation. Adding these cues to the feature-based system narrows its gap to the CNN. This gives a measurable explanation of the network's advantage and a reusable recipe for inspecting CNNs in other NLP tasks.

## Key points
- Accuracy (Wiki / SE):
  - Bag-of-Words: 80.9% / 64.6%
  - Linguistic Features: 82.6% / 65.2%
  - Linguistic Features with discovered features: 83.8% / 65.7%
  - CNN: 85.8% / 66.4%
- CNN setup:
  - filter windows of 3, 4 and 5 words, 75 feature maps each, ReLU
  - Adam with learning rate 0.001, dropout 0.5, batch size 32
  - all tuning done on the Wiki dev set
- New strategies:
  - Indefinite Pronouns (e.g., something, somebody) score −0.13 for politeness, with 39% in the top quartile. This beats 2nd Person at 30%.
  - Punctuation (e.g., "???", "...") scores −0.71, the strongest negative strategy, with 62% in the top quartile.
- Saliency and embedding analyses highlight the importance of "thanks for", "help" and "why would you". After training, "please" and modals like "could" drift away from the positive cluster.
- The visualizations also prompted dropping the older pronoun features (#14–18). That removal and the two new features each account for roughly half of the baseline's gain.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

Fields: [Linguistics & languages](https://md.research.elamur.ai/free/shelves/linguistics/)

## Cited by

Cited by

28 papers of our library cite it (of the 140673 whose bibliographies we read); 7 of them have a page here, 21 not yet.

- [Contextualized Embeddings for Enriching Linguistic Analyses on Politeness](https://md.research.elamur.ai/free/papers/aljanaideh-2020-contextualized-embeddings-enriching-linguistic-analyses/)
- [Gender and Politeness Perception: A Novel Approach for Exploring Annotations Disagreement](https://md.research.elamur.ai/free/papers/aljanaideh-2026-gender-politeness-perception-novel-approach/)
- [Analysis Methods in Neural Language Processing: A Survey](https://md.research.elamur.ai/free/papers/belinkov-2019-analysis-methods-neural-language-processing/)
- [Analyzing Politeness in Arabic Tweets: A Preliminary Study](https://md.research.elamur.ai/free/papers/khalifa-2024-analyzing-politeness-arabic-tweets-preliminary/)
- [TyDiP: A Dataset for Politeness Classification in Nine Typologically Diverse Languages](https://md.research.elamur.ai/free/papers/srinivasan-2022-tydip-dataset-politeness-classification-nine/)
- [Conversations Gone Awry: Detecting Early Signs of Conversational Failure](https://md.research.elamur.ai/free/papers/zhang-2018-conversations-gone-awry-detecting-early/)
- [Comparing human and LLM politeness strategies in free production](https://md.research.elamur.ai/free/papers/zhao-2025-comparing-human-llm-politeness-strategies/)

## Cite

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{aubakirova2016interpreting,
  title = {{Interpreting Neural Networks to Improve Politeness Comprehension}},
  author = {Aubakirova, Malika and Bansal, Mohit},
  booktitle = {Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing},
  year = {2016},
  doi = {10.18653/v1/d16-1216},
  eprint = {1610.02683},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.18653/v1/d16-1216}
}
```

- [← previous International translation in the Grammatical Framework (GF)](https://md.research.elamur.ai/free/papers/ranta-2016-international-translation-grammatical-framework-gf/)
- [next → Intra-Sentential Subject Zero Anaphora Resolution using Multi-Column Convolutional Neural Network](https://md.research.elamur.ai/free/papers/iida-2016-intra-sentential-subject-zero-anaphora/)
- [↑ back to papers of 2016](https://md.research.elamur.ai/free/papers/2016/page/2/)

## References

31 references, 8 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Francesca Bargiela-Chiappini. 2003. Face and politeness: new (insights) for old (concepts). Journal of pragmatics, 35(10):1453–1469. [↗](https://doi.org/10.1016/s0378-2166(02)00173-x)
2. Penelope Brown and Stephen C Levinson. 1987. Politeness: Some universals in language usage, volume 4. Cambridge university press. [find](https://scholar.google.com/scholar?q=Penelope+Brown+and+Stephen+C+Levinson%2E+1987%2E+Politeness%3A+Some+universals+in+language+usage%2C+volume+4%2E+Cambridge+university+press%2E)
3. Long-Sheng Chen, Cheng-Hsiang Liu, and Hui-Ju Chiu. 2011. A neural network based approach for sentiment classification in the blogosphere. Journal of Informet-rics, 5(2):313–322. [↗](https://doi.org/10.1016/j.joi.2011.01.003)
4. Ronan Collobert, Jason Weston, Léon Bottou, Michael Karlen, Koray Kavukcuoglu, and Pavel Kuksa. 2011. Natural language processing (almost) from scratch. The Journal of Machine Learning Research, 12:2493– 2537. [find](https://scholar.google.com/scholar?q=Ronan+Collobert%2C+Jason+Weston%2C+Le%CC%81on+Bottou%2C+Michael+Karlen%2C+Koray+Kavukcuoglu%2C+and+Pavel+Kuksa%2E+2011%2E+Natural+language+processing+%28almost%29+from+scratch%2E+The+Journal+of+Machine+Learning+Research%2C+12%3A2) [our card](https://md.research.elamur.ai/free/papers/collobert-2011-natural-language-processing-almost-scratch/)
5. Cristian Danescu-Niculescu-Mizil, Moritz Sudhof, Dan Jurafsky, Jure Leskovec, and Christopher Potts. 2013. A computational approach to politeness with application to social factors. In Proceedings of ACL. [↗](https://doi.org/10.48550/arxiv.1306.6078) [our card](https://md.research.elamur.ai/free/papers/mizil-2013-computational-approach-politeness-application-social/)
6. Li Dong, Furu Wei, Chuanqi Tan, Duyu Tang, Ming Zhou, and Ke Xu. 2014. Adaptive recursive neural network for target-dependent twitter sentiment classification. In Proceedings of ACL, pages 49–54. [find](https://scholar.google.com/scholar?q=Li+Dong%2C+Furu+Wei%2C+Chuanqi+Tan%2C+Duyu+Tang%2C+Ming+Zhou%2C+and+Ke+Xu%2E+2014%2E+Adaptive+recursive+neural+network+for+target%2Ddependent+twitter+sentiment+classification%2E+In+Proceedings+of+ACL%2C+pages+49%E2%80%9354%2E)
7. Cı́cero Nogueira dos Santos and Maira Gatti. 2014. Deep convolutional neural networks for sentiment analysis of short texts. In Proceedings of COLING, pages 69– 78. [find](https://scholar.google.com/scholar?q=C%C4%B1%CC%81cero+Nogueira+dos+Santos+and+Maira+Gatti%2E+2014%2E+Deep+convolutional+neural+networks+for+sentiment+analysis+of+short+texts%2E+In+Proceedings+of+COLING%2C+pages+69%E2%80%93+78%2E)
8. Dumitru Erhan, Yoshua Bengio, Aaron Courville, and Pascal Vincent. 2009. Visualizing higher-layer features of a deep network. University of Montreal, 1341. [find](https://scholar.google.com/scholar?q=Dumitru+Erhan%2C+Yoshua+Bengio%2C+Aaron+Courville%2C+and+Pascal+Vincent%2E+2009%2E+Visualizing+higher%2Dlayer+features+of+a+deep+network%2E+University+of+Montreal%2C+1341%2E)
9. M Ghiassi, J Skinner, and D Zimbra. 2013. Twitter brand sentiment analysis: A hybrid system using ngram analysis and dynamic artificial neural network. Expert Systems with applications, 40(16):6266–6282. [find](https://scholar.google.com/scholar?q=M+Ghiassi%2C+J+Skinner%2C+and+D+Zimbra%2E+2013%2E+Twitter+brand+sentiment+analysis%3A+A+hybrid+system+using+ngram+analysis+and+dynamic+artificial+neural+network%2E+Expert+Systems+with+applications%2C+40%2816%29%3A6266%E2%80%9362)
10. Ross Girshick, Jeff Donahue, Trevor Darrell, and Jiten-dra Malik. 2014. Rich feature hierarchies for accurate object detection and semantic segmentation. In Proceedings of CVPR, pages 580–587. [find](https://scholar.google.com/scholar?q=Ross+Girshick%2C+Jeff+Donahue%2C+Trevor+Darrell%2C+and+Jiten%2Ddra+Malik%2E+2014%2E+Rich+feature+hierarchies+for+accurate+object+detection+and+semantic+segmentation%2E+In+Proceedings+of+CVPR%2C+pages+580%E2%80%93587%2E) [our card](https://md.research.elamur.ai/free/papers/girshick-2013-rich-feature-hierarchies-accurate-object/)
11. Alec Go, Richa Bhayani, and Lei Huang. 2009. Twitter sentiment classification using distant supervision. CS224N Project Report, Stanford, 1:12. [find](https://scholar.google.com/scholar?q=Alec+Go%2C+Richa+Bhayani%2C+and+Lei+Huang%2E+2009%2E+Twitter+sentiment+classification+using+distant+supervision%2E+CS224N+Project+Report%2C+Stanford%2C+1%3A12%2E)
12. Daena J Goldsmith. 2007. Brown and levinsons politeness theory. Explaining communication: Contemporary theories and exemplars, pages 219–236. [find](https://scholar.google.com/scholar?q=Daena+J+Goldsmith%2E+2007%2E+Brown+and+levinsons+politeness+theory%2E+Explaining+communication%3A+Contemporary+theories+and+exemplars%2C+pages+219%E2%80%93236%2E)
13. Yueguo Gu. 1990. Politeness phenomena in modern chinese. Journal of pragmatics, 14(2):237–257. [↗](https://doi.org/10.1016/0378-2166(90)90082-o)
14. Dániel Z Kádár and Michael Haugh. 2013. Understanding politeness. Cambridge University Press. [find](https://scholar.google.com/scholar?q=Da%CC%81niel+Z+Ka%CC%81da%CC%81r+and+Michael+Haugh%2E+2013%2E+Understanding+politeness%2E+Cambridge+University+Press%2E)
15. Akos Kádár, Grzegorz Chrupała, and Afra Alishahi. 2016. Representation of linguistic form and function in recurrent neural networks. arXiv preprint arXiv:1602.08952. [↗](https://arxiv.org/abs/1602.08952)
16. Nal Kalchbrenner, Edward Grefenstette, and Phil Blunsom. 2014. A convolutional neural network for modelling sentences. In Proceedings of ACL. [↗](https://doi.org/10.48550/arxiv.1404.2188) [our card](https://md.research.elamur.ai/free/papers/kalchbrenner-2014-convolutional-neural-network-modelling-sentences/)
17. Andrej Karpathy, Justin Johnson, and Fei-Fei Li. 2016. Visualizing and understanding recurrent networks. In Proceedings of ICLR Workshop. [↗](https://doi.org/10.48550/arxiv.1506.02078) [our card](https://md.research.elamur.ai/free/papers/karpathy-2015-visualizing-understanding-recurrent-networks/)
18. Alistair Kennedy and Diana Inkpen. 2006. Sentiment classification of movie reviews using contextual valence shifters. Computational intelligence, 22(2):110– 125. [↗](https://doi.org/10.1111/j.1467-8640.2006.00277.x)
19. Diederik Kingma and Jimmy Ba. 2015. Adam: A method for stochastic optimization. In Proceedings of ICLR. [↗](https://doi.org/10.48550/arxiv.1412.6980)
20. Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. 2012. Imagenet classification with deep convolutional neural networks. In Proceedings of NIPS, pages 1097– 1105. [find](https://scholar.google.com/scholar?q=Alex+Krizhevsky%2C+Ilya+Sutskever%2C+and+Geoffrey+E+Hinton%2E+2012%2E+Imagenet+classification+with+deep+convolutional+neural+networks%2E+In+Proceedings+of+NIPS%2C+pages+1097%E2%80%93+1105%2E)
21. Jiwei Li, Xinlei Chen, Eduard Hovy, and Dan Jurafsky. 2016. Visualizing and understanding neural models in NLP. In Proceedings of NAACL. [find](https://scholar.google.com/scholar?q=Jiwei+Li%2C+Xinlei+Chen%2C+Eduard+Hovy%2C+and+Dan+Jurafsky%2E+2016%2E+Visualizing+and+understanding+neural+models+in+NLP%2E+In+Proceedings+of+NAACL%2E) [our card](https://md.research.elamur.ai/free/papers/li-2016-visualizing-understanding-neural-models-nlp/)
22. Miriam A Locher and Richard J Watts. 2005. Politeness theory and relational work. Journal of Politeness Research. Language, Behaviour, Culture, 1(1):9–33. [↗](https://doi.org/10.1515/jplr.2005.1.1.9)
23. Aravindh Mahendran and Andrea Vedaldi. 2015. Understanding deep image representations by inverting them. In Proceedings of CVPR, pages 5188–5196. IEEE. [↗](https://doi.org/10.48550/arxiv.1412.0035)
24. Rodrigo Moraes, Joao Francisco Valiati, and Wilson P GaviãO Neto. 2013. Document-level sentiment classification: An empirical comparison between svm and ann. Expert Systems with Applications, 40(2):621–633. [↗](https://doi.org/10.1016/j.eswa.2012.07.059)
25. Vinod Nair and Geoffrey E Hinton. 2010. Rectified linear units improve restricted boltzmann machines. In Proceedings of ICML, pages 807–814. [find](https://scholar.google.com/scholar?q=Vinod+Nair+and+Geoffrey+E+Hinton%2E+2010%2E+Rectified+linear+units+improve+restricted+boltzmann+machines%2E+In+Proceedings+of+ICML%2C+pages+807%E2%80%93814%2E)
26. Bo Pang and Lillian Lee. 2004. A sentimental education: Sentiment analysis using subjectivity summarization based on minimum cuts. In Proceedings of ACL, page 271. [↗](https://doi.org/10.48550/arxiv.cs/0409058)
27. Bo Pang, Lillian Lee, and Shivakumar Vaithyanathan. 2002. Thumbs up?: sentiment classification using machine learning techniques. In Proceedings of EMNLP, pages 79–86. [↗](https://doi.org/10.48550/arxiv.cs/0205070)
28. Wojciech Samek, Alexander Binder, Grégoire Montavon, Sebastian Bach, and Klaus-Robert Müller. 2016. Evaluating the visualization of what a deep neural network has learned. IEEE Transactions on Neural Networks and Learning Systems. [↗](https://doi.org/10.1109/tnnls.2016.2599820)
29. Karen Simonyan, Andrea Vedaldi, and Andrew Zisserman. 2014. Deep inside convolutional networks: Visualising image classification models and saliency maps. In Proceedings of ICLR Workshop. [↗](https://doi.org/10.48550/arxiv.1312.6034) [our card](https://md.research.elamur.ai/free/papers/simonyan-2013-deep-inside-convolutional-networks-visualising/)
30. Richard Socher, Alex Perelygin, Jean Y Wu, Jason Chuang, Christopher D Manning, Andrew Y Ng, and Christopher Potts. 2013. Recursive deep models for semantic compositionality over a sentiment treebank. In Proceedings of EMNLP. [find](https://scholar.google.com/scholar?q=Richard+Socher%2C+Alex+Perelygin%2C+Jean+Y+Wu%2C+Jason+Chuang%2C+Christopher+D+Manning%2C+Andrew+Y+Ng%2C+and+Christopher+Potts%2E+2013%2E+Recursive+deep+models+for+semantic+compositionality+over+a+sentiment+treebank%2E)
31. Matthew D Zeiler and Rob Fergus. 2014. Visualizing and understanding convolutional networks. In Proceedings of ECCV, pages 818–833. Springer. [↗](https://doi.org/10.48550/arxiv.1311.2901) [our card](https://md.research.elamur.ai/free/papers/zeiler-2013-visualizing-understanding-convolutional-networks/)