---
title: "Human Rationales as Attribution Priors for — Jayaram 2021"
description: "Stance classifiers often reach correct labels for reasons humans would not accept."
authors: ["Sahil Jayaram", "Emily Allaway"]
year: 2021
venue: "Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing"
source: "https://aclanthology.org/2021.emnlp-main.450/"
doi: "10.18653/v1/2021.emnlp-main.450"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/papers/jayaram-2021-human-rationales-as-attribution-priors/"
---
[Elamur Research](https://md.research.elamur.ai/) · [this page in HTML](https://research.elamur.ai/free/papers/jayaram-2021-human-rationales-as-attribution-priors/)

[home](https://md.research.elamur.ai/) / [papers](https://md.research.elamur.ai/free/papers/) / [2021](https://md.research.elamur.ai/free/papers/2021/page/2/) / Human Rationales as Attribution Priors for Explainable Stance Detection

# Human Rationales as Attribution Priors for Explainable Stance Detection

**Authors:** Sahil Jayaram, [Emily Allaway](https://md.research.elamur.ai/free/authors/emily-allaway/) · **Year:** 2021 · **Venue:** Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2021.emnlp-main.450/) · [PDF](https://aclanthology.org/2021.emnlp-main.450.pdf) · [DOI 10.18653/v1/2021.emnlp-main.450](https://doi.org/10.18653/v1/2021.emnlp-main.450) · [OpenAlex](https://openalex.org/W3211640232)

**Citations:** 17 citations (openalex-snapshot · 2026-10-14) · [13](#cited-by) in our library, of the 140673 papers whose bibliographies we read

## Summary
Stance classifiers often reach correct labels for reasons humans would not accept. This matters for sensitive political or religious topics and when models must handle topics never seen in training. Jayaram and Allaway address this by adding an attribution prior: an extra loss term that penalizes a classifier when its word-importance scores differ from human judgements. Using Amazon Mechanical Turk, they collected human rationales (the words most indicative of stance) for roughly 500 training arguments and a small test sample from VAST, a corpus of New York Times comments with zero-shot and few-shot test topics. The model is a fine-tuned version of BERT-joint. Its attributions come from mean attention weights (MAW), and the prior is a weighted mean squared error between normalized model and human scores. Experiments were run on the full training set and on reduced versions. When data is scarce, the prior makes rationales closer to human ones, including for sarcasm, without lowering accuracy. Contrary to much recent criticism of attention, MAW also gave more faithful explanations than Gradient × Input (GI).

## Key points
- **Annotation process:** Three qualified MTurk workers annotated each example. Oracle attributions weight each worker by their worker quality score (average 0.58). Annotators flagged about 26% of tokens as important.
- **What humans selected:** Topic words appeared among the selections in only 51% of examples (267/519). Stopwords made up about 44% of important words. MPQA sentiment words accounted for only about 10% (positive) and 9% (negative), showing that human rationales are not simple topic or sentiment cues.
- **Rationale quality:** In MAW-based rationale loss, the prior model significantly beat the baseline in all data settings (full, reduced 25, reduced 10). It also produced better rationales for all five VAST challenge phenomena (Imp, mlT, mlS, Qte, Sarc) while keeping comparable accuracy.
- **Error analysis:** Of 50 manually inspected errors, 68% were semantic (missed negations, rhetorical questions, sarcasm), 46% selected too few or too many words, and 30% stemmed from data annotation problems.
- **Availability:** Data and models are released at github.com/SahilJ97/Explainable-Stance-Detection.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

Fields: [Linguistics & languages](https://md.research.elamur.ai/free/shelves/linguistics/)

## Cited by

Cited by

13 papers of our library cite it (of the 140673 whose bibliographies we read); 2 of them have a page here, 11 not yet.

- [POLITICS: Pretraining with Same-story Article Comparison for Ideology Prediction and Stance Detection](https://md.research.elamur.ai/free/papers/liu-2022-politics-pretraining-same-story-article/)
- [Exploiting Sentiment and Common Sense for Zero-shot Stance Detection](https://md.research.elamur.ai/free/papers/luo-2022-exploiting-sentiment-common-sense-zero/)

## Cite

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{jayaram2021human,
  title = {{Human Rationales as Attribution Priors for Explainable Stance Detection}},
  author = {Jayaram, Sahil and Allaway, Emily},
  booktitle = {Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing},
  year = {2021},
  doi = {10.18653/v1/2021.emnlp-main.450},
  url = {https://doi.org/10.18653/v1/2021.emnlp-main.450}
}
```

- [← previous HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Units](https://md.research.elamur.ai/free/papers/hsu-2021-hubert-self-supervised-speech-representation/)
- [next → Hurdles to Progress in Long-form Question Answering](https://md.research.elamur.ai/free/papers/krishna-2021-hurdles-progress-long-form-question/)
- [↑ back to papers of 2021](https://md.research.elamur.ai/free/papers/2021/page/2/)

## References

40 references, 10 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Rob Abbott, Brian Ecker, Pranav Anand, and Marilyn A. Walker. 2016. Internet argument corpus 2.0: An sql schema for dialogic social media and the corpora to go with it. In LREC. [find](https://scholar.google.com/scholar?q=Rob+Abbott%2C+Brian+Ecker%2C+Pranav+Anand%2C+and+Marilyn+A%2E+Walker%2E+2016%2E+Internet+argument+corpus+2%2E0%3A+An+sql+schema+for+dialogic+social+media+and+the+corpora+to+go+with+it%2E+In+LREC%2E)
2. Samira Abnar and W. Zuidema. 2020. tifying attention flow in transformers. abs/2005.00928. — [arXiv:2005.00928](https://arxiv.org/abs/2005.00928) [find](https://scholar.google.com/scholar?q=Samira+Abnar+and+W%2E+Zuidema%2E+2020%2E+tifying+attention+flow+in+transformers%2E+abs%2F2005%2E00928%2E+%E2%80%94+%5BarXiv%3A2005%2E00928%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2005%2E00928%29)
3. Quan-ArXiv, Emily Allaway and Kathleen McKeown. 2020. Zero-shot stance detection: A dataset and model using generalized topic representations. In EMNLP. [find](https://scholar.google.com/scholar?q=Quan%2DArXiv%2C+Emily+Allaway+and+Kathleen+McKeown%2E+2020%2E+Zero%2Dshot+stance+detection%3A+A+dataset+and+model+using+generalized+topic+representations%2E+In+EMNLP%2E) [our card](https://md.research.elamur.ai/free/papers/allaway-2020-zero-shot-stance-detection-dataset/)
4. Emily Allaway, Malavika Srikanth, and K. McKeown. 2021. Adversarial learning for zero-shot stance detection on social media. ArXiv, abs/2105.06603. [find](https://scholar.google.com/scholar?q=Emily+Allaway%2C+Malavika+Srikanth%2C+and+K%2E+McKeown%2E+2021%2E+Adversarial+learning+for+zero%2Dshot+stance+detection+on+social+media%2E+ArXiv%2C+abs%2F2105%2E06603%2E) [our card](https://md.research.elamur.ai/free/papers/allaway-2021-adversarial-learning-zero-shot-stance/)
5. Pepa Atanasova, Jakob Grue Simonsen, Christina Li-oma, and Isabelle Augenstein. 2020. A diagnostic study of explainability techniques for text classification. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 3256–3274, Online. Association for C… [find](https://scholar.google.com/scholar?q=Pepa+Atanasova%2C+Jakob+Grue+Simonsen%2C+Christina+Li%2Doma%2C+and+Isabelle+Augenstein%2E+2020%2E+A+diagnostic+study+of+explainability+techniques+for+text+classification%2E+In+Proceedings+of+the+2020+Conference+on+)
6. Isabelle Augenstein, Tim Rocktäschel, Andreas Vlachos, and Kalina Bontcheva. 2016. Stance detection with bidirectional conditional encoding. In EMNLP. [find](https://scholar.google.com/scholar?q=Isabelle+Augenstein%2C+Tim+Rockt%C3%A4schel%2C+Andreas+Vlachos%2C+and+Kalina+Bontcheva%2E+2016%2E+Stance+detection+with+bidirectional+conditional+encoding%2E+In+EMNLP%2E) [our card](https://md.research.elamur.ai/free/papers/augenstein-2016-stance-detection-bidirectional-conditional-encoding/)
7. S. Bach, Alexander Binder, Grégoire Montavon, F. Klauschen, K. Müller, and W. Samek. 2015. On pixel-wise explanations for non-linear classifier decisions by layer-wise relevance propagation. PLoS ONE, 10. [find](https://scholar.google.com/scholar?q=S%2E+Bach%2C+Alexander+Binder%2C+Gr%C3%A9goire+Montavon%2C+F%2E+Klauschen%2C+K%2E+M%C3%BCller%2C+and+W%2E+Samek%2E+2015%2E+On+pixel%2Dwise+explanations+for+non%2Dlinear+classifier+decisions+by+layer%2Dwise+relevance+propagation%2E+PLoS+ONE%2C)
8. J. Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. Bert: Pre-training of deep bidirectional transformers for language understanding. In NAACL-HLT. [find](https://scholar.google.com/scholar?q=J%2E+Devlin%2C+Ming%2DWei+Chang%2C+Kenton+Lee%2C+and+Kristina+Toutanova%2E+2019%2E+Bert%3A+Pre%2Dtraining+of+deep+bidirectional+transformers+for+language+understanding%2E+In+NAACL%2DHLT%2E)
9. We use a dataset collected and distributed by Allaway and McKeown (2020): [https://github.com/emilyallaway/zero-shot-stance](https://github.com/emilyallaway/zero-shot-stance). Jay DeYoung, Sarthak Jain, Nazneen Fatema Rajani, Data was collected from publicly available Eric Lehman, Caiming Xiong, Richard Socher, and Byron C. Wallace. 2020. ERASER: A… — [arXiv:1808.06080](https://arxiv.org/abs/1808.06080) [↗](https://github.com/emilyallaway/zero-shot-stance)
10. G. Erion, J. Janizek, Pascal Sturmfels, Scott M. Lundberg, and Su-In Lee. 2020. Improving performance of deep learning models with axiomatic attribution priors and expected gradients. arXiv: Learning. [find](https://scholar.google.com/scholar?q=G%2E+Erion%2C+J%2E+Janizek%2C+Pascal+Sturmfels%2C+Scott+M%2E+Lundberg%2C+and+Su%2DIn+Lee%2E+2020%2E+Improving+performance+of+deep+learning+models+with+axiomatic+attribution+priors+and+expected+gradients%2E+arXiv%3A+Learning%2E)
11. Amirata Ghorbani, Abubakar Abid, and James Zou. 2018. Interpretation of neural networks is fragile. [find](https://scholar.google.com/scholar?q=Amirata+Ghorbani%2C+Abubakar+Abid%2C+and+James+Zou%2E+2018%2E+Interpretation+of+neural+networks+is+fragile%2E)
12. Momchil Hardalov, Arnav Arora, Preslav Nakov, and Isabelle Augenstein. 2021. Cross-domain label-adaptive stance detection. ArXiv, abs/2104.07467. [find](https://scholar.google.com/scholar?q=Momchil+Hardalov%2C+Arnav+Arora%2C+Preslav+Nakov%2C+and+Isabelle+Augenstein%2E+2021%2E+Cross%2Ddomain+label%2Dadaptive+stance+detection%2E+ArXiv%2C+abs%2F2104%2E07467%2E) [our card](https://md.research.elamur.ai/free/papers/hardalov-2021-cross-domain-label-adaptive-stance/)
13. Kazi Saidul Hasan and Vincent Ng. 2014. Why are you taking this stance? identifying and classifying reasons in ideological debates. In EMNLP. [find](https://scholar.google.com/scholar?q=Kazi+Saidul+Hasan+and+Vincent+Ng%2E+2014%2E+Why+are+you+taking+this+stance%3F+identifying+and+classifying+reasons+in+ideological+debates%2E+In+EMNLP%2E)
14. Bernease Herman. 2017. The promise and peril of human evaluation for model interpretability. ArXiv, abs/1711.07414. — [arXiv:1711.07414](https://arxiv.org/abs/1711.07414) [find](https://scholar.google.com/scholar?q=Bernease+Herman%2E+2017%2E+The+promise+and+peril+of+human+evaluation+for+model+interpretability%2E+ArXiv%2C+abs%2F1711%2E07414%2E+%E2%80%94+%5BarXiv%3A1711%2E07414%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F1711%2E07414%29)
15. Alon Jacovi and Yoav Goldberg. 2020. Towards faithfully interpretable nlp systems: How should we define and evaluate faithfulness? In ACL. [find](https://scholar.google.com/scholar?q=Alon+Jacovi+and+Yoav+Goldberg%2E+2020%2E+Towards+faithfully+interpretable+nlp+systems%3A+How+should+we+define+and+evaluate+faithfulness%3F+In+ACL%2E) [our card](https://md.research.elamur.ai/free/papers/jacovi-2020-faithfully-interpretable-nlp-systems-how/)
16. Alon Jacovi and Yoav Goldberg. 2021. Aligning Faithful Interpretations with their Social Attribution. Transactions of the Association for Computational Linguistics, 9:294–310. [find](https://scholar.google.com/scholar?q=Alon+Jacovi+and+Yoav+Goldberg%2E+2021%2E+Aligning+Faithful+Interpretations+with+their+Social+Attribution%2E+Transactions+of+the+Association+for+Computational+Linguistics%2C+9%3A294%E2%80%93310%2E)
17. Sarthak Jain and Byron C. Wallace. 2019. Attention is not explanation. In NAACL-HLT. [find](https://scholar.google.com/scholar?q=Sarthak+Jain+and+Byron+C%2E+Wallace%2E+2019%2E+Attention+is+not+explanation%2E+In+NAACL%2DHLT%2E) [our card](https://md.research.elamur.ai/free/papers/jain-2019-attention-is-not-explanation/)
18. Teja Kanchinadam, Keith Westpfahl, Qian You, and Glenn M. Fung. 2020. Rationale-based human-in-the-loop via supervised attention. In DaSH@KDD. [find](https://scholar.google.com/scholar?q=Teja+Kanchinadam%2C+Keith+Westpfahl%2C+Qian+You%2C+and+Glenn+M%2E+Fung%2E+2020%2E+Rationale%2Dbased+human%2Din%2Dthe%2Dloop+via+supervised+attention%2E+In+DaSH%40KDD%2E)
19. Klaus Krippendorff. 1980. Content analysis an introduction to its methodology. SAGE. [find](https://scholar.google.com/scholar?q=Klaus+Krippendorff%2E+1980%2E+Content+analysis+an+introduction+to+its+methodology%2E+SAGE%2E)
20. Mirko Lai, Alessandra Teresa Cignarella, D. I. H. Farías, C. Bosco, V. Patti, and P. Rosso. 2020. Multilingual stance detection in social media political debates. Comput. Speech Lang., 63:101075. [find](https://scholar.google.com/scholar?q=Mirko+Lai%2C+Alessandra+Teresa+Cignarella%2C+D%2E+I%2E+H%2E+Far%C3%ADas%2C+C%2E+Bosco%2C+V%2E+Patti%2C+and+P%2E+Rosso%2E+2020%2E+Multilingual+stance+detection+in+social+media+political+debates%2E+Comput%2E+Speech+Lang%2E%2C+63%3A101075%2E)
21. Frederick Liu and Besim Avci. 2019. Incorporating priors with feature attribution on text classification. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 6274–6283, Florence, Italy. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Frederick+Liu+and+Besim+Avci%2E+2019%2E+Incorporating+priors+with+feature+attribution+on+text+classification%2E+In+Proceedings+of+the+57th+Annual+Meeting+of+the+Association+for+Computational+Linguistics%2C+pa)
22. Saif M. Mohammad, Svetlana Kiritchenko, Parinaz Sobhani, Xiao-Dan Zhu, and Colin Cherry. 2016. Semeval-2016 task 6: Detecting stance in tweets. In SemEval@NAACL-HLT. [find](https://scholar.google.com/scholar?q=Saif+M%2E+Mohammad%2C+Svetlana+Kiritchenko%2C+Parinaz+Sobhani%2C+Xiao%2DDan+Zhu%2C+and+Colin+Cherry%2E+2016%2E+Semeval%2D2016+task+6%3A+Detecting+stance+in+tweets%2E+In+SemEval%40NAACL%2DHLT%2E)
23. Saif M. Mohammad, Parinaz Sobhani, and Svetlana Kiritchenko. 2017. Stance and sentiment in tweets. ACM Trans. Internet Techn., 17:26:1–26:23. [find](https://scholar.google.com/scholar?q=Saif+M%2E+Mohammad%2C+Parinaz+Sobhani%2C+and+Svetlana+Kiritchenko%2E+2017%2E+Stance+and+sentiment+in+tweets%2E+ACM+Trans%2E+Internet+Techn%2E%2C+17%3A26%3A1%E2%80%9326%3A23%2E)
24. Danish Pruthi, Bhuwan Dhingra, Graham Neubig, and Zachary C. Lipton. 2020. Weakly- and semi-supervised evidence extraction. In Findings of the Association for Computational Linguistics: EMNLP 2020, pages 3965–3970, Online. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Danish+Pruthi%2C+Bhuwan+Dhingra%2C+Graham+Neubig%2C+and+Zachary+C%2E+Lipton%2E+2020%2E+Weakly%2D+and+semi%2Dsupervised+evidence+extraction%2E+In+Findings+of+the+Association+for+Computational+Linguistics%3A+EMNLP+2020%2C+pa)
25. Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin. 2016. "why should i trust you?": Explaining the predictions of any classifier. Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. [find](https://scholar.google.com/scholar?q=Marco+Tulio+Ribeiro%2C+Sameer+Singh%2C+and+Carlos+Guestrin%2E+2016%2E+%22why+should+i+trust+you%3F%22%3A+Explaining+the+predictions+of+any+classifier%2E+Proceedings+of+the+22nd+ACM+SIGKDD+International+Conference+on+Kn) [our card](https://md.research.elamur.ai/free/papers/ribeiro-2016-why-should-i-trust-explaining/)
26. Benjamin Schiller, Johannes Daxenberger, and Iryna Gurevych. 2020. Stance detection benchmark: How robust is your stance detection? ArXiv, abs/2001.01565. [find](https://scholar.google.com/scholar?q=Benjamin+Schiller%2C+Johannes+Daxenberger%2C+and+Iryna+Gurevych%2E+2020%2E+Stance+detection+benchmark%3A+How+robust+is+your+stance+detection%3F+ArXiv%2C+abs%2F2001%2E01565%2E)
27. Jost Tobias Springenberg, A. Dosovitskiy, T. Brox, and Martin A. Riedmiller. 2015. Striving for simplicity: The all convolutional net. CoRR, abs/1412.6806. — [arXiv:1412.6806](https://arxiv.org/abs/1412.6806) [find](https://scholar.google.com/scholar?q=Jost+Tobias+Springenberg%2C+A%2E+Dosovitskiy%2C+T%2E+Brox%2C+and+Martin+A%2E+Riedmiller%2E+2015%2E+Striving+for+simplicity%3A+The+all+convolutional+net%2E+CoRR%2C+abs%2F1412%2E6806%2E+%E2%80%94+%5BarXiv%3A1412%2E6806%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F14)
28. Mukund Sundararajan, Ankur Taly, and Qiqi Yan. 2017. Axiomatic attribution for deep networks. In Proceedings of the 34th International Conference on Machine Learning - Volume 70, ICML’17, page 3319–3328. JMLR.org. [find](https://scholar.google.com/scholar?q=Mukund+Sundararajan%2C+Ankur+Taly%2C+and+Qiqi+Yan%2E+2017%2E+Axiomatic+attribution+for+deep+networks%2E+In+Proceedings+of+the+34th+International+Conference+on+Machine+Learning+%2D+Volume+70%2C+ICML%E2%80%9917%2C+page+3319%E2%80%9333)
29. Mariona Taulé, Maria Antònia Martí, Francisco M. Rangel Pardo, Paolo Rosso, Cristina Bosco, and Viviana Patti. 2017. Overview of the task on stance and gender detection in tweets on catalan independence. In IberEval@SEPLN. [find](https://scholar.google.com/scholar?q=Mariona+Taul%C3%A9%2C+Maria+Ant%C3%B2nia+Mart%C3%AD%2C+Francisco+M%2E+Rangel+Pardo%2C+Paolo+Rosso%2C+Cristina+Bosco%2C+and+Viviana+Patti%2E+2017%2E+Overview+of+the+task+on+stance+and+gender+detection+in+tweets+on+catalan+independen)
30. Adam Tsakalidis, Nikolaos Aletras, Alexandra I. Cristea, and Maria Liakata. 2018. Nowcasting the stance of social media users in a sudden vote: The case of the greek referendum. Proceedings of the 27th ACM International Conference on Information and Knowledge Management. [find](https://scholar.google.com/scholar?q=Adam+Tsakalidis%2C+Nikolaos+Aletras%2C+Alexandra+I%2E+Cristea%2C+and+Maria+Liakata%2E+2018%2E+Nowcasting+the+stance+of+social+media+users+in+a+sudden+vote%3A+The+case+of+the+greek+referendum%2E+Proceedings+of+the+27t)
31. Jannis Vamvas and Rico Sennrich. 2020. X -stance: A multilingual multi-target dataset for stance detection. ArXiv, abs/2003.08385. — [arXiv:2003.08385](https://arxiv.org/abs/2003.08385) [find](https://scholar.google.com/scholar?q=Jannis+Vamvas+and+Rico+Sennrich%2E+2020%2E+X+%2Dstance%3A+A+multilingual+multi%2Dtarget+dataset+for+stance+detection%2E+ArXiv%2C+abs%2F2003%2E08385%2E+%E2%80%94+%5BarXiv%3A2003%2E08385%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2003%2E08385%29)
32. Marilyn A. Walker, Jean E. Fox Tree, Pranav Anand, Rob Abbott, and Joseph King. 2012. A corpus for research on deliberation and debate. In LREC. [find](https://scholar.google.com/scholar?q=Marilyn+A%2E+Walker%2C+Jean+E%2E+Fox+Tree%2C+Pranav+Anand%2C+Rob+Abbott%2C+and+Joseph+King%2E+2012%2E+A+corpus+for+research+on+deliberation+and+debate%2E+In+LREC%2E) [our card](https://md.research.elamur.ai/free/papers/walker-2012-corpus-research-deliberation-debate/)
33. Sarah Wiegreffe and Yuval Pinter. 2019. Attention is not not explanation. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 11–20, Hong Kong, China. Association for… [find](https://scholar.google.com/scholar?q=Sarah+Wiegreffe+and+Yuval+Pinter%2E+2019%2E+Attention+is+not+not+explanation%2E+In+Proceedings+of+the+2019+Conference+on+Empirical+Methods+in+Natural+Language+Processing+and+the+9th+International+Joint+Conf) [our card](https://md.research.elamur.ai/free/papers/wiegreffe-2019-attention-is-not-not-explanation/)
34. Theresa Wilson, Janyce Wiebe, and Claire Cardie. 2017. MPQA Opinion Corpus, page 813–832. Springer. [find](https://scholar.google.com/scholar?q=Theresa+Wilson%2C+Janyce+Wiebe%2C+and+Claire+Cardie%2E+2017%2E+MPQA+Opinion+Corpus%2C+page+813%E2%80%93832%2E+Springer%2E)
35. Zhengxuan Wu and Desmond C. Ong. 2021. On explaining your explanations of BERT: an empirical study with sequence classification. CoRR, abs/2101.00196. — [arXiv:2101.00196](https://arxiv.org/abs/2101.00196) [find](https://scholar.google.com/scholar?q=Zhengxuan+Wu+and+Desmond+C%2E+Ong%2E+2021%2E+On+explaining+your+explanations+of+BERT%3A+an+empirical+study+with+sequence+classification%2E+CoRR%2C+abs%2F2101%2E00196%2E+%E2%80%94+%5BarXiv%3A2101%2E00196%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2101%2E0)
36. Chang Xu, Cécile Paris, Surya Nepal, and Ross Sparks. 2018. Cross-target stance classification with self-attention networks. In ACL. [find](https://scholar.google.com/scholar?q=Chang+Xu%2C+C%C3%A9cile+Paris%2C+Surya+Nepal%2C+and+Ross+Sparks%2E+2018%2E+Cross%2Dtarget+stance+classification+with+self%2Dattention+networks%2E+In+ACL%2E)
37. Omar Zaidan, Jason Eisner, and Christine Piatko. 2007. Using “annotator rationales” to improve machine learning for text categorization. In Human Language Technologies 2007: The Conference of the North American Chapter of the Association for Computational Linguistics; Proceedings of the Main Confere… [find](https://scholar.google.com/scholar?q=Omar+Zaidan%2C+Jason+Eisner%2C+and+Christine+Piatko%2E+2007%2E+Using+%E2%80%9Cannotator+rationales%E2%80%9D+to+improve+machine+learning+for+text+categorization%2E+In+Human+Language+Technologies+2007%3A+The+Conference+of+the+Nort)
38. B. Zhang, M. Yang, Xutao Li, Yunming Ye, Xiaofei Xu, and Kuai Dai. 2020. Enhancing cross-target stance detection with transferable semantic-emotion knowledge. In ACL. [find](https://scholar.google.com/scholar?q=B%2E+Zhang%2C+M%2E+Yang%2C+Xutao+Li%2C+Yunming+Ye%2C+Xiaofei+Xu%2C+and+Kuai+Dai%2E+2020%2E+Enhancing+cross%2Dtarget+stance+detection+with+transferable+semantic%2Demotion+knowledge%2E+In+ACL%2E) [our card](https://md.research.elamur.ai/free/papers/zhang-2020-enhancing-cross-target-stance-detection/)
39. Ye Zhang, Iain Marshall, and Byron C. Wallace. 2016. Rationale-augmented convolutional neural networks for text classification. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 795–804, Austin, Texas. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Ye+Zhang%2C+Iain+Marshall%2C+and+Byron+C%2E+Wallace%2E+2016%2E+Rationale%2Daugmented+convolutional+neural+networks+for+text+classification%2E+In+Proceedings+of+the+2016+Conference+on+Empirical+Methods+in+Natural+La)
40. Ruiqi Zhong, S. Shao, and K. McKeown. 2019. Fine-grained sentiment analysis with faithful attention. ArXiv, abs/1908.06870. *Figure 2: HIT instructions.* — [arXiv:1908.06870](https://arxiv.org/abs/1908.06870) [find](https://scholar.google.com/scholar?q=Ruiqi+Zhong%2C+S%2E+Shao%2C+and+K%2E+McKeown%2E+2019%2E+Fine%2Dgrained+sentiment+analysis+with+faithful+attention%2E+ArXiv%2C+abs%2F1908%2E06870%2E+%2AFigure+2%3A+HIT+instructions%2E%2A+%E2%80%94+%5BarXiv%3A1908%2E06870%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F190)