---
title: "Understanding by Understanding Not: Modeling Negation in"
description: "Pre-trained language models such as BERT often ignore negation. 42 references, cited by 54 papers here."
authors: ["Arian Hosseini", "Siva Reddy", "Dzmitry Bahdanau", "R Devon Hjelm", "Alessandro Sordoni", "Aaron Courville"]
year: 2021
venue: "Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies"
source: "https://aclanthology.org/2021.naacl-main.102/"
doi: "10.18653/v1/2021.naacl-main.102"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/hosseini-2021-understanding-understanding-not-modeling-negation/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/hosseini-2021-understanding-understanding-not-modeling-negation/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2021](https://md.research.elamur.ai/free/en/papers/2021/page/4/) / Understanding by Understanding Not: Modeling Negation in Language Models

# Understanding by Understanding Not: Modeling Negation in Language Models

**Authors:** [Arian Hosseini](https://md.research.elamur.ai/free/en/authors/arian-hosseini/), [Siva Reddy](https://md.research.elamur.ai/free/en/authors/siva-reddy/), [Dzmitry Bahdanau](https://md.research.elamur.ai/free/en/authors/dzmitry-bahdanau/), R Devon Hjelm, [Alessandro Sordoni](https://md.research.elamur.ai/free/en/authors/alessandro-sordoni/) and [Aaron Courville](https://md.research.elamur.ai/free/en/authors/aaron-courville/) · **Year:** 2021 · **Venue:** [Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies](https://md.research.elamur.ai/free/en/venues/naacl/) ([NAACL 2021](https://md.research.elamur.ai/free/en/venues/naacl/2021/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2021.naacl-main.102/) · [PDF](https://aclanthology.org/2021.naacl-main.102.pdf) · [DOI 10.18653/v1/2021.naacl-main.102](https://doi.org/10.18653/v1/2021.naacl-main.102) · [OpenAlex](https://openalex.org/W3161374759)

**Citations:** 44 citations (OpenAlex, 2026-10-14) · [54](https://md.research.elamur.ai/free/en/papers/hosseini-2021-understanding-understanding-not-modeling-negation/cited-by/) in our library, of the 141,840 papers whose bibliographies we read

## Summary
Pre-trained language models such as BERT often ignore negation. For example, they may fill the blank in both a positive and a negated factual query with the same answer. Hosseini et al. address this by fine-tuning BERT with an extra training signal built from automatically negated Wikipedia sentences.

A rule-based negator works from Stanza dependency parses, POS tags and morphology. It uses Semgrex patterns to rewrite sentences and to select a target token. Each negated sentence is paired with its original, which serves as a reference. This pairing lets generic, non-factual text be used. An unlikelihood loss then pushes down the probability of the selected token in the negated sentence. A KL-divergence distillation loss on unchanged pairs and raw sentences keeps the model close to its original behaviour.

The resulting model, BERTNOT, cuts the mean top-1 error on negated LAMA to about 4%, roughly 5 points better than BERT. It keeps original LAMA precision intact and improves NLI test splits focused on negation. This shows negation handling can be taught cheaply from unlabeled text.

## Key points
- **Negation augmentation quality:**
  - Of 930 sampled Wikipedia sentences, only 31 matched no Semgrex rule.
  - A native speaker checked 100 negated outputs and found 94% of them correct.
  - Input sentences are limited to 20 words.
- **Training setup:** BERT-base cased was trained for 5 epochs with 20k examples per objective, maximum sequence length 128, and learning rate 1e-5.
- **Negated LAMA top-1 error (BERT → BERTNOT):**
  - SQuAD: 8.61 → 2.10
  - ConceptNet: 2.24 → 0.73
  - T-REx: 21.42 → 11.86
  - Google-RE: 3.76 → 1.10
- **Original LAMA:** P@1 stays essentially unchanged after training.
- **NLI accuracy on the negation splits of Hossain et al. (2020b):**
  - RTE: 65.47 → 74.47
  - SNLI: 44.18 → 45.96
  - MNLI: 60.62 → 60.89
  - The authors attribute the smaller gains on the large datasets to forgetting during extensive fine-tuning.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Université de Montréal](https://md.research.elamur.ai/free/en/universities/universite-de-montreal/) — University, Montreal, Canada · 47 works · 73,845 citations
- [McGill University](https://md.research.elamur.ai/free/en/universities/mcgill-university/) — University, Montreal, Canada · 51 works · 10,476 citations
- [Mila - Quebec Artificial Intelligence Institute](https://md.research.elamur.ai/free/en/labs/mila-quebec-artificial-intelligence-institute/) — Research institute, Montreal, Canada · 31 works · 13,061 citations
- [Microsoft Research Montréal](https://md.research.elamur.ai/free/en/companies/microsoft-research-montreal/) — Company, Montreal, Canada · 5 works · 704 citations

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 9,085 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 8,888 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 9,296 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 9,206 works

🧭 Topics (1): [🧠 Language models & pretraining](https://md.research.elamur.ai/free/en/topics/language-models/) · 1,344 works

## Cite

Cite as: Arian Hosseini, Siva Reddy, Dzmitry Bahdanau, R Devon Hjelm, Alessandro Sordoni and Aaron Courville. 2021. Understanding by Understanding Not: Modeling Negation in Language Models. Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{hosseini2021understanding,
  title = {{Understanding by Understanding Not: Modeling Negation in Language Models}},
  author = {Arian Hosseini, Siva Reddy, Dzmitry Bahdanau, R Devon Hjelm, Alessandro Sordoni and Aaron Courville},
  booktitle = {Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies},
  year = {2021},
  doi = {10.18653/v1/2021.naacl-main.102},
  url = {https://doi.org/10.18653/v1/2021.naacl-main.102}
}
```

- [← previous Data-to-text Generation with Macro Planning](https://md.research.elamur.ai/free/en/papers/puduppully-2021-data-text-generation-macro-planning/)
- [next → AraELECTRA: Pre-Training Text Discriminators for Arabic Language Understanding](https://md.research.elamur.ai/free/en/papers/antoun-2021-araelectra-pre-training-text-discriminators/)
- [↑ back to papers of 2021](https://md.research.elamur.ai/free/en/papers/2021/page/4/)

## References

42 references, 17 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Chris Alberti, Kenton Lee, and Michael Collins. 2019. A BERT baseline for the natural questions. CoRR, abs/1901.08634. — [arXiv:1901.08634](https://arxiv.org/abs/1901.08634) [find](https://scholar.google.com/scholar?q=Chris+Alberti%2C+Kenton+Lee%2C+and+Michael+Collins%2E+2019%2E+A+BERT+baseline+for+the+natural+questions%2E+CoRR%2C+abs%2F1901%2E08634%2E+%E2%80%94+%5BarXiv%3A1901%2E08634%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F1901%2E08634%29)
2. Jacob Andreas. 2019. [Good-enough compositional data augmentation](https://md.research.elamur.ai/free/en/papers/andreas-2020-good-enough-compositional-data-augmentation/). CoRR, abs/1904.09545.
3. Roy Bar-Haim, Ido Dagan, Bill Dolan, Lisa Ferro, and Danilo Giampiccolo. 2006. The second pascal recognising textual entailment challenge. Proceedings of the Second PASCAL Challenges Workshop on Recognising Textual Entailment. [find](https://scholar.google.com/scholar?q=Roy+Bar%2DHaim%2C+Ido+Dagan%2C+Bill+Dolan%2C+Lisa+Ferro%2C+and+Danilo+Giampiccolo%2E+2006%2E+The+second+pascal+recognising+textual+entailment+challenge%2E+Proceedings+of+the+Second+PASCAL+Challenges+Workshop+on+Recog)
4. Luisa Bentivogli, Peter Clark, Ido Dagan, and Danilo Giampiccolo. 2009. B.: The fifth pascal recognizing textual entailment challenge. In Proceedings of TAC 9. [find](https://scholar.google.com/scholar?q=Luisa+Bentivogli%2C+Peter+Clark%2C+Ido+Dagan%2C+and+Danilo+Giampiccolo%2E+2009%2E+B%2E%3A+The+fifth+pascal+recognizing+textual+entailment+challenge%2E+In+Proceedings+of+TAC+9%2E)
5. Antoine Bosselut, Hannah Rashkin, Maarten Sap, Chaitanya Malaviya, Asli Celikyilmaz, and Yejin Choi. 2019. [COMET: Commonsense transformers for automatic knowledge graph construction](https://md.research.elamur.ai/free/en/papers/bosselut-2019-comet-commonsense-transformers-automatic-knowledge/). In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 4762–4779, Florence…
6. Samuel R. Bowman, Gabor Angeli, Christopher Potts, and Christopher D. Manning. 2015. A large annotated corpus for learning natural language inference. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages 632–642, Lisbon, Portugal. Association for Computati… [find](https://scholar.google.com/scholar?q=Samuel+R%2E+Bowman%2C+Gabor+Angeli%2C+Christopher+Potts%2C+and+Christopher+D%2E+Manning%2E+2015%2E+A+large+annotated+corpus+for+learning+natural+language+inference%2E+In+Proceedings+of+the+2015+Conference+on+Empirica)
7. Nathanael Chambers, Daniel M. Cer, Trond Grenager, David Hall, Chloé Kiddon, Bill MacCartney, Marie-Catherine de Marneffe, Daniel Ramage, Eric Yeh, and Christopher D. Manning. 2007. Learning alignments and leveraging natural logic. In ACL-PASCAL@ACL, pages 165–170. Association for Computational Ling… [find](https://scholar.google.com/scholar?q=Nathanael+Chambers%2C+Daniel+M%2E+Cer%2C+Trond+Grenager%2C+David+Hall%2C+Chlo%C3%A9+Kiddon%2C+Bill+MacCartney%2C+Marie%2DCatherine+de+Marneffe%2C+Daniel+Ramage%2C+Eric+Yeh%2C+and+Christopher+D%2E+Manning%2E+2007%2E+Learning+alignment)
8. Long Chen. 2019. Attention-based deep learning system for negation and assertion detection in clinical notes. International Journal of Artificial Intelligence & Applications, 10:1–9. [find](https://scholar.google.com/scholar?q=Long+Chen%2E+2019%2E+Attention%2Dbased+deep+learning+system+for+negation+and+assertion+detection+in+clinical+notes%2E+International+Journal+of+Artificial+Intelligence+%26+Applications%2C+10%3A1%E2%80%939%2E)
9. Ido Dagan, Oren Glickman, and Bernardo Magnini. 2005. The PASCAL recognising textual entailment challenge. In Machine Learning Challenges, Evaluating Predictive Uncertainty, Visual Object Classification and Recognizing Textual Entailment, First PASCAL Machine Learning Challenges Workshop, MLCW 2005,… [find](https://scholar.google.com/scholar?q=Ido+Dagan%2C+Oren+Glickman%2C+and+Bernardo+Magnini%2E+2005%2E+The+PASCAL+recognising+textual+entailment+challenge%2E+In+Machine+Learning+Challenges%2C+Evaluating+Predictive+Uncertainty%2C+Visual+Object+Classificati)
10. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. [BERT: pre-training of deep bidirectional transformers for language understanding](https://md.research.elamur.ai/free/en/papers/devlin-2019-bert-pre-training-deep-bidirectional/). In NAACL-HLT (1), pages 4171–4186. Association for Computational Linguistics.
11. Hady ElSahar, Pavlos Vougiouklis, Arslen Remaci, Christophe Gravier, Jonathon S. Hare, Frédérique Laforest, and Elena Simperl. 2018. [T-rex: A large scale alignment of natural language with knowledge base triples](https://md.research.elamur.ai/free/en/papers/elsahar-2018-t-rex-large-scale-alignment/). In Proceedings of the Eleventh International Conference on Language Resources and Evalu…
12. Federico Fancellu, Adam Lopez, and Bonnie Webber. 2016. Neural networks for negation scope detection. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 495–504, Berlin, Germany. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Federico+Fancellu%2C+Adam+Lopez%2C+and+Bonnie+Webber%2E+2016%2E+Neural+networks+for+negation+scope+detection%2E+In+Proceedings+of+the+54th+Annual+Meeting+of+the+Association+for+Computational+Linguistics+%28Volume)
13. Federico Fancellu, Adam Lopez, Bonnie Webber, and Hangfeng He. 2017. Detecting negation scope is easy, except when it isn’t. In Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 2, Short Papers, pages 58–63, Valencia, Spain. Associati… [find](https://scholar.google.com/scholar?q=Federico+Fancellu%2C+Adam+Lopez%2C+Bonnie+Webber%2C+and+Hangfeng+He%2E+2017%2E+Detecting+negation+scope+is+easy%2C+except+when+it+isn%E2%80%99t%2E+In+Proceedings+of+the+15th+Conference+of+the+European+Chapter+of+the+Associ)
14. Danilo Giampiccolo, Bernardo Magnini, Ido Dagan, and Bill Dolan. 2007. The third PASCAL recognizing textual entailment challenge. In Proceedings of the ACL-PASCAL Workshop on Textual Entailment and Paraphrasing, pages 1–9, Prague. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Danilo+Giampiccolo%2C+Bernardo+Magnini%2C+Ido+Dagan%2C+and+Bill+Dolan%2E+2007%2E+The+third+PASCAL+recognizing+textual+entailment+challenge%2E+In+Proceedings+of+the+ACL%2DPASCAL+Workshop+on+Textual+Entailment+and+Pa)
15. Kristina Gulordava, Piotr Bojanowski, Edouard Grave, Tal Linzen, and Marco Baroni. 2018. [Colorless green recurrent networks dream hierarchically](https://md.research.elamur.ai/free/en/papers/gulordava-2018-colorless-green-recurrent-networks-dream/). In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume…
16. Md Mosharaf Hossain, Kathleen Hamilton, Alexis Palmer, and Eduardo Blanco. 2020a. Predicting the focus of negation: Model and error analysis. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 8389– 8401, Online. Association for Computational Linguistic… [find](https://scholar.google.com/scholar?q=Md+Mosharaf+Hossain%2C+Kathleen+Hamilton%2C+Alexis+Palmer%2C+and+Eduardo+Blanco%2E+2020a%2E+Predicting+the+focus+of+negation%3A+Model+and+error+analysis%2E+In+Proceedings+of+the+58th+Annual+Meeting+of+the+Associati)
17. Md Mosharaf Hossain, Venelin Kovatchev, Pranoy Dutta, Tiffany Kao, Elizabeth Wei, and Eduardo Blanco. 2020b. [An analysis of natural language inference benchmarks through the lens of negation](https://md.research.elamur.ai/free/en/papers/hossain-2020-analysis-natural-language-inference-benchmarks/). In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, EMNLP 2020, Onlin…
18. Jaap Jumelet and Dieuwke Hupkes. 2018. Do language models understand anything? on the ability of LSTMs to understand negative polarity items. In Proceedings of the 2018 EMNLP Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP, pages 222–231, Brussels, Belgium. Association for C… [find](https://scholar.google.com/scholar?q=Jaap+Jumelet+and+Dieuwke+Hupkes%2E+2018%2E+Do+language+models+understand+anything%3F+on+the+ability+of+LSTMs+to+understand+negative+polarity+items%2E+In+Proceedings+of+the+2018+EMNLP+Workshop+BlackboxNLP%3A+Ana)
19. Nora Kassner and Hinrich Schütze. 2019. [Negated and misprimed probes for pretrained language models: Birds can talk, but cannot fly](https://md.research.elamur.ai/free/en/papers/kassner-2020-negated-misprimed-probes-pretrained-language/).
20. Hao Li and Wei Lu. 2018. Learning with structured representations for negation scope extraction. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 533–539, Melbourne, Australia. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Hao+Li+and+Wei+Lu%2E+2018%2E+Learning+with+structured+representations+for+negation+scope+extraction%2E+In+Proceedings+of+the+56th+Annual+Meeting+of+the+Association+for+Computational+Linguistics+%28Volume+2%3A+S)
21. Xiang Li, Aynaz Taheri, Lifu Tu, and Kevin Gimpel. 2016. [Commonsense knowledge base completion](https://md.research.elamur.ai/free/en/papers/li-2016-commonsense-knowledge-base-completion/). In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, ACL 2016, August 7-12, 2016, Berlin, Germany, Volume 1: Long Papers. The Association for Computer Linguistics.
22. Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. Roberta: A robustly optimized BERT pretraining approach. CoRR, abs/1907.11692. [find](https://scholar.google.com/scholar?q=Yinhan+Liu%2C+Myle+Ott%2C+Naman+Goyal%2C+Jingfei+Du%2C+Mandar+Joshi%2C+Danqi+Chen%2C+Omer+Levy%2C+Mike+Lewis%2C+Luke+Zettlemoyer%2C+and+Veselin+Stoyanov%2E+2019%2E+Roberta%3A+A+robustly+optimized+BERT+pretraining+approach%2E+C)
23. Rebecca Marvin and Tal Linzen. 2018. Targeted syntactic evaluation of language models. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 1192–1202, Brussels, Belgium. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Rebecca+Marvin+and+Tal+Linzen%2E+2018%2E+Targeted+syntactic+evaluation+of+language+models%2E+In+Proceedings+of+the+2018+Conference+on+Empirical+Methods+in+Natural+Language+Processing%2C+pages+1192%E2%80%931202%2C+Bruss)
24. Tom McCoy, Ellie Pavlick, and Tal Linzen. 2019. [Right for the wrong reasons: Diagnosing syntactic heuristics in natural language inference](https://md.research.elamur.ai/free/en/papers/mccoy-2019-right-wrong-reasons-diagnosing-syntactic/). In Proceedings of the 57th Conference of the Association for Computational Linguistics, ACL 2019, Florence, Italy, July 28- August 2, 2019, Volume 1: Long Paper…
25. Junghyun Min, R. Thomas McCoy, Dipanjan Das, Emily Pitler, and Tal Linzen. 2020. [Syntactic data augmentation increases robustness to inference heuristics](https://md.research.elamur.ai/free/en/papers/min-2020-syntactic-data-augmentation-increases-robustness/). CoRR, abs/2004.11999.
26. Roser Morante and Walter Daelemans. 2009. A metalearning approach to processing the scope of negation. In Proceedings of the Thirteenth Conference on Computational Natural Language Learning, CoNLL 2009, Boulder, Colorado, USA, June 4-5, 2009, pages 21–29. ACL. [find](https://scholar.google.com/scholar?q=Roser+Morante+and+Walter+Daelemans%2E+2009%2E+A+metalearning+approach+to+processing+the+scope+of+negation%2E+In+Proceedings+of+the+Thirteenth+Conference+on+Computational+Natural+Language+Learning%2C+CoNLL+200)
27. Aakanksha Naik, Abhilasha Ravichander, Norman M. Sadeh, Carolyn Penstein Rosé, and Graham Neubig. 2018. Stress test evaluation for natural language inference. In COLING, pages 2340–2353. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Aakanksha+Naik%2C+Abhilasha+Ravichander%2C+Norman+M%2E+Sadeh%2C+Carolyn+Penstein+Ros%C3%A9%2C+and+Graham+Neubig%2E+2018%2E+Stress+test+evaluation+for+natural+language+inference%2E+In+COLING%2C+pages+2340%E2%80%932353%2E+Association+f)
28. Hiroshi Noji and Hiroya Takamura. 2020. An analysis of the utility of explicit negative examples to improve the syntactic abilities of neural language models. CoRR, abs/2004.02451. [find](https://scholar.google.com/scholar?q=Hiroshi+Noji+and+Hiroya+Takamura%2E+2020%2E+An+analysis+of+the+utility+of+explicit+negative+examples+to+improve+the+syntactic+abilities+of+neural+language+models%2E+CoRR%2C+abs%2F2004%2E02451%2E)
29. Fabio Petroni, Tim Rocktäschel, Sebastian Riedel, Patrick S. H. Lewis, Anton Bakhtin, Yuxiang Wu, and Alexander H. Miller. 2019. [Language models as knowledge bases?](https://md.research.elamur.ai/free/en/papers/petroni-2019-language-models-as-knowledge-bases/) In EMNLP/IJCNLP (1), pages 2463–2473. Association for Computational Linguistics.
30. Peng Qi, Yuhao Zhang, Yuhui Zhang, Jason Bolton, and Christopher D. Manning. 2020. [Stanza: A python natural language processing toolkit for many human languages](https://md.research.elamur.ai/free/en/papers/qi-2020-stanza-python-natural-language-processing/). CoRR, abs/2003.07082.
31. Alec Radford, Jeff Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. 2019. Language models are unsupervised multitask learners. [find](https://scholar.google.com/scholar?q=Alec+Radford%2C+Jeff+Wu%2C+Rewon+Child%2C+David+Luan%2C+Dario+Amodei%2C+and+Ilya+Sutskever%2E+2019%2E+Language+models+are+unsupervised+multitask+learners%2E)
32. Pranav Rajpurkar, Jian Zhang, Konstantin Lopyrev, and Percy Liang. 2016. [Squad: 100, 000+ questions for machine comprehension of text](https://md.research.elamur.ai/free/en/papers/rajpurkar-2016-squad-100-000-questions-machine/). In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, EMNLP 2016, Austin, Texas, USA, November 1-4, 2016, pages 2383–2392. The A…
33. Longxiang Shen, Bowei Zou, Yu Hong, Guodong Zhou, Qiaoming Zhu, and AiTi Aw. 2019. Negative focus detection via contextual attention mechanism. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Pro… [find](https://scholar.google.com/scholar?q=Longxiang+Shen%2C+Bowei+Zou%2C+Yu+Hong%2C+Guodong+Zhou%2C+Qiaoming+Zhu%2C+and+AiTi+Aw%2E+2019%2E+Negative+focus+detection+via+contextual+attention+mechanism%2E+In+Proceedings+of+the+2019+Conference+on+Empirical+Metho)
34. Alon Talmor, Yanai Elazar, Yoav Goldberg, and Jonathan Berant. 2019. [olmpics - on what language model pre-training captures](https://md.research.elamur.ai/free/en/papers/talmor-2019-olmpics-language-model-pre-training/). CoRR, abs/1912.13283.
35. Ian Tenney, Dipanjan Das, and Ellie Pavlick. 2019. [Bert rediscovers the classical nlp pipeline](https://md.research.elamur.ai/free/en/papers/tenney-2019-bert-rediscovers-classical-nlp-pipeline/). In Association for Computational Linguistics.
36. Alex Warstadt and Samuel R. Bowman. 2019. Grammatical analysis of pretrained sentence encoders with acceptability judgments. CoRR, abs/1901.03438. — [arXiv:1901.03438](https://arxiv.org/abs/1901.03438) [find](https://scholar.google.com/scholar?q=Alex+Warstadt+and+Samuel+R%2E+Bowman%2E+2019%2E+Grammatical+analysis+of+pretrained+sentence+encoders+with+acceptability+judgments%2E+CoRR%2C+abs%2F1901%2E03438%2E+%E2%80%94+%5BarXiv%3A1901%2E03438%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F1901%2E03438)
37. Jason W. Wei and Kai Zou. 2019. [EDA: easy data augmentation techniques for boosting performance on text classification tasks](https://md.research.elamur.ai/free/en/papers/wei-2019-eda-easy-data-augmentation-techniques/). In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, EMNLP-IJC…
38. Sean Welleck, Ilia Kulikov, Stephen Roller, Emily Dinan, Kyunghyun Cho, and Jason Weston. 2020. [Neural text generation with unlikelihood training](https://md.research.elamur.ai/free/en/papers/welleck-2019-neural-text-generation-unlikelihood-training/). In ICLR. OpenReview.net.
39. Adina Williams, Nikita Nangia, and Samuel Bowman. 2018. A broad-coverage challenge corpus for sentence understanding through inference. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Pa… [find](https://scholar.google.com/scholar?q=Adina+Williams%2C+Nikita+Nangia%2C+and+Samuel+Bowman%2E+2018%2E+A+broad%2Dcoverage+challenge+corpus+for+sentence+understanding+through+inference%2E+In+Proceedings+of+the+2018+Conference+of+the+North+American+Chap)
40. Yiyun Zhao and Steven Bethard. 2020. How does BERT’s attention change when you fine-tune? an analysis methodology and a case study in negation scope. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 4729–4747, Online. Association for Computational Lin… [find](https://scholar.google.com/scholar?q=Yiyun+Zhao+and+Steven+Bethard%2E+2020%2E+How+does+BERT%E2%80%99s+attention+change+when+you+fine%2Dtune%3F+an+analysis+methodology+and+a+case+study+in+negation+scope%2E+In+Proceedings+of+the+58th+Annual+Meeting+of+the+A)
41. Bowei Zou, Guodong Zhou, and Qiaoming Zhu. 2014. Negation focus identification with contextual discourse information. In Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics, ACL 2014, June 22-27, 2014, Baltimore, MD, USA, Volume 1: Long Papers, pages 522–530. The… [find](https://scholar.google.com/scholar?q=Bowei+Zou%2C+Guodong+Zhou%2C+and+Qiaoming+Zhu%2E+2014%2E+Negation+focus+identification+with+contextual+discourse+information%2E+In+Proceedings+of+the+52nd+Annual+Meeting+of+the+Association+for+Computational+Lin)
42. Bowei Zou, Guodong Zhou, and Qiaoming Zhu. 2015. Unsupervised negation focus identification with word-topic graph model. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, EMNLP 2015, Lisbon, Portugal, September 17-21, 2015, pages 1632–1636. The Association fo… [find](https://scholar.google.com/scholar?q=Bowei+Zou%2C+Guodong+Zhou%2C+and+Qiaoming+Zhu%2E+2015%2E+Unsupervised+negation+focus+identification+with+word%2Dtopic+graph+model%2E+In+Proceedings+of+the+2015+Conference+on+Empirical+Methods+in+Natural+Language+)

*Updated 24 October 2026*