---
title: "Supporting Context Monotonicity Abstractions in Neural NLI Models"
description: "Neural NLI models still reason badly when a word is swapped for a broader or narrower one inside a… 23 references, cited by 5 papers here."
authors: ["Julia Rozanova", "Deborah Ferreira", "Mokanarangan Thayaparan", "Marco Valentino", "André Freitas"]
year: 2021
venue: "Proceedings of the 1st and 2nd Workshops on Natural Logic Meets Machine Learning (NALOMA)"
source: "https://aclanthology.org/2021.naloma-1.6/"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/rozanova-2021-supporting-context-monotonicity-abstractions-neural/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/rozanova-2021-supporting-context-monotonicity-abstractions-neural/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2021](https://md.research.elamur.ai/free/en/papers/2021/page/7/) / Supporting Context Monotonicity Abstractions in Neural NLI Models

# Supporting Context Monotonicity Abstractions in Neural NLI Models

**Authors:** [Julia Rozanova](https://md.research.elamur.ai/free/en/authors/julia-rozanova/), [Deborah Ferreira](https://md.research.elamur.ai/free/en/authors/deborah-ferreira/), [Mokanarangan Thayaparan](https://md.research.elamur.ai/free/en/authors/mokanarangan-thayaparan/), [Marco Valentino](https://md.research.elamur.ai/free/en/authors/marco-valentino/) and [André Freitas](https://md.research.elamur.ai/free/en/authors/andre-freitas/) · **Year:** 2021 · **Venue:** [Proceedings of the 1st and 2nd Workshops on Natural Logic Meets Machine Learning (NALOMA)](https://md.research.elamur.ai/free/en/venues/naloma/) ([NALOMA 2021](https://md.research.elamur.ai/free/en/venues/naloma/2021/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2021.naloma-1.6/) · [PDF](https://aclanthology.org/2021.naloma-1.6.pdf) · [OpenAlex](https://openalex.org/W3163896859)

**Citations:** 0 citations (OpenAlex, 2026-10-14) · [5](https://md.research.elamur.ai/free/en/papers/rozanova-2021-supporting-context-monotonicity-abstractions-neural/cited-by/) in our library, of the 141,840 papers whose bibliographies we read

## Summary
Neural NLI models still reason badly when a word is swapped for a broader or narrower one inside a downward monotone context, such as one created by negation or quantifiers like "every". The authors treat a context as a sentence with an open noun slot. For this class of problems, the entailment label follows from two things: the containment relation between the two fillers and whether the slot is upward or downward monotone. They give this view a formal basis by extending Moss's syllogistic logic L(all) with abstract monotone contexts, and they prove the extended logic sound and complete. On the practical side, they convert HELP into HELP-Contexts, recast context monotonicity classification so that a pretrained transformer NLI model can learn it, and add it as an intermediate fine-tuning stage before NLI training on HELP. The intermediate stage clearly helps bert-base-uncased-snli on downward contexts, gives roberta-large-mnli only small gains, and usually costs less accuracy on SNLI/MNLI than training on HELP alone. The broader point is that targeting the hidden variable of a logical formalism can be an effective training signal.

## Key points
- The problems studied are pairs (p(a), p(b)) in which the label depends only on the monotonicity of context p and the containment relation between a and b (for example, apples ⊑ fruit, or couch ≡ sofa).
- The logic adds relation symbols ⊑_p for each context and has three rules: BARBARA, ↑ (upward), and ↓ (downward, which reverses the direction). Completeness is proved through a canonical model built from down-sets.
- The roberta-large-mnli baseline scores above 93% on MNLI dev but only 5.32% on MoNLI downward items, which suggests it effectively assumes every context is upward monotone.
- For bert-base-uncased-snli, adding HELP-Contexts lifts downward accuracy to 97.67 on MoNLI (baseline 56.49, HELP alone 85.02) and to 87.13 on MED (baseline 43.91, HELP alone 72.43). Upward accuracy falls at the same time, showing a clear trade-off between the two directions.
- Benchmark losses for BERT on the SNLI test set are −3.58 with HELP + HELP-Contexts versus −14.86 with HELP alone. The one exception to the smaller-loss pattern is RoBERTa on MNLI, which loses 13.11 compared with 11.45.

## Interesting ideas
- Model a context as a single unit, a sentence with a gap, and attach one monotonicity label to it instead of composing the profile of every operator; this works well whenever an outside component can supply the label.
- Split an entailment decision into two independent, explainable factors: the lexical containment between fillers and the polarity of the context.
- Before training the end task, train the model on the intermediate abstraction that the formalism says determines the label, building that dataset by restructuring an existing corpus (here HELP into HELP-Contexts).
- Report results split by phenomenon direction (upward vs downward) and always track the drop on the original benchmarks, because a gain on a challenge set can simply hide a trade-off.

## For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

- **I1** (world): A text clause could carry an explicit up/down flag on each entity slot, so that WordNet hypernym substitutions are licensed or blocked with a traceable reason.
- **I2** (answer): Readers answering "does it follow that…" questions could look up hypernymy in the ontology, look up slot polarity, and combine the two as a short, auditable proof step.
- **I3** (learn): Silver data for subtasks such as polarity or negation scope could be carved out of existing NLI or QA sets and used as a staged curriculum before full answering.
- **I4** (learn): Any repair rule or retraining step should be scored on both polarities and on a general control set, so that regressions show up as a negative control.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [University of Manchester](https://md.research.elamur.ai/free/en/universities/university-of-manchester/) — University, Manchester, United Kingdom · 30 works · 43,942 citations
- [Idiap Research Institute](https://md.research.elamur.ai/free/en/labs/idiap-research-institute/) — Research institute, Martigny-Combe, Switzerland · 20 works · 1,757 citations

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 9,086 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 8,889 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 9,296 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 9,206 works

🧭 Topics (1): [🧮 Reasoning](https://md.research.elamur.ai/free/en/topics/reasoning/) · 510 works

## Cite

Cite as: Julia Rozanova, Deborah Ferreira, Mokanarangan Thayaparan, Marco Valentino and André Freitas. 2021. Supporting Context Monotonicity Abstractions in Neural NLI Models. Proceedings of the 1st and 2nd Workshops on Natural Logic Meets Machine Learning (NALOMA).

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{rozanova2021supporting,
  title = {{Supporting Context Monotonicity Abstractions in Neural NLI Models}},
  author = {Julia Rozanova, Deborah Ferreira, Mokanarangan Thayaparan, Marco Valentino and André Freitas},
  booktitle = {Proceedings of the 1st and 2nd Workshops on Natural Logic Meets Machine Learning (NALOMA)},
  year = {2021},
  url = {https://aclanthology.org/2021.naloma-1.6/}
}
```

- [← previous Speech Technology for Everyone: Automatic Speech Recognition for Non-Native English with Transfer Learning](https://md.research.elamur.ai/free/en/papers/shibano-2021-speech-technology-everyone-automatic-speech-recognition/)
- [next → WANLP 2021 Shared-Task: Towards Irony and Sentiment Detection in Arabic Tweets using Multi-headed-LSTM-CNN-GRU and MaRBERT](https://md.research.elamur.ai/free/en/papers/salam-2021-wanlp-2021-shared-task-irony/)
- [↑ back to papers of 2021](https://md.research.elamur.ai/free/en/papers/2021/page/7/)

## References

23 references, 7 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Hai Hu and Larry Moss. 2018. Polarity computations in flexible categorial grammar. In Proceedings of the Seventh Joint Conference on Lexical and Computational Semantics, pages 124–129, New Orleans, Louisiana. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Hai+Hu+and+Larry+Moss%2E+2018%2E+Polarity+computations+in+flexible+categorial+grammar%2E+In+Proceedings+of+the+Seventh+Joint+Conference+on+Lexical+and+Computational+Semantics%2C+pages+124%E2%80%93129%2C+New+Orleans%2C+Lo)
2. Lasha Abzianidze. 2015. A tableau prover for natural logic and language. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages 2492–2502, Lisbon, Portugal. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Lasha+Abzianidze%2E+2015%2E+A+tableau+prover+for+natural+logic+and+language%2E+In+Proceedings+of+the+2015+Conference+on+Empirical+Methods+in+Natural+Language+Processing%2C+pages+2492%E2%80%932502%2C+Lisbon%2C+Portugal%2E+A)
3. Gabor Angeli and Christopher D Manning. 2014. [Naturalli: Natural logic inference for common sense reasoning](https://md.research.elamur.ai/free/en/papers/angeli-2014-naturalli-natural-logic-inference-common/). In Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP), pages 534–545.
4. Thomas Icard, Lawrence Moss, and William Tune. 2017. A monotonicity calculus and its completeness. In Proceedings of the 15th Meeting on the Mathematics of Language, pages 75–87, London, UK. Association for Computational Linguistics. [↗](https://doi.org/10.18653/v1/w17-3408)
5. Johan van Benthem. 1988. Essays in logical semantics. Studia Logica, 47(2):172–173. [find](https://scholar.google.com/scholar?q=Johan+van+Benthem%2E+1988%2E+Essays+in+logical+semantics%2E+Studia+Logica%2C+47%282%29%3A172%E2%80%93173%2E)
6. Aikaterini-Lida Kalouli, Richard Crouch, and Valeria de Paiva. 2020. Hy-NLI: a hybrid system for natural language inference. In Proceedings of the 28th International Conference on Computational Linguistics, pages 5235–5249, Barcelona, Spain (Online). International Committee on Computational Linguistics. [find](https://scholar.google.com/scholar?q=Aikaterini%2DLida+Kalouli%2C+Richard+Crouch%2C+and+Valeria+de+Paiva%2E+2020%2E+Hy%2DNLI%3A+a+hybrid+system+for+natural+language+inference%2E+In+Proceedings+of+the+28th+International+Conference+on+Computational+Lingui)
7. D. Bobrow, Bob Cheslow, C. Condoravdi, Tracy Holloway King, R. Nairn, and A. Zaenen. 2007. Parc’s bridge and question answering system. [find](https://scholar.google.com/scholar?q=D%2E+Bobrow%2C+Bob+Cheslow%2C+C%2E+Condoravdi%2C+Tracy+Holloway+King%2C+R%2E+Nairn%2C+and+A%2E+Zaenen%2E+2007%2E+Parc%E2%80%99s+bridge+and+question+answering+system%2E)
8. Samuel R. Bowman, Gabor Angeli, Christopher Potts, and Christopher D. Manning. 2015. A large annotated corpus for learning natural language inference. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing (EMNLP). Association for Computational Linguistics. [↗](https://doi.org/10.48550/arxiv.1508.05326)
9. Nelson F. Liu, Roy Schwartz, and Noah A. Smith. 2019a. [Inoculation by fine-tuning: A method for analyzing challenge datasets](https://md.research.elamur.ai/free/en/papers/liu-2019-inoculation-fine-tuning-method-analyzing/). CoRR, abs/1904.02668. [↗](https://arxiv.org/abs/1904.02668)
10. Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019b. Roberta: A robustly optimized BERT pretraining approach. CoRR, abs/1907.11692. [↗](https://arxiv.org/abs/1907.11692)
11. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. [BERT: Pre-training of deep bidirectional transformers for language understanding](https://md.research.elamur.ai/free/en/papers/devlin-2019-bert-pre-training-deep-bidirectional/). In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 4171–4186, Minneapolis, Minnesota. Association for Computational Linguistics. [↗](https://doi.org/10.18653/v1/n19-1423)
12. Bill MacCartney and Christopher D. Manning. 2009. An extended model of natural logic. In Proceedings of the Eight International Conference on Computational Semantics, pages 140–156, Tilburg, The Netherlands. Association for Computational Linguistics. [↗](https://doi.org/10.3115/1693756.1693772)
13. A. Geiger, Kyle Richardson, and Christopher Potts. 2020. [Neural natural language inference models partially embed theories of lexical entailment and negation](https://md.research.elamur.ai/free/en/papers/geiger-2020-neural-natural-language-inference-models/). arXiv: Computation and Language.
14. Lawrence S. Moss. 2008a. Completeness theorems for syllogistic fragments, pages 143–174. De Gruyter Mouton. [find](https://scholar.google.com/scholar?q=Lawrence+S%2E+Moss%2E+2008a%2E+Completeness+theorems+for+syllogistic+fragments%2C+pages+143%E2%80%93174%2E+De+Gruyter+Mouton%2E)
15. Lawrence S. Moss. 2008b. Syllogistic Logics with Verbs. Journal of Logic and Computation, 20(4):947–967. [↗](https://doi.org/10.1093/logcom/exn086)
16. Emily Goodwin, Koustuv Sinha, and Timothy J. O’Donnell. 2020. Probing linguistic systematicity. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 1958–1969, Online. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Emily+Goodwin%2C+Koustuv+Sinha%2C+and+Timothy+J%2E+O%E2%80%99Donnell%2E+2020%2E+Probing+linguistic+systematicity%2E+In+Proceedings+of+the+58th+Annual+Meeting+of+the+Association+for+Computational+Linguistics%2C+pages+1958%E2%80%931)
17. Kyle Richardson, H. Hu, L. Moss, and A. Sabharwal. 2020. [Probing natural language inference models through semantic fragments](https://md.research.elamur.ai/free/en/papers/richardson-2019-probing-natural-language-inference-models/). ArXiv, abs/1909.07521. [↗](https://arxiv.org/abs/1909.07521)
18. John Hewitt and Christopher D. Manning. 2019. [A structural probe for finding syntax in word representations](https://md.research.elamur.ai/free/en/papers/hewitt-2019-structural-probe-finding-syntax-word/). In Proceedings of the 2019 Conference of the North American Chapter of the Association V.M.S.
19. Valencia. 1991. Studies on Natural Logic and Categorial Grammar. Universiteit van Amsterdam. [find](https://scholar.google.com/scholar?q=Valencia%2E+1991%2E+Studies+on+Natural+Logic+and+Categorial+Grammar%2E+Universiteit+van+Amsterdam%2E)
20. Adina Williams, Nikita Nangia, and Samuel Bowman. 2018. A broad-coverage challenge corpus for sentence understanding through inference. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers), pages 1112–1122. Association for Computational Linguistics. [↗](https://doi.org/10.18653/v1/n18-1101)
21. Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pier-ric Cistac, Tim Rault, Rémi Louf, Morgan Funtowicz, Joe Davison, Sam Shleifer, Patrick von Platen, Clara Ma, Yacine Jernite, Julien Plu, Canwen Xu, Teven Le Scao, Sylvain Gugger, Mariama Drame, Quentin Lhoest, and Alexander M. Rush. 2020. [Transformers: State-of-the-art natural language processing](https://md.research.elamur.ai/free/en/papers/wolf-2020-transformers-state-art-natural-language/). In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, pages 38–45, Online. Association for Computational Linguistics. [↗](https://doi.org/10.18653/v1/2020.emnlp-demos.6)
22. Hitomi Yanaka, Koji Mineshima, Daisuke Bekki, and Kentaro Inui. 2020. Do neural models learn systematicity of monotonicity inference in natural language? In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 6105–6117, Online. Association for Computational Linguistics. [↗](https://doi.org/10.5715/jnlp.27.653)
23. Hitomi Yanaka, Koji Mineshima, Daisuke Bekki, Kentaro Inui, Satoshi Sekine, Lasha Abzianidze, and Johan Bos. 2019. HELP: A dataset for identifying shortcomings of neural models in monotonicity reasoning. In Proceedings of the Eighth Joint Conference on Lexical and Computational Semantics (*SEM 2019), pages 250–255, Minneapolis, Minnesota. Association for Computational Linguistics. [↗](https://doi.org/10.48550/arxiv.1904.12166)

*Updated 24 October 2026*