---
title: "Adversarial Analysis of Natural Language Inference Systems — paper by Chien and Kalita (2019)"
description: "They test all of them on three existing challenge sets: Glockner et al.: single-word lexical swaps based on WordNet. 22 references, cited by 8 papers here."
authors: ["Tiffany Chien", "Jugal Kalita"]
year: 2019
venue: "arXiv:1912.03441"
source: "https://arxiv.org/abs/1912.03441"
doi: "10.48550/arxiv.1912.03441"
arxiv: "1912.03441"
licence: "arXiv non-exclusive distribution licence (read only, no redistribution)"
licence_url: "http://arxiv.org/licenses/nonexclusive-distrib/1.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/chien-2019-adversarial-analysis-natural-language-inference/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/chien-2019-adversarial-analysis-natural-language-inference/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2019](https://md.research.elamur.ai/free/en/papers/2019/page/6/) / Adversarial Analysis of Natural Language Inference Systems

# Adversarial Analysis of Natural Language Inference Systems

**Authors:** Tiffany Chien and Jugal Kalita · **Year:** 2019 · **Venue:** arXiv:1912.03441 · **Licence:** [arXiv non-exclusive distribution licence (read only, no redistribution)](http://arxiv.org/licenses/nonexclusive-distrib/1.0/)

**Links:** [Publisher's page](https://arxiv.org/abs/1912.03441) · [PDF](https://arxiv.org/pdf/1912.03441) · [DOI 10.48550/arxiv.1912.03441](https://doi.org/10.48550/arxiv.1912.03441) · [arXiv 1912.03441](https://arxiv.org/abs/1912.03441) · [OpenAlex](https://openalex.org/W2993439425)

**Citations:** 7 citations (OpenAlex, 2026-10-14) · [8](https://md.research.elamur.ai/free/en/papers/chien-2019-adversarial-analysis-natural-language-inference/cited-by/) in our library, of the 141,840 papers whose bibliographies we read

## Summary
The paper asks how robust NLI models with heavy pre-training really are, given that SNLI and MNLI contain annotation artifacts that allow shortcut solutions. Chien and Kalita fine-tune base and large versions of BERT and MT-DNN on MNLI and compare them with the older recurrent ESIM. They test all of them on three existing challenge sets:

- Glockner et al.: single-word lexical swaps based on WordNet.
- Naik et al.: stress tests built from an error analysis.
- McCoy et al.: syntactic-heuristic templates, examined across 30 subcases.

The models score almost the same on MNLI, yet their robustness differs a lot. Lexical knowledge turns out to be strong. Syntax and logic remain the weak point for every model, including garden-path sentences and propositional logic.

The authors also run two extra experiments. In one, they feed flattened dependency parses into unchanged pre-trained models, which helps some structure-dependent subcases but not all. In the other, they track how growing overfitting to MNLI exposes dataset biases.

The work matters because headline test accuracy hides real differences in understanding. Model size, multi-task learning and syntactic signals affect robustness in ways that standard benchmarks do not show.

## Key points
- **MNLI matched dev accuracy:**
  - ESIM 76.80
  - BERT base 84.17, BERT large 85.84
  - MT-DNN base 84.20, MT-DNN large 86.69
- **Glockner et al. single-word replacements:** BERT/MT-DNN models reach 92.2–94.8%, against 83.5 for the WordNet-augmented KIM and 65.6 for ESIM. Model size and multi-task learning make little difference, which suggests lexical semantics is a low-level skill.
- **Naik et al. stress tests:** averaged accuracies range from 68.39 (ESIM) to 77.21 (BERT large). All models show the same profile of strengths and weaknesses across categories, so the authors consider this set too coarse to tell models apart.
- **Naik et al. test types:** "competence" tests cover antonyms and spelling. "Distraction" tests append tautologies such as "and true is true" to probe word overlap, negation and length mismatch.
- **McCoy et al. dataset:** templates target nested shallow heuristics, starting with lexical overlap, where shared words push the model toward entailment. This set separated the strong models best and received a fine-grained analysis by model size (base vs. large) and by MT-DNN vs. BERT.

## Interesting ideas
- The inoculation diagnostic (Liu et al.) exposes a model to a small slice of a challenge set and then re-tests it on both the original and the challenge data; the pattern of gains and losses shows whether the model, the training data or the challenge set is at fault.
- A dependency parse can be flattened into a token string and fed to an unmodified pre-trained model, adding a structural bias with no change to the architecture.
- Models that tie on a standard benchmark can differ sharply on phenomenon-level subcases, so evaluation should be broken down by linguistic phenomenon rather than reported as a single score.
- Distraction tests append content that carries no meaning, such as tautologies, and check that the label does not change; a robust system must recognize irrelevant material.
- Tracking behaviour over increasing amounts of fine-tuning, as the model overfits more and more, reveals which dataset artifacts the model is learning.

## For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

- **I1** (learn): When a repair rule or a new silver batch fixes a failure class, re-run the original evaluation as well, to tell a real fix from a skewed test set.
- **I2** (tree): gparse or LLM teacher tiers could receive linearized UD trees from witness parsers as extra input, a cheap way to test whether explicit structure helps on hard attachments.
- **I3** (answer): School tests and fable QA results could be reported per phenomenon (negation, passives, embedded clauses, garden paths), so regressions hidden by the total score become visible.
- **I4** (world): Appending "and true is true" style clauses to a text should add no entities, states or events; this makes an easy invariance check for the world builder.
- **I5** (learn): Evaluate checkpoints along training against negative controls, to catch the point where the parser starts absorbing treebank quirks instead of general syntax.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [University of California, Berkeley](https://md.research.elamur.ai/free/en/universities/university-of-california-berkeley/) — University, Berkeley, United States · 127 works · 212,523 citations
- [University of Colorado System](https://md.research.elamur.ai/free/en/universities/university-of-colorado-system/) — University, Boulder, United States · 7 works · 1,348 citations

Also on the byline: [University of Colorado Colorado Springs](https://ror.org/054spjc55).

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 9,086 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 8,889 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 9,296 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 9,206 works

🧭 Topics (1): [🧮 Reasoning](https://md.research.elamur.ai/free/en/topics/reasoning/) · 510 works

## Cite

Cite as: Tiffany Chien and Jugal Kalita. 2019. Adversarial Analysis of Natural Language Inference Systems. arXiv:1912.03441.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@misc{chien2019adversarial,
  title = {{Adversarial Analysis of Natural Language Inference Systems}},
  author = {Tiffany Chien and Jugal Kalita},
  year = {2019},
  doi = {10.48550/arxiv.1912.03441},
  eprint = {1912.03441},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.1912.03441}
}
```

- [← previous The First Shared Task on Discourse Representation Structure Parsing](https://md.research.elamur.ai/free/en/papers/abzianidze-2019-first-shared-task-discourse-representation/)
- [next → Cross-document coreference: An approach to capturing coreference without context](https://md.research.elamur.ai/free/en/papers/bettner-2019-cross-document-coreference-approach-capturing/)
- [↑ back to papers of 2019](https://md.research.elamur.ai/free/en/papers/2019/page/6/)

## References

22 references, 12 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. T. McCoy, E. Pavlick, and T. Linzen, “Right for the Wrong Reasons: Diagnosing Syntactic Heuristics in Natural Language Inference,” in Proceedings of the 57th Conference of the Association for Computational Linguistics. Florence, Italy: Association for Computational Linguistics, Jul. 2019, pp. 3428–3448. [our card](https://md.research.elamur.ai/free/en/papers/mccoy-2019-right-wrong-reasons-diagnosing-syntactic/)
2. M. Glockner, V. Shwartz, and Y. Goldberg, “Breaking NLI Systems with Sentences that Require Simple Lexical Inferences,” in Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers). Melbourne, Australia: Association for Computational Linguistics, Jul. 2018, pp. 650–655. [our card](https://md.research.elamur.ai/free/en/papers/glockner-2018-breaking-nli-systems-sentences-that/)
3. A. Naik, A. Ravichander, N. Sadeh, C. Rose, and G. Neubig, “Stress Test Evaluation for Natural Language Inference,” arXiv:1806.00692 [cs], Jun. 2018, arXiv: 1806.00692. [↗](https://arxiv.org/abs/1806.00692)
4. A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, L. Kaiser, and I. Polosukhin, “Attention Is All You Need,” arXiv:1706.03762 [cs], Jun. 2017, arXiv: 1706.03762. [↗](https://arxiv.org/abs/1706.03762) [our card](https://md.research.elamur.ai/free/en/papers/vaswani-2017-attention-all-need/)
5. J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova, “BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding,” arXiv:1810.04805 [cs], Oct. 2018, arXiv: 1810.04805. [↗](https://arxiv.org/abs/1810.04805) [our card](https://md.research.elamur.ai/free/en/papers/devlin-2019-bert-pre-training-deep-bidirectional/)
6. X. Liu, P. He, W. Chen, and J. Gao, “Multi-Task Deep Neural Networks for Natural Language Understanding,” arXiv:1901.11504 [cs], Jan. 2019, arXiv: 1901.11504. [↗](https://arxiv.org/abs/1901.11504) [our card](https://md.research.elamur.ai/free/en/papers/liu-2019-multi-task-deep-neural-networks/)
7. Y. Belinkov and J. Glass, “Analysis Methods in Neural Language Processing: A Survey,” Transactions of the Association for Computational Linguistics, vol. 7, pp. 49–72, Apr. 2019. [↗](https://doi.org/10.1162/tacl_a_00254) [our card](https://md.research.elamur.ai/free/en/papers/belinkov-2019-analysis-methods-neural-language-processing/)
8. N. F. Liu, R. Schwartz, and N. A. Smith, “Inoculation by Fine-Tuning: A Method for Analyzing Challenge Datasets,” arXiv:1904.02668 [cs], Apr. 2019, arXiv: 1904.02668. [↗](https://arxiv.org/abs/1904.02668) [our card](https://md.research.elamur.ai/free/en/papers/liu-2019-inoculation-fine-tuning-method-analyzing/)
9. Y. Nie, Y. Wang, and M. Bansal, “Analyzing Compositionality-Sensitivity of NLI Models,” arXiv:1811.07033 [cs], Nov. 2018, arXiv: 1811.07033. [↗](https://arxiv.org/abs/1811.07033) [our card](https://md.research.elamur.ai/free/en/papers/nie-2018-analyzing-compositionality-sensitivity-nli-models/)
10. S. Gururangan, S. Swayamdipta, O. Levy, R. Schwartz, S. R. Bowman, and N. A. Smith, “Annotation Artifacts in Natural Language Inference Data,” arXiv:1803.02324 [cs], Mar. 2018, arXiv: 1803.02324. [↗](https://arxiv.org/abs/1803.02324)
11. A. Wang, A. Singh, J. Michael, F. Hill, O. Levy, and S. R. Bowman, “GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding,” arXiv:1804.07461 [cs], Apr. 2018, arXiv: 1804.07461. [↗](https://arxiv.org/abs/1804.07461) [our card](https://md.research.elamur.ai/free/en/papers/wang-2018-glue-multi-task-benchmark-analysis/)
12. Q. Chen, X. Zhu, Z. Ling, S. Wei, H. Jiang, and D. Inkpen, “Enhanced LSTM for Natural Language Inference,” arXiv:1609.06038 [cs], Sep. 2016, arXiv: 1609.06038. [↗](https://arxiv.org/abs/1609.06038)
13. Y. Goldberg, “Assessing BERT’s Syntactic Abilities,” arXiv:1901.05287 [cs], Jan. 2019, arXiv: 1901.05287. [↗](https://arxiv.org/abs/1901.05287) [our card](https://md.research.elamur.ai/free/en/papers/goldberg-2019-assessing-bert-s-syntactic-abilities/)
14. G. Jawahar, B. Sagot, and D. Seddah, “What Does BERT Learn about the Structure of Language?” in Proceedings of the 57th Conference of the Association for Computational Linguistics. Florence, Italy: Association for Computational Linguistics, Jul. 2019, pp. 3651–3657. [our card](https://md.research.elamur.ai/free/en/papers/jawahar-2019-what-does-bert-learn-about/)
15. S. R. Bowman, J. Gauthier, A. Rastogi, R. Gupta, C. D. Manning, and C. Potts, “A Fast Unified Model for Parsing and Sentence Understanding,” arXiv:1603.06021 [cs], Mar. 2016, arXiv: 1603.06021. [↗](https://arxiv.org/abs/1603.06021)
16. Y. Nie and M. Bansal, “Shortcut-Stacked Sentence Encoders for Multi-Domain Inference,” in Proceedings of the 2nd Workshop on Evaluating Vector Space Representations for NLP. Copenhagen, Denmark: Association for Computational Linguistics, Sep. 2017, pp. 41–45. [find](https://scholar.google.com/scholar?q=Y%2E+Nie+and+M%2E+Bansal%2C+%E2%80%9CShortcut%2DStacked+Sentence+Encoders+for+Multi%2DDomain+Inference%2C%E2%80%9D+in+Proceedings+of+the+2nd+Workshop+on+Evaluating+Vector+Space+Representations+for+NLP%2E+Copenhagen%2C+Denmark%3A+Assoc)
17. R. Evans, D. Saxton, D. Amos, P. Kohli, and E. Grefenstette, “Can Neural Networks Understand Logical Entailment?” arXiv:1802.08535 [cs], Feb. 2018, arXiv: 1802.08535. [↗](https://arxiv.org/abs/1802.08535)
18. P. Minervini and S. Riedel, “Adversarially Regularising Neural NLI Models to Integrate Logical Background Knowledge,” arXiv:1808.08609 [cs, stat], Aug. 2018, arXiv: 1808.08609. [↗](https://arxiv.org/abs/1808.08609)
19. Z. Hu, X. Ma, Z. Liu, E. Hovy, and E. Xing, “Harnessing Deep Neural Networks with Logic Rules,” in Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). Berlin, Germany: Association for Computational Linguistics, Aug. 2016, pp. 2410–2420. [our card](https://md.research.elamur.ai/free/en/papers/hu-2016-harnessing-deep-neural-networks-logic/)
20. H. Wang and H. Poon, “Deep Probabilistic Logic: A Unifying Framework for Indirect Supervision,” in Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing. Brussels, Belgium: Association for Computational Linguistics, Oct. 2018, pp. 1891–1902. [find](https://scholar.google.com/scholar?q=H%2E+Wang+and+H%2E+Poon%2C+%E2%80%9CDeep+Probabilistic+Logic%3A+A+Unifying+Framework+for+Indirect+Supervision%2C%E2%80%9D+in+Proceedings+of+the+2018+Conference+on+Empirical+Methods+in+Natural+Language+Processing%2E+Brussels%2C+Bel)
21. F. Ferreira and J. M. Henderson, “Recovery from misanalyses of garden-path sentences,” Journal of Memory and Language, vol. 30, no. 6, pp. 725–745, Dec. 1991. [↗](https://doi.org/10.1016/0749-596x(91)90034-h)
22. L. Osterhout, P. J. Holcomb, and D. A. Swinney, “Brain potentials elicited by garden-path sentences: Evidence of the application of verb information during parsing,” Journal of Experimental Psychology: Learning, Memory, and Cognition, vol. 20, no. 4, pp. 786–803, 1994. [↗](https://doi.org/10.1037//0278-7393.20.4.786)

*Updated 24 October 2026*