---
title: "On Adversarial Removal of Hypothesis-only Bias in Natural"
description: "AdvDat replaces some premises with random ones and reverses gradients into the hypothesis… 23 references, cited by 39 papers here."
authors: ["Yonatan Belinkov", "Adam Poliak", "Stuart Shieber", "Benjamin Van Durme", "Alexander Rush"]
year: 2019
venue: "Proceedings of the Eighth Joint Conference on Lexical and Computational Semantics (*SEM 2019)"
source: "https://aclanthology.org/S19-1028/"
doi: "10.18653/v1/s19-1028"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/belinkov-2019-adversarial-removal-hypothesis-only-bias/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/belinkov-2019-adversarial-removal-hypothesis-only-bias/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2019](https://md.research.elamur.ai/free/en/papers/2019/page/5/) / On Adversarial Removal of Hypothesis-only Bias in Natural Language Inference

# On Adversarial Removal of Hypothesis-only Bias in Natural Language Inference

**Authors:** [Yonatan Belinkov](https://md.research.elamur.ai/free/en/authors/yonatan-belinkov/), [Adam Poliak](https://md.research.elamur.ai/free/en/authors/adam-poliak/), [Stuart Shieber](https://md.research.elamur.ai/free/en/authors/stuart-m-shieber/), [Benjamin Van Durme](https://md.research.elamur.ai/free/en/authors/benjamin-van-durme/) and [Alexander Rush](https://md.research.elamur.ai/free/en/authors/alexander-m-rush/) · **Year:** 2019 · **Venue:** [Proceedings of the Eighth Joint Conference on Lexical and Computational Semantics (\*SEM 2019)](https://md.research.elamur.ai/free/en/venues/starsem/) ([\*SEM 2019](https://md.research.elamur.ai/free/en/venues/starsem/2019/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/S19-1028/) · [PDF](https://aclanthology.org/S19-1028.pdf) · [DOI 10.18653/v1/s19-1028](https://doi.org/10.18653/v1/s19-1028) · [OpenAlex](https://openalex.org/W2955503152)

**Citations:** 56 citations (OpenAlex, 2026-10-14) · [39](https://md.research.elamur.ai/free/en/papers/belinkov-2019-adversarial-removal-hypothesis-only-bias/cited-by/) in our library, of the 141,840 papers whose bibliographies we read

## Summary
The paper asks whether adversarial training can make NLI models stop exploiting hypothesis-only biases, which are cues in the hypothesis sentence that predict the label without reading the premise. The authors work on SNLI, where such cues are especially strong, and add two adversarial components to InferSent, a BiLSTM sentence-encoder model. AdvCls attaches a hypothesis-only classifier behind a gradient reversal layer, so the hypothesis encoder is pushed to hide label-predictive cues. AdvDat replaces some premises with random ones and reverses gradients into the hypothesis encoder for those noisy pairs. AdvCls keeps NLI accuracy almost unchanged, but its frozen hypothesis vectors still let a newly trained probe recover the bias. AdvDat removes more bias, sometimes leaving the hypothesis less informative than a random encoder, and it weakens the pull of "give-away" words. The price is a few points of accuracy. The work matters because it shows that fooling one adversary head is not the same as removing information, so removal has to be checked with fresh probes.

## Key points
- SNLI was chosen because a hypothesis-only model reaches about 69% there, compared with roughly 34% for the majority class. The authors also report results on SNLI-hard (Gururangan et al.).
- The base model is InferSent: GloVe inputs, BiLSTM encoders with max-pooling, and features built from concatenation, difference and element-wise product. Hyper-parameters (λ Loss, λ Enc, λ Rand) were swept over {0.05, 0.1, 0.2, 0.4, 0.8, 1.0}.
- Test accuracy (Val / Test / Hard):

  | Model | Val | Test | Hard |
  |---|---|---|---|
  | Baseline | 84.25 | 84.22 | 68.02 |
  | AdvCls | 84.58 | 83.56 | 66.27 |
  | AdvDat | 78.45 | 78.30 | 55.60 |

- A probe trained on a frozen random encoder already scores 63.26%, against a majority baseline of 34.28%. The authors attribute this to the pre-trained, non-updated word embeddings.
- On frequent contradiction cue words ("sleeping", "Nobody", "no", "naked"…), AdvDat with λ Rand=0.4, λ Enc=1 cut the rate of contradiction predictions by about 43–88% relative to the baseline. Other settings did not show this effect.

## Interesting ideas
- Check whether an adversary truly removed a signal by freezing the encoder and training a new probe from scratch, because a jointly trained adversary head can be fooled while the information stays in the vectors.
- Use a frozen random encoder over the same embeddings as a reference floor, since even random encoders can carry a lot of signal from their inputs.
- Swap the context for a random one and see whether the model's answer changes; if it doesn't, the model is answering from the question alone. The authors warn that a swapped pair may legitimately keep a neutral label.
- Measure how often a model predicts a label given that a specific cue word appears, and compare that rate with the word's empirical label correlation. This audits reliance on individual surface cues.
- The bias may live in fixed pre-trained embeddings that adversarial training never touches, so debiasing should extend down to the input representations.

## For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

- **I1** (learn): Whenever we claim a learned component ignores some feature, retrain a fresh probe on its frozen outputs as a negative control rather than trusting the training-time loss.
- **I2** (learn): Add a random-weights version of gparse or the tagger as a control row in evaluations, so we can tell how much a probe's score comes from embeddings alone.
- **I3** (answer): For fable QA and school tests, pair each question with an unrelated story and flag items the readers still answer "correctly" as solvable without the text, excluding cases where "unknown" is the honest answer.
- **I4** (answer): Compute answer-type rates conditioned on cue tokens like "not", "nobody" or "all" in test questions, to catch readers that react to negation words instead of the built world state.
- **I5** (tree): When checking gparse for dataset artifacts, inspect the frozen embedding layer too, not only the trainable layers above it.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Johns Hopkins University](https://md.research.elamur.ai/free/en/universities/johns-hopkins-university/) — University, Baltimore, United States · 187 works · 92,194 citations
- [Massachusetts Institute of Technology](https://md.research.elamur.ai/free/en/universities/massachusetts-institute-of-technology/) — University, Cambridge, United States · 159 works · 65,157 citations
- [Harvard University](https://md.research.elamur.ai/free/en/universities/harvard-university/) — University, Cambridge, United States · 73 works · 37,309 citations

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 9,086 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 8,889 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 9,296 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 9,206 works

🧭 Topics (1): [🧮 Reasoning](https://md.research.elamur.ai/free/en/topics/reasoning/) · 510 works

## Cite

Cite as: Yonatan Belinkov, Adam Poliak, Stuart Shieber, Benjamin Van Durme and Alexander Rush. 2019. On Adversarial Removal of Hypothesis-only Bias in Natural Language Inference. Proceedings of the Eighth Joint Conference on Lexical and Computational Semantics (\*SEM 2019).

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{belinkov2019adversarial,
  title = {{On Adversarial Removal of Hypothesis-only Bias in Natural Language Inference}},
  author = {Yonatan Belinkov, Adam Poliak, Stuart Shieber, Benjamin {Van Durme} and Alexander Rush},
  booktitle = {Proceedings of the Eighth Joint Conference on Lexical and Computational Semantics (*SEM 2019)},
  year = {2019},
  doi = {10.18653/v1/s19-1028},
  url = {https://doi.org/10.18653/v1/s19-1028}
}
```

- [← previous A Unified Linear-Time Framework for Sentence-Level Discourse Parsing](https://md.research.elamur.ai/free/en/papers/lin-2019-unified-linear-time-framework-sentence/)
- [next → The NiuTrans Machine Translation Systems for WMT19](https://md.research.elamur.ai/free/en/papers/li-2019-niutrans-machine-translation-systems-wmt19/)
- [↑ back to papers of 2019](https://md.research.elamur.ai/free/en/papers/2019/page/5/)

## References

23 references, 8 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Moustafa Alzantot, Yash Sharma, Ahmed Elgohary, Bo-Jhang Ho, Mani Srivastava, and Kai-Wei Chang. [Generating Natural Language Adversarial Examples](https://md.research.elamur.ai/free/en/papers/alzantot-2018-generating-natural-language-adversarial-examples/). In Proceedings of the 2018 Conference on Strengths, Shortcomings, and Side Effects. In Proceedings of the 2nd Workshop on Shortcomings in Vision and Language (SiVL) at NAACL-HLT, June 2019.
2. Empirical Methods in Natural Language Processing, pp. 2890–2896, 2018. [find](https://scholar.google.com/scholar?q=Empirical+Methods+in+Natural+Language+Processing%2C+pp%2E+2890%E2%80%932896%2C+2018%2E)
3. Yonatan Belinkov and Yonatan Bisk. Synthetic and Natural Noise Both Break Neural Machine Translation. In International Conference on Learning Representations, 2018. URL https://openreview.net/forum?id=BJ8vJebC-. [↗](https://doi.org/10.48550/arxiv.1711.02173)
4. Suchin Gururangan, Swabha Swayamdipta, Omer Levy, Roy Schwartz, Samuel Bowman, and Noah A. Smith. Annotation Artifacts in Natural Language Inference Data. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 2 (Short Papers), pp. 107–112, New Orleans, Louisiana, June 2018. Association for Computational Linguistics. URL http://www.aclweb.org/anthology/N18-2017. [↗](https://aclanthology.org/N18-2017/)
5. Samuel R. Bowman, Gabor Angeli, Christopher Potts, and Christopher D. Manning. A large annotated corpus for learning natural language inference. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing (EMNLP). Association for Computational Linguistics, 2015. [↗](https://doi.org/10.48550/arxiv.1508.05326)
6. Sepp Hochreiter and Jürgen Schmidhuber. Long short-term memory. Neural computation, 9(8):1735– 1780, 1997. [↗](https://doi.org/10.1162/neco.1997.9.8.1735)
7. Xilun Chen and Claire Cardie. Multinomial adversarial networks for multi-domain text classification. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers), pp. 1226–1240. Association for Computational Linguistics, 2018. URL http://aclweb.org/ anthology/N18-1111. [↗](https://aclanthology.org/N18-1111/)
8. Dongyeop Kang, Tushar Khot, Ashish Sabharwal, and Eduard Hovy. AdvEntuRe: Adversarial Training for Textual Entailment with Knowledge-Guided Examples. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 2418–2428, Melbourne, Australia, July 2018. Association for Computational Linguistics. URL http://www.aclweb.org/ anthology/P18-1225. [↗](https://aclanthology.org/P18-1225/)
9. Alexis Conneau, Douwe Kiela, Holger Schwenk, Loı̈c Barrault, and Antoine Bordes. [Supervised Learning of Universal Sentence Representations from Natural Language Inference Data](https://md.research.elamur.ai/free/en/papers/conneau-2017-supervised-learning-universal-sentence-representations/). In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pp. 670–680. Association for Computational Linguistics, 2017. URL http://aclweb.org/anthology/D17-1070. [↗](https://aclanthology.org/D17-1070/)
10. Pasquale Minervini and Sebastian Riedel. Adversarially Regularising Neural NLI Models to Integrate Logical Background Knowledge. In Proceedings of the 22nd Conference on Computational Natural Language Learning (CoNLL 2018). Association for Computational Linguistics, 2018. [↗](https://doi.org/10.48550/arxiv.1808.08609)
11. [Alexis Conneau, Germán Kruszewski, Guillaume Lample, Loı̈c Barrault, and Marco Baroni. What you can cram into a single $&!#* vector: Probing sentence embeddings for linguistic properties. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 2126–2136. Association for Computational Linguistics, 2018. URL http://aclweb.org/anthology/P18-1198.](https://md.research.elamur.ai/free/en/papers/conneau-2018-what-you-can-cram-into/) [↗](https://aclanthology.org/P18-1198/)
12. Lili Mou, Rui Men, Ge Li, Yan Xu, Lu Zhang, Rui Yan, and Zhi Jin. [Natural Language Inference by Tree-Based Convolution and Heuristic Matching](https://md.research.elamur.ai/free/en/papers/mou-2016-natural-language-inference-tree-based/). In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pp. 130–136, Berlin, Germany, August 2016. Association for Computational Linguistics. URL http://anthology.aclweb.org/ P16-2022. [↗](https://aclanthology.org/P16-2022/)
13. Yanai Elazar and Yoav Goldberg. [Adversarial Removal of Demographic Attributes from Text Data](https://md.research.elamur.ai/free/en/papers/elazar-2018-adversarial-removal-demographic-attributes-text/). In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pp. 11–21. Association for Computational Linguistics, 2018. URL http://aclweb.org/ anthology/D18-1002. [↗](https://aclanthology.org/D18-1002/)
14. Jeffrey Pennington, Richard Socher, and Christopher Manning. [Glove: Global vectors for word representation](https://md.research.elamur.ai/free/en/papers/pennington-2014-glove-global-vectors-word-representation/). In Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP), pp. 1532–1543, 2014.
15. Adam Poliak, Jason Naradowsky, Aparajita Haldar, Rachel Rudinger, and Benjamin Van Durme. Hypothesis Only Baselines in Natural Language Inference. In Proceedings of the Seventh Joint Conference on Lexical and Computational Semantics, pp. 180–191, New Orleans, Louisiana, June 2018. Association for Computational Linguistics. URL http://www.aclweb.org/anthology/S18-2023. [↗](https://aclanthology.org/S18-2023/)
16. Yaroslav Ganin and Victor Lempitsky. Unsupervised Domain Adaptation by Backpropagation. In International Conference on Machine Learning, pp. 1180–1189, 2015. [↗](https://doi.org/10.48550/arxiv.1409.7495)
17. Ian J Goodfellow, Jonathon Shlens, and Christian Szegedy. [Explaining and Harnessing Adversarial Examples](https://md.research.elamur.ai/free/en/papers/goodfellow-2014-explaining-harnessing-adversarial-examples/). In International Conference on Learning Representations (ICLR), 2015. [↗](https://doi.org/10.48550/arxiv.1412.6572)
18. Sainandan Ramakrishnan, Aishwarya Agrawal, and Stefan Lee. Overcoming Language Priors in Visual Question Answering with Adversarial Regularization. In NIPS, pp. 1548–1558, 2018. [↗](https://doi.org/10.48550/arxiv.1810.03649)
19. Gabriel Grand and Yonatan Belinkov. Adversarial Regularization for Visual Question Answering: A Roy Schwartz, Maarten Sap, Ioannis Konstas, Leila Zilles, Yejin Choi, and Noah A. Smith. Story Cloze Task: UW NLP System. In Proceedings of LSDSem, 2017. [find](https://scholar.google.com/scholar?q=Gabriel+Grand+and+Yonatan+Belinkov%2E+Adversarial+Regularization+for+Visual+Question+Answering%3A+A+Roy+Schwartz%2C+Maarten+Sap%2C+Ioannis+Konstas%2C+Leila+Zilles%2C+Yejin+Choi%2C+and+Noah+A%2E+Smith%2E+Story+Cloze+Tas)
20. Stronger hyper-parameters for AdvCls Figure 3 provides validation results using AdvCls with stronger hyper-parameters to complement the discussion in §4.2. While it is difficult to notice trends, all configurations perform similarly and slightly below the baseline. These models seem to be less stable compared to using smaller hyperparameters, as discussed in §4.2. Masatoshi Tsuchiya. [Performance Impact Caused by Hidden Bias of Training Data for Recognizing Textual Entailment](https://md.research.elamur.ai/free/en/papers/tsuchiya-2018-performance-impact-caused-hidden-bias/). In 11th International Conference on Language Resources and Evaluation (LREC2018), 2018.
21. Qizhe Xie, Zihang Dai, Yulun Du, Eduard Hovy, and Graham Neubig. Controllable invariance through adversarial feature learning. In Advances in Neural Information Processing Systems, pp. 585–596, 2017. [↗](https://doi.org/10.48550/arxiv.1705.11122)
22. Brian Hu Zhang, Blake Lemoine, and Margaret Mitchell. Mitigating unwanted biases with adversarial learning. In Proceedings of the 2018 AAAI/ACM Conference on AI, Ethics, and Society, pp. 335–340. [find](https://scholar.google.com/scholar?q=Brian+Hu+Zhang%2C+Blake+Lemoine%2C+and+Margaret+Mitchell%2E+Mitigating+unwanted+biases+with+adversarial+learning%2E+In+Proceedings+of+the+2018+AAAI%2FACM+Conference+on+AI%2C+Ethics%2C+and+Society%2C+pp%2E+335%E2%80%93340%2E)
23. Kelly Zhang and Samuel Bowman. Language Modeling Teaches You More than Translation Does: Lessons Learned Through Auxiliary Task Analysis. In Proceedings of the 2018 EMNLP Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP, pp. 359–361, Brussels, Belgium, November 2018. Association for Computational Linguistics. URL http://www.aclweb.org/ anthology/W18-5448. *Figure 3: Validation results using AdvCls with stronger hyper-parameters.* [↗](https://aclanthology.org/W18-5448/)

*Updated 24 October 2026*