---
title: "Performance Impact Caused by Hidden Bias of — Tsuchiya 2018"
description: "This paper asks whether a large RTE corpus contains systematic artifacts that let a model guess the entailment label without doing real inference."
authors: ["Masatoshi Tsuchiya"]
year: 2018
venue: "Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018)"
source: "https://aclanthology.org/L18-1239/"
licence: "CC BY-NC 4.0 (ELRA, LREC proceedings)"
licence_url: "https://creativecommons.org/licenses/by-nc/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/papers/tsuchiya-2018-performance-impact-caused-hidden-bias/"
---
[Elamur Research](https://md.research.elamur.ai/) · [this page in HTML](https://research.elamur.ai/free/papers/tsuchiya-2018-performance-impact-caused-hidden-bias/)

[home](https://md.research.elamur.ai/) / [papers](https://md.research.elamur.ai/free/papers/) / [2018](https://md.research.elamur.ai/free/papers/2018/page/3/) / Performance Impact Caused by Hidden Bias of Training Data for Recognizing Textual Entailment

# Performance Impact Caused by Hidden Bias of Training Data for Recognizing Textual Entailment

**Authors:** Masatoshi Tsuchiya · **Year:** 2018 · **Venue:** Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018) · **Licence:** [CC BY-NC 4.0 (ELRA, LREC proceedings)](https://creativecommons.org/licenses/by-nc/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/L18-1239/) · [PDF](https://aclanthology.org/L18-1239.pdf) · [OpenAlex](https://openalex.org/W2798617936)

**Citations:** 30 citations (openalex-snapshot · 2026-10-14) · [95](#cited-by) in our library, of the 140673 papers whose bibliographies we read

## Summary
This paper asks whether a large RTE corpus contains systematic artifacts that let a model guess the entailment label without doing real inference. The author proposes a quality check modelled on a statistical hypothesis test. Since an entailment label should, by definition, depend on the premise, the "null hypothesis" states that the label can be determined from the hypothesis sentence alone. This is tested by comparing a multinomial Naive Bayes classifier that sees only hypothesis-sentence unigrams with a majority-class baseline, using a sign test.

The check was applied to SNLI and SICK. On SICK, the hypothesis-only model is no better than the baseline. On SNLI it is far better, which reveals a hidden bias. The SNLI test set is then split into an "easy" part (pairs the hypothesis-only model gets right) and a "hard" part. Two LSTM-based RTE models show a large performance drop on the hard part. This suggests that much of their apparent skill comes from exploiting the bias rather than from reasoning over sentence pairs.

## Key points
- **Hypothesis-only model:** multinomial NB (Wang and Manning, 2012) with word unigrams of the hypothesis as features, implemented with scikit-learn. It is compared to a baseline that always outputs the most frequent label.
- **SNLI result:** 63.3% accuracy versus 34.3% for the baseline. The sign test gives p = 5.7e−202, and the confusion matrix shows real per-sentence predictions across all three labels.
- **SICK result:** both models reach 56.7% (p = 0.65). The NB model essentially outputs "neutral" for nearly every item, so no exploitable bias is detected.
- **SNLI test split:**
  - empirical easy set E_e: 6,219 pairs (63.3%);
  - empirical hard set H_e: 3,605 pairs (36.7%);
  - labels are spread fairly evenly across entailment, neutral and contradiction in both subsets.
- **Models probed:**
  - a "parallel" LSTM model: 300-d GloVe embeddings, two 100-d sentence LSTMs concatenated, three tanh layers and softmax;
  - an attention-less "sequential" LSTM model following Rocktäschel et al. (2015).
  - Both perform much worse on H_e than on E_e.

## Interesting ideas
- **Remove the input that should be essential.** Deliberately withholding a piece of the input that the task definition requires makes a cheap probe for annotation artifacts. If a classifier still beats chance, the data leaks label information. The same probe could be applied to QA (answers without the question), parallel-corpus filtering, or any paired-input dataset.
- **Use a deliberately weak probe.** A bag-of-words Naive Bayes model is a strong choice for an artifact detector precisely because it is weak. Any success it has can only come from surface lexical cues, not from genuine understanding.
- **Frame the audit as a significance test.** Comparing the probe against a majority baseline with a sign test, rather than just reporting accuracy, gives a principled yes/no verdict. The test needs only the data itself, with no multiple annotations or agreement scores required.
- **Inspect the confusion matrix, not just accuracy.** On SICK, identical accuracies hide a degenerate always-"neutral" model. Checking the matrix distinguishes true predictive signal from majority-label behaviour.
- **Build easy/hard splits from the probe.** Splitting a test set by whether the artifact model succeeds yields a hard subset for stress-testing stronger models. This is reusable for evaluating any new benchmark or model, including for low-resource languages such as Ukrainian.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

Fields: [Linguistics & languages](https://md.research.elamur.ai/free/shelves/linguistics/)

## Cited by

Cited by

95 papers of our library cite it (of the 140673 whose bibliographies we read); 11 of them have a page here, 84 not yet.

- [Don't Take the Premise for Granted: Mitigating Artifacts in Natural Language Inference](https://md.research.elamur.ai/free/papers/belinkov-2019-don-t-take-premise-granted/)
- [Abductive Commonsense Reasoning](https://md.research.elamur.ai/free/papers/bhagavatula-2019-abductive-commonsense-reasoning/)
- [On the Opportunities and Risks of Foundation Models](https://md.research.elamur.ai/free/papers/bommasani-2021-opportunities-risks-foundation-models/)
- [Uncertain Natural Language Inference](https://md.research.elamur.ai/free/papers/chen-2020-uncertain-natural-language-inference/)
- [JGLUE: Japanese General Language Understanding Evaluation](https://md.research.elamur.ai/free/papers/kurihara-2022-jglue-japanese-general-language-understanding/)
- [WANLI: Worker and AI Collaboration for Natural Language Inference Dataset Creation](https://md.research.elamur.ai/free/papers/liu-2022-wanli-worker-ai-collaboration-natural/)
- [Collecting Diverse Natural Language Inference Problems for Sentence Representation Evaluation](https://md.research.elamur.ai/free/papers/poliak-2018-collecting-diverse-natural-language-inference/)
- [Get Your Vitamin C! Robust Fact Verification with Contrastive Evidence](https://md.research.elamur.ai/free/papers/schuster-2021-get-your-vitamin-c-robust/)
- [Masked Language Modeling and the Distributional Hypothesis: Order Word Matters Pre-training for Little](https://md.research.elamur.ai/free/papers/sinha-2021-masked-language-modeling-distributional-hypothesis/)
- [Symbolic Knowledge Distillation: from General Language Models to Commonsense Models](https://md.research.elamur.ai/free/papers/west-2022-symbolic-knowledge-distillation-general-language/)
- [DocNLI: A Large-scale Dataset for Document-level Natural Language Inference](https://md.research.elamur.ai/free/papers/yin-2021-docnli-large-scale-dataset-document/)

## Cite

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{tsuchiya2018performance,
  title = {{Performance Impact Caused by Hidden Bias of Training Data for Recognizing Textual Entailment}},
  author = {Tsuchiya, Masatoshi},
  booktitle = {Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018)},
  year = {2018},
  doi = {10.63317/26mjmiipm2qf},
  url = {https://doi.org/10.63317/26mjmiipm2qf}
}
```

- [← previous Parsing Tweets into Universal Dependencies](https://md.research.elamur.ai/free/papers/ud-2018-liu-parsing-tweets-into-universal-dependencies/)
- [next → Personalizing Dialogue Agents: I have a dog, do you have pets too?](https://md.research.elamur.ai/free/papers/zhang-2018-personalizing-dialogue-agents-i-have/)
- [↑ back to papers of 2018](https://md.research.elamur.ai/free/papers/2018/page/3/)

## References

30 references, 7 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Al Khatib, K., Wachsmuth, H., Kiesel, J., Hagen, M., and Stein, B. (2016). A news editorial corpus for mining argumentation strategies. In Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers, pages 3433–3443, Osaka, Japan, December. [find](https://scholar.google.com/scholar?q=Al+Khatib%2C+K%2E%2C+Wachsmuth%2C+H%2E%2C+Kiesel%2C+J%2E%2C+Hagen%2C+M%2E%2C+and+Stein%2C+B%2E+%282016%29%2E+A+news+editorial+corpus+for+mining+argumentation+strategies%2E+In+Proceedings+of+COLING+2016%2C+the+26th+International+Conference)
2. The COLING 2016 Organizing Committee. [find](https://scholar.google.com/scholar?q=The+COLING+2016+Organizing+Committee%2E)
3. Artstein, R. and Poesio, M. (2008). Inter-coder agreement for computational linguistics. Computational Linguistics, 34(4):555–596. [↗](https://doi.org/10.1162/coli.07-034-r2)
4. Bos, J. and Markert, K. (2005). Recognising textual entailment with logical inference. In Proceedings of Human Language Technology Conference and Conference on Empirical Methods in Natural Language Processing, pages 628–635, Vancouver, British Columbia, Canada, October. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Bos%2C+J%2E+and+Markert%2C+K%2E+%282005%29%2E+Recognising+textual+entailment+with+logical+inference%2E+In+Proceedings+of+Human+Language+Technology+Conference+and+Conference+on+Empirical+Methods+in+Natural+Language+Pr) [our card](https://md.research.elamur.ai/free/papers/bos-2005-recognising-textual-entailment-logical-inference/)
5. Bowman, S. R., Angeli, G., Potts, C., and Manning, C. D. (2015). A large annotated corpus for learning natural language inference. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages 632–642, Lisbon, Portugal, September. Association for Computational Linguistics. [↗](https://doi.org/10.48550/arxiv.1508.05326)
6. Cheng, J., Dong, L., and Lapata, M. (2016). Long short-term memory-networks for machine reading. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 551–561, Austin, Texas, November. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Cheng%2C+J%2E%2C+Dong%2C+L%2E%2C+and+Lapata%2C+M%2E+%282016%29%2E+Long+short%2Dterm+memory%2Dnetworks+for+machine+reading%2E+In+Proceedings+of+the+2016+Conference+on+Empirical+Methods+in+Natural+Language+Processing%2C+pages+551%E2%80%9356) [our card](https://md.research.elamur.ai/free/papers/cheng-2016-long-short-term-memory-networks/)
7. Condoravdi, C., Crouch, D., de Paiva, V., Stolle, R., and Bobrow, D. G. (2003). Entailment, intensionality and text understanding. In Graeme Hirst et al., editors, Proceedings of the HLT-NAACL 2003 Workshop on Text Meaning, pages 38–45. [find](https://scholar.google.com/scholar?q=Condoravdi%2C+C%2E%2C+Crouch%2C+D%2E%2C+de+Paiva%2C+V%2E%2C+Stolle%2C+R%2E%2C+and+Bobrow%2C+D%2E+G%2E+%282003%29%2E+Entailment%2C+intensionality+and+text+understanding%2E+In+Graeme+Hirst+et+al%2E%2C+editors%2C+Proceedings+of+the+HLT%2DNAACL+2003+Wo)
8. Craggs, R. and Wood, M. M. (2005). Evaluating discourse and dialogue coding schemes. Computational Linguistics, 31(3):289–296, September. [↗](https://doi.org/10.1162/089120105774321109)
9. Liu, P., Qiu, X., Chen, J., and Huang, X. (2016a). Deep fusion lstms for text semantic matching. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1034–1043, Berlin, Germany, August. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Liu%2C+P%2E%2C+Qiu%2C+X%2E%2C+Chen%2C+J%2E%2C+and+Huang%2C+X%2E+%282016a%29%2E+Deep+fusion+lstms+for+text+semantic+matching%2E+In+Proceedings+of+the+54th+Annual+Meeting+of+the+Association+for+Computational+Linguistics+%28Volume+1%3A+L)
10. Liu, P., Qiu, X., Zhou, Y., Chen, J., and Huang, X. (2016b). Modelling interaction of sentence pair with coupled-lstms. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 1703–1712, Austin, Texas, November. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Liu%2C+P%2E%2C+Qiu%2C+X%2E%2C+Zhou%2C+Y%2E%2C+Chen%2C+J%2E%2C+and+Huang%2C+X%2E+%282016b%29%2E+Modelling+interaction+of+sentence+pair+with+coupled%2Dlstms%2E+In+Proceedings+of+the+2016+Conference+on+Empirical+Methods+in+Natural+Language+P)
11. MacCartney, B. and Manning, C. D. (2009). An extended model of natural logic. In Proceedings of the Eight International Conference on Computational Semantics, pages 140–156, Tilburg, The Netherlands, January. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=MacCartney%2C+B%2E+and+Manning%2C+C%2E+D%2E+%282009%29%2E+An+extended+model+of+natural+logic%2E+In+Proceedings+of+the+Eight+International+Conference+on+Computational+Semantics%2C+pages+140%E2%80%93156%2C+Tilburg%2C+The+Netherlands%2C+)
12. Marelli, M., Bentivogli, L., Baroni, M., Bernardi, R., Menini, S., and Zamparelli, R. (2014a). Semeval-2014 task 1: Evaluation of compositional distributional semantic models on full sentences through semantic relatedness and textual entailment. In Proceedings of the 8th International Workshop on Semantic Evaluation (SemEval 2014), pages 1–8, Dublin, Ireland, August. Association for Computational Linguistics and Dublin City University. [find](https://scholar.google.com/scholar?q=Marelli%2C+M%2E%2C+Bentivogli%2C+L%2E%2C+Baroni%2C+M%2E%2C+Bernardi%2C+R%2E%2C+Menini%2C+S%2E%2C+and+Zamparelli%2C+R%2E+%282014a%29%2E+Semeval%2D2014+task+1%3A+Evaluation+of+compositional+distributional+semantic+models+on+full+sentences+through) [our card](https://md.research.elamur.ai/free/papers/marelli-2014-semeval-2014-task-1-evaluation/)
13. Marelli, M., Menini, S., Baroni, M., Bentivogli, L., Bernardi, R., and Zamparelli, R. (2014b). A SICK cure for the evaluation of compositional distributional semantic models. In Nicoletta Calzolari (Conference Chair), et al., editors, Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC’14), pages 216–223, Reykjavik, Iceland, may. European Language Resources Association (ELRA). [find](https://scholar.google.com/scholar?q=Marelli%2C+M%2E%2C+Menini%2C+S%2E%2C+Baroni%2C+M%2E%2C+Bentivogli%2C+L%2E%2C+Bernardi%2C+R%2E%2C+and+Zamparelli%2C+R%2E+%282014b%29%2E+A+SICK+cure+for+the+evaluation+of+compositional+distributional+semantic+models%2E+In+Nicoletta+Calzolari+%28C)
14. Mathet, Y., Widlöcher, A., Fort, K., François, C., Galibert, O., Grouin, C., Kahn, J., Rosset, S., and Zweigenbaum, P. (2012). Manual corpus annotation: Giving meaning to the evaluation metrics. In Proceedings of COLING 2012: Posters, pages 809–818, Mumbai, India, December. [find](https://scholar.google.com/scholar?q=Mathet%2C+Y%2E%2C+WidlA%CC%83%C2%B6cher%2C+A%2E%2C+Fort%2C+K%2E%2C+Franc%CC%A7ois%2C+C%2E%2C+Galibert%2C+O%2E%2C+Grouin%2C+C%2E%2C+Kahn%2C+J%2E%2C+Rosset%2C+S%2E%2C+and+Zweigenbaum%2C+P%2E+%282012%29%2E+Manual+corpus+annotation%3A+Giving+meaning+to+the+evaluation+metrics%2E+In)
15. The COLING 2012 Organizing Committee. [find](https://scholar.google.com/scholar?q=The+COLING+2012+Organizing+Committee%2E)
16. Mou, L., Men, R., Li, G., Xu, Y., Zhang, L., Yan, R., and Jin, Z. (2016a). Natural language inference by tree-based convolution and heuristic matching. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 130–136, Berlin, Germany, August. Association for Computational Linguistics. [↗](https://doi.org/10.60692/webgv-2rq18) [our card](https://md.research.elamur.ai/free/papers/mou-2016-natural-language-inference-tree-based/)
17. Mou, L., Meng, Z., Yan, R., Li, G., Xu, Y., Zhang, L., and Jin, Z. (2016b). How transferable are neural networks in nlp applications? In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 479–489, Austin, Texas, November. Association for Computational Linguistics. [↗](https://doi.org/10.60692/89tvw-g5k46)
18. Parikh, A., Täckström, O., Das, D., and Uszkoreit, J. (2016). A decomposable attention model for natural language inference. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 2249–2255, Austin, Texas, November. Association for Computational Linguistics. [↗](https://doi.org/10.48550/arxiv.1606.01933) [our card](https://md.research.elamur.ai/free/papers/parikh-2016-decomposable-attention-model-natural-language/)
19. Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., Blondel, M., Prettenhofer, P., Weiss, R., Dubourg, V., Vanderplas, J., Passos, A., Cournapeau, D., Brucher, M., Perrot, M., and Duchesnay, E. (2011). Scikit-learn: Machine learning in Python. Journal of Machine Learning Research, 12:2825–2830. [↗](https://doi.org/10.48550/arxiv.1201.0490)
20. Pennington, J., Socher, R., and Manning, C. (2014). Glove: Global vectors for word representation. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 1532– 1543, Doha, Qatar, October. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Pennington%2C+J%2E%2C+Socher%2C+R%2E%2C+and+Manning%2C+C%2E+%282014%29%2E+Glove%3A+Global+vectors+for+word+representation%2E+In+Proceedings+of+the+2014+Conference+on+Empirical+Methods+in+Natural+Language+Processing+%28EMNLP%29%2C+pa)
21. Reidsma, D. and Carletta, J. (2008). Reliability measurement without limits. Computational Linguistics, 34(3):319–326, September. [↗](https://doi.org/10.1162/coli.2008.34.3.319)
22. Rocktäschel, T., Grefenstette, E., Hermann, K. M., Kociský, T., and Blunsom, P. (2015). Reasoning about entailment with neural attention. In Proceedings of The International Conference on Learning Representations (ICLR2016). [↗](https://doi.org/10.48550/arxiv.1509.06664) [our card](https://md.research.elamur.ai/free/papers/rocktaschel-2015-reasoning-about-entailment-neural-attention/)
23. Sabou, M., Bontcheva, K., Derczynski, L., and Scharl, A. (2014). Corpus annotation through crowdsourcing: Towards best practice guidelines. pages 859–866. [find](https://scholar.google.com/scholar?q=Sabou%2C+M%2E%2C+Bontcheva%2C+K%2E%2C+Derczynski%2C+L%2E%2C+and+Scharl%2C+A%2E+%282014%29%2E+Corpus+annotation+through+crowdsourcing%3A+Towards+best+practice+guidelines%2E+pages+859%E2%80%93866%2E)
24. Sha, L., Chang, B., Sui, Z., and Li, S. (2016). Reading and thinking: Re-read lstm unit for textual entailment recognition. In Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers, pages 2870–2879, Osaka, Japan, December. [find](https://scholar.google.com/scholar?q=Sha%2C+L%2E%2C+Chang%2C+B%2E%2C+Sui%2C+Z%2E%2C+and+Li%2C+S%2E+%282016%29%2E+Reading+and+thinking%3A+Re%2Dread+lstm+unit+for+textual+entailment+recognition%2E+In+Proceedings+of+COLING+2016%2C+the+26th+International+Conference+on+Computat)
25. The COLING 2016 Organizing Committee. [find](https://scholar.google.com/scholar?q=The+COLING+2016+Organizing+Committee%2E)
26. Wang, S. and Jiang, J. (2016). Learning natural language inference with lstm. In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 1442–1451, San Diego, California, June. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Wang%2C+S%2E+and+Jiang%2C+J%2E+%282016%29%2E+Learning+natural+language+inference+with+lstm%2E+In+Proceedings+of+the+2016+Conference+of+the+North+American+Chapter+of+the+Association+for+Computational+Linguistics%3A+Huma)
27. Wang, S. and Manning, C. (2012). Baselines and bigrams: Simple, good sentiment and topic classification. In Proceedings of the 50th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 90–94, Jeju Island, Korea, July. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Wang%2C+S%2E+and+Manning%2C+C%2E+%282012%29%2E+Baselines+and+bigrams%3A+Simple%2C+good+sentiment+and+topic+classification%2E+In+Proceedings+of+the+50th+Annual+Meeting+of+the+Association+for+Computational+Linguistics+%28Vol)
28. Yin, W., Schà 14 tze, H., Xiang, B., and Zhou, B. (2016). Abcnn: Attention-based convolutional neural network for modeling sentence pairs. Transactions of the Association for Computational Linguistics, 4:259–272. [↗](https://doi.org/10.1162/tacl_a_00097) [our card](https://md.research.elamur.ai/free/papers/yin-2016-abcnn-attention-based-convolutional-neural/)
29. Young, P., Lai, A., Hodosh, M., and Hockenmaier, J. (2014). From image descriptions to visual denotations: New similarity metrics for semantic inference over event descriptions. Transactions of the Association for Computational Linguistics, 2:67–78. [↗](https://doi.org/10.1162/tacl_a_00166)
30. Zhang, C., Bengio, S., Hardt, M., Recht, B., and Vinyals, O. (2017). Understanding deep learning requires rethinking generalization. In Proceedings of The International Conference on Learning Representations (ICLR2016). [↗](https://doi.org/10.48550/arxiv.1611.03530)