---
title: "Mind the Trade-off — paper by Utama et al. (2020)"
description: "The authors propose confidence regularization to remove this trade-off. 31 references, cited by 67 papers here."
authors: ["Prasetya Ajie Utama", "Nafise Sadat Moosavi", "Iryna Gurevych"]
year: 2020
venue: "Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics"
source: "https://aclanthology.org/2020.acl-main.770/"
doi: "10.18653/v1/2020.acl-main.770"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/utama-2020-mind-trade-off-debiasing-nlu/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/utama-2020-mind-trade-off-debiasing-nlu/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2020](https://md.research.elamur.ai/free/en/papers/2020/page/4/) / Mind the Trade-off: Debiasing NLU Models without Degrading the In-distribution Performance

# Mind the Trade-off: Debiasing NLU Models without Degrading the In-distribution Performance

**Authors:** [Prasetya Ajie Utama](https://md.research.elamur.ai/free/en/authors/prasetya-ajie-utama/), [Nafise Sadat Moosavi](https://md.research.elamur.ai/free/en/authors/nafise-sadat-moosavi/) and [Iryna Gurevych](https://md.research.elamur.ai/free/en/authors/iryna-gurevych/) · **Year:** 2020 · **Venue:** [Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics](https://md.research.elamur.ai/free/en/venues/acl/) ([ACL 2020](https://md.research.elamur.ai/free/en/venues/acl/2020/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2020.acl-main.770/) · [PDF](https://aclanthology.org/2020.acl-main.770.pdf) · [DOI 10.18653/v1/2020.acl-main.770](https://doi.org/10.18653/v1/2020.acl-main.770) · [OpenAlex](https://openalex.org/W3023038345)

**Citations:** 92 citations (OpenAlex, 2026-10-14) · [67](https://md.research.elamur.ai/free/en/papers/utama-2020-mind-trade-off-debiasing-nlu/cited-by/) in our library, of the 141,840 papers whose bibliographies we read

## Summary
Fine-tuned language models such as BERT often succeed on NLU benchmarks by exploiting dataset biases, such as word overlap or indicative words, and they then fail on out-of-distribution test sets. Earlier debiasing methods, product-of-expert and learned-mixin, improve robustness by down-weighting examples that a bias-only model already solves. This sacrifices accuracy on the original in-distribution data. The authors propose confidence regularization to remove this trade-off. A teacher model trained in the usual way provides soft labels through knowledge distillation. These targets are then scaled according to how confidently a biased model predicts each example. The main model therefore learns from every training example but is pushed toward lower confidence on biased ones. The method was tested on three tasks: natural language inference (MNLI/HANS), fact verification (FEVER/Fever-Symmetric) and paraphrase identification (QQP/PAWS). It matched earlier methods on out-of-distribution sets while keeping in-distribution accuracy. It also improved calibration. This matters because the gains are less likely to reflect a model that has simply become biased in another direction.

## Key points
- Confidence regularization raises accuracy on HANS by about 7 points and slightly improves in-distribution MNLI accuracy.
- Expected calibration error (ECE) falls by 3.0–3.6 on the MNLI-m and MNLI-mm dev sets.
- QQP is heavily biased: about 80% of training examples can be solved from lexical overlap alone, and 53% are predicted with confidence above 0.8, versus 12% for MNLI. This explains a slight in-distribution drop on QQP.
- When QQP training data is augmented with 2,500 PAWS examples and tested on 10K held-out PAWS examples, the method reaches 77.05%, compared with 71.63% for baseline BERT.
- Code: github.com/UKPLab/acl2020-confidence-regularization.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Technische Universität Darmstadt](https://md.research.elamur.ai/free/en/universities/technische-universitat-darmstadt/) — University, Darmstadt, Germany · 76 works · 35,008 citations

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 9,086 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 8,889 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 9,296 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 9,206 works

🧭 Topics (1): [🛡️ Bias, fairness & safety](https://md.research.elamur.ai/free/en/topics/bias-fairness-safety/) · 209 works

## Cite

Cite as: Prasetya Ajie Utama, Nafise Sadat Moosavi and Iryna Gurevych. 2020. Mind the Trade-off: Debiasing NLU Models without Degrading the In-distribution Performance. Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{utama2020mind,
  title = {{Mind the Trade-off: Debiasing NLU Models without Degrading the In-distribution Performance}},
  author = {Prasetya Ajie Utama, Nafise Sadat Moosavi and Iryna Gurevych},
  booktitle = {Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics},
  year = {2020},
  doi = {10.18653/v1/2020.acl-main.770},
  url = {https://doi.org/10.18653/v1/2020.acl-main.770}
}
```

- [← previous I Feel Offended, Don't Be Abusive! Implicit/Explicit Messages in Offensive and Abusive Language](https://md.research.elamur.ai/free/en/papers/caselli-2020-i-feel-offended-don-t/)
- [next → A Sentiment Analysis Dataset for Code-Mixed Malayalam-English](https://md.research.elamur.ai/free/en/papers/chakravarthi-2020-sentiment-analysis-dataset-code-mixed/)
- [↑ back to papers of 2020](https://md.research.elamur.ai/free/en/papers/2020/page/4/)

## References

31 references, 14 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Aishwarya Agrawal, Dhruv Batra, and Devi Parikh. 2016. [Analyzing the behavior of visual question answering models](https://md.research.elamur.ai/free/en/papers/agrawal-2016-analyzing-behavior-visual-question-answering/). In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 1955–1960, Austin, Texas. Association for Computational Linguistics.
2. Yonatan Belinkov, Adam Poliak, Stuart M. Shieber, Benjamin Van Durme, and Alexander M. Rush. 2019. [On adversarial removal of hypothesis-only bias in natural language inference](https://md.research.elamur.ai/free/en/papers/belinkov-2019-adversarial-removal-hypothesis-only-bias/). In Proceedings of the Eighth Joint Conference on Lexical and Computational Semantics, *SEM@NAACL-HLT 2019, Minneapolis, MN,…
3. Emily Bender and Alexander Koller. 2020. [Climbing towards NLU: On meaning, form, and understanding in the age of data](https://md.research.elamur.ai/free/en/papers/bender-2020-climbing-nlu-meaning-form-understanding/). In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, page to appear, virtual conference. Association for Computational Linguistics.
4. Christopher Clark, Mark Yatskar, and Luke Zettlemoyer. 2019a. Don’t take the easy way out: Ensemble based methods for avoiding known dataset biases. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Languag… [find](https://scholar.google.com/scholar?q=Christopher+Clark%2C+Mark+Yatskar%2C+and+Luke+Zettlemoyer%2E+2019a%2E+Don%E2%80%99t+take+the+easy+way+out%3A+Ensemble+based+methods+for+avoiding+known+dataset+biases%2E+In+Proceedings+of+the+2019+Conference+on+Empirical+)
5. Kevin Clark, Minh-Thang Luong, Urvashi Khandelwal, Christopher D. Manning, and Quoc V. Le. 2019b. [BAM! born-again multi-task networks for natural language understanding](https://md.research.elamur.ai/free/en/papers/clark-2019-bam-born-again-multi-task/). In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 5931–5937, Florence, Italy. Asso…
6. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. [BERT: Pre-training of deep bidirectional transformers for language understanding](https://md.research.elamur.ai/free/en/papers/devlin-2019-bert-pre-training-deep-bidirectional/). In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies,…
7. Tobias Falke, Leonardo F. R. Ribeiro, Prasetya Ajie Utama, Ido Dagan, and Iryna Gurevych. 2019. [Ranking generated summaries by correctness: An interesting but challenging application for natural language inference](https://md.research.elamur.ai/free/en/papers/falke-2019-ranking-generated-summaries-correctness-interesting/). In Proceedings of the 57th Annual Meeting of the Association for Computational Lingui…
8. Shi Feng, Eric Wallace, Alvin Grissom II, Mohit Iyyer, Pedro Rodriguez, and Jordan Boyd-Graber. 2018. Pathologies of neural models make interpretations difficult. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 3719–3728, Brussels, Belgium. Associatio… [find](https://scholar.google.com/scholar?q=Shi+Feng%2C+Eric+Wallace%2C+Alvin+Grissom+II%2C+Mohit+Iyyer%2C+Pedro+Rodriguez%2C+and+Jordan+Boyd%2DGraber%2E+2018%2E+Pathologies+of+neural+models+make+interpretations+difficult%2E+In+Proceedings+of+the+2018+Conference)
9. Tommaso Furlanello, Zachary Chase Lipton, Michael Tschannen, Laurent Itti, and Anima Anandkumar. 2018. Born-again neural networks. In Proceedings of the 35th International Conference on Machine Learning, ICML 2018, Stockholmsmässan, Stockholm, Sweden, July 10-15, 2018, volume 80 of Proceedings of M… [find](https://scholar.google.com/scholar?q=Tommaso+Furlanello%2C+Zachary+Chase+Lipton%2C+Michael+Tschannen%2C+Laurent+Itti%2C+and+Anima+Anandkumar%2E+2018%2E+Born%2Dagain+neural+networks%2E+In+Proceedings+of+the+35th+International+Conference+on+Machine+Learni)
10. Chuan Guo, Geoff Pleiss, Yu Sun, and Kilian Q. Weinberger. 2017. On calibration of modern neural networks. In Proceedings of the 34th International Conference on Machine Learning, ICML 2017, Sydney, NSW, Australia, 6-11 August 2017, volume 70 of Proceedings of Machine Learning Research, pages 1321–1… [find](https://scholar.google.com/scholar?q=Chuan+Guo%2C+Geoff+Pleiss%2C+Yu+Sun%2C+and+Kilian+Q%2E+Weinberger%2E+2017%2E+On+calibration+of+modern+neural+networks%2E+In+Proceedings+of+the+34th+International+Conference+on+Machine+Learning%2C+ICML+2017%2C+Sydney%2C+N)
11. Suchin Gururangan, Swabha Swayamdipta, Omer Levy, Roy Schwartz, Samuel Bowman, and Noah A. Smith. 2018. Annotation artifacts in natural language inference data. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Techno… [find](https://scholar.google.com/scholar?q=Suchin+Gururangan%2C+Swabha+Swayamdipta%2C+Omer+Levy%2C+Roy+Schwartz%2C+Samuel+Bowman%2C+and+Noah+A%2E+Smith%2E+2018%2E+Annotation+artifacts+in+natural+language+inference+data%2E+In+Proceedings+of+the+2018+Conference+o)
12. He He, Sheng Zha, and Haohan Wang. 2019. [Unlearn dataset bias in natural language inference by fitting the residual](https://md.research.elamur.ai/free/en/papers/he-2019-unlearn-dataset-bias-natural-language/). In Proceedings of the 2nd Workshop on Deep Learning Approaches for Low-Resource NLP, DeepLo@EMNLP-IJCNLP 2019, Hong Kong, China, November 3, 2019, pages 132–142. Association for Compu…
13. Geoffrey E. Hinton, Oriol Vinyals, and Jeffrey Dean. 2015. [Distilling the knowledge in a neural network](https://md.research.elamur.ai/free/en/papers/hinton-2015-distilling-knowledge-neural-network/). CoRR, abs/1503.02531.
14. Divyansh Kaushik, Eduard Hovy, and Zachary Lipton. 2020. Learning the difference that makes a difference with counterfactually-augmented data. In 8th International Conference on Learning Representations, ICLR 2020, Virtual Conference, 26 April - 1 May, 2019. OpenReview.net. [find](https://scholar.google.com/scholar?q=Divyansh+Kaushik%2C+Eduard+Hovy%2C+and+Zachary+Lipton%2E+2020%2E+Learning+the+difference+that+makes+a+difference+with+counterfactually%2Daugmented+data%2E+In+8th+International+Conference+on+Learning+Representatio)
15. Rabeeh Karimi Mahabadi and James Henderson. 2019. Simple but effective techniques to reduce biases. CoRR, abs/1909.06321. — [arXiv:1909.06321](https://arxiv.org/abs/1909.06321) [find](https://scholar.google.com/scholar?q=Rabeeh+Karimi+Mahabadi+and+James+Henderson%2E+2019%2E+Simple+but+effective+techniques+to+reduce+biases%2E+CoRR%2C+abs%2F1909%2E06321%2E+%E2%80%94+%5BarXiv%3A1909%2E06321%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F1909%2E06321%29)
16. R Thomas McCoy, Junghyun Min, and Tal Linzen. 2019a. Berts of a feather do not generalize together: Large variability in generalization across models with similar test set performance. arXiv preprint arXiv:1911.02969. [find](https://scholar.google.com/scholar?q=R+Thomas+McCoy%2C+Junghyun+Min%2C+and+Tal+Linzen%2E+2019a%2E+Berts+of+a+feather+do+not+generalize+together%3A+Large+variability+in+generalization+across+models+with+similar+test+set+performance%2E+arXiv+preprint+)
17. Tom McCoy, Ellie Pavlick, and Tal Linzen. 2019b. [Right for the wrong reasons: Diagnosing syntactic heuristics in natural language inference](https://md.research.elamur.ai/free/en/papers/mccoy-2019-right-wrong-reasons-diagnosing-syntactic/). In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 3428–3448, Florence, Italy. Association for Computational Lin…
18. Mahdi Pakdaman Naeini, Gregory F. Cooper, and Milos Hauskrecht. 2015. Obtaining well calibrated probabilities using bayesian binning. In Proceedings of the Twenty-Ninth AAAI Conference on Artificial Intelligence, January 25-30, 2015, Austin, Texas, USA, pages 2901–2907. AAAI Press. [find](https://scholar.google.com/scholar?q=Mahdi+Pakdaman+Naeini%2C+Gregory+F%2E+Cooper%2C+and+Milos+Hauskrecht%2E+2015%2E+Obtaining+well+calibrated+probabilities+using+bayesian+binning%2E+In+Proceedings+of+the+Twenty%2DNinth+AAAI+Conference+on+Artificial+I)
19. Timothy Niven and Hung-Yu Kao. 2019. Probing neural network comprehension of natural language arguments. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 4658–4664, Florence, Italy. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Timothy+Niven+and+Hung%2DYu+Kao%2E+2019%2E+Probing+neural+network+comprehension+of+natural+language+arguments%2E+In+Proceedings+of+the+57th+Annual+Meeting+of+the+Association+for+Computational+Linguistics%2C+pag)
20. Nicolas Papernot, Patrick D. McDaniel, Xi Wu, Somesh Jha, and Ananthram Swami. 2016. Distillation as a defense to adversarial perturbations against deep neural networks. In IEEE Symposium on Security and Privacy, SP 2016, San Jose, CA, USA, May 22-26, 2016, pages 582–597. IEEE Computer Society. [find](https://scholar.google.com/scholar?q=Nicolas+Papernot%2C+Patrick+D%2E+McDaniel%2C+Xi+Wu%2C+Somesh+Jha%2C+and+Ananthram+Swami%2E+2016%2E+Distillation+as+a+defense+to+adversarial+perturbations+against+deep+neural+networks%2E+In+IEEE+Symposium+on+Security+)
21. Gabriel Pereyra, George Tucker, Jan Chorowski, Lukasz Kaiser, and Geoffrey E. Hinton. 2017. [Regularizing neural networks by penalizing confident output distributions](https://md.research.elamur.ai/free/en/papers/pereyra-2017-regularizing-neural-networks-penalizing-confident/). In 5th International Conference on Learning Representations, ICLR 2017, Toulon, France, April 24-26, 2017, Workshop Track Proceedings…
22. Adam Poliak, Jason Naradowsky, Aparajita Haldar, Rachel Rudinger, and Benjamin Van Durme. 2018. Hypothesis only baselines in natural language inference. In Proceedings of the Seventh Joint Conference on Lexical and Computational Semantics, pages 180–191, New Orleans, Louisiana. Association for Compu… [find](https://scholar.google.com/scholar?q=Adam+Poliak%2C+Jason+Naradowsky%2C+Aparajita+Haldar%2C+Rachel+Rudinger%2C+and+Benjamin+Van+Durme%2E+2018%2E+Hypothesis+only+baselines+in+natural+language+inference%2E+In+Proceedings+of+the+Seventh+Joint+Conference+)
23. Keisuke Sakaguchi, Ronan Le Bras, Chandra Bhagavatula, and Yejin Choi. 2019. WINOGRANDE: an adversarial winograd schema challenge at scale. CoRR, abs/1907.10641. [find](https://scholar.google.com/scholar?q=Keisuke+Sakaguchi%2C+Ronan+Le+Bras%2C+Chandra+Bhagavatula%2C+and+Yejin+Choi%2E+2019%2E+WINOGRANDE%3A+an+adversarial+winograd+schema+challenge+at+scale%2E+CoRR%2C+abs%2F1907%2E10641%2E)
24. Fact Extraction and VERification (FEVER), pages 1–9, Brussels, Belgium. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Fact+Extraction+and+VERification+%28FEVER%29%2C+pages+1%E2%80%939%2C+Brussels%2C+Belgium%2E+Association+for+Computational+Linguistics%2E)
25. Alex Wang, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel Bowman. 2018. [GLUE: A multi-task benchmark and analysis platform for natural language understanding](https://md.research.elamur.ai/free/en/papers/wang-2018-glue-multi-task-benchmark-analysis/). In Proceedings of the 2018 EMNLP Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP, pages 353–355,…
26. Adina Williams, Nikita Nangia, and Samuel Bowman. 2018. A broad-coverage challenge corpus for sentence understanding through inference. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Pa… [find](https://scholar.google.com/scholar?q=Adina+Williams%2C+Nikita+Nangia%2C+and+Samuel+Bowman%2E+2018%2E+A+broad%2Dcoverage+challenge+corpus+for+sentence+understanding+through+inference%2E+In+Proceedings+of+the+2018+Conference+of+the+North+American+Chap)
27. Rowan Zellers, Ari Holtzman, Yonatan Bisk, Ali Farhadi, and Yejin Choi. 2019. Hellaswag: Can a machine really finish your sentence? In Proceedings of the 57th Conference of the Association for Computational Linguistics, ACL 2019, Florence, Italy, July 28- August 2, 2019, Volume 1: Long Papers, pages… [find](https://scholar.google.com/scholar?q=Rowan+Zellers%2C+Ari+Holtzman%2C+Yonatan+Bisk%2C+Ali+Farhadi%2C+and+Yejin+Choi%2E+2019%2E+Hellaswag%3A+Can+a+machine+really+finish+your+sentence%3F+In+Proceedings+of+the+57th+Conference+of+the+Association+for+Computa)
28. Yuan Zhang, Jason Baldridge, and Luheng He. 2019. [PAWS: Paraphrase adversaries from word scrambling](https://md.research.elamur.ai/free/en/papers/zhang-2019-paws-paraphrase-adversaries-word-scrambling/). In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 1298–1308, M…
29. Tal Schuster, Darsh Shah, Yun Jie Serene Yeo, Daniel Roberto Filizzola Ortiz, Enrico Santus, and Regina Barzilay. 2019. [Towards debiasing fact verification models](https://md.research.elamur.ai/free/en/papers/schuster-2019-debiasing-fact-verification-models/). In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on…
30. Christian Szegedy, Vincent Vanhoucke, Sergey Ioffe, Jonathon Shlens, and Zbigniew Wojna. 2016. Rethinking the inception architecture for computer vision. In 2016 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2016, Las Vegas, NV, USA, June 27-30, 2016, pages 2818–2826. IEEE Compute… [find](https://scholar.google.com/scholar?q=Christian+Szegedy%2C+Vincent+Vanhoucke%2C+Sergey+Ioffe%2C+Jonathon+Shlens%2C+and+Zbigniew+Wojna%2E+2016%2E+Rethinking+the+inception+architecture+for+computer+vision%2E+In+2016+IEEE+Conference+on+Computer+Vision+and)
31. James Thorne, Andreas Vlachos, Oana Cocarascu, Christos Christodoulopoulos, and Arpit Mittal. 2018. [The fact extraction and VERification (FEVER) shared task](https://md.research.elamur.ai/free/en/papers/thorne-2018-fact-extraction-verification-fever-shared/). In Proceedings of the First Workshop on

*Updated 24 October 2026*