---
title: "Mitigating Label Biases for In-context Learning — Fei 2023"
description: "In-context learning (ICL) with large language models is unstable: the prompt and the data can push a… 35 references, cited by 51 papers here."
authors: ["Yu Fei", "Yifan Hou", "Zeming Chen", "Antoine Bosselut"]
year: 2023
venue: "Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)"
source: "https://aclanthology.org/2023.acl-long.783/"
doi: "10.18653/v1/2023.acl-long.783"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/fei-2023-mitigating-label-biases-context-learning/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/fei-2023-mitigating-label-biases-context-learning/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2023](https://md.research.elamur.ai/free/en/papers/2023/page/7/) / Mitigating Label Biases for In-context Learning

# Mitigating Label Biases for In-context Learning

**Authors:** Yu Fei, Yifan Hou, [Zeming Chen](https://md.research.elamur.ai/free/en/authors/zeming-chen/) and [Antoine Bosselut](https://md.research.elamur.ai/free/en/authors/antoine-bosselut/) · **Year:** 2023 · **Venue:** [Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)](https://md.research.elamur.ai/free/en/venues/acl/) ([ACL 2023](https://md.research.elamur.ai/free/en/venues/acl/2023/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2023.acl-long.783/) · [PDF](https://aclanthology.org/2023.acl-long.783.pdf) · [DOI 10.18653/v1/2023.acl-long.783](https://doi.org/10.18653/v1/2023.acl-long.783) · [OpenAlex](https://openalex.org/W4385570446)

**Citations:** 28 citations (OpenAlex, 2026-10-14) · [51](https://md.research.elamur.ai/free/en/papers/fei-2023-mitigating-label-biases-context-learning/cited-by/) in our library, of the 141,840 papers whose bibliographies we read

## Summary
In-context learning (ICL) with large language models is unstable: the prompt and the data can push a model toward certain label names whatever the input. The authors sort these effects into three kinds. Vanilla-label bias is a preference for some label words even without context. Context-label bias comes from the demonstrations, such as their order or template. Domain-label bias, which they identify for the first time, is a skew caused by the vocabulary of the task corpus itself. They measure this third bias by comparing label priors when the model sees random generic English words versus random words drawn from the dataset. On tasks where this bias is high, GPT-J often scores at chance level, and Contextual Calibration (CC) does not help. Their fix, Domain-context Calibration (DC), estimates label priors from several random bags of in-domain words and divides predictions by those priors. Across 24 classification datasets, DC clearly improves GPT-J and GPT-3. This shows that ICL evaluations should report results by dataset bias level rather than one aggregate score.

## Key points
- **Bias metric:** domain-label bias is half the L1 distance between two sets of label priors. One set comes from random English words, the other from random in-domain words, each matched to the dataset's average text length. On Tweet hate, in-domain words shift GPT-J's priors to roughly 0.95 hate versus 0.05 neutral.
- **DC procedure:** DC samples dataset-length strings from a bag-of-words built on unlabeled task texts. It repeats this M times, averages the resulting label probabilities, and uses the average to rescale each prediction.
- **Average gains:** DC raises average ICL performance by about 20% for GPT-J and 18% for GPT-3. Gains reach up to 37% Macro-F1 on high-bias tasks.
- **Tweet hate vs. SST-2:** on Tweet hate, GPT-J Macro-F1 goes from 31.4 (original) and 31.8 (CC) to 63.0 with DC. On SST-2, which has low domain bias, all methods score about 94–95.
- **Generalization and label names:** benefits carry over to different model scales, Instruct-GPT, manually written task instructions, and zero-shot prompting of RoBERTa. After calibration, labels that look sensible to humans can perform worse than arbitrary ones such as X/Y.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [University of California, Irvine](https://md.research.elamur.ai/free/en/universities/university-of-california-irvine/) — University, Irvine, United States · 29 works · 36,049 citations
- [École Polytechnique Fédérale de Lausanne](https://md.research.elamur.ai/free/en/universities/ecole-polytechnique-federale-de-lausanne/) — University, Lausanne, Switzerland · 23 works · 6,687 citations
- [ETH Zurich](https://md.research.elamur.ai/free/en/universities/eth-zurich/) — University, Zurich, Switzerland · 36 works · 4,534 citations

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 9,086 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 8,889 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 9,296 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 9,206 works

🧭 Topics (1): [🧠 Language models & pretraining](https://md.research.elamur.ai/free/en/topics/language-models/) · 1,343 works

## Cite

Cite as: Yu Fei, Yifan Hou, Zeming Chen and Antoine Bosselut. 2023. Mitigating Label Biases for In-context Learning. Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers).

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{fei2023mitigating,
  title = {{Mitigating Label Biases for In-context Learning}},
  author = {Yu Fei, Yifan Hou, Zeming Chen and Antoine Bosselut},
  booktitle = {Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2023},
  doi = {10.18653/v1/2023.acl-long.783},
  url = {https://doi.org/10.18653/v1/2023.acl-long.783}
}
```

- [← previous D4: Improving LLM Pretraining via Document De-Duplication and Diversification](https://md.research.elamur.ai/free/en/papers/tirumala-2023-d4-improving-llm-pretraining-document/)
- [next → Scaling Laws vs Model Architectures: How does Inductive Bias Influence Scaling?](https://md.research.elamur.ai/free/en/papers/tay-2023-scaling-laws-vs-model-architectures/)
- [↑ back to papers of 2023](https://md.research.elamur.ai/free/en/papers/2023/page/7/)

## References

35 references, 24 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Francesco Barbieri, Jose Camacho-Collados, Luis Espinosa Anke, and Leonardo Neves. 2020. [TweetEval: Unified benchmark and comparative evaluation for tweet classification](https://md.research.elamur.ai/free/en/papers/barbieri-2020-tweeteval-unified-benchmark-comparative-evaluation/). In Findings of the Association for Computational Linguistics: EMNLP 2020, pages 1644–1650, Online. Association for Computational…
2. Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. 2020. [Language models are few-shot learners](https://md.research.elamur.ai/free/en/papers/brown-2020-language-models-few-shot-learners/). Advances in neural information processing systems, 33:1877–1901.
3. Aakanksha Chowdhery, Sharan Narang, Jacob Devlin, Maarten Bosma, Gaurav Mishra, Adam Roberts, Paul Barham, Hyung Won Chung, Charles Sutton, Sebastian Gehrmann, et al. 2022. [Palm: Scaling language modeling with pathways](https://md.research.elamur.ai/free/en/papers/chowdhery-2022-palm-scaling-language-modeling-pathways/). arXiv preprint arXiv:2204.02311.
4. Hyung Won Chung, Le Hou, Shayne Longpre, Barret Zoph, Yi Tay, William Fedus, Eric Li, Xuezhi Wang, Mostafa Dehghani, Siddhartha Brahma, et al. 2022. [Scaling instruction-finetuned language models](https://md.research.elamur.ai/free/en/papers/chung-2022-scaling-instruction-finetuned-language-models/). arXiv preprint arXiv:2210.11416.
5. Ido Dagan, Oren Glickman, and Bernardo Magnini. 2005. The pascal recognising textual entailment challenge. In Machine learning challenges workshop, pages 177–190. Springer. [find](https://scholar.google.com/scholar?q=Ido+Dagan%2C+Oren+Glickman%2C+and+Bernardo+Magnini%2E+2005%2E+The+pascal+recognising+textual+entailment+challenge%2E+In+Machine+learning+challenges+workshop%2C+pages+177%E2%80%93190%2E+Springer%2E)
6. Ona de Gibert, Naiara Perez, Aitor García-Pablos, and Montse Cuadros. 2018. [Hate speech dataset from a white supremacy forum](https://md.research.elamur.ai/free/en/papers/gibert-2018-hate-speech-dataset-white-supremacy/). In Proceedings of the 2nd Workshop on Abusive Language Online (ALW2), pages 11–20, Brussels, Belgium. Association for Computational Linguistics.
7. Marie-Catherine De Marneffe, Mandy Simons, and Judith Tonhauser. 2019. [The commitmentbank: Investigating projection in naturally occurring discourse](https://md.research.elamur.ai/free/en/papers/marneffe-2019-commitmentbank-investigating-projection-naturally/). In proceedings of Sinn und Bedeutung, volume 23, pages 107–124.
8. Tianyu Gao, Adam Fisch, and Danqi Chen. 2020. [Making pre-trained language models better few-shot learners](https://md.research.elamur.ai/free/en/papers/gao-2021-making-pre-trained-language-models/). arXiv preprint arXiv:2012.15723.
9. Ari Holtzman, Peter West, Vered Shwartz, Yejin Choi, and Luke Zettlemoyer. 2021. [Surface form competition: Why the highest probability answer isn’t always right](https://md.research.elamur.ai/free/en/papers/holtzman-2021-surface-form-competition-why-highest/). arXiv preprint arXiv:2104.08315.
10. Minqing Hu and Bing Liu. 2004. Mining and summarizing customer reviews. In Proceedings of the tenth ACM SIGKDD international conference on Knowledge discovery and data mining, pages 168–177. [find](https://scholar.google.com/scholar?q=Minqing+Hu+and+Bing+Liu%2E+2004%2E+Mining+and+summarizing+customer+reviews%2E+In+Proceedings+of+the+tenth+ACM+SIGKDD+international+conference+on+Knowledge+discovery+and+data+mining%2C+pages+168%E2%80%93177%2E)
11. Sewon Min, Xinxi Lyu, Ari Holtzman, Mikel Artetxe, Mike Lewis, Hannaneh Hajishirzi, and Luke Zettlemoyer. 2022. [Rethinking the role of demonstrations: What makes in-context learning work?](https://md.research.elamur.ai/free/en/papers/min-2022-rethinking-role-demonstrations-what-makes/) arXiv preprint arXiv:2202.12837.
12. Swaroop Mishra, Daniel Khashabi, Chitta Baral, Yejin Choi, and Hannaneh Hajishirzi. 2021. [Reframing instructional prompts to gptk’s language](https://md.research.elamur.ai/free/en/papers/mishra-2022-reframing-instructional-prompts-gptk-s/). arXiv preprint arXiv:2109.07830.
13. Ioannis Mollas, Zoe Chrysopoulou, Stamatis Karlos, and Grigorios Tsoumakas. 2022. Ethos: a multi-label hate speech detection dataset. Complex & Intelligent Systems, pages 1–16. [find](https://scholar.google.com/scholar?q=Ioannis+Mollas%2C+Zoe+Chrysopoulou%2C+Stamatis+Karlos%2C+and+Grigorios+Tsoumakas%2E+2022%2E+Ethos%3A+a+multi%2Dlabel+hate+speech+detection+dataset%2E+Complex+%26+Intelligent+Systems%2C+pages+1%E2%80%9316%2E)
14. Long Ouyang, Jeff Wu, Xu Jiang, Diogo Almeida, Carroll L Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. 2022. [Training language models to follow instructions with human feedback](https://md.research.elamur.ai/free/en/papers/ouyang-2022-training-language-models-follow-instructions/). arXiv preprint arXiv:2203.02155.
15. Hector Levesque, Ernest Davis, and Leora Morgenstern. 2012. The winograd schema challenge. In Thirteenth international conference on the principles of knowledge representation and reasoning. [find](https://scholar.google.com/scholar?q=Hector+Levesque%2C+Ernest+Davis%2C+and+Leora+Morgenstern%2E+2012%2E+The+winograd+schema+challenge%2E+In+Thirteenth+international+conference+on+the+principles+of+knowledge+representation+and+reasoning%2E)
16. Bo Pang and Lillian Lee. 2004. [A sentimental education: Sentiment analysis using subjectivity summarization based on minimum cuts](https://md.research.elamur.ai/free/en/papers/pang-2004-sentimental-education-sentiment-analysis-subjectivity/). arXiv preprint cs/0409058.
17. Quentin Lhoest, Albert Villanova del Moral, Yacine Jernite, Abhishek Thakur, Patrick von Platen, Suraj Patil, Julien Chaumond, Mariama Drame, Julien Plu, Lewis Tunstall, et al. 2021. [Datasets: A community library for natural language processing](https://md.research.elamur.ai/free/en/papers/lhoest-2021-datasets-community-library-natural-language/). arXiv preprint arXiv:2109.02846.
18. Bo Pang and Lillian Lee. 2005. [Seeing stars: Exploiting class relationships for sentiment categorization with respect to rating scales](https://md.research.elamur.ai/free/en/papers/pang-2005-seeing-stars-exploiting-class-relationships/). arXiv preprint cs/0506075.
19. Jiachang Liu, Dinghan Shen, Yizhe Zhang, Bill Dolan, Lawrence Carin, and Weizhu Chen. 2021. [What makes good in-context examples for gpt-3?](https://md.research.elamur.ai/free/en/papers/liu-2022-what-makes-good-context-examples/) arXiv preprint arXiv:2101.06804.
20. Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. Roberta: A robustly optimized bert pretraining approach. arXiv preprint arXiv:1907.11692. [find](https://scholar.google.com/scholar?q=Yinhan+Liu%2C+Myle+Ott%2C+Naman+Goyal%2C+Jingfei+Du%2C+Mandar+Joshi%2C+Danqi+Chen%2C+Omer+Levy%2C+Mike+Lewis%2C+Luke+Zettlemoyer%2C+and+Veselin+Stoyanov%2E+2019%2E+Roberta%3A+A+robustly+optimized+bert+pretraining+approach%2E+a)
21. Yao Lu, Max Bartolo, Alastair Moore, Sebastian Riedel, and Pontus Stenetorp. 2021. [Fantastically ordered prompts and where to find them: Overcoming few-shot prompt order sensitivity](https://md.research.elamur.ai/free/en/papers/lu-2022-fantastically-ordered-prompts-where-find/). arXiv preprint arXiv:2104.08786.
22. Yasaman Razeghi, Robert L Logan IV, Matt Gardner, and Sameer Singh. 2022. Impact of pretraining term frequencies on few-shot reasoning. arXiv preprint arXiv:2202.07206. — [arXiv:2202.07206](https://arxiv.org/abs/2202.07206) [find](https://scholar.google.com/scholar?q=Yasaman+Razeghi%2C+Robert+L+Logan+IV%2C+Matt+Gardner%2C+and+Sameer+Singh%2E+2022%2E+Impact+of+pretraining+term+frequencies+on+few%2Dshot+reasoning%2E+arXiv+preprint+arXiv%3A2202%2E07206%2E+%E2%80%94+%5BarXiv%3A2202%2E07206%5D%28https%3A%2F%2Far)
23. Timo Schick and Hinrich Schütze. 2020. [Exploiting cloze questions for few shot text classification and natural language inference](https://md.research.elamur.ai/free/en/papers/schick-2021-exploiting-cloze-questions-few-shot/). arXiv preprint arXiv:2001.07676.
24. Emily Sheng and David Uthus. 2020. Investigating societal biases in a poetry composition system. In Proceedings of the Second Workshop on Gender Bias in Natural Language Processing, pages 93–106, Barcelona, Spain (Online). Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Emily+Sheng+and+David+Uthus%2E+2020%2E+Investigating+societal+biases+in+a+poetry+composition+system%2E+In+Proceedings+of+the+Second+Workshop+on+Gender+Bias+in+Natural+Language+Processing%2C+pages+93%E2%80%93106%2C+Barc)
25. Pekka Malo, Ankur Sinha, Pekka Korhonen, Jyrki Wallenius, and Pyry Takala. 2014. Good debt or bad debt: Detecting semantic orientations in economic texts. Journal of the Association for Information Science and Technology, 65(4):782–796. [find](https://scholar.google.com/scholar?q=Pekka+Malo%2C+Ankur+Sinha%2C+Pekka+Korhonen%2C+Jyrki+Wallenius%2C+and+Pyry+Takala%2E+2014%2E+Good+debt+or+bad+debt%3A+Detecting+semantic+orientations+in+economic+texts%2E+Journal+of+the+Association+for+Information+Sc)
26. Richard Socher, Alex Perelygin, Jean Wu, Jason Chuang, Christopher D Manning, Andrew Y Ng, and Christopher Potts. 2013. Recursive deep models for semantic compositionality over a sentiment treebank. In Proceedings of the 2013 conference on empirical methods in natural language processing, pages 1631… [find](https://scholar.google.com/scholar?q=Richard+Socher%2C+Alex+Perelygin%2C+Jean+Wu%2C+Jason+Chuang%2C+Christopher+D+Manning%2C+Andrew+Y+Ng%2C+and+Christopher+Potts%2E+2013%2E+Recursive+deep+models+for+semantic+compositionality+over+a+sentiment+treebank%2E+I)
27. Marco Marelli, Stefano Menini, Marco Baroni, Luisa Bentivogli, Raffaella Bernardi, and Roberto Zamparelli. 2014. [A sick cure for the evaluation of compositional distributional semantic models](https://md.research.elamur.ai/free/en/papers/marelli-2014-sick-cure-evaluation-compositional-distributional/). In Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC’14), pages…
28. Taylor Sorensen, Joshua Robinson, Christopher Michael Rytting, Alexander Glenn Shaw, Kyle Jeffrey Rogers, Alexia Pauline Delorey, Mahmoud Khalil, Nancy Fulda, and David Wingate. 2022. [An information-theoretic approach to prompt engineering without ground truth labels](https://md.research.elamur.ai/free/en/papers/sorensen-2022-information-theoretic-approach-prompt-engineering/). arXiv preprint arXiv:2203.11364…
29. Ellen M Voorhees and Dawn M Tice. 2000. Building a question answering test collection. In Proceedings of the 23rd annual international ACM SIGIR conference on Research and development in information retrieval, pages 200–207. [find](https://scholar.google.com/scholar?q=Ellen+M+Voorhees+and+Dawn+M+Tice%2E+2000%2E+Building+a+question+answering+test+collection%2E+In+Proceedings+of+the+23rd+annual+international+ACM+SIGIR+conference+on+Research+and+development+in+information+r)
30. Ben Wang and Aran Komatsuzaki. 2021. GPT-J-6B: A 6 Billion Parameter Autoregressive Language Model. [https://github.com/kingoflolz/](https://github.com/kingoflolz/) mesh-transformer-jax. [↗](https://github.com/kingoflolz/)
31. Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Ed Chi, Quoc Le, and Denny Zhou. 2022. [Chain of thought prompting elicits reasoning in large language models](https://md.research.elamur.ai/free/en/papers/wei-2022-chain-thought-prompting-elicits-reasoning/). arXiv preprint arXiv:2201.11903.
32. Jerry Wei, Jason Wei, Yi Tay, Dustin Tran, Albert Webson, Yifeng Lu, Xinyun Chen, Hanxiao Liu, Da Huang, Denny Zhou, et al. 2023. [Larger language models do in-context learning differently](https://md.research.elamur.ai/free/en/papers/wei-2023-larger-language-models-context-learning/). arXiv preprint arXiv:2303.03846.
33. Susan Zhang, Stephen Roller, Naman Goyal, Mikel Artetxe, Moya Chen, Shuohui Chen, Christopher Dewan, Mona Diab, Xian Li, Xi Victoria Lin, et al. 2022. [Opt: Open pre-trained transformer language models](https://md.research.elamur.ai/free/en/papers/zhang-2022-opt-open-pre-trained-transformer/). arXiv preprint arXiv:2205.01068.
34. Xiang Zhang, Junbo Zhao, and Yann LeCun. 2015. [Character-level convolutional networks for text classification](https://md.research.elamur.ai/free/en/papers/zhang-2015-character-level-convolutional-networks-text/). Advances in neural information processing systems, 28.
35. Zihao Zhao, Eric Wallace, Shi Feng, Dan Klein, and Sameer Singh. 2021. [Calibrate before use: Improving few-shot performance of language models](https://md.research.elamur.ai/free/en/papers/zhao-2021-calibrate-before-use-improving-few/). In International Conference on Machine Learning, pages 12697–12706. PMLR.

*Updated 24 October 2026*