---
title: "Want To Reduce Labeling Cost? GPT-3 Can Help — Wang 2021"
description: "Manual annotation for NLP is slow and expensive. Most pseudo-labeling methods are tied to one task and need a sizable labeled seed set. This paper asks…"
authors: ["Shuohang Wang", "Yang Liu", "Yichong Xu", "Chenguang Zhu", "Michael Zeng"]
year: 2021
venue: "Findings of the Association for Computational Linguistics: EMNLP 2021"
source: "https://aclanthology.org/2021.findings-emnlp.354/"
doi: "10.18653/v1/2021.findings-emnlp.354"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/wang-2021-want-reduce-labeling-cost-gpt/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/wang-2021-want-reduce-labeling-cost-gpt/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2021](https://md.research.elamur.ai/free/en/papers/2021/page/5/) / Want To Reduce Labeling Cost? GPT-3 Can Help

# Want To Reduce Labeling Cost? GPT-3 Can Help

**Authors:** [Shuohang Wang](https://md.research.elamur.ai/free/en/authors/shuohang-wang/), [Yang Liu](https://md.research.elamur.ai/free/en/authors/yang-liu/), [Yichong Xu](https://md.research.elamur.ai/free/en/authors/yichong-xu/), [Chenguang Zhu](https://md.research.elamur.ai/free/en/authors/chenguang-zhu/), [Michael Zeng](https://md.research.elamur.ai/free/en/authors/michael-zeng/) · **Year:** 2021 · **Venue:** Findings of the Association for Computational Linguistics: EMNLP 2021 · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2021.findings-emnlp.354/) · [PDF](https://aclanthology.org/2021.findings-emnlp.354.pdf) · [DOI 10.18653/v1/2021.findings-emnlp.354](https://doi.org/10.18653/v1/2021.findings-emnlp.354) · [OpenAlex](https://openalex.org/W3196820561)

**Citations:** 170 citations (openalex-snapshot · 2026-10-14) · [127](#cited-by) in our library, of the 140673 papers whose bibliographies we read

## Summary
Manual annotation for NLP is slow and expensive. Most pseudo-labeling methods are tied to one task and need a sizable labeled seed set. This paper asks whether GPT-3 can serve as a cheap annotator whose outputs train smaller deployable models, as an alternative to running the huge model itself. The authors prompt GPT-3 (Davinci) with a few human-labeled examples to label unannotated data. They then fine-tune RoBERTa large on the labeled data for understanding tasks and PEGASUS for generation tasks. Costs are compared using OpenAI API pricing against crowd-sourcing rates from Google Cloud. They also study mixing GPT-3 and human labels under a fixed budget, and active labeling, in which humans re-check the GPT-3 labels that the model was least confident about. Across nine tasks, GPT-3 labels matched human-label performance at 50–96% lower cost. The student models frequently outperformed few-shot GPT-3, and a theoretical error bound supports this result. Mixed labeling beat either source used alone. This matters for budget-limited projects, especially in low-resource settings.

## Key points
- **Benchmarks and models:**
  - Nine datasets were used: SST-2, CB, RTE, TREC, AGNews, DBPedia, Gigaword, XSum, and SQuAD (question generation).
  - Prompts used 2/4/8 shots for NLU tasks and 1/2/3 shots for NLG tasks.
- **Cost per label:** On SST-2, a GPT-3 label costs roughly $0.002, compared with about $0.11 for a crowd-sourced label.
- **Reported savings at equal performance:**
  - 96% on SST-2.
  - 93.8% on Gigaword.
  - About 50–75% on the other tasks.
- **Active labeling:**
  - GPT-3's confidence (logit) correlates well with label accuracy. Its top-10% most confident labels reached roughly 95%, 90%, and 95% accuracy on TREC, AGNews, and DBPedia.
  - Having humans re-label the least confident items raised TREC accuracy from 77% to 80% under a $2.2 budget.
- **Limitations:**
  - With a large or unlimited budget, labeling everything by hand still gives the best results.
  - Fewer shots are often more cost-efficient for NLU, but 1-shot labeling hurts NLG quality.
  - The authors caution against relying on GPT-3 for high-stakes labeling, such as detecting toxic language.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

Fields: [Natural language processing](https://md.research.elamur.ai/free/en/shelves/nlp/)

## Cited by

Cited by

127 papers of our library cite it (of the 140673 whose bibliographies we read); 14 of them have a page here, 113 not yet.

- [Large language models are few-shot clinical information extractors](https://md.research.elamur.ai/free/en/papers/agrawal-2022-large-language-models-are-few/)
- [Is GPT-3 a Good Data Annotator?](https://md.research.elamur.ai/free/en/papers/ding-2022-gpt-3-good-data-annotator/)
- [A Survey on In-context Learning](https://md.research.elamur.ai/free/en/papers/dong-2023-survey-context-learning/)
- [AnnoLLM: Making Large Language Models to Be Better Crowdsourced Annotators](https://md.research.elamur.ai/free/en/papers/he-2023-annollm-making-large-language-models/)
- [Distilling Step-by-Step! Outperforming Larger Language Models with Less Training Data and Smaller Model Sizes](https://md.research.elamur.ai/free/en/papers/hsieh-2023-distilling-step-step-outperforming-larger/)
- [Impossible Distillation: from Low-Quality Model to High-Quality Dataset & Model for Summarization and Paraphrasing](https://md.research.elamur.ai/free/en/papers/jung-2023-impossible-distillation-low-quality-model/)
- [Challenges and Applications of Large Language Models](https://md.research.elamur.ai/free/en/papers/kaddour-2023-challenges-applications-large-language-models/)
- [On LLMs-Driven Synthetic Data Generation, Curation, and Evaluation: A Survey](https://md.research.elamur.ai/free/en/papers/long-2024-llms-driven-synthetic-data-generation/)
- [A Holistic Approach to Undesired Content Detection in the Real World](https://md.research.elamur.ai/free/en/papers/markov-2022-holistic-approach-undesired-content-detection/)
- [SALOMO: An Annotation Tool for Complex Annotation Tasks with a Large Number of Labels](https://md.research.elamur.ai/free/en/papers/menzner-2026-salomo-annotation-tool-complex-annotation/)
- [What happens before and after: Multi-Event Commonsense in Event Coreference Resolution](https://md.research.elamur.ai/free/en/papers/ravi-2023-what-happens-before-after-multi/)
- [mGPT: Few-Shot Learners Go Multilingual](https://md.research.elamur.ai/free/en/papers/shliazhko-2022-mgpt-few-shot-learners-go/)
- [ToolAlpaca: Generalized Tool Learning for Language Models with 3000 Simulated Cases](https://md.research.elamur.ai/free/en/papers/tang-2023-toolalpaca-generalized-tool-learning-language/)
- [OpenStance: Real-world Zero-shot Stance Detection](https://md.research.elamur.ai/free/en/papers/xu-2022-openstance-real-world-zero-shot/)

## Cite

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{wang2021want,
  title = {{Want To Reduce Labeling Cost? GPT-3 Can Help}},
  author = {Wang, Shuohang and Liu, Yang and Xu, Yichong and Zhu, Chenguang and Zeng, Michael},
  booktitle = {Findings of the Association for Computational Linguistics: EMNLP 2021},
  year = {2021},
  doi = {10.18653/v1/2021.findings-emnlp.354},
  url = {https://doi.org/10.18653/v1/2021.findings-emnlp.354}
}
```

- [← previous WANLP 2021 Shared-Task: Towards Irony and Sentiment Detection in Arabic Tweets using Multi-headed-LSTM-CNN-GRU and MaRBERT](https://md.research.elamur.ai/free/en/papers/salam-2021-wanlp-2021-shared-task-irony/)
- [next → We Need to Consider Disagreement in Evaluation](https://md.research.elamur.ai/free/en/papers/basile-2021-we-need-consider-disagreement-evaluation/)
- [↑ back to papers of 2021](https://md.research.elamur.ai/free/en/papers/2021/page/5/)

## References

33 references, 10 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Josh Attenberg, Prem Melville, and Foster Provost. 2010. A unified approach to active dual supervision for labeling features and examples. In Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pages 40–55. Springer. [find](https://scholar.google.com/scholar?q=Josh+Attenberg%2C+Prem+Melville%2C+and+Foster+Provost%2E+2010%2E+A+unified+approach+to+active+dual+supervision+for+labeling+features+and+examples%2E+In+Joint+European+Conference+on+Machine+Learning+and+Knowledg)
2. Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel Ziegler, Jeffrey Wu, Clemens Winter, Chri… [find](https://scholar.google.com/scholar?q=Tom+Brown%2C+Benjamin+Mann%2C+Nick+Ryder%2C+Melanie+Subbiah%2C+Jared+D+Kaplan%2C+Prafulla+Dhariwal%2C+Arvind+Neelakantan%2C+Pranav+Shyam%2C+Girish+Sastry%2C+Amanda+Askell%2C+Sandhini+Agarwal%2C+Ariel+Herbert%2DVoss%2C+Gretchen)
3. Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Ilya Sutskever, and Dario Amodei. 2020. Language models are few-shot learners. In Advances in Neural Information Processing Systems, volume 33, pages 1877–1901. [find](https://scholar.google.com/scholar?q=Sigler%2C+Mateusz+Litwin%2C+Scott+Gray%2C+Benjamin+Chess%2C+Jack+Clark%2C+Christopher+Berner%2C+Sam+McCandlish%2C+Alec+Radford%2C+Ilya+Sutskever%2C+and+Dario+Amodei%2E+2020%2E+Language+models+are+few%2Dshot+learners%2E+In+Adva)
4. Ido Dagan, Oren Glickman, and Bernardo Magnini. 2005. The pascal recognising textual entailment challenge. In Machine Learning Challenges Workshop, pages 177–190. Springer. [find](https://scholar.google.com/scholar?q=Ido+Dagan%2C+Oren+Glickman%2C+and+Bernardo+Magnini%2E+2005%2E+The+pascal+recognising+textual+entailment+challenge%2E+In+Machine+Learning+Challenges+Workshop%2C+pages+177%E2%80%93190%2E+Springer%2E)
5. Marie-Catherine De Marneffe, Mandy Simons, and Judith Tonhauser. 2019. The commitmentbank: Investigating projection in naturally occurring discourse. In proceedings of Sinn und Bedeutung, volume 23, pages 107–124. [find](https://scholar.google.com/scholar?q=Marie%2DCatherine+De+Marneffe%2C+Mandy+Simons%2C+and+Judith+Tonhauser%2E+2019%2E+The+commitmentbank%3A+Investigating+projection+in+naturally+occurring+discourse%2E+In+proceedings+of+Sinn+und+Bedeutung%2C+volume+23%2C+p) [our card](https://md.research.elamur.ai/free/en/papers/enprag-2019-marneffe-commitmentbank-investigating-projection-naturally/)
6. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies,… [find](https://scholar.google.com/scholar?q=Jacob+Devlin%2C+Ming%2DWei+Chang%2C+Kenton+Lee%2C+and+Kristina+Toutanova%2E+2019%2E+BERT%3A+Pre%2Dtraining+of+deep+bidirectional+transformers+for+language+understanding%2E+In+Proceedings+of+the+2019+Conference+of+the+N)
7. Pinar Donmez and Jaime G Carbonell. 2008. Proactive learning: cost-sensitive active learning with multiple imperfect oracles. In Proceedings of the 17th ACM conference on Information and knowledge management, pages 619–628. [find](https://scholar.google.com/scholar?q=Pinar+Donmez+and+Jaime+G+Carbonell%2E+2008%2E+Proactive+learning%3A+cost%2Dsensitive+active+learning+with+multiple+imperfect+oracles%2E+In+Proceedings+of+the+17th+ACM+conference+on+Information+and+knowledge+man)
8. Liat Ein Dor, Alon Halfon, Ariel Gera, Eyal Shnarch, Lena Dankin, Leshem Choshen, Marina Danilevsky, Ranit Aharonov, Yoav Katz, and Noam Slonim. 2020. Active learning for bert: An empirical study. In Proceedings of the Conference on Empirical Methods in Natural Language Processing, pages 7949–7962. [find](https://scholar.google.com/scholar?q=Liat+Ein+Dor%2C+Alon+Halfon%2C+Ariel+Gera%2C+Eyal+Shnarch%2C+Lena+Dankin%2C+Leshem+Choshen%2C+Marina+Danilevsky%2C+Ranit+Aharonov%2C+Yoav+Katz%2C+and+Noam+Slonim%2E+2020%2E+Active+learning+for+bert%3A+An+empirical+study%2E+In+)
9. Junxian He, Jiatao Gu, Jiajun Shen, and Marc’Aurelio Ranzato. 2019. Revisiting self-training for neural sequence generation. arXiv preprint arXiv:1909.13788. — [arXiv:1909.13788](https://arxiv.org/abs/1909.13788) [find](https://scholar.google.com/scholar?q=Junxian+He%2C+Jiatao+Gu%2C+Jiajun+Shen%2C+and+Marc%E2%80%99Aurelio+Ranzato%2E+2019%2E+Revisiting+self%2Dtraining+for+neural+sequence+generation%2E+arXiv+preprint+arXiv%3A1909%2E13788%2E+%E2%80%94+%5BarXiv%3A1909%2E13788%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs)
10. Woohwan Jung and Kyuseok Shim. 2020. Dual supervision framework for relation extraction with distant supervision and human annotation. In Proceedings of the 28th International Conference on Computational Linguistics, pages 6411–6423. [find](https://scholar.google.com/scholar?q=Woohwan+Jung+and+Kyuseok+Shim%2E+2020%2E+Dual+supervision+framework+for+relation+extraction+with+distant+supervision+and+human+annotation%2E+In+Proceedings+of+the+28th+International+Conference+on+Computatio)
11. Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. Roberta: A robustly optimized bert pretraining approach. arXiv preprint arXiv:1907.11692. [find](https://scholar.google.com/scholar?q=Yinhan+Liu%2C+Myle+Ott%2C+Naman+Goyal%2C+Jingfei+Du%2C+Mandar+Joshi%2C+Danqi+Chen%2C+Omer+Levy%2C+Mike+Lewis%2C+Luke+Zettlemoyer%2C+and+Veselin+Stoyanov%2E+2019%2E+Roberta%3A+A+robustly+optimized+bert+pretraining+approach%2E+a)
12. Yao Lu, Max Bartolo, Alastair Moore, Sebastian Riedel, and Pontus Stenetorp. 2021. Fantastically ordered prompts and where to find them: Overcoming few-shot prompt order sensitivity. arXiv preprint arXiv:2104.08786. [find](https://scholar.google.com/scholar?q=Yao+Lu%2C+Max+Bartolo%2C+Alastair+Moore%2C+Sebastian+Riedel%2C+and+Pontus+Stenetorp%2E+2021%2E+Fantastically+ordered+prompts+and+where+to+find+them%3A+Overcoming+few%2Dshot+prompt+order+sensitivity%2E+arXiv+preprint+ar) [our card](https://md.research.elamur.ai/free/en/papers/lu-2022-fantastically-ordered-prompts-where-find/)
13. Prem Melville and Vikas Sindhwani. 2009. Active dual supervision: Reducing the cost of annotating examples and features. In Proceedings of the NAACL HLT 2009 workshop on active learning for natural language processing, pages 49–57. [find](https://scholar.google.com/scholar?q=Prem+Melville+and+Vikas+Sindhwani%2E+2009%2E+Active+dual+supervision%3A+Reducing+the+cost+of+annotating+examples+and+features%2E+In+Proceedings+of+the+NAACL+HLT+2009+workshop+on+active+learning+for+natural+la)
14. Subhabrata Mukherjee and Ahmed Awadallah. 2020. Uncertainty-aware self-training for few-shot text classification. Advances in Neural Information Processing Systems, 33. [find](https://scholar.google.com/scholar?q=Subhabrata+Mukherjee+and+Ahmed+Awadallah%2E+2020%2E+Uncertainty%2Daware+self%2Dtraining+for+few%2Dshot+text+classification%2E+Advances+in+Neural+Information+Processing+Systems%2C+33%2E)
15. Shashi Narayan, Shay B Cohen, and Mirella Lapata. 2018. Don’t give me the details, just the summary! topic-aware convolutional neural networks for extreme summarization. In Proceedings of the Conference on Empirical Methods in Natural Language Processing. [find](https://scholar.google.com/scholar?q=Shashi+Narayan%2C+Shay+B+Cohen%2C+and+Mirella+Lapata%2E+2018%2E+Don%E2%80%99t+give+me+the+details%2C+just+the+summary%21+topic%2Daware+convolutional+neural+networks+for+extreme+summarization%2E+In+Proceedings+of+the+Conferen)
16. Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. 2020. Exploring the limits of transfer learning with a unified text-to-text transformer. Journal of Machine Learning Research, 21(140):1–67. [find](https://scholar.google.com/scholar?q=Colin+Raffel%2C+Noam+Shazeer%2C+Adam+Roberts%2C+Katherine+Lee%2C+Sharan+Narang%2C+Michael+Matena%2C+Yanqi+Zhou%2C+Wei+Li%2C+and+Peter+J%2E+Liu%2E+2020%2E+Exploring+the+limits+of+transfer+learning+with+a+unified+text%2Dto%2Dtex)
17. Pranav Rajpurkar, Jian Zhang, Konstantin Lopyrev, and Percy Liang. 2016. Squad: 100,000+ questions for machine comprehension of text. In Proceedings of the Conference on Empirical Methods in Natural Language Processing. [find](https://scholar.google.com/scholar?q=Pranav+Rajpurkar%2C+Jian+Zhang%2C+Konstantin+Lopyrev%2C+and+Percy+Liang%2E+2016%2E+Squad%3A+100%2C000%2B+questions+for+machine+comprehension+of+text%2E+In+Proceedings+of+the+Conference+on+Empirical+Methods+in+Natural+L) [our card](https://md.research.elamur.ai/free/en/papers/rajpurkar-2016-squad-100-000-questions-machine/)
18. Sebastian Ruder, Matthew E Peters, Swabha Swayamdipta, and Thomas Wolf. 2019. Transfer learning in natural language processing. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Tutorials, pages 15–18. [find](https://scholar.google.com/scholar?q=Sebastian+Ruder%2C+Matthew+E+Peters%2C+Swabha+Swayamdipta%2C+and+Thomas+Wolf%2E+2019%2E+Transfer+learning+in+natural+language+processing%2E+In+Proceedings+of+the+2019+Conference+of+the+North+American+Chapter+of+t) [our card](https://md.research.elamur.ai/free/en/papers/ruder-2019-transfer-learning-natural-language-processing/)
19. Alexander M Rush, Sumit Chopra, and Jason Weston. 2015. A neural attention model for abstractive sentence summarization. In Proceedings of the Conference on Empirical Methods in Natural Language Processing. [find](https://scholar.google.com/scholar?q=Alexander+M+Rush%2C+Sumit+Chopra%2C+and+Jason+Weston%2E+2015%2E+A+neural+attention+model+for+abstractive+sentence+summarization%2E+In+Proceedings+of+the+Conference+on+Empirical+Methods+in+Natural+Language+Proce)
20. Burr Settles. 2009. Active learning literature survey. [find](https://scholar.google.com/scholar?q=Burr+Settles%2E+2009%2E+Active+learning+literature+survey%2E) [our card](https://md.research.elamur.ai/free/en/papers/settles-2009-active-learning-literature-survey/)
21. Richard Socher, Alex Perelygin, Jean Wu, Jason Chuang, Christopher D Manning, Andrew Y Ng, and Christopher Potts. 2013. Recursive deep models for semantic compositionality over a sentiment treebank. In Proceedings of the 2013 conference on empirical methods in natural language processing, pages 1631… [find](https://scholar.google.com/scholar?q=Richard+Socher%2C+Alex+Perelygin%2C+Jean+Wu%2C+Jason+Chuang%2C+Christopher+D+Manning%2C+Andrew+Y+Ng%2C+and+Christopher+Potts%2E+2013%2E+Recursive+deep+models+for+semantic+compositionality+over+a+sentiment+treebank%2E+I)
22. Jialin Song, Yuxin Chen, and Yisong Yue. 2019. A general framework for multi-fidelity bayesian optimization with gaussian processes. In The 22nd International Conference on Artificial Intelligence and Statistics, pages 3158–3167. PMLR. [find](https://scholar.google.com/scholar?q=Jialin+Song%2C+Yuxin+Chen%2C+and+Yisong+Yue%2E+2019%2E+A+general+framework+for+multi%2Dfidelity+bayesian+optimization+with+gaussian+processes%2E+In+The+22nd+International+Conference+on+Artificial+Intelligence+and)
23. Shang-Yu Su, Chao-Wei Huang, and Yun-Nung Chen. 2019. Dual supervised learning for natural language understanding and generation. arXiv preprint arXiv:1905.06196. [find](https://scholar.google.com/scholar?q=Shang%2DYu+Su%2C+Chao%2DWei+Huang%2C+and+Yun%2DNung+Chen%2E+2019%2E+Dual+supervised+learning+for+natural+language+understanding+and+generation%2E+arXiv+preprint+arXiv%3A1905%2E06196%2E)
24. Ellen M Voorhees and Dawn M Tice. 2000. Building a question answering test collection. In Proceedings of the 23rd annual international ACM SIGIR conference on Research and development in information retrieval, pages 200–207. [find](https://scholar.google.com/scholar?q=Ellen+M+Voorhees+and+Dawn+M+Tice%2E+2000%2E+Building+a+question+answering+test+collection%2E+In+Proceedings+of+the+23rd+annual+international+ACM+SIGIR+conference+on+Research+and+development+in+information+r)
25. Yaqing Wang, Subhabrata Mukherjee, Haoda Chu, Yuancheng Tu, Ming Wu, Jing Gao, and Ahmed Hassan Awadallah. 2020. Adaptive self-training for few-shot neural sequence labeling. arXiv preprint arXiv:2010.03680. — [arXiv:2010.03680](https://arxiv.org/abs/2010.03680) [find](https://scholar.google.com/scholar?q=Yaqing+Wang%2C+Subhabrata+Mukherjee%2C+Haoda+Chu%2C+Yuancheng+Tu%2C+Ming+Wu%2C+Jing+Gao%2C+and+Ahmed+Hassan+Awadallah%2E+2020%2E+Adaptive+self%2Dtraining+for+few%2Dshot+neural+sequence+labeling%2E+arXiv+preprint+arXiv%3A2010)
26. Colin Wei, Kendrick Shen, Yining Chen, and Tengyu Ma. 2021. Theoretical analysis of self-training with deep networks on unlabeled data. In International Conference on Learning Representations. [find](https://scholar.google.com/scholar?q=Colin+Wei%2C+Kendrick+Shen%2C+Yining+Chen%2C+and+Tengyu+Ma%2E+2021%2E+Theoretical+analysis+of+self%2Dtraining+with+deep+networks+on+unlabeled+data%2E+In+International+Conference+on+Learning+Representations%2E)
27. Yichong Xu, Sivaraman Balakrishnan, Arthur Dubrawski, and Aarti Singh. 2020. Regression with comparisons: Escaping the curse of dimensionality with ordinal information. Journal of machine learning research. [find](https://scholar.google.com/scholar?q=Yichong+Xu%2C+Sivaraman+Balakrishnan%2C+Arthur+Dubrawski%2C+and+Aarti+Singh%2E+2020%2E+Regression+with+comparisons%3A+Escaping+the+curse+of+dimensionality+with+ordinal+information%2E+Journal+of+machine+learning+res)
28. David Yarowsky. 1995. Unsupervised word sense disambiguation rivaling supervised methods. In 33rd annual meeting of the association for computational linguistics, pages 189–196. [find](https://scholar.google.com/scholar?q=David+Yarowsky%2E+1995%2E+Unsupervised+word+sense+disambiguation+rivaling+supervised+methods%2E+In+33rd+annual+meeting+of+the+association+for+computational+linguistics%2C+pages+189%E2%80%93196%2E) [our card](https://md.research.elamur.ai/free/en/papers/yarowsky-1995-unsupervised-word-sense-disambiguation-rivaling/)
29. Wenpeng Yin. 2020. Meta-learning for few-shot natural language processing: A survey. arXiv preprint arXiv:2007.09604. — [arXiv:2007.09604](https://arxiv.org/abs/2007.09604) [find](https://scholar.google.com/scholar?q=Wenpeng+Yin%2E+2020%2E+Meta%2Dlearning+for+few%2Dshot+natural+language+processing%3A+A+survey%2E+arXiv+preprint+arXiv%3A2007%2E09604%2E+%E2%80%94+%5BarXiv%3A2007%2E09604%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2007%2E09604%29)
30. Kang Min Yoo, Dongju Park, Jaewook Kang, Sang-Woo Lee, and Woomyeong Park. 2021. Gpt3mix: Leveraging large-scale language models for text augmentation. arXiv preprint arXiv:2104.08826. [find](https://scholar.google.com/scholar?q=Kang+Min+Yoo%2C+Dongju+Park%2C+Jaewook+Kang%2C+Sang%2DWoo+Lee%2C+and+Woomyeong+Park%2E+2021%2E+Gpt3mix%3A+Leveraging+large%2Dscale+language+models+for+text+augmentation%2E+arXiv+preprint+arXiv%3A2104%2E08826%2E) [our card](https://md.research.elamur.ai/free/en/papers/yoo-2021-gpt3mix-leveraging-large-scale-language/)
31. Jingqing Zhang, Yao Zhao, Mohammad Saleh, and Peter Liu. 2020. Pegasus: Pre-training with extracted gap-sentences for abstractive summarization. In International Conference on Machine Learning, pages 11328–11339. PMLR. [find](https://scholar.google.com/scholar?q=Jingqing+Zhang%2C+Yao+Zhao%2C+Mohammad+Saleh%2C+and+Peter+Liu%2E+2020%2E+Pegasus%3A+Pre%2Dtraining+with+extracted+gap%2Dsentences+for+abstractive+summarization%2E+In+International+Conference+on+Machine+Learning%2C+pages+) [our card](https://md.research.elamur.ai/free/en/papers/zhang-2019-pegasus-pre-training-extracted-gap/)
32. Xiang Zhang, Junbo Zhao, and Yann LeCun. 2015. Character-level convolutional networks for text classification. In Proceedings of Advances in Neural Information Processing Systems. [find](https://scholar.google.com/scholar?q=Xiang+Zhang%2C+Junbo+Zhao%2C+and+Yann+LeCun%2E+2015%2E+Character%2Dlevel+convolutional+networks+for+text+classification%2E+In+Proceedings+of+Advances+in+Neural+Information+Processing+Systems%2E) [our card](https://md.research.elamur.ai/free/en/papers/zhang-2015-character-level-convolutional-networks-text/)
33. Tony Z Zhao, Eric Wallace, Shi Feng, Dan Klein, and Sameer Singh. 2021. Calibrate before use: Improving few-shot performance of language models. arXiv preprint arXiv:2102.09690. [find](https://scholar.google.com/scholar?q=Tony+Z+Zhao%2C+Eric+Wallace%2C+Shi+Feng%2C+Dan+Klein%2C+and+Sameer+Singh%2E+2021%2E+Calibrate+before+use%3A+Improving+few%2Dshot+performance+of+language+models%2E+arXiv+preprint+arXiv%3A2102%2E09690%2E) [our card](https://md.research.elamur.ai/free/en/papers/zhao-2021-calibrate-before-use-improving-few/)