---
title: "Knowledgeable Prompt-tuning — Hu 2022"
description: "Prompt-tuning turns text classification into a masked language modeling task."
authors: ["Shengding Hu", "Ning Ding", "Huadong Wang", "Zhiyuan Liu", "Jingang Wang", "Juanzi Li", "Wei Wu", "Maosong Sun"]
year: 2022
venue: "Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)"
source: "https://aclanthology.org/2022.acl-long.158/"
doi: "10.18653/v1/2022.acl-long.158"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/hu-2022-knowledgeable-prompt-tuning-incorporating-knowledge/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/hu-2022-knowledgeable-prompt-tuning-incorporating-knowledge/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2022](https://md.research.elamur.ai/free/en/papers/2022/) / Knowledgeable Prompt-tuning: Incorporating Knowledge into Prompt Verbalizer for Text Classification

# Knowledgeable Prompt-tuning: Incorporating Knowledge into Prompt Verbalizer for Text Classification

**Authors:** [Shengding Hu](https://md.research.elamur.ai/free/en/authors/shengding-hu/), [Ning Ding](https://md.research.elamur.ai/free/en/authors/ning-ding/), Huadong Wang, [Zhiyuan Liu](https://md.research.elamur.ai/free/en/authors/zhiyuan-liu/), Jingang Wang, [Juanzi Li](https://md.research.elamur.ai/free/en/authors/juanzi-li/), [Wei Wu](https://md.research.elamur.ai/free/en/authors/wei-wu/), [Maosong Sun](https://md.research.elamur.ai/free/en/authors/maosong-sun/) · **Year:** 2022 · **Venue:** Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2022.acl-long.158/) · [PDF](https://aclanthology.org/2022.acl-long.158.pdf) · [DOI 10.18653/v1/2022.acl-long.158](https://doi.org/10.18653/v1/2022.acl-long.158) · [OpenAlex](https://openalex.org/W3188542058)

**Citations:** 265 citations (OpenAlex, 2026-10-14) · [95](https://md.research.elamur.ai/free/en/papers/hu-2022-knowledgeable-prompt-tuning-incorporating-knowledge/cited-by/) in our library, of the 140884 papers whose bibliographies we read

## Summary
Prompt-tuning turns text classification into a masked language modeling task. A template wraps the input, and a verbalizer maps predicted words at the [MASK] position to class labels. Verbalizers are usually handcrafted with a single class-name word or learned from data. Hand-built ones cover too few words and carry their designer's bias. Learned ones need labeled data and cannot be used in zero-shot settings.

Hu et al. propose knowledgeable prompt-tuning (KPT), which works in three stages:
- **Construction:** external knowledge bases supply many related label words per class at different granularities, such as "physics" for SCIENCE.
- **Refinement:** the PLM itself prunes the noisy expanded vocabulary using four techniques.
- **Utilization:** scores over each word set are combined into a label score by plain or weighted averaging.

On zero- and few-shot topic and sentiment classification, KPT consistently beats manual, soft and automatic verbalizers. It also yields more stable results. This makes it useful wherever labeled data is scarce.

## Key points
- Average error-rate reduction of about 16%, 18%, 10% and 7% in 0-, 1-, 5- and 10-shot settings. Against the strongest baseline specifically, the reductions are 17.8%, 10.3% and 7.4% for 1, 5 and 10 shots.
- The four refinement methods filter label words by frequency and by relevance, calibrate predictions using context, and add a refinement step that is learned from data.
- Baselines are manual (PT), soft (SOFT) and automatic (AUTO) verbalizers. AUTO is weakest when few shots are available.
- Refinement mainly helps topic classification. Sentiment tasks work well without it because sentiment lexicons are already fairly clean.
- Code is released at github.com/thunlp/KnowledgeablePromptTuning. The authors warn that knowledge bases may contain errors or even malicious words.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

Fields: [Natural language processing](https://md.research.elamur.ai/free/en/shelves/nlp/)

## Cite

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{hu2022knowledgeable,
  title = {{Knowledgeable Prompt-tuning: Incorporating Knowledge into Prompt Verbalizer for Text Classification}},
  author = {Hu, Shengding and Ding, Ning and Wang, Huadong and Liu, Zhiyuan and Wang, Jingang and Li, Juanzi and Wu, Wei and Sun, Maosong},
  booktitle = {Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2022},
  doi = {10.18653/v1/2022.acl-long.158},
  url = {https://doi.org/10.18653/v1/2022.acl-long.158}
}
```

- [← previous The “Problem” of Human Label Variation: On Ground Truth in Data, Modeling and Evaluation](https://md.research.elamur.ai/free/en/papers/plank-2022-problem-human-label-variation-ground/)
- [next → BRIO: Bringing Order to Abstractive Summarization](https://md.research.elamur.ai/free/en/papers/liu-2022-brio-bringing-order-abstractive-summarization/)
- [↑ back to papers of 2022](https://md.research.elamur.ai/free/en/papers/2022/)

## References

35 references, 15 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter,… [find](https://scholar.google.com/scholar?q=Tom+B%2E+Brown%2C+Benjamin+Mann%2C+Nick+Ryder%2C+Melanie+Subbiah%2C+Jared+Kaplan%2C+Prafulla+Dhariwal%2C+Arvind+Neelakantan%2C+Pranav+Shyam%2C+Girish+Sastry%2C+Amanda+Askell%2C+Sandhini+Agarwal%2C+Ariel+Herbert%2DVoss%2C+Gretche)
2. Xiang Chen, Xin Xie, Ningyu Zhang, Jiahuan Yan, Shumin Deng, Chuanqi Tan, Fei Huang, Luo Si, and Huajun Chen. 2021. Adaprompt: Adaptive prompt-based finetuning for relation extraction. ArXiv preprint, abs/2104.07650. — [arXiv:2104.07650](https://arxiv.org/abs/2104.07650) [find](https://scholar.google.com/scholar?q=Xiang+Chen%2C+Xin+Xie%2C+Ningyu+Zhang%2C+Jiahuan+Yan%2C+Shumin+Deng%2C+Chuanqi+Tan%2C+Fei+Huang%2C+Luo+Si%2C+and+Huajun+Chen%2E+2021%2E+Adaprompt%3A+Adaptive+prompt%2Dbased+finetuning+for+relation+extraction%2E+ArXiv+preprint%2C)
3. Joe Davison, Joshua Feldman, and Alexander Rush. 2019. Commonsense knowledge mining from pre-trained models. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 1173–… [find](https://scholar.google.com/scholar?q=Joe+Davison%2C+Joshua+Feldman%2C+and+Alexander+Rush%2E+2019%2E+Commonsense+knowledge+mining+from+pre%2Dtrained+models%2E+In+Proceedings+of+the+2019+Conference+on+Empirical+Methods+in+Natural+Language+Processing+a)
4. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies,… [find](https://scholar.google.com/scholar?q=Jacob+Devlin%2C+Ming%2DWei+Chang%2C+Kenton+Lee%2C+and+Kristina+Toutanova%2E+2019%2E+BERT%3A+Pre%2Dtraining+of+deep+bidirectional+transformers+for+language+understanding%2E+In+Proceedings+of+the+2019+Conference+of+the+N)
5. Ning Ding, Shengding Hu, Weilin Zhao, Yulin Chen, Zhiyuan Liu, Hai-Tao Zheng, and Maosong Sun. 2021. Openprompt: An open-source framework for prompt-learning. ArXiv preprint, abs/2111.01998. [find](https://scholar.google.com/scholar?q=Ning+Ding%2C+Shengding+Hu%2C+Weilin+Zhao%2C+Yulin+Chen%2C+Zhiyuan+Liu%2C+Hai%2DTao+Zheng%2C+and+Maosong+Sun%2E+2021%2E+Openprompt%3A+An+open%2Dsource+framework+for+prompt%2Dlearning%2E+ArXiv+preprint%2C+abs%2F2111%2E01998%2E) [our card](https://md.research.elamur.ai/free/en/papers/ding-2022-openprompt-open-source-framework-prompt/)
6. Tianyu Gao, Adam Fisch, and Danqi Chen. 2021. Making pre-trained language models better few-shot learners. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pa… [find](https://scholar.google.com/scholar?q=Tianyu+Gao%2C+Adam+Fisch%2C+and+Danqi+Chen%2E+2021%2E+Making+pre%2Dtrained+language+models+better+few%2Dshot+learners%2E+In+Proceedings+of+the+59th+Annual+Meeting+of+the+Association+for+Computational+Linguistics+an) [our card](https://md.research.elamur.ai/free/en/papers/gao-2021-making-pre-trained-language-models/)
7. Karen Hambardzumyan, Hrant Khachatrian, and Jonathan May. 2021. WARP: Word-level Adversarial ReProgramming. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), p… [find](https://scholar.google.com/scholar?q=Karen+Hambardzumyan%2C+Hrant+Khachatrian%2C+and+Jonathan+May%2E+2021%2E+WARP%3A+Word%2Dlevel+Adversarial+ReProgramming%2E+In+Proceedings+of+the+59th+Annual+Meeting+of+the+Association+for+Computational+Linguistics+a)
8. Xu Han, Weilin Zhao, Ning Ding, Zhiyuan Liu, and Maosong Sun. 2021. Ptr: Prompt tuning with rules for text classification. ArXiv preprint, abs/2105.11259. — [arXiv:2105.11259](https://arxiv.org/abs/2105.11259) [find](https://scholar.google.com/scholar?q=Xu+Han%2C+Weilin+Zhao%2C+Ning+Ding%2C+Zhiyuan+Liu%2C+and+Maosong+Sun%2E+2021%2E+Ptr%3A+Prompt+tuning+with+rules+for+text+classification%2E+ArXiv+preprint%2C+abs%2F2105%2E11259%2E+%E2%80%94+%5BarXiv%3A2105%2E11259%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F21)
9. Ari Holtzman, Peter West, Vered Shwartz, Yejin Choi, and Luke Zettlemoyer. 2021. Surface form competition: Why the highest probability answer isn’t always right. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 7038–7051. [find](https://scholar.google.com/scholar?q=Ari+Holtzman%2C+Peter+West%2C+Vered+Shwartz%2C+Yejin+Choi%2C+and+Luke+Zettlemoyer%2E+2021%2E+Surface+form+competition%3A+Why+the+highest+probability+answer+isn%E2%80%99t+always+right%2E+In+Proceedings+of+the+2021+Conference+) [our card](https://md.research.elamur.ai/free/en/papers/holtzman-2021-surface-form-competition-why-highest/)
10. Karen Sparck Jones. 1972. A statistical interpretation of term specificity and its application in retrieval. Journal of documentation. [find](https://scholar.google.com/scholar?q=Karen+Sparck+Jones%2E+1972%2E+A+statistical+interpretation+of+term+specificity+and+its+application+in+retrieval%2E+Journal+of+documentation%2E)
11. Lingkai Kong, Haoming Jiang, Yuchen Zhuang, Jie Lyu, Tuo Zhao, and Chao Zhang. 2020. Calibrated language model fine-tuning for in- and out-of-distribution data. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 1326–1340, Online. Association for… [find](https://scholar.google.com/scholar?q=Lingkai+Kong%2C+Haoming+Jiang%2C+Yuchen+Zhuang%2C+Jie+Lyu%2C+Tuo+Zhao%2C+and+Chao+Zhang%2E+2020%2E+Calibrated+language+model+fine%2Dtuning+for+in%2D+and+out%2Dof%2Ddistribution+data%2E+In+Proceedings+of+the+2020+Conference+o)
12. Kamran Kowsari, Kiana Jafari Meimandi, Mojtaba Heidarysafa, Sanjana Mendu, Laura Barnes, and Donald Brown. 2019. Text classification algorithms: A survey. Information, 10(4):150. [find](https://scholar.google.com/scholar?q=Kamran+Kowsari%2C+Kiana+Jafari+Meimandi%2C+Mojtaba+Heidarysafa%2C+Sanjana+Mendu%2C+Laura+Barnes%2C+and+Donald+Brown%2E+2019%2E+Text+classification+algorithms%3A+A+survey%2E+Information%2C+10%284%29%3A150%2E)
13. Zhenzhong Lan, Mingda Chen, Sebastian Goodman, Kevin Gimpel, Piyush Sharma, and Radu Soricut. 2020. ALBERT: A lite BERT for self-supervised learning of language representations. In 8th International Conference on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia, April 26-30, 2020. OpenRevi… [find](https://scholar.google.com/scholar?q=Zhenzhong+Lan%2C+Mingda+Chen%2C+Sebastian+Goodman%2C+Kevin+Gimpel%2C+Piyush+Sharma%2C+and+Radu+Soricut%2E+2020%2E+ALBERT%3A+A+lite+BERT+for+self%2Dsupervised+learning+of+language+representations%2E+In+8th+International+C)
14. Jens Lehmann, Robert Isele, Max Jakob, Anja Jentzsch, Dimitris Kontokostas, Pablo N Mendes, Sebastian Hellmann, Mohamed Morsey, Patrick Van Kleef, Sören Auer, et al. 2015. Dbpedia–a large-scale, multilingual knowledge base extracted from wikipedia. Semantic web, 6(2):167–195. [find](https://scholar.google.com/scholar?q=Jens+Lehmann%2C+Robert+Isele%2C+Max+Jakob%2C+Anja+Jentzsch%2C+Dimitris+Kontokostas%2C+Pablo+N+Mendes%2C+Sebastian+Hellmann%2C+Mohamed+Morsey%2C+Patrick+Van+Kleef%2C+S%C3%B6ren+Auer%2C+et+al%2E+2015%2E+Dbpedia%E2%80%93a+large%2Dscale%2C+multi)
15. Xiao Liu, Yanan Zheng, Zhengxiao Du, Ming Ding, Yujie Qian, Zhilin Yang, and Jie Tang. 2021. Gpt understands, too. ArXiv preprint, abs/2103.10385. [find](https://scholar.google.com/scholar?q=Xiao+Liu%2C+Yanan+Zheng%2C+Zhengxiao+Du%2C+Ming+Ding%2C+Yujie+Qian%2C+Zhilin+Yang%2C+and+Jie+Tang%2E+2021%2E+Gpt+understands%2C+too%2E+ArXiv+preprint%2C+abs%2F2103%2E10385%2E) [our card](https://md.research.elamur.ai/free/en/papers/liu-2021-gpt-understands-too/)
16. Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. Roberta: A robustly optimized bert pretraining approach. ArXiv preprint, abs/1907.11692. [find](https://scholar.google.com/scholar?q=Yinhan+Liu%2C+Myle+Ott%2C+Naman+Goyal%2C+Jingfei+Du%2C+Mandar+Joshi%2C+Danqi+Chen%2C+Omer+Levy%2C+Mike+Lewis%2C+Luke+Zettlemoyer%2C+and+Veselin+Stoyanov%2E+2019%2E+Roberta%3A+A+robustly+optimized+bert+pretraining+approach%2E+A)
17. Andrew L. Maas, Raymond E. Daly, Peter T. Pham, Dan Huang, Andrew Y. Ng, and Christopher Potts. 2011. Learning word vectors for sentiment analysis. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies, pages 142–150, Portland, Oregon… [find](https://scholar.google.com/scholar?q=Andrew+L%2E+Maas%2C+Raymond+E%2E+Daly%2C+Peter+T%2E+Pham%2C+Dan+Huang%2C+Andrew+Y%2E+Ng%2C+and+Christopher+Potts%2E+2011%2E+Learning+word+vectors+for+sentiment+analysis%2E+In+Proceedings+of+the+49th+Annual+Meeting+of+the+Ass) [our card](https://md.research.elamur.ai/free/en/papers/maas-2011-learning-word-vectors-sentiment-analysis/)
18. Julian J. McAuley and Jure Leskovec. 2013. Hidden factors and hidden topics: understanding rating dimensions with review text. In Seventh ACM Conference on Recommender Systems, RecSys ’13, Hong Kong, China, October 12-16, 2013, pages 165–172. ACM. [find](https://scholar.google.com/scholar?q=Julian+J%2E+McAuley+and+Jure+Leskovec%2E+2013%2E+Hidden+factors+and+hidden+topics%3A+understanding+rating+dimensions+with+review+text%2E+In+Seventh+ACM+Conference+on+Recommender+Systems%2C+RecSys+%E2%80%9913%2C+Hong+Kong%2C+)
19. Yu Meng, Yunyi Zhang, Jiaxin Huang, Chenyan Xiong, Heng Ji, Chao Zhang, and Jiawei Han. 2020. Text classification using label names only: A language model self-training approach. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 9006–9017, Onlin… [find](https://scholar.google.com/scholar?q=Yu+Meng%2C+Yunyi+Zhang%2C+Jiaxin+Huang%2C+Chenyan+Xiong%2C+Heng+Ji%2C+Chao+Zhang%2C+and+Jiawei+Han%2E+2020%2E+Text+classification+using+label+names+only%3A+A+language+model+self%2Dtraining+approach%2E+In+Proceedings+of+the) [our card](https://md.research.elamur.ai/free/en/papers/meng-2020-text-classification-label-names-only/)
20. Tri Nguyen, Mir Rosenberg, Xia Song, Jianfeng Gao, Saurabh Tiwary, Rangan Majumder, and Li Deng. 2016. Ms marco: A human generated machine reading comprehension dataset. In Proceedings of CoCo@ NeurIPS. the 2016 Conference on Empirical Methods in Natural Language Processing, pages 2383–2392, Austin,… [find](https://scholar.google.com/scholar?q=Tri+Nguyen%2C+Mir+Rosenberg%2C+Xia+Song%2C+Jianfeng+Gao%2C+Saurabh+Tiwary%2C+Rangan+Majumder%2C+and+Li+Deng%2E+2016%2E+Ms+marco%3A+A+human+generated+machine+reading+comprehension+dataset%2E+In+Proceedings+of+CoCo%40+NeurIP) [our card](https://md.research.elamur.ai/free/en/papers/bajaj-2016-ms-marco-human-generated/)
21. Timo Schick, Helmut Schmid, and Hinrich Schütze. 2020. Automatically identifying words that can serve as labels for few-shot text classification. In Proceedings of COLING, pages 5569–5578. [find](https://scholar.google.com/scholar?q=Timo+Schick%2C+Helmut+Schmid%2C+and+Hinrich+Sch%C3%BCtze%2E+2020%2E+Automatically+identifying+words+that+can+serve+as+labels+for+few%2Dshot+text+classification%2E+In+Proceedings+of+COLING%2C+pages+5569%E2%80%935578%2E)
22. Timo Schick and Hinrich Schütze. 2021a. Exploiting cloze-questions for few-shot text classification and natural language inference. In Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume, pages 255–269, Online. Association for Comp… [find](https://scholar.google.com/scholar?q=Timo+Schick+and+Hinrich+Sch%C3%BCtze%2E+2021a%2E+Exploiting+cloze%2Dquestions+for+few%2Dshot+text+classification+and+natural+language+inference%2E+In+Proceedings+of+the+16th+Conference+of+the+European+Chapter+of+the) [our card](https://md.research.elamur.ai/free/en/papers/schick-2021-exploiting-cloze-questions-few-shot/)
23. Ted Pedersen, Siddharth Patwardhan, and Jason Michelizzi. 2004. WordNet::Similarity - measuring the relatedness of concepts. In Demonstration Papers at HLT-NAACL 2004, pages 38–41, Boston, Massachusetts, USA. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Ted+Pedersen%2C+Siddharth+Patwardhan%2C+and+Jason+Michelizzi%2E+2004%2E+WordNet%3A%3ASimilarity+%2D+measuring+the+relatedness+of+concepts%2E+In+Demonstration+Papers+at+HLT%2DNAACL+2004%2C+pages+38%E2%80%9341%2C+Boston%2C+Massachuset)
24. Timo Schick and Hinrich Schütze. 2021b. It’s not just size that matters: Small language models are also few-shot learners. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 2339–2352, Online. Assoc… [find](https://scholar.google.com/scholar?q=Timo+Schick+and+Hinrich+Sch%C3%BCtze%2E+2021b%2E+It%E2%80%99s+not+just+size+that+matters%3A+Small+language+models+are+also+few%2Dshot+learners%2E+In+Proceedings+of+the+2021+Conference+of+the+North+American+Chapter+of+the+As) [our card](https://md.research.elamur.ai/free/en/papers/schick-2021-it-s-not-just-size/)
25. Matthew E. Peters, Mark Neumann, Mohit Iyyer, Matt Gardner, Christopher Clark, Kenton Lee, and Luke Zettlemoyer. 2018. Deep contextualized word representations. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Techno… [find](https://scholar.google.com/scholar?q=Matthew+E%2E+Peters%2C+Mark+Neumann%2C+Mohit+Iyyer%2C+Matt+Gardner%2C+Christopher+Clark%2C+Kenton+Lee%2C+and+Luke+Zettlemoyer%2E+2018%2E+Deep+contextualized+word+representations%2E+In+Proceedings+of+the+2018+Conference+o)
26. Taylor Shin, Yasaman Razeghi, Robert L. Logan IV, Eric Wallace, and Sameer Singh. 2020. AutoPrompt: Eliciting Knowledge from Language Models with Automatically Generated Prompts. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 4222–4235, Onlin… [find](https://scholar.google.com/scholar?q=Taylor+Shin%2C+Yasaman+Razeghi%2C+Robert+L%2E+Logan+IV%2C+Eric+Wallace%2C+and+Sameer+Singh%2E+2020%2E+AutoPrompt%3A+Eliciting+Knowledge+from+Language+Models+with+Automatically+Generated+Prompts%2E+In+Proceedings+of+the) [our card](https://md.research.elamur.ai/free/en/papers/shin-2020-autoprompt-eliciting-knowledge-language-models/)
27. Fabio Petroni, Patrick Lewis, Aleksandra Piktus, Tim Rocktäschel, Yuxiang Wu, Alexander H. Miller, and Sebastian Riedel. 2020. How context affects language models’ factual predictions. In Automated Knowledge Base Construction. [find](https://scholar.google.com/scholar?q=Fabio+Petroni%2C+Patrick+Lewis%2C+Aleksandra+Piktus%2C+Tim+Rockt%C3%A4schel%2C+Yuxiang+Wu%2C+Alexander+H%2E+Miller%2C+and+Sebastian+Riedel%2E+2020%2E+How+context+affects+language+models%E2%80%99+factual+predictions%2E+In+Automated+Kn) [our card](https://md.research.elamur.ai/free/en/papers/petroni-2020-context-affects-language-models-factual/)
28. Fabio Petroni, Tim Rocktäschel, Sebastian Riedel, Patrick Lewis, Anton Bakhtin, Yuxiang Wu, and Alexander Miller. 2019. Language models as knowledge bases? In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural… [find](https://scholar.google.com/scholar?q=Fabio+Petroni%2C+Tim+Rockt%C3%A4schel%2C+Sebastian+Riedel%2C+Patrick+Lewis%2C+Anton+Bakhtin%2C+Yuxiang+Wu%2C+and+Alexander+Miller%2E+2019%2E+Language+models+as+knowledge+bases%3F+In+Proceedings+of+the+2019+Conference+on+Emp) [our card](https://md.research.elamur.ai/free/en/papers/petroni-2019-language-models-as-knowledge-bases/)
29. Robyn Speer, Joshua Chin, and Catherine Havasi. 2017. Conceptnet 5.5: An open multilingual graph of general knowledge. In Proceedings of the Thirty-First AAAI Conference on Artificial Intelligence, February 4-9, 2017, San Francisco, California, USA, pages 4444–4451. AAAI Press. [find](https://scholar.google.com/scholar?q=Robyn+Speer%2C+Joshua+Chin%2C+and+Catherine+Havasi%2E+2017%2E+Conceptnet+5%2E5%3A+An+open+multilingual+graph+of+general+knowledge%2E+In+Proceedings+of+the+Thirty%2DFirst+AAAI+Conference+on+Artificial+Intelligence%2C+Fe)
30. Han Xu, Zhang Zhengyan, Ding Ning, Gu Yuxian, Liu Xiao, Huo Yuqi, Qiu Jiezhong, Zhang Liang, Han Wentao, Huang Minlie, et al. 2021. Pre-trained models: Past, present and future. ArXiv preprint, abs/2106.07139. — [arXiv:2106.07139](https://arxiv.org/abs/2106.07139) [find](https://scholar.google.com/scholar?q=Han+Xu%2C+Zhang+Zhengyan%2C+Ding+Ning%2C+Gu+Yuxian%2C+Liu+Xiao%2C+Huo+Yuqi%2C+Qiu+Jiezhong%2C+Zhang+Liang%2C+Han+Wentao%2C+Huang+Minlie%2C+et+al%2E+2021%2E+Pre%2Dtrained+models%3A+Past%2C+present+and+future%2E+ArXiv+preprint%2C+abs%2F21)
31. Alec Radford, Karthik Narasimhan, Tim Salimans, and Ilya Sutskever. 2018. Improving language understanding by generative pre-training. [find](https://scholar.google.com/scholar?q=Alec+Radford%2C+Karthik+Narasimhan%2C+Tim+Salimans%2C+and+Ilya+Sutskever%2E+2018%2E+Improving+language+understanding+by+generative+pre%2Dtraining%2E)
32. Wenpeng Yin, Jamaal Hay, and Dan Roth. 2019. Benchmarking zero-shot text classification: Datasets, evaluation and entailment approach. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (… [find](https://scholar.google.com/scholar?q=Wenpeng+Yin%2C+Jamaal+Hay%2C+and+Dan+Roth%2E+2019%2E+Benchmarking+zero%2Dshot+text+classification%3A+Datasets%2C+evaluation+and+entailment+approach%2E+In+Proceedings+of+the+2019+Conference+on+Empirical+Methods+in+Nat) [our card](https://md.research.elamur.ai/free/en/papers/yin-2019-benchmarking-zero-shot-text-classification/)
33. Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J Liu. 2020. Exploring the limits of transfer learning with a unified text-to-text transformer. Journal of Machine Learning Research, 21:1– 67. [find](https://scholar.google.com/scholar?q=Colin+Raffel%2C+Noam+Shazeer%2C+Adam+Roberts%2C+Katherine+Lee%2C+Sharan+Narang%2C+Michael+Matena%2C+Yanqi+Zhou%2C+Wei+Li%2C+and+Peter+J+Liu%2E+2020%2E+Exploring+the+limits+of+transfer+learning+with+a+unified+text%2Dto%2Dtext)
34. Xiang Zhang, Junbo Jake Zhao, and Yann LeCun. 2015. Character-level convolutional networks for text classification. In Advances in Neural Information Processing Systems 28: Annual Conference on Neural Information Processing Systems 2015, December 7-12, 2015, Montreal, Quebec, Canada, pages 649–657. [find](https://scholar.google.com/scholar?q=Xiang+Zhang%2C+Junbo+Jake+Zhao%2C+and+Yann+LeCun%2E+2015%2E+Character%2Dlevel+convolutional+networks+for+text+classification%2E+In+Advances+in+Neural+Information+Processing+Systems+28%3A+Annual+Conference+on+Neural) [our card](https://md.research.elamur.ai/free/en/papers/zhang-2015-character-level-convolutional-networks-text/)
35. Pranav Rajpurkar, Jian Zhang, Konstantin Lopyrev, and Percy Liang. 2016. SQuAD: 100,000+ questions for machine comprehension of text. In Proceedings of Zihao Zhao, Eric Wallace, Shi Feng, Dan Klein, and Sameer Singh. 2021. Calibrate before use: Improving few-shot performance of language models. In P… [find](https://scholar.google.com/scholar?q=Pranav+Rajpurkar%2C+Jian+Zhang%2C+Konstantin+Lopyrev%2C+and+Percy+Liang%2E+2016%2E+SQuAD%3A+100%2C000%2B+questions+for+machine+comprehension+of+text%2E+In+Proceedings+of+Zihao+Zhao%2C+Eric+Wallace%2C+Shi+Feng%2C+Dan+Klein%2C+a) [our card](https://md.research.elamur.ai/free/en/papers/rajpurkar-2016-squad-100-000-questions-machine/)