---
title: "InPars — paper by Bonifacio et al. (2022) — summary and ideas"
description: "Neural retrievers often perform poorly outside the domain of their training data. 54 references, cited by 53 papers here."
authors: ["Luiz Bonifacio", "Hugo Abonizio", "Marzieh Fadaee", "Rodrigo Nogueira"]
year: 2022
venue: "arXiv:2202.05144"
source: "https://arxiv.org/abs/2202.05144"
doi: "10.48550/arxiv.2202.05144"
arxiv: "2202.05144"
licence: "CC BY 4.0 (arXiv)"
licence_url: "http://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/bonifacio-2022-inpars-data-augmentation-information-retrieval/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/bonifacio-2022-inpars-data-augmentation-information-retrieval/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2022](https://md.research.elamur.ai/free/en/papers/2022/page/5/) / InPars: Data Augmentation for Information Retrieval using Large Language Models

# InPars: Data Augmentation for Information Retrieval using Large Language Models

**Authors:** Luiz Bonifacio, Hugo Abonizio, [Marzieh Fadaee](https://md.research.elamur.ai/free/en/authors/marzieh-fadaee/) and [Rodrigo Nogueira](https://md.research.elamur.ai/free/en/authors/rodrigo-nogueira/) · **Year:** 2022 · **Venue:** arXiv:2202.05144 · **Licence:** [CC BY 4.0 (arXiv)](http://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://arxiv.org/abs/2202.05144) · [PDF](https://arxiv.org/pdf/2202.05144) · [DOI 10.48550/arxiv.2202.05144](https://doi.org/10.48550/arxiv.2202.05144) · [arXiv 2202.05144](https://arxiv.org/abs/2202.05144) · [OpenAlex](https://openalex.org/W4221151914)

**Citations:** 19 citations (OpenAlex, 2026-10-14) · [53](https://md.research.elamur.ai/free/en/papers/bonifacio-2022-inpars-data-augmentation-information-retrieval/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
Neural retrievers often perform poorly outside the domain of their training data. Collecting in-domain relevance labels is costly, and billion-parameter language models are too expensive to run directly as rerankers or dense retrievers. InPars ("Inquisitive Parrots for Search") uses a large LM offline, as a generator of training data rather than as the retriever. GPT-3 is given a fixed prompt containing three document–query examples plus a new document, and it writes a plausible query for that document. The resulting synthetic pairs are ranked by the model's mean token log-probability, and only the most confident ones are kept. These pairs then serve as positives for finetuning monoT5 rerankers, which reorder first-stage BM25 results. Rerankers trained purely on this synthetic data beat BM25 and recent self-supervised dense retrievers. When the synthetic data is combined with supervised MS MARCO training, zero-shot transfer improves, reaching state of the art on two of the three transfer collections. The approach shows a cheap, nearly unsupervised way to adapt retrievers to new domains.

## Key points
- **Data generation:** 100,000 documents were sampled per collection. GPT-3 Curie generated one question per document using greedy decoding, at a cost of roughly 100–300 USD. Only the top K = 10,000 pairs by probability were kept, since training on all pairs lost about 4 MRR@10 points on MS MARCO.
- **Prompts:** Two prompt types were compared, "Vanilla" and "GBQ" (good/bad question). GBQ produces more specific queries and worked best with in-domain documents. Vanilla was best for MS MARCO and TREC-DL.
- **Evaluation:** Rerankers were tested on MS MARCO, TREC-DL 2020, Robust04, Natural Questions (BEIR) and TREC-COVID (BEIR). The 3B monoT5 model gained most from in-domain synthetic queries.
- **Model size:** Larger GPT-3 generators gave slowly increasing MRR@10.
- **Contamination check:** To test whether GPT-3 had memorized MS MARCO, generated questions were matched against the dataset. Under 5.7% of Davinci's 93,200 unique questions appeared in MS MARCO. Code, models and data are released on GitHub (zetaalphavector/inpars).

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [University of Waterloo](https://md.research.elamur.ai/free/en/universities/university-of-waterloo/) — University, Waterloo, Canada · 80 works · 15,030 citations
- [Universidade Estadual de Campinas (UNICAMP)](https://md.research.elamur.ai/free/en/universities/universidade-estadual-de-campinas-unicamp/) — University, Campinas, Brazil · 3 works · 74 citations

📚 Shelves (5): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 18,614 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 17,965 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 17,450 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 17,296 works · [🗄️ Information Systems](https://md.research.elamur.ai/free/en/shelves/computer-science/information-systems/) · 1,730 works

🧭 Topics (1): [🔍 Information retrieval & search](https://md.research.elamur.ai/free/en/topics/information-retrieval/) · 552 works

## Cite

Cite as: Luiz Bonifacio, Hugo Abonizio, Marzieh Fadaee and Rodrigo Nogueira. 2022. InPars: Data Augmentation for Information Retrieval using Large Language Models. arXiv:2202.05144.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@misc{bonifacio2022inpars,
  title = {{InPars: Data Augmentation for Information Retrieval using Large Language Models}},
  author = {Luiz Bonifacio, Hugo Abonizio, Marzieh Fadaee and Rodrigo Nogueira},
  year = {2022},
  doi = {10.48550/arxiv.2202.05144},
  eprint = {2202.05144},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2202.05144}
}
```

- [← previous Infusing Knowledge from Wikipedia to Enhance Stance Detection](https://md.research.elamur.ai/free/en/papers/he-2022-infusing-knowledge-wikipedia-enhance-stance/)
- [next → The Ghost in the Machine has an American accent: value conflict in GPT-3](https://md.research.elamur.ai/free/en/papers/johnson-2022-ghost-machine-has-american-accent/)
- [↑ back to papers of 2022](https://md.research.elamur.ai/free/en/papers/2022/page/5/)

## References

54 references, 27 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Ateret Anaby-Tavor, Boaz Carmeli, Esther Goldbraich, Amir Kantor, George Kour, Segev Shlomov, Naama Tepper, and Naama Zwerdling. 2020. Do not have enough data? deep learning to the rescue! In Proceedings of the AAAI Conference on Artificial Intelligence, volume 34, pages 7383–7390. [find](https://scholar.google.com/scholar?q=Ateret+Anaby%2DTavor%2C+Boaz+Carmeli%2C+Esther+Goldbraich%2C+Amir+Kantor%2C+George+Kour%2C+Segev+Shlomov%2C+Naama+Tepper%2C+and+Naama+Zwerdling%2E+2020%2E+Do+not+have+enough+data%3F+deep+learning+to+the+rescue%21+In+Proceedi)
2. Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter,… — [Language Models are Few-Shot Learners](https://md.research.elamur.ai/free/en/papers/brown-2020-language-models-few-shot-learners/)
3. Ruey-Cheng Chen, Luke Gallagher, Roi Blanco, and J Shane Culpepper. 2017. Efficient cost-aware cascade ranking in multi-stage retrieval. In Proceedings of the 40th International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 445–454. [find](https://scholar.google.com/scholar?q=Ruey%2DCheng+Chen%2C+Luke+Gallagher%2C+Roi+Blanco%2C+and+J+Shane+Culpepper%2E+2017%2E+Efficient+cost%2Daware+cascade+ranking+in+multi%2Dstage+retrieval%2E+In+Proceedings+of+the+40th+International+ACM+SIGIR+Conference+o)
4. Nick Craswell, Bhaskar Mitra, Emine Yilmaz, and Daniel Campos. 2021. Overview of the TREC 2020 deep learning track. CoRR, abs/2102.07662. — [arXiv:2102.07662](https://arxiv.org/abs/2102.07662) [find](https://scholar.google.com/scholar?q=Nick+Craswell%2C+Bhaskar+Mitra%2C+Emine+Yilmaz%2C+and+Daniel+Campos%2E+2021%2E+Overview+of+the+TREC+2020+deep+learning+track%2E+CoRR%2C+abs%2F2102%2E07662%2E+%E2%80%94+%5BarXiv%3A2102%2E07662%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2102%2E07662%29)
5. Zhuyun Dai and Jamie Callan. 2019. Deeper text understanding for ir with contextual neural language modeling. In Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 985–988. [find](https://scholar.google.com/scholar?q=Zhuyun+Dai+and+Jamie+Callan%2E+2019%2E+Deeper+text+understanding+for+ir+with+contextual+neural+language+modeling%2E+In+Proceedings+of+the+42nd+International+ACM+SIGIR+Conference+on+Research+and+Development+)
6. Marzieh Fadaee, Arianna Bisazza, and Christof Monz. 2017. [Data augmentation for low-resource neural machine translation](https://md.research.elamur.ai/free/en/papers/fadaee-2017-data-augmentation-low-resource-neural/). In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 567–573, Vancouver, Canada. Association for Computationa…
7. Luyu Gao and Jamie Callan. 2021. [Unsupervised corpus aware language model pre-training for dense passage retrieval](https://md.research.elamur.ai/free/en/papers/gao-2022-unsupervised-corpus-aware-language-model/). arXiv preprint arXiv:2108.05540.
8. Jesse Michael Han, Igor Babuschkin, Harrison Edwards, Arvind Neelakantan, Tao Xu, Stanislas Polu, Alex Ray, Pranav Shyam, Aditya Ramesh, Alec Radford, et al. 2021. Unsupervised neural machine translation with generative language models only. arXiv preprint arXiv:2110.05448. [find](https://scholar.google.com/scholar?q=Jesse+Michael+Han%2C+Igor+Babuschkin%2C+Harrison+Edwards%2C+Arvind+Neelakantan%2C+Tao+Xu%2C+Stanislas+Polu%2C+Alex+Ray%2C+Pranav+Shyam%2C+Aditya+Ramesh%2C+Alec+Radford%2C+et+al%2E+2021%2E+Unsupervised+neural+machine+translat)
9. Ari Holtzman, Jan Buys, Li Du, Maxwell Forbes, and Yejin Choi. 2019. [The curious case of neural text degeneration](https://md.research.elamur.ai/free/en/papers/holtzman-2019-curious-case-neural-text-degeneration/). In International Conference on Learning Representations.
10. Gautier Izacard, Mathilde Caron, Lucas Hosseini, Sebastian Riedel, Piotr Bojanowski, Armand Joulin, and Edouard Grave. 2021. Towards [unsupervised dense information retrieval with contrastive learning](https://md.research.elamur.ai/free/en/papers/izacard-2021-unsupervised-dense-information-retrieval-contrastive/). arXiv preprint arXiv:2112.09118.
11. Jeff Johnson, Matthijs Douze, and Hervé Jégou. 2019. [Billion-scale similarity search with gpus](https://md.research.elamur.ai/free/en/papers/johnson-2017-billion-scale-similarity-search-gpus/). IEEE Transactions on Big Data.
12. Vladimir Karpukhin, Barlas Oguz, Sewon Min, Ledell Wu, Sergey Edunov, Danqi Chen, and Wen-tau Yih. 2020. [Dense passage retrieval for open-domain question answering](https://md.research.elamur.ai/free/en/papers/karpukhin-2020-dense-passage-retrieval-open-domain/). CoRR, abs/2004.04906.
13. Omar Khattab and Matei Zaharia. 2020. [Colbert: Efficient and effective passage search via contextualized late interaction over BERT](https://md.research.elamur.ai/free/en/papers/khattab-2020-colbert-efficient-effective-passage/). CoRR, abs/2004.12832.
14. Sosuke Kobayashi. 2018. [Contextual augmentation: Data augmentation by words with paradigmatic relations](https://md.research.elamur.ai/free/en/papers/kobayashi-2018-contextual-augmentation-data-augmentation-words/). arXiv preprint arXiv:1805.06201.
15. Varun Kumar, Ashutosh Choudhary, and Eunah Cho. 2020. [Data augmentation using pre-trained transformer models](https://md.research.elamur.ai/free/en/papers/kumar-2020-data-augmentation-using-pre-trained/). In Proceedings of the 2nd Workshop on Life-long Learning for Spoken Language Systems, pages 18–26.
16. Tom Kwiatkowski, Jennimaria Palomaki, Olivia Redfield, Michael Collins, Ankur Parikh, Chris Alberti, Danielle Epstein, Illia Polosukhin, Matthew Kelcey, Jacob Devlin, Kenton Lee, Kristina N. Toutanova, Llion Jones, Ming-Wei Chang, Andrew Dai, Jakob Uszkoreit, Quoc Le, and Slav Petrov. 2019. [Natural…](https://md.research.elamur.ai/free/en/papers/kwiatkowski-2019-natural-questions-benchmark-question-answering/)
17. Canjia Li, Andrew Yates, Sean MacAvaney, Ben He, and Yingfei Sun. 2020. Parade: Passage representation aggregation for document reranking. arXiv preprint arXiv:2008.09093. — [arXiv:2008.09093](https://arxiv.org/abs/2008.09093) [find](https://scholar.google.com/scholar?q=Canjia+Li%2C+Andrew+Yates%2C+Sean+MacAvaney%2C+Ben+He%2C+and+Yingfei+Sun%2E+2020%2E+Parade%3A+Passage+representation+aggregation+for+document+reranking%2E+arXiv+preprint+arXiv%3A2008%2E09093%2E+%E2%80%94+%5BarXiv%3A2008%2E09093%5D%28https%3A%2F)
18. Jimmy Lin, Xueguang Ma, Sheng-Chieh Lin, Jheng-Hong Yang, Ronak Pradeep, and Rodrigo Nogueira. 2021. Pyserini: A python toolkit for reproducible information retrieval research with sparse and dense representations. In Proceedings of the 44th Annual International ACM SI-GIR Conference on Research and… [find](https://scholar.google.com/scholar?q=Jimmy+Lin%2C+Xueguang+Ma%2C+Sheng%2DChieh+Lin%2C+Jheng%2DHong+Yang%2C+Ronak+Pradeep%2C+and+Rodrigo+Nogueira%2E+2021%2E+Pyserini%3A+A+python+toolkit+for+reproducible+information+retrieval+research+with+sparse+and+dense+re)
19. Alisa Liu, Swabha Swayamdipta, Noah A. Smith, and Yejin Choi. 2022. [Wanli: Worker and ai collaboration for natural language inference dataset creation](https://md.research.elamur.ai/free/en/papers/liu-2022-wanli-worker-ai-collaboration-natural/).
20. Shichen Liu, Fei Xiao, Wenwu Ou, and Luo Si. 2017. Cascade ranking for operational ecommerce search. In Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pages 1557–1565. [find](https://scholar.google.com/scholar?q=Shichen+Liu%2C+Fei+Xiao%2C+Wenwu+Ou%2C+and+Luo+Si%2E+2017%2E+Cascade+ranking+for+operational+ecommerce+search%2E+In+Proceedings+of+the+23rd+ACM+SIGKDD+International+Conference+on+Knowledge+Discovery+and+Data+Mini)
21. Ji Ma, Ivan Korotkov, Yinfei Yang, Keith Hall, and Ryan McDonald. 2021. [Zero-shot neural passage retrieval via domain-targeted synthetic question generation](https://md.research.elamur.ai/free/en/papers/ma-2021-zero-shot-neural-passage-retrieval/). In Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume, pages 1075–1088,…
22. Irina Matveeva, Chris Burges, Timo Burkard, Andy Laucius, and Leon Wong. 2006. High accuracy retrieval with multiple nested ranker. In Proceedings of the 29th annual international ACM SIGIR conference on Research and development in information retrieval, pages 437–444. [find](https://scholar.google.com/scholar?q=Irina+Matveeva%2C+Chris+Burges%2C+Timo+Burkard%2C+Andy+Laucius%2C+and+Leon+Wong%2E+2006%2E+High+accuracy+retrieval+with+multiple+nested+ranker%2E+In+Proceedings+of+the+29th+annual+international+ACM+SIGIR+conference)
23. Yu Meng, Jiaxin Huang, Yu Zhang, and Jiawei Han. 2022. [Generating training data with language models: Towards zero-shot language understanding](https://md.research.elamur.ai/free/en/papers/meng-2022-generating-training-data-language-models/).
24. Biswesh Mohapatra, Gaurav Pandey, Danish Contractor, and Sachindra Joshi. 2021. Simulated chats for building dialog systems: Learning to generate conversations from instructions. In Findings of the Association for Computational Linguistics: EMNLP 2021, pages 1190–1203. [find](https://scholar.google.com/scholar?q=Biswesh+Mohapatra%2C+Gaurav+Pandey%2C+Danish+Contractor%2C+and+Sachindra+Joshi%2E+2021%2E+Simulated+chats+for+building+dialog+systems%3A+Learning+to+generate+conversations+from+instructions%2E+In+Findings+of+the+As)
25. Arvind Neelakantan, Tao Xu, Raul Puri, Alec Radford, Jesse Michael Han, Jerry Tworek, Qiming Yuan, Nikolas Tezak, Jong Wook Kim, Chris Hallacy, et al. 2022. [Text and code embeddings by contrastive pre-training](https://md.research.elamur.ai/free/en/papers/neelakantan-2022-text-code-embeddings-contrastive-pre/). arXiv preprint arXiv:2201.10005.
26. Tri Nguyen, Mir Rosenberg, Xia Song, Jianfeng Gao, Saurabh Tiwary, Rangan Majumder, and Li Deng. 2016. [MS MARCO: A human generated machine reading comprehension dataset](https://md.research.elamur.ai/free/en/papers/bajaj-2016-ms-marco-human-generated/). CoRR, abs/1611.09268.
27. Rodrigo Nogueira, Zhiying Jiang, Ronak Pradeep, and Jimmy Lin. 2020. [Document ranking with a pretrained sequence-to-sequence model](https://md.research.elamur.ai/free/en/papers/nogueira-2020-document-ranking-pretrained-sequence-sequence/). In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: Findings, pages 708–718.
28. Yannis Papanikolaou and Andrea Pierleoni. 2020. Dare: Data augmented relation extraction with gpt-2. arXiv preprint arXiv:2004.13845. — [arXiv:2004.13845](https://arxiv.org/abs/2004.13845) [find](https://scholar.google.com/scholar?q=Yannis+Papanikolaou+and+Andrea+Pierleoni%2E+2020%2E+Dare%3A+Data+augmented+relation+extraction+with+gpt%2D2%2E+arXiv+preprint+arXiv%3A2004%2E13845%2E+%E2%80%94+%5BarXiv%3A2004%2E13845%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2004%2E13845%29)
29. Ronak Pradeep, Xueguang Ma, Xinyu Zhang, H. Cui, Ruizhou Xu, Rodrigo Nogueira, Jimmy J. Lin, and D. Cheriton. 2020a. H2oloo at trec 2020: When all you got is a hammer... deep learning, health misinformation, and precision medicine. In TREC. [find](https://scholar.google.com/scholar?q=Ronak+Pradeep%2C+Xueguang+Ma%2C+Xinyu+Zhang%2C+H%2E+Cui%2C+Ruizhou+Xu%2C+Rodrigo+Nogueira%2C+Jimmy+J%2E+Lin%2C+and+D%2E+Cheriton%2E+2020a%2E+H2oloo+at+trec+2020%3A+When+all+you+got+is+a+hammer%2E%2E%2E+deep+learning%2C+health+misinfor)
30. Ronak Pradeep, Xueguang Ma, Xinyu Zhang, Hang Cui, Ruizhou Xu, Rodrigo Nogueira, and Jimmy Lin. 2020b. H2oloo at trec 2020: When all you got is a hammer... deep learning, health misinformation, and precision medicine. Corpus, 5(d3):d2. [find](https://scholar.google.com/scholar?q=Ronak+Pradeep%2C+Xueguang+Ma%2C+Xinyu+Zhang%2C+Hang+Cui%2C+Ruizhou+Xu%2C+Rodrigo+Nogueira%2C+and+Jimmy+Lin%2E+2020b%2E+H2oloo+at+trec+2020%3A+When+all+you+got+is+a+hammer%2E%2E%2E+deep+learning%2C+health+misinformation%2C+and+pr)
31. Ronak Pradeep, Rodrigo Nogueira, and Jimmy Lin. 2021. The expando-mono-duo design pattern for text ranking with pretrained sequence-to-sequence models. arXiv preprint arXiv:2101.05667. — [arXiv:2101.05667](https://arxiv.org/abs/2101.05667) [find](https://scholar.google.com/scholar?q=Ronak+Pradeep%2C+Rodrigo+Nogueira%2C+and+Jimmy+Lin%2E+2021%2E+The+expando%2Dmono%2Dduo+design+pattern+for+text+ranking+with+pretrained+sequence%2Dto%2Dsequence+models%2E+arXiv+preprint+arXiv%3A2101%2E05667%2E+%E2%80%94+%5BarXiv%3A2101%2E0)
32. Yanru Qu, Dinghan Shen, Yelong Shen, Sandra Sajeev, Weizhu Chen, and Jiawei Han. 2021. Co{da}: Contrast-enhanced and diversity-promoting data augmentation for natural language understanding. In International Conference on Learning Representations. [find](https://scholar.google.com/scholar?q=Yanru+Qu%2C+Dinghan+Shen%2C+Yelong+Shen%2C+Sandra+Sajeev%2C+Weizhu+Chen%2C+and+Jiawei+Han%2E+2021%2E+Co%7Bda%7D%3A+Contrast%2Denhanced+and+diversity%2Dpromoting+data+augmentation+for+natural+language+understanding%2E+In+Intern)
33. Jack W Rae, Sebastian Borgeaud, Trevor Cai, Katie Millican, Jordan Hoffmann, Francis Song, John Aslanides, Sarah Henderson, Roman Ring, Susannah Young, et al. 2021. Scaling language models: Methods, analysis & insights from training gopher. arXiv preprint arXiv:2112.11446. [find](https://scholar.google.com/scholar?q=Jack+W+Rae%2C+Sebastian+Borgeaud%2C+Trevor+Cai%2C+Katie+Millican%2C+Jordan+Hoffmann%2C+Francis+Song%2C+John+Aslanides%2C+Sarah+Henderson%2C+Roman+Ring%2C+Susannah+Young%2C+et+al%2E+2021%2E+Scaling+language+models%3A+Methods%2C+a)
34. Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J Liu. 2020. [Exploring the limits of transfer learning with a unified text-to-text transformer](https://md.research.elamur.ai/free/en/papers/raffel-2019-exploring-limits-transfer-learning-unified/). Journal of Machine Learning Research, 21:1–67.
35. Ori Ram, Gal Shachaf, Omer Levy, Jonathan Berant, and Amir Globerson. 2021. Learning to retrieve passages without supervision. arXiv preprint arXiv:2112.07708. [find](https://scholar.google.com/scholar?q=Ori+Ram%2C+Gal+Shachaf%2C+Omer+Levy%2C+Jonathan+Berant%2C+and+Amir+Globerson%2E+2021%2E+Learning+to+retrieve+passages+without+supervision%2E+arXiv+preprint+arXiv%3A2112%2E07708%2E)
36. Nils Reimers and Iryna Gurevych. 2020. The curse of dense low-dimensional information retrieval for large index sizes. arXiv preprint arXiv:2012.14210. [find](https://scholar.google.com/scholar?q=Nils+Reimers+and+Iryna+Gurevych%2E+2020%2E+The+curse+of+dense+low%2Ddimensional+information+retrieval+for+large+index+sizes%2E+arXiv+preprint+arXiv%3A2012%2E14210%2E)
37. Kirk Roberts, Tasmeer Alam, Steven Bedrick, Dina Demner-Fushman, Kyle Lo, Ian Soboroff, Ellen Voorhees, Lucy Lu Wang, and William R Hersh. 2020. TREC-COVID: rationale and structure of an information retrieval shared task for COVID-19. Journal of the American Medical Informatics Association, 27(9):14… [find](https://scholar.google.com/scholar?q=Kirk+Roberts%2C+Tasmeer+Alam%2C+Steven+Bedrick%2C+Dina+Demner%2DFushman%2C+Kyle+Lo%2C+Ian+Soboroff%2C+Ellen+Voorhees%2C+Lucy+Lu+Wang%2C+and+William+R+Hersh%2E+2020%2E+TREC%2DCOVID%3A+rationale+and+structure+of+an+information+r)
38. SE ROBERTSON, S WALKER, S JONES, MM HANCOCK-BEAULIEU, and M GAT-FORD. 1995. [Okapi at trec-3](https://md.research.elamur.ai/free/en/papers/robertson-1994-okapi-trec-3/). NIST special publication, (500225):109–123.
39. Victor Sanh, Albert Webson, Colin Raffel, Stephen H. Bach, Lintang Sutawika, Zaid Alyafeai, Antoine Chaffin, Arnaud Stiegler, Teven Le Scao, Arun Raja, Manan Dey, M Saiful Bari, Canwen Xu, Urmish Thakker, Shanya Sharma Sharma, Eliza Szczechla, Taewoon Kim, Gunjan Chhablani, Nihal Nayak, Debajyoti Da… — [Multitask Prompted Training Enables Zero-Shot Task Generalization](https://md.research.elamur.ai/free/en/papers/sanh-2021-multitask-prompted-training-enables-zero/)
40. Keshav Santhanam, Omar Khattab, Jon Saad-Falcon, Christopher Potts, and Matei Zaharia. 2021. [Colbertv2: Effective and efficient retrieval via lightweight late interaction](https://md.research.elamur.ai/free/en/papers/santhanam-2022-colbertv2-effective-efficient-retrieval-lightweight/). arXiv preprint arXiv:2112.01488.
41. Timo Schick and Hinrich Schütze. 2021a. [Generating datasets with pretrained language models](https://md.research.elamur.ai/free/en/papers/schick-2021-generating-datasets-pretrained-language-models/). arXiv preprint arXiv:2104.07540.
42. Timo Schick and Hinrich Schütze. 2021b. [It’s not just size that matters: Small language models are also few-shot learners](https://md.research.elamur.ai/free/en/papers/schick-2021-it-s-not-just-size/). In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 2339–2352, Online. Asso…
43. Javad Pourmostafa Roshan Sharami, Dimitar Shterionov, and Pieter Spronck. 2022. Selecting parallel in-domain sentences for neural machine translation using monolingual texts. [find](https://scholar.google.com/scholar?q=Javad+Pourmostafa+Roshan+Sharami%2C+Dimitar+Shterionov%2C+and+Pieter+Spronck%2E+2022%2E+Selecting+parallel+in%2Ddomain+sentences+for+neural+machine+translation+using+monolingual+texts%2E)
44. Nandan Thakur, Nils Reimers, Andreas Rücklé, Abhishek Srivastava, and Iryna Gurevych. 2021. BEIR: A heterogeneous benchmark for zero-shot evaluation of information retrieval models. In Thirty-fifth Conference on Neural Information Processing Systems Datasets and Benchmarks Track (Round 2). [find](https://scholar.google.com/scholar?q=Nandan+Thakur%2C+Nils+Reimers%2C+Andreas+Ru%CC%88ckle%CC%81%2C+Abhishek+Srivastava%2C+and+Iryna+Gurevych%2E+2021%2E+BEIR%3A+A+heterogeneous+benchmark+for+zero%2Dshot+evaluation+of+information+retrieval+models%2E+In+Thirty%2Dfifth+)
45. Ellen Voorhees. 2005. Overview of the trec 2004 robust retrieval track. [find](https://scholar.google.com/scholar?q=Ellen+Voorhees%2E+2005%2E+Overview+of+the+trec+2004+robust+retrieval+track%2E)
46. Ellen M. Voorhees. 2004. Overview of trec 2004. In TREC. [find](https://scholar.google.com/scholar?q=Ellen+M%2E+Voorhees%2E+2004%2E+Overview+of+trec+2004%2E+In+TREC%2E)
47. Ellen M. Voorhees and Donna K. Harman. 1999. Overview of the eighth text retrieval conference (trec-8). In TREC. [find](https://scholar.google.com/scholar?q=Ellen+M%2E+Voorhees+and+Donna+K%2E+Harman%2E+1999%2E+Overview+of+the+eighth+text+retrieval+conference+%28trec%2D8%29%2E+In+TREC%2E)
48. Kexin Wang, Nandan Thakur, Nils Reimers, and Iryna Gurevych. 2021. [Gpl: Generative pseudo labeling for unsupervised domain adaptation of dense retrieval](https://md.research.elamur.ai/free/en/papers/wang-2022-gpl-generative-pseudo-labeling-unsupervised/). arXiv preprint arXiv:2112.07577.
49. Lidan Wang, Jimmy Lin, and Donald Metzler. 2011. A cascade ranking model for efficient ranked retrieval. In Proceedings of the 34th international ACM SIGIR conference on Research and development in Information Retrieval, pages 105–114. [find](https://scholar.google.com/scholar?q=Lidan+Wang%2C+Jimmy+Lin%2C+and+Donald+Metzler%2E+2011%2E+A+cascade+ranking+model+for+efficient+ranked+retrieval%2E+In+Proceedings+of+the+34th+international+ACM+SIGIR+conference+on+Research+and+development+in+In)
50. Jason Wei, Maarten Bosma, Vincent Zhao, Kelvin Guu, Adams Wei Yu, Brian Lester, Nan Du, Andrew M. Dai, and Quoc V Le. 2022. [Finetuned language models are zero-shot learners](https://md.research.elamur.ai/free/en/papers/wei-2021-finetuned-language-models-zero-shot/). In International Conference on Learning Representations.
51. Genta Indra Winata, Andrea Madotto, Zhaojiang Lin, Rosanne Liu, Jason Yosinski, and Pascale Fung. 2021. [Language models are few-shot multilingual learners](https://md.research.elamur.ai/free/en/papers/winata-2021-language-models-are-few-shot/). In Proceedings of the 1st Workshop on Multilingual Representation Learning, pages 1–15, Punta Cana, Dominican Republic. Association for Computat…
52. Yiben Yang, Chaitanya Malaviya, Jared Fernandez, Swabha Swayamdipta, Ronan Le Bras, Ji-Ping Wang, Chandra Bhagavatula, Yejin Choi, and Doug Downey. 2020. G-daug: [Generative data augmentation for commonsense reasoning](https://md.research.elamur.ai/free/en/papers/yang-2020-generative-data-augmentation-commonsense-reasoning/). In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Pro…
53. Emine Yilmaz, Nick Craswell, Bhaskar Mitra, and Daniel Campos. 2020. On the reliability of test collections for evaluating systems of different types. In proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 2101– 2104. [find](https://scholar.google.com/scholar?q=Emine+Yilmaz%2C+Nick+Craswell%2C+Bhaskar+Mitra%2C+and+Daniel+Campos%2E+2020%2E+On+the+reliability+of+test+collections+for+evaluating+systems+of+different+types%2E+In+proceedings+of+the+43rd+International+ACM+SIGI)
54. Tiezheng Yu, Zihan Liu, and Pascale Fung. 2021. Adaptsum: Towards low-resource domain adaptation for abstractive summarization. [find](https://scholar.google.com/scholar?q=Tiezheng+Yu%2C+Zihan+Liu%2C+and+Pascale+Fung%2E+2021%2E+Adaptsum%3A+Towards+low%2Dresource+domain+adaptation+for+abstractive+summarization%2E)

*Updated 2 November 2026*