---
title: "Precise Zero-Shot Dense Retrieval without — Gao 2022"
description: "The paper tackles fully zero-shot dense retrieval, where no relevance judgments are available, not even from a large transfer dataset such as MS-MARCO…"
authors: ["Luyu Gao", "Xueguang Ma", "Jimmy Lin", "Jamie Callan"]
year: 2022
venue: "arXiv:2212.10496"
source: "https://arxiv.org/abs/2212.10496"
doi: "10.48550/arxiv.2212.10496"
arxiv: "2212.10496"
licence: "arXiv — non-exclusive license to distribute"
licence_url: "http://arxiv.org/licenses/nonexclusive-distrib/1.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/gao-2022-precise-zero-shot-dense-retrieval/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/gao-2022-precise-zero-shot-dense-retrieval/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2022](https://md.research.elamur.ai/free/en/papers/2022/page/4/) / Precise Zero-Shot Dense Retrieval without Relevance Labels

# Precise Zero-Shot Dense Retrieval without Relevance Labels

**Authors:** [Luyu Gao](https://md.research.elamur.ai/free/en/authors/luyu-gao/), [Xueguang Ma](https://md.research.elamur.ai/free/en/authors/xueguang-ma/), [Jimmy Lin](https://md.research.elamur.ai/free/en/authors/jimmy-lin/), [Jamie Callan](https://md.research.elamur.ai/free/en/authors/jamie-callan/) · **Year:** 2022 · **Venue:** arXiv:2212.10496 · **Licence:** [arXiv — non-exclusive license to distribute](http://arxiv.org/licenses/nonexclusive-distrib/1.0/)

**Links:** [Publisher's page](https://arxiv.org/abs/2212.10496) · [PDF](https://arxiv.org/pdf/2212.10496) · [DOI 10.48550/arxiv.2212.10496](https://doi.org/10.48550/arxiv.2212.10496) · [arXiv 2212.10496](https://arxiv.org/abs/2212.10496) · [OpenAlex](https://openalex.org/W4385565351)

**Citations:** 342 citations (openalex-snapshot · 2026-10-14) · [202](#cited-by) in our library, of the 140673 papers whose bibliographies we read

## Summary

The paper tackles fully zero-shot dense retrieval, where no relevance judgments are available, not even from a large transfer dataset such as MS-MARCO, whose licence restricts commercial use. The authors propose HyDE (Hypothetical Document Embeddings), which avoids learning a query–document relevance space. An instruction-following LLM (InstructGPT) is first prompted to write a passage answering the query. This invented passage may contain false facts but resembles a relevant document. An unsupervised contrastive encoder (Contriever, or mContriever for non-English data) then embeds the passage, optionally averaging several samples. The resulting vector is used to find nearby real documents in the corpus. The encoder's compact embedding is expected to discard the hallucinated specifics. No model is trained or fine-tuned. Across 11 query sets covering web search, QA, fact verification and several non-English languages, HyDE clearly beats plain Contriever and often matches retrievers fine-tuned on MS-MARCO. This makes it a practical choice for launching a search system before any search logs or labels exist.

## Key points

- **Method:** retrieval is split into two steps. An LLM generates a hypothetical document, and Contriever handles document–document similarity over a standard MIPS index. Query–document relevance is never scored directly. Code is available at github.com/texttron/hyde.
- **TREC DL19/DL20 (NDCG@10):** Contriever scores 44.5/42.1. HyDE with InstructGPT (175b) reaches 61.3/57.9, close to the in-domain fine-tuned Contriever FT at 62.1/63.2.
- **Smaller LLMs help less:** FLAN-T5-xxl (11b) reaches 48.9/52.9, and Cohere command-xlarge (52b) reaches 53.8/53.8. Pairing InstructGPT with Contriever FT gives 67.4 on DL19.
- **Mr.TyDi (MRR@100):** HyDE improves on mContriever for Swahili (41.7 vs 38.3), Korean (30.6 vs 22.3), Japanese (30.7 vs 19.5) and Bengali (41.3 vs 35.3). It still trails mContriever FT, which the authors attribute to these languages being under-trained in the LLM.
- **Comparison with supervised baselines:** HyDE generally outperforms the MS-MARCO-trained DPR and ANCE. Contriever FT keeps the lead on FiQA and DBPedia.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

Fields: [Other](https://md.research.elamur.ai/free/en/shelves/other/)

## Cited by

Cited by

202 papers of our library cite it (of the 140673 whose bibliographies we read); 9 of them have a page here, 193 not yet.

- [A Survey on RAG Meeting LLMs: Towards Retrieval-Augmented Large Language Models](https://md.research.elamur.ai/free/en/papers/fan-2024-survey-rag-meeting-llms-towards/)
- [Retrieval-Augmented Generation for Large Language Models: A Survey](https://md.research.elamur.ai/free/en/papers/gao-2023-retrieval-augmented-generation-large-language/)
- [LightRAG: Simple and Fast Retrieval-Augmented Generation](https://md.research.elamur.ai/free/en/papers/guo-2024-lightrag-simple-fast-retrieval-augmented/)
- [Large Language Models are Effective Text Rankers with Pairwise Ranking Prompting](https://md.research.elamur.ai/free/en/papers/qin-2024-large-language-models-are-effective/)
- [Enhancing Retrieval-Augmented Large Language Models with Iterative Retrieval-Generation Synergy](https://md.research.elamur.ai/free/en/papers/shao-2023-enhancing-retrieval-augmented-large-language/)
- [Is ChatGPT Good at Search? Investigating Large Language Models as Re-Ranking Agents](https://md.research.elamur.ai/free/en/papers/sun-2023-chatgpt-good-search-investigating-large/)
- [Query2doc: Query Expansion with Large Language Models](https://md.research.elamur.ai/free/en/papers/wang-2023-query2doc-query-expansion-large-language/)
- [A Survey of Large Language Models](https://md.research.elamur.ai/free/en/papers/zhao-2023-survey-large-language-models/)
- [Large Language Models for Information Retrieval: A Survey](https://md.research.elamur.ai/free/en/papers/zhu-2023-large-language-models-information-retrieval/)

## Cite

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@misc{gao2022precise,
  title = {{Precise Zero-Shot Dense Retrieval without Relevance Labels}},
  author = {Gao, Luyu and Ma, Xueguang and Lin, Jimmy and Callan, Jamie},
  year = {2022},
  doi = {10.48550/arxiv.2212.10496},
  eprint = {2212.10496},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2212.10496}
}
```

- [← previous Pragmatic and Logical Inferences in NLI Systems: The Case of Conjunction Buttressing](https://md.research.elamur.ai/free/en/papers/pedinotti-2022-pragmatic-logical-inferences-nli-systems/)
- [next → Predicting the Proficiency Level of Nonnative Hebrew Authors](https://md.research.elamur.ai/free/en/papers/lingbulk-2022-nguyen-predicting-proficiency-level-nonnative-hebrew/)
- [↑ back to papers of 2022](https://md.research.elamur.ai/free/en/papers/2022/page/4/)

## References

39 references, 21 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Akari Asai, Timo Schick, Patrick Lewis, Xilun Chen, Gautier Izacard, Sebastian Riedel, Hannaneh Hajishirzi, and Wen-tau Yih. 2022. Task-aware retrieval with instructions. [find](https://scholar.google.com/scholar?q=Akari+Asai%2C+Timo+Schick%2C+Patrick+Lewis%2C+Xilun+Chen%2C+Gautier+Izacard%2C+Sebastian+Riedel%2C+Hannaneh+Hajishirzi%2C+and+Wen%2Dtau+Yih%2E+2022%2E+Task%2Daware+retrieval+with+instructions%2E) [our card](https://md.research.elamur.ai/free/en/papers/asai-2023-task-aware-retrieval-instructions/)
2. Payal Bajaj, Daniel Campos, Nick Craswell, Li Deng, Jianfeng Gao, Xiaodong Liu, Rangan Majumder, Andrew McNamara, Bhaskar Mitra, Tri Nguyen, Mir Rosenberg, Xia Song, Alina Stoica, Saurabh Tiwary, and Tong Wang. 2016. Ms marco: A human generated machine reading comprehension dataset. [find](https://scholar.google.com/scholar?q=Payal+Bajaj%2C+Daniel+Campos%2C+Nick+Craswell%2C+Li+Deng%2C+Jianfeng+Gao%2C+Xiaodong+Liu%2C+Rangan+Majumder%2C+Andrew+McNamara%2C+Bhaskar+Mitra%2C+Tri+Nguyen%2C+Mir+Rosenberg%2C+Xia+Song%2C+Alina+Stoica%2C+Saurabh+Tiwary%2C+and+) [our card](https://md.research.elamur.ai/free/en/papers/bajaj-2016-ms-marco-human-generated/)
3. Michele Bevilacqua, Giuseppe Ottaviano, Patrick Lewis, Wen-tau Yih, Sebastian Riedel, and Fabio Petroni. 2022. Autoregressive search engines: Generating substrings as document identifiers. CoRR, abs/2204.10628. — [arXiv:2204.10628](https://arxiv.org/abs/2204.10628) [find](https://scholar.google.com/scholar?q=Michele+Bevilacqua%2C+Giuseppe+Ottaviano%2C+Patrick+Lewis%2C+Wen%2Dtau+Yih%2C+Sebastian+Riedel%2C+and+Fabio+Petroni%2E+2022%2E+Autoregressive+search+engines%3A+Generating+substrings+as+document+identifiers%2E+CoRR%2C+abs%2F2)
4. Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter,… [find](https://scholar.google.com/scholar?q=Tom+B%2E+Brown%2C+Benjamin+Mann%2C+Nick+Ryder%2C+Melanie+Subbiah%2C+Jared+Kaplan%2C+Prafulla+Dhariwal%2C+Arvind+Neelakantan%2C+Pranav+Shyam%2C+Girish+Sastry%2C+Amanda+Askell%2C+Sandhini+Agarwal%2C+Ariel+Herbert%2DVoss%2C+Gretche)
5. Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde de Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, Alex Ray, Raul Puri, Gretchen Krueger, Michael Petrov, Heidy Khlaaf, Girish Sastry, Pamela Mishkin, Brooke Chan, Scott Gray, Nick Ryder, Mikhai… [find](https://scholar.google.com/scholar?q=Mark+Chen%2C+Jerry+Tworek%2C+Heewoo+Jun%2C+Qiming+Yuan%2C+Henrique+Ponde+de+Oliveira+Pinto%2C+Jared+Kaplan%2C+Harri+Edwards%2C+Yuri+Burda%2C+Nicholas+Joseph%2C+Greg+Brockman%2C+Alex+Ray%2C+Raul+Puri%2C+Gretchen+Krueger%2C+Mich) [our card](https://md.research.elamur.ai/free/en/papers/chen-2021-evaluating-large-language-models-trained/)
6. Aakanksha Chowdhery, Sharan Narang, Jacob Devlin, Maarten Bosma, Gaurav Mishra, Adam Roberts, Paul Barham, Hyung Won Chung, Charles Sutton, Sebastian Gehrmann, Parker Schuh, Kensen Shi, Sasha Tsvyashchenko, Joshua Maynez, Abhishek Rao, Parker Barnes, Yi Tay, Noam Shazeer, Vinodkumar Prabhakaran, Emi… [find](https://scholar.google.com/scholar?q=Aakanksha+Chowdhery%2C+Sharan+Narang%2C+Jacob+Devlin%2C+Maarten+Bosma%2C+Gaurav+Mishra%2C+Adam+Roberts%2C+Paul+Barham%2C+Hyung+Won+Chung%2C+Charles+Sutton%2C+Sebastian+Gehrmann%2C+Parker+Schuh%2C+Kensen+Shi%2C+Sasha+Tsvyashc)
7. Alexis Conneau, Kartikay Khandelwal, Naman Goyal, Vishrav Chaudhary, Guillaume Wenzek, Francisco Guzmán, Edouard Grave, Myle Ott, Luke Zettlemoyer, and Veselin Stoyanov. 2020. Unsupervised cross-lingual representation learning at scale. In Proceedings of the 58th Annual Meeting of the Association fo… [find](https://scholar.google.com/scholar?q=Alexis+Conneau%2C+Kartikay+Khandelwal%2C+Naman+Goyal%2C+Vishrav+Chaudhary%2C+Guillaume+Wenzek%2C+Francisco+Guzm%C3%A1n%2C+Edouard+Grave%2C+Myle+Ott%2C+Luke+Zettlemoyer%2C+and+Veselin+Stoyanov%2E+2020%2E+Unsupervised+cross%2Dlingu)
8. Nick Craswell, Bhaskar Mitra, Emine Yilmaz, Daniel Campos, and Ellen M. Voorhees. 2020a. Overview of the trec 2019 deep learning track. [find](https://scholar.google.com/scholar?q=Nick+Craswell%2C+Bhaskar+Mitra%2C+Emine+Yilmaz%2C+Daniel+Campos%2C+and+Ellen+M%2E+Voorhees%2E+2020a%2E+Overview+of+the+trec+2019+deep+learning+track%2E) [our card](https://md.research.elamur.ai/free/en/papers/craswell-2020-overview-trec-2019-deep-learning/)
9. Nick Craswell, Bhaskar Mitra, Emine Yilmaz, Daniel Fernando Campos, and Ellen M. Voorhees. 2020b. Overview of the trec 2020 deep learning track. ArXiv, abs/2003.07820. [find](https://scholar.google.com/scholar?q=Nick+Craswell%2C+Bhaskar+Mitra%2C+Emine+Yilmaz%2C+Daniel+Fernando+Campos%2C+and+Ellen+M%2E+Voorhees%2E+2020b%2E+Overview+of+the+trec+2020+deep+learning+track%2E+ArXiv%2C+abs%2F2003%2E07820%2E) [our card](https://md.research.elamur.ai/free/en/papers/craswell-2020-overview-trec-2019-deep-learning/)
10. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies,… [find](https://scholar.google.com/scholar?q=Jacob+Devlin%2C+Ming%2DWei+Chang%2C+Kenton+Lee%2C+and+Kristina+Toutanova%2E+2019%2E+BERT%3A+Pre%2Dtraining+of+deep+bidirectional+transformers+for+language+understanding%2E+In+Proceedings+of+the+2019+Conference+of+the+N)
11. Luyu Gao and Jamie Callan. 2021. Condenser: a pre-training architecture for dense retrieval. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 981–993, Online and Punta Cana, Dominican Republic. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Luyu+Gao+and+Jamie+Callan%2E+2021%2E+Condenser%3A+a+pre%2Dtraining+architecture+for+dense+retrieval%2E+In+Proceedings+of+the+2021+Conference+on+Empirical+Methods+in+Natural+Language+Processing%2C+pages+981%E2%80%93993%2C+O) [our card](https://md.research.elamur.ai/free/en/papers/gao-2021-condenser-pre-training-architecture-dense/)
12. Luyu Gao and Jamie Callan. 2022. Unsupervised corpus aware language model pre-training for dense passage retrieval. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 2843–2853, Dublin, Ireland. Association for Computational Ling… [find](https://scholar.google.com/scholar?q=Luyu+Gao+and+Jamie+Callan%2E+2022%2E+Unsupervised+corpus+aware+language+model+pre%2Dtraining+for+dense+passage+retrieval%2E+In+Proceedings+of+the+60th+Annual+Meeting+of+the+Association+for+Computational+Lingu) [our card](https://md.research.elamur.ai/free/en/papers/gao-2022-unsupervised-corpus-aware-language-model/)
13. Tianyu Gao, Xingcheng Yao, and Danqi Chen. 2021. SimCSE: Simple contrastive learning of sentence embeddings. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 6894–6910, Online and Punta Cana, Dominican Republic. Association for Computational Linguistic… [find](https://scholar.google.com/scholar?q=Tianyu+Gao%2C+Xingcheng+Yao%2C+and+Danqi+Chen%2E+2021%2E+SimCSE%3A+Simple+contrastive+learning+of+sentence+embeddings%2E+In+Proceedings+of+the+2021+Conference+on+Empirical+Methods+in+Natural+Language+Processing%2C+) [our card](https://md.research.elamur.ai/free/en/papers/gao-2021-simcse-simple-contrastive-learning-sentence/)
14. Jordan Hoffmann, Sebastian Borgeaud, Arthur Mensch, Elena Buchatskaya, Trevor Cai, Eliza Rutherford, Diego de Las Casas, Lisa Anne Hendricks, Johannes Welbl, Aidan Clark, Tom Hennigan, Eric Noland, Katie Millican, George van den Driessche, Bogdan Damoc, Aurelia Guy, Simon Osindero, Karen Simonyan, E… [find](https://scholar.google.com/scholar?q=Jordan+Hoffmann%2C+Sebastian+Borgeaud%2C+Arthur+Mensch%2C+Elena+Buchatskaya%2C+Trevor+Cai%2C+Eliza+Rutherford%2C+Diego+de+Las+Casas%2C+Lisa+Anne+Hendricks%2C+Johannes+Welbl%2C+Aidan+Clark%2C+Tom+Hennigan%2C+Eric+Noland%2C+Ka)
15. Sebastian Hofstätter, Sheng-Chieh Lin, Jheng-Hong Yang, Jimmy Lin, and Allan Hanbury. 2021. Efficiently teaching an effective dense retriever with balanced topic aware sampling. In Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR… [find](https://scholar.google.com/scholar?q=Sebastian+Hofst%C3%A4tter%2C+Sheng%2DChieh+Lin%2C+Jheng%2DHong+Yang%2C+Jimmy+Lin%2C+and+Allan+Hanbury%2E+2021%2E+Efficiently+teaching+an+effective+dense+retriever+with+balanced+topic+aware+sampling%2E+In+Proceedings+of+the+) [our card](https://md.research.elamur.ai/free/en/papers/hofstatter-2021-efficiently-teaching-effective-dense-retriever/)
16. Gautier Izacard, Mathilde Caron, Lucas Hosseini, Sebastian Riedel, Piotr Bojanowski, Armand Joulin, and Edouard Grave. 2021. Towards unsupervised dense information retrieval with contrastive learning. CoRR, abs/2112.09118. [find](https://scholar.google.com/scholar?q=Gautier+Izacard%2C+Mathilde+Caron%2C+Lucas+Hosseini%2C+Sebastian+Riedel%2C+Piotr+Bojanowski%2C+Armand+Joulin%2C+and+Edouard+Grave%2E+2021%2E+Towards+unsupervised+dense+information+retrieval+with+contrastive+learning%2E) [our card](https://md.research.elamur.ai/free/en/papers/izacard-2021-unsupervised-dense-information-retrieval-contrastive/)
17. Jeff Johnson, Matthijs Douze, and Hervé Jégou. 2017. Billion-scale similarity search with gpus. CoRR, abs/1702.08734. [find](https://scholar.google.com/scholar?q=Jeff+Johnson%2C+Matthijs+Douze%2C+and+Herv%C3%A9+J%C3%A9gou%2E+2017%2E+Billion%2Dscale+similarity+search+with+gpus%2E+CoRR%2C+abs%2F1702%2E08734%2E) [our card](https://md.research.elamur.ai/free/en/papers/johnson-2017-billion-scale-similarity-search-gpus/)
18. Vladimir Karpukhin, Barlas Oguz, Sewon Min, Patrick Lewis, Ledell Wu, Sergey Edunov, Danqi Chen, and Wen-tau Yih. 2020. Dense passage retrieval for open-domain question answering. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 6769– 6781, Onl… [find](https://scholar.google.com/scholar?q=Vladimir+Karpukhin%2C+Barlas+Oguz%2C+Sewon+Min%2C+Patrick+Lewis%2C+Ledell+Wu%2C+Sergey+Edunov%2C+Danqi+Chen%2C+and+Wen%2Dtau+Yih%2E+2020%2E+Dense+passage+retrieval+for+open%2Ddomain+question+answering%2E+In+Proceedings+of+th) [our card](https://md.research.elamur.ai/free/en/papers/karpukhin-2020-dense-passage-retrieval-open-domain/)
19. Hyunji Lee, Sohee Yang, Hanseok Oh, and Minjoon Seo. 2022. Generative multi-hop retrieval. [find](https://scholar.google.com/scholar?q=Hyunji+Lee%2C+Sohee+Yang%2C+Hanseok+Oh%2C+and+Minjoon+Seo%2E+2022%2E+Generative+multi%2Dhop+retrieval%2E)
20. Kenton Lee, Ming-Wei Chang, and Kristina Toutanova. 2019. Latent retrieval for weakly supervised open domain question answering. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 6086–6096, Florence, Italy. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Kenton+Lee%2C+Ming%2DWei+Chang%2C+and+Kristina+Toutanova%2E+2019%2E+Latent+retrieval+for+weakly+supervised+open+domain+question+answering%2E+In+Proceedings+of+the+57th+Annual+Meeting+of+the+Association+for+Comput) [our card](https://md.research.elamur.ai/free/en/papers/lee-2019-latent-retrieval-weakly-supervised-open/)
21. Jimmy Lin, Xueguang Ma, Sheng-Chieh Lin, Jheng-Hong Yang, Ronak Pradeep, and Rodrigo Nogueira. 2021a. Pyserini: A Python toolkit for reproducible information retrieval research with sparse and dense representations. In Proceedings of the 44th Annual International ACM SIGIR Conference on Research and… [find](https://scholar.google.com/scholar?q=Jimmy+Lin%2C+Xueguang+Ma%2C+Sheng%2DChieh+Lin%2C+Jheng%2DHong+Yang%2C+Ronak+Pradeep%2C+and+Rodrigo+Nogueira%2E+2021a%2E+Pyserini%3A+A+Python+toolkit+for+reproducible+information+retrieval+research+with+sparse+and+dense+r)
22. Sheng-Chieh Lin, Jheng-Hong Yang, and Jimmy Lin. 2021b. In-batch negatives for knowledge distillation with tightly-coupled teachers for dense retrieval. In Proceedings of the 6th Workshop on Representation Learning for NLP (RepL4NLP-2021), pages 163– 173, Online. Association for Computational Lingui… [find](https://scholar.google.com/scholar?q=Sheng%2DChieh+Lin%2C+Jheng%2DHong+Yang%2C+and+Jimmy+Lin%2E+2021b%2E+In%2Dbatch+negatives+for+knowledge+distillation+with+tightly%2Dcoupled+teachers+for+dense+retrieval%2E+In+Proceedings+of+the+6th+Workshop+on+Represent)
23. Zheng Liu and Yingxia Shao. 2022. Retromae: Pre-training retrieval-oriented transformers via masked auto-encoder. ArXiv, abs/2205.12035. — [arXiv:2205.12035](https://arxiv.org/abs/2205.12035) [find](https://scholar.google.com/scholar?q=Zheng+Liu+and+Yingxia+Shao%2E+2022%2E+Retromae%3A+Pre%2Dtraining+retrieval%2Doriented+transformers+via+masked+auto%2Dencoder%2E+ArXiv%2C+abs%2F2205%2E12035%2E+%E2%80%94+%5BarXiv%3A2205%2E12035%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2205%2E12035%29)
24. Shuqi Lu, Di He, Chenyan Xiong, Guolin Ke, Waleed Malik, Zhicheng Dou, Paul Bennett, Tie-Yan Liu, and Arnold Overwijk. 2021. Less is more: Pre-train a strong Siamese encoder for dense text retrieval using a weak decoder. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language… [find](https://scholar.google.com/scholar?q=Shuqi+Lu%2C+Di+He%2C+Chenyan+Xiong%2C+Guolin+Ke%2C+Waleed+Malik%2C+Zhicheng+Dou%2C+Paul+Bennett%2C+Tie%2DYan+Liu%2C+and+Arnold+Overwijk%2E+2021%2E+Less+is+more%3A+Pre%2Dtrain+a+strong+Siamese+encoder+for+dense+text+retrieval+u)
25. Donald Metzler, Yi Tay, Dara Bahri, and Marc Najork. 2021. Rethinking search: making domain experts out of dilettantes. SIGIR Forum, 55(1):13:1–13:27. [find](https://scholar.google.com/scholar?q=Donald+Metzler%2C+Yi+Tay%2C+Dara+Bahri%2C+and+Marc+Najork%2E+2021%2E+Rethinking+search%3A+making+domain+experts+out+of+dilettantes%2E+SIGIR+Forum%2C+55%281%29%3A13%3A1%E2%80%9313%3A27%2E)
26. Sewon Min, Mike Lewis, Luke Zettlemoyer, and Hannaneh Hajishirzi. 2022. MetaICL: Learning to learn in context. In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 2791–2809, Seattle, United States. A… [find](https://scholar.google.com/scholar?q=Sewon+Min%2C+Mike+Lewis%2C+Luke+Zettlemoyer%2C+and+Hannaneh+Hajishirzi%2E+2022%2E+MetaICL%3A+Learning+to+learn+in+context%2E+In+Proceedings+of+the+2022+Conference+of+the+North+American+Chapter+of+the+Association+fo)
27. Long Ouyang, Jeff Wu, Xu Jiang, Diogo Almeida, Carroll L. Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, John Schulman, Jacob Hilton, Fraser Kelton, Luke Miller, Maddie Simens, Amanda Askell, Peter Welinder, Paul Christiano, Jan Leike, and Ryan Lowe. 2022. Train… [find](https://scholar.google.com/scholar?q=Long+Ouyang%2C+Jeff+Wu%2C+Xu+Jiang%2C+Diogo+Almeida%2C+Carroll+L%2E+Wainwright%2C+Pamela+Mishkin%2C+Chong+Zhang%2C+Sandhini+Agarwal%2C+Katarina+Slama%2C+Alex+Ray%2C+John+Schulman%2C+Jacob+Hilton%2C+Fraser+Kelton%2C+Luke+Miller%2C+)
28. Yingqi Qu, Yuchen Ding, Jing Liu, Kai Liu, Ruiyang Ren, Wayne Xin Zhao, Daxiang Dong, Hua Wu, and Haifeng Wang. 2021. RocketQA: An optimized training approach to dense passage retrieval for open-domain question answering. In Proceedings of the 2021 Conference of the North American Chapter of the Ass… [find](https://scholar.google.com/scholar?q=Yingqi+Qu%2C+Yuchen+Ding%2C+Jing+Liu%2C+Kai+Liu%2C+Ruiyang+Ren%2C+Wayne+Xin+Zhao%2C+Daxiang+Dong%2C+Hua+Wu%2C+and+Haifeng+Wang%2E+2021%2E+RocketQA%3A+An+optimized+training+approach+to+dense+passage+retrieval+for+open%2Ddomai) [our card](https://md.research.elamur.ai/free/en/papers/qu-2021-rocketqa-optimized-training-approach-dense/)
29. Jack W. Rae, Sebastian Borgeaud, Trevor Cai, Katie Millican, Jordan Hoffmann, Francis Song, John Aslanides, Sarah Henderson, Roman Ring, Susannah Young, Eliza Rutherford, Tom Hennigan, Jacob Menick, Albin Cassirer, Richard Powell, George van den Driessche, Lisa Anne Hendricks, Maribeth Rauh, Po-Sen… [find](https://scholar.google.com/scholar?q=Jack+W%2E+Rae%2C+Sebastian+Borgeaud%2C+Trevor+Cai%2C+Katie+Millican%2C+Jordan+Hoffmann%2C+Francis+Song%2C+John+Aslanides%2C+Sarah+Henderson%2C+Roman+Ring%2C+Susannah+Young%2C+Eliza+Rutherford%2C+Tom+Hennigan%2C+Jacob+Menick%2C+A)
30. Devendra Singh Sachan, Mike Lewis, Mandar Joshi, Armen Aghajanyan, Wen-tau Yih, Joelle Pineau, and Luke Zettlemoyer. 2022. Improving passage retrieval with zero-shot question generation. [find](https://scholar.google.com/scholar?q=Devendra+Singh+Sachan%2C+Mike+Lewis%2C+Mandar+Joshi%2C+Armen+Aghajanyan%2C+Wen%2Dtau+Yih%2C+Joelle+Pineau%2C+and+Luke+Zettlemoyer%2E+2022%2E+Improving+passage+retrieval+with+zero%2Dshot+question+generation%2E) [our card](https://md.research.elamur.ai/free/en/papers/sachan-2022-improving-passage-retrieval-zero-shot/)
31. Victor Sanh, Albert Webson, Colin Raffel, Stephen Bach, Lintang Sutawika, Zaid Alyafeai, Antoine Chaffin, Arnaud Stiegler, Arun Raja, Manan Dey, M Saiful Bari, Canwen Xu, Urmish Thakker, Shanya Sharma Sharma, Eliza Szczechla, Taewoon Kim, Gunjan Chhablani, Nihal V. Nayak, Debajyoti Datta, Jonathan C… [find](https://scholar.google.com/scholar?q=Victor+Sanh%2C+Albert+Webson%2C+Colin+Raffel%2C+Stephen+Bach%2C+Lintang+Sutawika%2C+Zaid+Alyafeai%2C+Antoine+Chaffin%2C+Arnaud+Stiegler%2C+Arun+Raja%2C+Manan+Dey%2C+M+Saiful+Bari%2C+Canwen+Xu%2C+Urmish+Thakker%2C+Shanya+Sharma) [our card](https://md.research.elamur.ai/free/en/papers/sanh-2021-multitask-prompted-training-enables-zero/)
32. Yi Tay, Vinh Q. Tran, Mostafa Dehghani, Jianmo Ni, Dara Bahri, Harsh Mehta, Zhen Qin, Kai Hui, Zhe Zhao, Jai Prakash Gupta, Tal Schuster, William W. Cohen, and Donald Metzler. 2022. Transformer memory as a differentiable search index. CoRR, abs/2202.06991. [find](https://scholar.google.com/scholar?q=Yi+Tay%2C+Vinh+Q%2E+Tran%2C+Mostafa+Dehghani%2C+Jianmo+Ni%2C+Dara+Bahri%2C+Harsh+Mehta%2C+Zhen+Qin%2C+Kai+Hui%2C+Zhe+Zhao%2C+Jai+Prakash+Gupta%2C+Tal+Schuster%2C+William+W%2E+Cohen%2C+and+Donald+Metzler%2E+2022%2E+Transformer+memory) [our card](https://md.research.elamur.ai/free/en/papers/tay-2022-transformer-memory-differentiable-search-index/)
33. Nandan Thakur, Nils Reimers, Andreas Rücklé, Abhishek Srivastava, and Iryna Gurevych. 2021. BEIR: A heterogenous benchmark for zero-shot evaluation of information retrieval models. CoRR, abs/2104.08663. [find](https://scholar.google.com/scholar?q=Nandan+Thakur%2C+Nils+Reimers%2C+Andreas+R%C3%BCckl%C3%A9%2C+Abhishek+Srivastava%2C+and+Iryna+Gurevych%2E+2021%2E+BEIR%3A+A+heterogenous+benchmark+for+zero%2Dshot+evaluation+of+information+retrieval+models%2E+CoRR%2C+abs%2F2104%2E0866) [our card](https://md.research.elamur.ai/free/en/papers/thakur-2021-beir-heterogenous-benchmark-zero/)
34. Romal Thoppilan, Daniel De Freitas, Jamie Hall, Noam Shazeer, Apoorv Kulshreshtha, Heng-Tze Cheng, Alicia Jin, Taylor Bos, Leslie Baker, Yu Du, YaGuang Li, Hongrae Lee, Huaixiu Steven Zheng, Amin Ghafouri, Marcelo Menegali, Yanping Huang, Maxim Krikun, Dmitry Lepikhin, James Qin, Dehao Chen, Yuanzho… [find](https://scholar.google.com/scholar?q=Romal+Thoppilan%2C+Daniel+De+Freitas%2C+Jamie+Hall%2C+Noam+Shazeer%2C+Apoorv+Kulshreshtha%2C+Heng%2DTze+Cheng%2C+Alicia+Jin%2C+Taylor+Bos%2C+Leslie+Baker%2C+Yu+Du%2C+YaGuang+Li%2C+Hongrae+Lee%2C+Huaixiu+Steven+Zheng%2C+Amin+Ghaf) [our card](https://md.research.elamur.ai/free/en/papers/thoppilan-2022-lamda-language-models-dialog-applications/)
35. Kexin Wang, Nandan Thakur, Nils Reimers, and Iryna Gurevych. 2022. GPL: Generative pseudo labeling for unsupervised domain adaptation of dense retrieval. In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies,… [find](https://scholar.google.com/scholar?q=Kexin+Wang%2C+Nandan+Thakur%2C+Nils+Reimers%2C+and+Iryna+Gurevych%2E+2022%2E+GPL%3A+Generative+pseudo+labeling+for+unsupervised+domain+adaptation+of+dense+retrieval%2E+In+Proceedings+of+the+2022+Conference+of+the+N) [our card](https://md.research.elamur.ai/free/en/papers/wang-2022-gpl-generative-pseudo-labeling-unsupervised/)
36. Jason Wei, Maarten Bosma, Vincent Y. Zhao, Kelvin Guu, Adams Wei Yu, Brian Lester, Nan Du, Andrew M. Dai, and Quoc V. Le. 2022. Finetuned language models are zero-shot learners. In The Tenth International Conference on Learning Representations, ICLR 2022, Virtual Event, April 25-29, 2022. OpenReview… [find](https://scholar.google.com/scholar?q=Jason+Wei%2C+Maarten+Bosma%2C+Vincent+Y%2E+Zhao%2C+Kelvin+Guu%2C+Adams+Wei+Yu%2C+Brian+Lester%2C+Nan+Du%2C+Andrew+M%2E+Dai%2C+and+Quoc+V%2E+Le%2E+2022%2E+Finetuned+language+models+are+zero%2Dshot+learners%2E+In+The+Tenth+Internati)
37. Lee Xiong, Chenyan Xiong, Ye Li, Kwok-Fung Tang, Jialin Liu, Paul N. Bennett, Junaid Ahmed, and Arnold Overwijk. 2021. Approximate nearest neighbor negative contrastive learning for dense text retrieval. In 9th International Conference on Learning Representations, ICLR 2021, Virtual Event, Austria,… [find](https://scholar.google.com/scholar?q=Lee+Xiong%2C+Chenyan+Xiong%2C+Ye+Li%2C+Kwok%2DFung+Tang%2C+Jialin+Liu%2C+Paul+N%2E+Bennett%2C+Junaid+Ahmed%2C+and+Arnold+Overwijk%2E+2021%2E+Approximate+nearest+neighbor+negative+contrastive+learning+for+dense+text+retriev) [our card](https://md.research.elamur.ai/free/en/papers/xiong-2020-approximate-nearest-neighbor-negative-contrastive/)
38. Yue Yu, Chenyan Xiong, Si Sun, Chao Zhang, and Arnold Overwijk. 2022. Coco-dr: Combating distribution shifts in zero-shot dense retrieval with contrastive and distributionally robust learning. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing. [find](https://scholar.google.com/scholar?q=Yue+Yu%2C+Chenyan+Xiong%2C+Si+Sun%2C+Chao+Zhang%2C+and+Arnold+Overwijk%2E+2022%2E+Coco%2Ddr%3A+Combating+distribution+shifts+in+zero%2Dshot+dense+retrieval+with+contrastive+and+distributionally+robust+learning%2E+In+Proc)
39. Xinyu Zhang, Xueguang Ma, Peng Shi, and Jimmy Lin. 2021. Mr. TyDi: A multi-lingual benchmark for dense retrieval. arXiv:2108.08787. [find](https://scholar.google.com/scholar?q=Xinyu+Zhang%2C+Xueguang+Ma%2C+Peng+Shi%2C+and+Jimmy+Lin%2E+2021%2E+Mr%2E+TyDi%3A+A+multi%2Dlingual+benchmark+for+dense+retrieval%2E+arXiv%3A2108%2E08787%2E)