---
title: "Questions Are All You Need to Train a Dense Passage Retriever"
description: "This paper presents ART (Autoencoding-based Retriever Training), an unsupervised alternative. 46 references, cited by 36 papers here."
authors: ["Devendra Singh Sachan", "Mike Lewis", "Dani Yogatama", "Luke Zettlemoyer", "Joelle Pineau", "Manzil Zaheer"]
year: 2023
venue: "Transactions of the Association for Computational Linguistics"
source: "https://aclanthology.org/2023.tacl-1.35/"
doi: "10.1162/tacl_a_00564"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/sachan-2023-questions-are-all-you-need/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/sachan-2023-questions-are-all-you-need/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2023](https://md.research.elamur.ai/free/en/papers/2023/page/10/) / Questions Are All You Need to Train a Dense Passage Retriever

# Questions Are All You Need to Train a Dense Passage Retriever

**Authors:** [Devendra Singh Sachan](https://md.research.elamur.ai/free/en/authors/devendra-sachan/), [Mike Lewis](https://md.research.elamur.ai/free/en/authors/mike-lewis/), [Dani Yogatama](https://md.research.elamur.ai/free/en/authors/dani-yogatama/), [Luke Zettlemoyer](https://md.research.elamur.ai/free/en/authors/luke-zettlemoyer/), [Joelle Pineau](https://md.research.elamur.ai/free/en/authors/joelle-pineau/) and [Manzil Zaheer](https://md.research.elamur.ai/free/en/authors/manzil-zaheer/) · **Year:** 2023 · **Venue:** [Transactions of the Association for Computational Linguistics](https://md.research.elamur.ai/free/en/venues/tacl/) ([Transactions of the Association for Computational Linguistics (TACL) 2023](https://md.research.elamur.ai/free/en/venues/tacl/2023/)) · **Publisher:** [The MIT Press](https://md.research.elamur.ai/free/en/publishers/the-mit-press/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2023.tacl-1.35/) · [PDF](https://aclanthology.org/2023.tacl-1.35.pdf) · [DOI 10.1162/tacl\_a\_00564](https://doi.org/10.1162/tacl_a_00564) · [OpenAlex](https://openalex.org/W4381686872)

**Citations:** 29 citations (OpenAlex, 2026-10-14) · [36](https://md.research.elamur.ai/free/en/papers/sachan-2023-questions-are-all-you-need/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
Dense passage retrievers for open-domain QA usually depend on large labeled sets of question–passage pairs, plus extra machinery such as hard-negative mining and cleaning of positive examples. This paper presents ART (Autoencoding-based Retriever Training), an unsupervised alternative. It needs only a pool of questions and an evidence collection, with no alignment between them. For each question, a dual-encoder retriever fetches a few candidate passages. A large frozen generative PLM then estimates how well each passage lets it regenerate the original question, using teacher-forcing and a simple instruction prompt. These reconstruction scores become soft labels. The retriever is trained to bring its own passage distribution close to them through a KL-divergence objective. On five QA retrieval benchmarks, ART matches or beats strongly supervised systems like DPR without seeing any gold pairs. It also transfers well to out-of-distribution data and is very sample-efficient. The work matters because it shows competitive dense retrieval can be built from raw questions alone. This is attractive for low-resource and multilingual settings.

## Key points
- **Training loop:**
  - Retrieve the top-K passages for a question.
  - Compute the retriever's softmax over those K passages (the student).
  - Score each passage with the frozen PLM's question likelihood (the teacher).
  - Minimize the KL divergence between teacher and student.
  - Rebuild the evidence embeddings every 500 steps to avoid stale indices.
- **QA results:** Using only training-set questions from WebQ, NQ-Open, SQuAD-Open, TriviaQA and EQ, ART beats DPR on average by about 5 points in top-20 accuracy and 4 points in top-100 accuracy.
- **Sample efficiency:** On NQ-Open, ART surpasses BM25 after only 100 questions and DPR after roughly 1,000 questions. Larger retriever models give consistent gains.
- **Noisy question sets:** Training on NQ-Full (307,373 questions, including unanswerable ones) still yields strong out-of-distribution generalization.
- **BEIR results (ART trained on MS MARCO questions):**
  - Average Recall@100 is 64.8, compared with BM25 at 63.6, Contriever at 60.1 and DPR at 48.2.
  - Average nDCG@10 is 40.4, roughly on par with BM25 (41.6).
- **Availability:** Code and checkpoints are released at github.com/DevSinghSachan/art.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [University of Washington](https://md.research.elamur.ai/free/en/universities/university-of-washington/) — University, Seattle, United States · 402 works · 212,762 citations
- [Meta](https://md.research.elamur.ai/free/en/companies/meta/) — Company, Menlo Park, United States · 214 works · 241,492 citations
- [McGill University](https://md.research.elamur.ai/free/en/universities/mcgill-university/) — University, Montreal, Canada · 91 works · 11,178 citations
- [Mila - Quebec Artificial Intelligence Institute](https://md.research.elamur.ai/free/en/labs/mila-quebec-artificial-intelligence-institute/) — Research institute, Montreal, Canada · 65 works · 14,425 citations

📚 Shelves (6): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 18,614 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 17,965 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 17,450 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 17,296 works · [👁️ Computer Vision and Pattern Recognition](https://md.research.elamur.ai/free/en/shelves/computer-science/computer-vision-and-pattern-recognition/) · 3,486 works · [🗄️ Information Systems](https://md.research.elamur.ai/free/en/shelves/computer-science/information-systems/) · 1,730 works

🧭 Topics (1): [🔍 Information retrieval & search](https://md.research.elamur.ai/free/en/topics/information-retrieval/) · 552 works

## Cite

Cite as: Devendra Singh Sachan, Mike Lewis, Dani Yogatama, Luke Zettlemoyer, Joelle Pineau and Manzil Zaheer. 2023. Questions Are All You Need to Train a Dense Passage Retriever. Transactions of the Association for Computational Linguistics.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@article{sachan2023questions,
  title = {{Questions Are All You Need to Train a Dense Passage Retriever}},
  author = {Devendra Singh Sachan, Mike Lewis, Dani Yogatama, Luke Zettlemoyer, Joelle Pineau and Manzil Zaheer},
  journal = {Transactions of the Association for Computational Linguistics},
  year = {2023},
  doi = {10.1162/tacl\_a\_00564},
  url = {https://doi.org/10.1162/tacl_a_00564}
}
```

- [← previous Online Speculative Decoding](https://md.research.elamur.ai/free/en/papers/liu-2023-online-speculative-decoding/)
- [next → Routing to the Expert: Efficient Reward-guided Ensemble of Large Language Models](https://md.research.elamur.ai/free/en/papers/lu-2023-routing-expert-efficient-reward-guided/)
- [↑ back to papers of 2023](https://md.research.elamur.ai/free/en/papers/2023/page/10/)

## References

46 references, 35 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Akari Asai, Jungo Kasai, Jonathan Clark, Kenton Lee, Eunsol Choi, and Hannaneh Hajishirzi. 2021. [XOR QA: Cross-lingual open-retrieval question answering](https://md.research.elamur.ai/free/en/papers/asai-2021-xor-qa-cross-lingual-open/). In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies.…
2. Payal Bajaj, Daniel Campos, Nick Craswell, Li Deng, Jianfeng Gao, Xiaodong Liu, Rangan Majumder, Andrew McNamara, Bhaskar Mitra, Tri Nguyen, et al. 2016. [MS MARCO: A human generated machine reading comprehension dataset](https://md.research.elamur.ai/free/en/papers/bajaj-2016-ms-marco-human-generated/). arXiv preprint arXiv:1611.09268. https://doi.org/10.48550/arXiv.1611 .09268 [↗](https://doi.org/10.48550/arXiv.1611)
3. Jonathan Berant, Andrew Chou, Roy Frostig, and Percy Liang. 2013. [Semantic parsing on Freebase from question-answer pairs](https://md.research.elamur.ai/free/en/papers/berant-2013-semantic-parsing-freebase-question-answer/). In Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing.
4. Aakanksha Chowdhery, Sharan Narang, Jacob Devlin, Maarten Bosma, Gaurav Mishra, Adam Roberts, Paul Barham, Hyung Won Chung, Charles Sutton, Sebastian Gehrmann, Parker Schuh, Kensen Shi, Sasha Tsvyashchenko, Joshua Maynez, Abhishek Rao, Parker Barnes, Yi Tay, Noam M. Shazeer, Vinodkumar Prabhakaran,… — [PaLM: Scaling Language Modeling with Pathways](https://md.research.elamur.ai/free/en/papers/chowdhery-2022-palm-scaling-language-modeling-pathways/)
5. Jonathan H. Clark, Eunsol Choi, Michael Collins, Dan Garrette, Tom Kwiatkowski, Vitaly Nikolaev, and Jennimaria Palomaki. 2020. [TyDi QA: A benchmark for information-seeking question answering in typologically diverse languages](https://md.research.elamur.ai/free/en/papers/clark-2020-tydi-qa-benchmark-information-seeking/). Transactions of the Association for Computational Linguistics, 8:454–470…
6. Zhuyun Dai, Arun Tejasvi Chaganty, Vincent Y. Zhao, Aida Amini, Qazi Mamunur Rashid, Mike Green, and Kelvin Guu. 2022. Dialog inpainting: Turning documents into dialogs. In Proceedings of the 39th International Conference on Machine Learning. question answering. In International Conference on Learni… [find](https://scholar.google.com/scholar?q=Zhuyun+Dai%2C+Arun+Tejasvi+Chaganty%2C+Vincent+Y%2E+Zhao%2C+Aida+Amini%2C+Qazi+Mamunur+Rashid%2C+Mike+Green%2C+and+Kelvin+Guu%2E+2022%2E+Dialog+inpainting%3A+Turning+documents+into+dialogs%2E+In+Proceedings+of+the+39th+Int)
7. Mandar Joshi, Eunsol Choi, Daniel Weld, and Luke Zettlemoyer. 2017. [TriviaQA: A large scale distantly supervised challenge dataset for reading comprehension](https://md.research.elamur.ai/free/en/papers/joshi-2017-triviaqa-large-scale-distantly-supervised/). In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). https://doi.org/10.18653/v…
8. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. [BERT: Pre-training of deep bidirectional transformers for language understanding](https://md.research.elamur.ai/free/en/papers/devlin-2019-bert-pre-training-deep-bidirectional/). In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies,…
9. Vladimir Karpukhin, Barlas Oğuz, Sewon Min, Ledell Wu, Sergey Edunov, Danqi Chen, and Wen-tau Yih. 2020. [Dense passage retrieval for open-domain question answering](https://md.research.elamur.ai/free/en/papers/karpukhin-2020-dense-passage-retrieval-open-domain/). In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP). https://doi.org/10.18653 /v1/2020.e…
10. Luyu Gao and Jamie Callan. 2022. [Unsupervised corpus aware language model pre-training for dense passage retrieval](https://md.research.elamur.ai/free/en/papers/gao-2022-unsupervised-corpus-aware-language-model/). In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). https://doi.org/10.18653/v1/2022.acl-long.203 [↗](https://doi.org/10.18653/v1/2022.acl-long.203)
11. Omar Khattab, Christopher Potts, and Matei Zaharia. 2021. [Relevance-guided supervision for openqa with colbert](https://md.research.elamur.ai/free/en/papers/khattab-2021-relevance-guided-supervision-openqa-colbert/). Transactions of the Association for Computational Linguistics, 9:929–944. https://doi.org/10.1162/tacl_a_00405 [↗](https://doi.org/10.1162/tacl_a_00405)
12. Daniel Gillick, Sayali Kulkarni, Larry Lansing, Alessandro Presta, Jason Baldridge, Eugene Ie, and Diego Garcia-Olano. 2019. [Learning dense representations for entity retrieval](https://md.research.elamur.ai/free/en/papers/gillick-2019-learning-dense-representations-entity-retrieval/). In Proceedings of the 23rd Conference on Computational Natural Language Learning (CoNLL). https://doi.org/10.18653/v1/K19… [↗](https://doi.org/10.18653/v1/K19…)
13. Omar Khattab and Matei Zaharia. 2020. [Colbert: Efficient and effective passage search via contextualized late interaction over BERT](https://md.research.elamur.ai/free/en/papers/khattab-2020-colbert-efficient-effective-passage/). In Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval. https://doi.org/10.1145/3397271.3401075 [↗](https://doi.org/10.1145/3397271.3401075)
14. Kelvin Guu, Kenton Lee, Zora Tung, Panupong Pasupat, and Mingwei Chang. 2020. [Retrieval augmented language model pre-training](https://md.research.elamur.ai/free/en/papers/guu-2020-realm-retrieval-augmented-language-model/). In Proceedings of the 37th International Conference on Machine Learning.
15. Tom Kwiatkowski, Jennimaria Palomaki, Olivia Redfield, Michael Collins, Ankur Parikh, Chris Alberti, Danielle Epstein, Illia Polosukhin, Matthew Kelcey, Jacob Devlin, Kenton Lee, Kristina N. Toutanova, Llion Jones, Ming-Wei Chang, Andrew Dai, Jakob Uszkoreit, Quoc Le, and Slav Petrov. 2019. [Natural…](https://md.research.elamur.ai/free/en/papers/kwiatkowski-2019-natural-questions-benchmark-question-answering/)
16. Samuel Humeau, Kurt Shuster, Marie-Anne Lachaux, and Jason Weston. 2020. Poly-encoders: Architectures and pre-training strategies for fast and accurate multi-sentence scoring. In International Conference on Learning Representations. [find](https://scholar.google.com/scholar?q=Samuel+Humeau%2C+Kurt+Shuster%2C+Marie%2DAnne+Lachaux%2C+and+Jason+Weston%2E+2020%2E+Poly%2Dencoders%3A+Architectures+and+pre%2Dtraining+strategies+for+fast+and+accurate+multi%2Dsentence+scoring%2E+In+International+Confere)
17. Gautier Izacard, Mathilde Caron, Lucas Hosseini, Sebastian Riedel, Piotr Bojanowski, Armand Joulin, and Edouard Grave. 2022. [Unsupervised dense information retrieval with contrastive learning](https://md.research.elamur.ai/free/en/papers/izacard-2021-unsupervised-dense-information-retrieval-contrastive/). Transactions on Machine Learning Research.
18. Kenton Lee, Ming-Wei Chang, and Kristina Toutanova. 2019. [Latent retrieval for weakly supervised open domain question answering](https://md.research.elamur.ai/free/en/papers/lee-2019-latent-retrieval-weakly-supervised-open/). In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. https://doi.org/10.18653/v1/P19 -1612 [↗](https://doi.org/10.18653/v1/P19)
19. Gautier Izacard and Edouard Grave. 2021. Distilling knowledge from reader to retriever for Brian Lester, Rami Al-Rfou, and Noah Constant. 2021. [The power of scale for parameter-efficient prompt tuning](https://md.research.elamur.ai/free/en/papers/lester-2021-power-scale-parameter-efficient-prompt/). In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing. https:/…
20. Arvind Neelakantan, Tao Xu, Raul Puri, Alec Radford, Jesse Michael Han, Jerry Tworek, Qiming Yuan, Nikolas Tezak, Jong Wook Kim, Chris Hallacy, et al. 2022. [Text and code embeddings by contrastive pre-training](https://md.research.elamur.ai/free/en/papers/neelakantan-2022-text-code-embeddings-contrastive-pre/). arXiv preprint arXiv:2201.10005. https://doi.org/10.48550/arXiv.2201.10005 [↗](https://doi.org/10.48550/arXiv.2201.10005)
21. Yujia Li, David Choi, Junyoung Chung, Nate Kushman, Julian Schrittwieser, Rémi Leblond, Tom Eccles, James Keeling, Felix Gimeno, Agustin Dal Lago, Thomas Hubert, Peter Choy, Cyprien de Masson d’Autume, Igor Babuschkin, Xinyun Chen, Po-Sen Huang, Johannes Welbl, Sven Gowal, Alexey Cherepanov, James… — [Competition-Level Code Generation with AlphaCode](https://md.research.elamur.ai/free/en/papers/li-2022-competition-level-code-generation-alphacode/)
22. Rodrigo Nogueira and Kyunghyun Cho. 2019. [Passage re-ranking with BERT](https://md.research.elamur.ai/free/en/papers/nogueira-2019-passage-re-ranking-bert/). arXiv preprint arXiv:1901.04085. https://doi.org/10 .48550/arXiv.1901.04085
23. Jimmy Lin, Rodrigo Nogueira, and Andrew Yates. 2021. Pretrained transformers for text ranking: BERT and beyond. Synthesis Lectures on Human Language Technologies, 14(4):1–325. https://doi.org/10.2200/S01123ED 1V01Y202108HLT053 [↗](https://doi.org/10.2200/S01123ED)
24. Erik Lindgren, Sashank J. Reddi, Ruiqi Guo, and Sanjiv Kumar. 2021. Efficient training of retrieval models using negative cache. In Advances in Neural Information Processing Systems. [find](https://scholar.google.com/scholar?q=Erik+Lindgren%2C+Sashank+J%2E+Reddi%2C+Ruiqi+Guo%2C+and+Sanjiv+Kumar%2E+2021%2E+Efficient+training+of+retrieval+models+using+negative+cache%2E+In+Advances+in+Neural+Information+Processing+Systems%2E)
25. Rodrigo Nogueira, Zhiying Jiang, Ronak Pradeep, and Jimmy Lin. 2020. [Document ranking with a pretrained sequence-to-sequence model](https://md.research.elamur.ai/free/en/papers/nogueira-2020-document-ranking-pretrained-sequence-sequence/). In Findings of the Association for Computational Linguistics: EMNLP 2020. https://doi.org/10.18653/v1/2020.findings-emnlp.63 [↗](https://doi.org/10.18653/v1/2020.findings-emnlp.63)
26. Cicero Nogueira dos Santos, Xiaofei Ma, Ramesh Nallapati, Zhiheng Huang, and Bing Xiang. 2020. Beyond [CLS] through ranking by generation. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP). https://doi.org/10 .18653/v1/2020.emnlp-main.134 [find](https://scholar.google.com/scholar?q=Cicero+Nogueira+dos+Santos%2C+Xiaofei+Ma%2C+Ramesh+Nallapati%2C+Zhiheng+Huang%2C+and+Bing+Xiang%2E+2020%2E+Beyond+%5BCLS%5D+through+ranking+by+generation%2E+In+Proceedings+of+the+2020+Conference+on+Empirical+Methods+in)
27. Aaron van den Oord, Yazhe Li, and Oriol Vinyals. 2018. [Representation learning with contrastive predictive coding](https://md.research.elamur.ai/free/en/papers/oord-2018-representation-learning-contrastive-predictive-coding/). arXiv preprint arXiv:1807.03748. https://doi.org/10 .48550/arXiv.1807.03748
28. Yi Luan, Jacob Eisenstein, Kristina Toutanova, and Michael Collins. 2021. [Sparse, dense, and attentional representations for text retrieval](https://md.research.elamur.ai/free/en/papers/luan-2021-sparse-dense-attentional-representations-text/). Transactions of the Association for Computational Linguistics, 9:329–345. https://doi.org/10.1162/tacl_a_00369 [↗](https://doi.org/10.1162/tacl_a_00369)
29. Yingqi Qu, Yuchen Ding, Jing Liu, Kai Liu, Ruiyang Ren, Wayne Xin Zhao, Daxiang Dong, Hua Wu, and Haifeng Wang. 2021. Rocket-QA: An optimized training approach to [dense passage retrieval for open-domain question answering](https://md.research.elamur.ai/free/en/papers/karpukhin-2020-dense-passage-retrieval-open-domain/). In Proceedings of the 2021 Conference of the North American Chapter of the As… — [doi:10.18653/v1/2021.naacl-main](https://doi.org/10.18653/v1/2021.naacl-main)
30. Ji Ma, Ivan Korotkov, Yinfei Yang, Keith Hall, and Ryan McDonald. 2021. [Zero-shot neural passage retrieval via domain-targeted synthetic question generation](https://md.research.elamur.ai/free/en/papers/ma-2021-zero-shot-neural-passage-retrieval/). In Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume. https://doi.org/1…
31. Yuning Mao, Pengcheng He, Xiaodong Liu, Yelong Shen, Jianfeng Gao, Jiawei Han, and Weizhu Chen. 2021. [Generation-augmented retrieval for open-domain question answering](https://md.research.elamur.ai/free/en/papers/mao-2020-generation-augmented-retrieval-open-domain/). In Proceedings of the 59th Annual Meeting of Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena…
32. Devendra Singh Sachan, Siva Reddy, William L. Hamilton, Chris Dyer, and Dani Yogatama. 2021b. [End-to-end training of multi-document reader and retriever for open-domain question answering](https://md.research.elamur.ai/free/en/papers/sachan-2021-end-end-training-multi-document/). In Advances in Neural Information Processing Systems.
33. Pranav Rajpurkar, Jian Zhang, Konstantin Lopyrev, and Percy Liang. 2016. [SQuAD: 100,000+ questions for machine comprehension of text](https://md.research.elamur.ai/free/en/papers/rajpurkar-2016-squad-100-000-questions-machine/). In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing. https://doi.org/10.18653/v1/D16-1264 [↗](https://doi.org/10.18653/v1/D16-1264)
34. Victor Sanh, Albert Webson, Colin Raffel, Stephen Bach, Lintang Sutawika, Zaid Alyafeai, Antoine Chaffin, Arnaud Stiegler, Arun Raja, Manan Dey, M Saiful Bari, Canwen Xu, Urmish Thakker, Shanya Sharma Sharma, Eliza Szczechla, Taewoon Kim, Gunjan Chhablani, Nihal Nayak, Debajyoti Datta, Jonathan Chan… — [Multitask Prompted Training Enables Zero-Shot Task Generalization](https://md.research.elamur.ai/free/en/papers/sanh-2021-multitask-prompted-training-enables-zero/)
35. Ori Ram, Gal Shachaf, Omer Levy, Jonathan Berant, and Amir Globerson. 2022. Learning to retrieve passages without supervision. In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. https://doi.org/10.18653 /… [find](https://scholar.google.com/scholar?q=Ori+Ram%2C+Gal+Shachaf%2C+Omer+Levy%2C+Jonathan+Berant%2C+and+Amir+Globerson%2E+2022%2E+Learning+to+retrieve+passages+without+supervision%2E+In+Proceedings+of+the+2022+Conference+of+the+North+American+Chapter+of+th)
36. Ruiyang Ren, Yingqi Qu, Jing Liu, Wayne Xin Zhao, QiaoQiao She, Hua Wu, Haifeng Wang, and Ji-Rong Wen. 2021. [RocketQAv2: A joint training method for dense passage retrieval and passage re-ranking](https://md.research.elamur.ai/free/en/papers/ren-2021-rocketqav2-joint-training-method-dense/). In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing. [https://doi.…](https://doi.…) [↗](https://doi.…)
37. Christopher Sciavolino, Zexuan Zhong, Jinhyuk Lee, and Danqi Chen. 2021. [Simple entity-centric questions challenge dense retrievers](https://md.research.elamur.ai/free/en/papers/sciavolino-2021-simple-entity-centric-questions-challenge/). In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing. https://doi.org/10.18653/v1/2021 .emnlp-main.496 [↗](https://doi.org/10.18653/v1/2021)
38. Stephen Robertson and Hugo Zaragoza. 2009. The probabilistic relevance framework: BM25 and beyond. Foundations and Trends in Information Retrieval. https://doi.org/10 .1561/1500000019 [find](https://scholar.google.com/scholar?q=Stephen+Robertson+and+Hugo+Zaragoza%2E+2009%2E+The+probabilistic+relevance+framework%3A+BM25+and+beyond%2E+Foundations+and+Trends+in+Information+Retrieval%2E+https%3A%2F%2Fdoi%2Eorg%2F10+%2E1561%2F1500000019)
39. Devendra Singh Sachan, Mike Lewis, Mandar Joshi, Armen Aghajanyan, Wen-tau Yih, Joelle Pineau, and Luke Zettlemoyer. 2022. [Improving passage retrieval with zero-shot question generation](https://md.research.elamur.ai/free/en/papers/sachan-2022-improving-passage-retrieval-zero-shot/). In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing. https://doi.org/10 .48…
40. Nandan Thakur, Nils Reimers, Andreas Rücklé, Abhishek Srivastava, and Iryna Gurevych. 2021. BEIR: A heterogeneous benchmark for zero-shot evaluation of information retrieval models. In Thirty-fifth Conference on Neural Information Processing Systems Datasets and Benchmarks Track (Round 2). [find](https://scholar.google.com/scholar?q=Nandan+Thakur%2C+Nils+Reimers%2C+Andreas+Ru%CC%88ckle%CC%81%2C+Abhishek+Srivastava%2C+and+Iryna+Gurevych%2E+2021%2E+BEIR%3A+A+heterogeneous+benchmark+for+zero%2Dshot+evaluation+of+information+retrieval+models%2E+In+Thirty%2Dfifth+)
41. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. [Attention is all you need](https://md.research.elamur.ai/free/en/papers/vaswani-2017-attention-all-need/). In Advances in Neural Information Processing Systems.
42. Devendra Singh Sachan, Mostofa Patwary, Mohammad Shoeybi, Neel Kant, Wei Ping, William L. Hamilton, and Bryan Catanzaro. 2021a. [End-to-end training of neural retrievers for open-domain question answering](https://md.research.elamur.ai/free/en/papers/sachan-2021-end-end-training-neural-retrievers/). In Joint Conference of the 59th Annual Meeting of the Association for Computational Linguistics…
43. Lee Xiong, Chenyan Xiong, Ye Li, Kwok-Fung Tang, Jialin Liu, Paul N. Bennett, Junaid Ahmed, and Arnold Overwijk. 2021. [Approximate nearest neighbor negative contrastive learning for dense text retrieval](https://md.research.elamur.ai/free/en/papers/xiong-2020-approximate-nearest-neighbor-negative-contrastive/). In International Conference on Learning Representations.
44. Hang Zhang, Yeyun Gong, Yelong Shen, Jiancheng Lv, Nan Duan, and Weizhu Chen. 2022. Adversarial retriever-ranker for dense text retrieval. In International Conference on Learning Representations. [find](https://scholar.google.com/scholar?q=Hang+Zhang%2C+Yeyun+Gong%2C+Yelong+Shen%2C+Jiancheng+Lv%2C+Nan+Duan%2C+and+Weizhu+Chen%2E+2022%2E+Adversarial+retriever%2Dranker+for+dense+text+retrieval%2E+In+International+Conference+on+Learning+Representations%2E)
45. Hao Jiang, Zhao Cao, Fan Yu, Xin Jiang, Qun Liu, and Lei Chen. 2022. Hyperlink-induced pre-training for passage retrieval in open-domain question answering. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). https://doi.org/10.18653 /v… [find](https://scholar.google.com/scholar?q=Hao+Jiang%2C+Zhao+Cao%2C+Fan+Yu%2C+Xin+Jiang%2C+Qun+Liu%2C+and+Lei+Chen%2E+2022%2E+Hyperlink%2Dinduced+pre%2Dtraining+for+passage+retrieval+in+open%2Ddomain+question+answering%2E+In+Proceedings+of+the+60th+Annual+Meeting+o)
46. Jiawei Zhou, Xiaoguang Li, Lifeng Shang, Lan Luo, Ke Zhan, Enrui Hu, Xinyu Zhang, [find](https://scholar.google.com/scholar?q=Jiawei+Zhou%2C+Xiaoguang+Li%2C+Lifeng+Shang%2C+Lan+Luo%2C+Ke+Zhan%2C+Enrui+Hu%2C+Xinyu+Zhang%2C)

*Updated 2 November 2026*