---
title: "How to Train Your Dragon — paper by Lin et al. (2023)"
description: "Dense retrievers (DR) of BERT-base size have usually forced a choice: models tuned for in-domain… 50 references, cited by 67 papers here."
authors: ["Sheng-Chieh Lin", "Akari Asai", "Minghan Li", "Barlas Oguz", "Jimmy Lin", "Yashar Mehdad", "Wen-tau Yih", "Xilun Chen"]
year: 2023
venue: "Findings of the Association for Computational Linguistics: EMNLP 2023"
source: "https://aclanthology.org/2023.findings-emnlp.423/"
doi: "10.18653/v1/2023.findings-emnlp.423"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/lin-2023-how-train-your-dragon-diverse/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/lin-2023-how-train-your-dragon-diverse/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2023](https://md.research.elamur.ai/free/en/papers/2023/page/6/) / How to Train Your Dragon: Diverse Augmentation Towards Generalizable Dense Retrieval

# How to Train Your Dragon: Diverse Augmentation Towards Generalizable Dense Retrieval

**Authors:** [Sheng-Chieh Lin](https://md.research.elamur.ai/free/en/authors/sheng-chieh-lin/), [Akari Asai](https://md.research.elamur.ai/free/en/authors/akari-asai/), Minghan Li, [Barlas Oguz](https://md.research.elamur.ai/free/en/authors/barlas-oguz/), [Jimmy Lin](https://md.research.elamur.ai/free/en/authors/jimmy-lin/), [Yashar Mehdad](https://md.research.elamur.ai/free/en/authors/yashar-mehdad/), [Wen-tau Yih](https://md.research.elamur.ai/free/en/authors/wen-tau-yih/) and [Xilun Chen](https://md.research.elamur.ai/free/en/authors/xilun-chen/) · **Year:** 2023 · **Venue:** [Findings of the Association for Computational Linguistics: EMNLP 2023](https://md.research.elamur.ai/free/en/venues/findings/) ([Findings of the Association for Computational Linguistics 2023](https://md.research.elamur.ai/free/en/venues/findings/2023/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2023.findings-emnlp.423/) · [PDF](https://aclanthology.org/2023.findings-emnlp.423.pdf) · [DOI 10.18653/v1/2023.findings-emnlp.423](https://doi.org/10.18653/v1/2023.findings-emnlp.423) · [OpenAlex](https://openalex.org/W4389520055)

**Citations:** 48 citations (OpenAlex, 2026-10-14) · [67](https://md.research.elamur.ai/free/en/papers/lin-2023-how-train-your-dragon-diverse/cited-by/) in our library, of the 141,840 papers whose bibliographies we read

## Summary
Dense retrievers (DR) of BERT-base size have usually forced a choice: models tuned for in-domain supervised retrieval on MS MARCO tend to transfer poorly to new domains, and the reverse also holds. Some researchers attributed this to limited model capacity. The authors dispute that explanation. They treat recent DR training tricks, such as distillation, contrastive pre-training and pseudo-query generation, as variants of data augmentation (DA) applied to queries and relevance labels. Their pilot studies show two things. First, labels drawn from several diverse retrievers work better than labels from a single strong cross-encoder. Second, cheap cropped sentences can serve as training queries as well as costly generated ones. The resulting recipe, DRAGON, combines 28M augmented queries (cropped sentences plus doct5query outputs) with relevance labels added progressively from sparse, dense and multi-vector teachers. DRAGON is the first 110M-parameter DR to reach state-of-the-art results in both settings, matching far larger models and late-interaction systems. This suggests better training data, not more parameters, is the key lever for general-purpose retrievers.

## Key points
- **Evaluation setup:** supervised results are measured on MS MARCO Dev (6,980 queries; RR@10, R@1K) using the 8.8M-passage corpus. Zero-shot results are the average nDCG@10 over BEIR-13.
- **Query augmentation:** about 28M sentences were cropped from the MS MARCO corpus and 28M synthetic queries were drawn from doct5query. The two types are mixed for training.
- **Progressive label augmentation:** five teacher retrievers of mixed types (sparse, dense, multi-vector) are used. Feeding all teachers' labels at once, or merging their scores into soft labels, gave worse results.
- **Why cropping helps:** cropped sentences cover more varied topics and cause more disagreement among teachers. This exposes the student to richer relevance signals, so DRAGON-S generalizes better than DRAGON-Q.
- **Cost and availability:** training takes roughly 5 days on 32 A100 (40 GB) GPUs and requires already-trained teacher models. Code and checkpoints are released in facebookresearch/dpr-scale.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [University of Washington](https://md.research.elamur.ai/free/en/universities/university-of-washington/) — University, Seattle, United States · 263 works · 180,581 citations
- [University of Waterloo](https://md.research.elamur.ai/free/en/universities/university-of-waterloo/) — University, Waterloo, Canada · 44 works · 12,783 citations

📚 Shelves (6): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 9,086 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 8,889 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 9,296 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 9,206 works · [👁️ Computer Vision and Pattern Recognition](https://md.research.elamur.ai/free/en/shelves/computer-science/computer-vision-and-pattern-recognition/) · 1,282 works · [🗄️ Information Systems](https://md.research.elamur.ai/free/en/shelves/computer-science/information-systems/) · 523 works

🧭 Topics (1): [🔍 Information retrieval & search](https://md.research.elamur.ai/free/en/topics/information-retrieval/) · 145 works

## Cite

Cite as: Sheng-Chieh Lin, Akari Asai, Minghan Li, Barlas Oguz, Jimmy Lin, Yashar Mehdad, Wen-tau Yih and Xilun Chen. 2023. How to Train Your Dragon: Diverse Augmentation Towards Generalizable Dense Retrieval. Findings of the Association for Computational Linguistics: EMNLP 2023.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{lin2023how,
  title = {{How to Train Your Dragon: Diverse Augmentation Towards Generalizable Dense Retrieval}},
  author = {Sheng-Chieh Lin, Akari Asai, Minghan Li, Barlas Oguz, Jimmy Lin, Yashar Mehdad, Wen-tau Yih and Xilun Chen},
  booktitle = {Findings of the Association for Computational Linguistics: EMNLP 2023},
  year = {2023},
  doi = {10.18653/v1/2023.findings-emnlp.423},
  url = {https://doi.org/10.18653/v1/2023.findings-emnlp.423}
}
```

- [← previous GlotLID: Language Identification for Low-Resource Languages](https://md.research.elamur.ai/free/en/papers/kargaran-2023-glotlid-language-identification-low-resource/)
- [next → GoLLIE: Annotation Guidelines improve Zero-Shot Information-Extraction](https://md.research.elamur.ai/free/en/papers/sainz-2023-gollie-annotation-guidelines-improve-zero/)
- [↑ back to papers of 2023](https://md.research.elamur.ai/free/en/papers/2023/page/6/)

## References

50 references, 23 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Akari Asai, Timo Schick, Patrick Lewis, Xilun Chen, Gautier Izacard, Sebastian Riedel, Hannaneh Hajishirzi, and Wen-tau Yih. 2023. [Task-aware retrieval with instructions](https://md.research.elamur.ai/free/en/papers/asai-2023-task-aware-retrieval-instructions/). In Proc. Findings of ACL, pages 3650–3675.
2. Payal Bajaj, Daniel Campos, Nick Craswell, Li Deng, Jianfeng Gao, Xiaodong Liu, Rangan Majumder, Andrew McNamara, Bhaskar Mitra, Tri Nguyen, et al. 2016. [MS MARCO: A human generated machine reading comprehension dataset](https://md.research.elamur.ai/free/en/papers/bajaj-2016-ms-marco-human-generated/). arXiv:1611.09268.
3. Yoshua Bengio, Jérôme Louradour, Ronan Collobert, and Jason Weston. 2009. [Curriculum learning](https://md.research.elamur.ai/free/en/papers/bengio-2009-curriculum-learning/). In Proc. ICML, page 41–48.
4. Wei-Cheng Chang, Felix X. Yu, Yin-Wen Chang, Yiming Yang, and Sanjiv Kumar. 2020. Pre-training tasks for embedding-based large-scale retrieval. In Proc. ICLR. [find](https://scholar.google.com/scholar?q=Wei%2DCheng+Chang%2C+Felix+X%2E+Yu%2C+Yin%2DWen+Chang%2C+Yiming+Yang%2C+and+Sanjiv+Kumar%2E+2020%2E+Pre%2Dtraining+tasks+for+embedding%2Dbased+large%2Dscale+retrieval%2E+In+Proc%2E+ICLR%2E)
5. Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey Hinton. 2020. [A simple framework for contrastive learning of visual representations](https://md.research.elamur.ai/free/en/papers/chen-2020-simple-framework-contrastive-learning-visual/). In Proc. ICML.
6. Xilun Chen, Kushal Lakhotia, Barlas Oguz, Anchit Gupta, Patrick Lewis, Stan Peshterliev, Yashar Mehdad, Sonal Gupta, and Wen-tau Yih. 2022. Salient phrase aware dense retrieval: Can a dense retriever imitate a sparse one? In Proc. Findings of EMNLP, pages 250–262. [find](https://scholar.google.com/scholar?q=Xilun+Chen%2C+Kushal+Lakhotia%2C+Barlas+Oguz%2C+Anchit+Gupta%2C+Patrick+Lewis%2C+Stan+Peshterliev%2C+Yashar+Mehdad%2C+Sonal+Gupta%2C+and+Wen%2Dtau+Yih%2E+2022%2E+Salient+phrase+aware+dense+retrieval%3A+Can+a+dense+retriever+)
7. Zhuyun Dai, Vincent Y. Zhao, Ji Ma, Yi Luan, Jianmo Ni, Jing Lu, Anton Bakalov, Kelvin Guu, Keith B. Hall, and Ming-Wei Chang. 2022. [Promptagator: Few-shot dense retrieval from 8 examples](https://md.research.elamur.ai/free/en/papers/dai-2022-promptagator-few-shot-dense-retrieval/). arXiv:2209.11755.
8. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018. [BERT: Pre-training of deep bidirectional transformers for language understanding](https://md.research.elamur.ai/free/en/papers/devlin-2019-bert-pre-training-deep-bidirectional/). arXiv:1810.04805.
9. Thibault Formal, Carlos Lassance, Benjamin Piwowarski, and Stéphane Clinchant. 2022. From distillation to hard negative sampling: Making sparse neural IR models more effective. In Proc. SIGIR, page 2353–2359. [find](https://scholar.google.com/scholar?q=Thibault+Formal%2C+Carlos+Lassance%2C+Benjamin+Piwowarski%2C+and+St%C3%A9phane+Clinchant%2E+2022%2E+From+distillation+to+hard+negative+sampling%3A+Making+sparse+neural+IR+models+more+effective%2E+In+Proc%2E+SIGIR%2C+page+23)
10. Thibault Formal, Carlos Lassance, Benjamin Piwowarski, and Stéphane Clinchant. 2021. SPLADE v2: Sparse lexical and expansion model for information retrieval. arXiv:2109.10086. [find](https://scholar.google.com/scholar?q=Thibault+Formal%2C+Carlos+Lassance%2C+Benjamin+Piwowarski%2C+and+St%C3%A9phane+Clinchant%2E+2021%2E+SPLADE+v2%3A+Sparse+lexical+and+expansion+model+for+information+retrieval%2E+arXiv%3A2109%2E10086%2E)
11. Luyu Gao and Jamie Callan. 2021. [Condenser: a pre-training architecture for dense retrieval](https://md.research.elamur.ai/free/en/papers/gao-2021-condenser-pre-training-architecture-dense/). In Proc. EMNLP, pages 981–993.
12. Luyu Gao and Jamie Callan. 2022. [Unsupervised corpus aware language model pre-training for dense passage retrieval](https://md.research.elamur.ai/free/en/papers/gao-2022-unsupervised-corpus-aware-language-model/). In Proc. ACL, pages 2843–2853.
13. Ruiqi Guo, Philip Sun, Erik Lindgren, Quan Geng, David Simcha, Felix Chern, and Sanjiv Kumar. 2020. Accelerating large-scale inference with anisotropic vector quantization. In Proc. ICML. [find](https://scholar.google.com/scholar?q=Ruiqi+Guo%2C+Philip+Sun%2C+Erik+Lindgren%2C+Quan+Geng%2C+David+Simcha%2C+Felix+Chern%2C+and+Sanjiv+Kumar%2E+2020%2E+Accelerating+large%2Dscale+inference+with+anisotropic+vector+quantization%2E+In+Proc%2E+ICML%2E)
14. Geoffrey Hinton, Oriol Vinyals, and Jeffrey Dean. 2015. [Distilling the knowledge in a neural network](https://md.research.elamur.ai/free/en/papers/hinton-2015-distilling-knowledge-neural-network/). In Proc. NIPS.
15. Sebastian Hofstätter, Sheng-Chieh Lin, Jheng-Hong Yang, Jimmy Lin, and Allan Hanbury. 2021. [Efficiently teaching an effective dense retriever with balanced topic aware sampling](https://md.research.elamur.ai/free/en/papers/hofstatter-2021-efficiently-teaching-effective-dense-retriever/). In Proc. SIGIR, page 113–122.
16. Sebastian Hofstätter, Sophia Althammer, Michael Schröder, Mete Sertkan, and Allan Hanbury. 2020. Improving efficient neural ranking models with cross-architecture knowledge distillation. arXiv:2010.02666. — [arXiv:2010.02666](https://arxiv.org/abs/2010.02666) [find](https://scholar.google.com/scholar?q=Sebastian+Hofst%C3%A4tter%2C+Sophia+Althammer%2C+Michael+Schr%C3%B6der%2C+Mete+Sertkan%2C+and+Allan+Hanbury%2E+2020%2E+Improving+efficient+neural+ranking+models+with+cross%2Darchitecture+knowledge+distillation%2E+arXiv%3A2010%2E02)
17. Gautier Izacard, Mathilde Caron, Lucas Hosseini, Sebastian Riedel, Piotr Bojanowski, Armand Joulin, and Edouard Grave. 2021. [Unsupervised dense information retrieval with contrastive learning](https://md.research.elamur.ai/free/en/papers/izacard-2021-unsupervised-dense-information-retrieval-contrastive/). arXiv:2112.09118.
18. Nick Craswell, Bhaskar Mitra, and Daniel Campos. 2019. [Overview of the TREC 2019 deep learning track](https://md.research.elamur.ai/free/en/papers/craswell-2020-overview-trec-2019-deep-learning/). In Proc. TREC.
19. Gautier Izacard, Patrick Lewis, Maria Lomeli, Lucas Hosseini, Fabio Petroni, Timo Schick, Jane Dwivedi-Yu, Armand Joulin, Sebastian Riedel, and Edouard Grave. 2022. Few-shot Learning with Retrieval Augmented Language Models. arXiv:2208.03299. [find](https://scholar.google.com/scholar?q=Gautier+Izacard%2C+Patrick+Lewis%2C+Maria+Lomeli%2C+Lucas+Hosseini%2C+Fabio+Petroni%2C+Timo+Schick%2C+Jane+Dwivedi%2DYu%2C+Armand+Joulin%2C+Sebastian+Riedel%2C+and+Edouard+Grave%2E+2022%2E+Few%2Dshot+Learning+with+Retrieval+Au)
20. Nick Craswell, Bhaskar Mitra, Emine Yilmaz, and Daniel Campos. 2020. Overview of the TREC 2020 deep learning track. In Proc. TREC. [find](https://scholar.google.com/scholar?q=Nick+Craswell%2C+Bhaskar+Mitra%2C+Emine+Yilmaz%2C+and+Daniel+Campos%2E+2020%2E+Overview+of+the+TREC+2020+deep+learning+track%2E+In+Proc%2E+TREC%2E)
21. Jeff Johnson, Matthijs Douze, and Hervé Jégou. 2021. Billion-scale similarity search with GPUs. IEEE Transactions on Big Data, pages 535–547. [find](https://scholar.google.com/scholar?q=Jeff+Johnson%2C+Matthijs+Douze%2C+and+Herv%C3%A9+J%C3%A9gou%2E+2021%2E+Billion%2Dscale+similarity+search+with+GPUs%2E+IEEE+Transactions+on+Big+Data%2C+pages+535%E2%80%93547%2E)
22. Vladimir Karpukhin, Barlas Oguz, Sewon Min, Patrick Lewis, Ledell Wu, Sergey Edunov, Danqi Chen, and Wen-tau Yih. 2020. [Dense passage retrieval for open-domain question answering](https://md.research.elamur.ai/free/en/papers/karpukhin-2020-dense-passage-retrieval-open-domain/). In Proc. EMNLP, pages 6769–6781.
23. Carlos Lassance and Stephane Clinchant. 2023. The tale of two MSMARCO - and their unfair comparisons. In Proc. SIGIR, page 2431–2435. [find](https://scholar.google.com/scholar?q=Carlos+Lassance+and+Stephane+Clinchant%2E+2023%2E+The+tale+of+two+MSMARCO+%2D+and+their+unfair+comparisons%2E+In+Proc%2E+SIGIR%2C+page+2431%E2%80%932435%2E)
24. Kenton Lee, Ming-Wei Chang, and Kristina Toutanova. 2019. [Latent retrieval for weakly supervised open domain question answering](https://md.research.elamur.ai/free/en/papers/lee-2019-latent-retrieval-weakly-supervised-open/). In Proc. ACL, pages 6086–6096.
25. Barlas Oguz, Kushal Lakhotia, Anchit Gupta, Patrick Lewis, Vladimir Karpukhin, Aleksandra Piktus, Xilun Chen, Sebastian Riedel, Scott Yih, Sonal Gupta, and Yashar Mehdad. 2022. Domain-matched pre-training tasks for dense retrieval. In Proc. NAACL, pages 1524–1534. [find](https://scholar.google.com/scholar?q=Barlas+Oguz%2C+Kushal+Lakhotia%2C+Anchit+Gupta%2C+Patrick+Lewis%2C+Vladimir+Karpukhin%2C+Aleksandra+Piktus%2C+Xilun+Chen%2C+Sebastian+Riedel%2C+Scott+Yih%2C+Sonal+Gupta%2C+and+Yashar+Mehdad%2E+2022%2E+Domain%2Dmatched+pre%2Dtrai)
26. Yingqi Qu, Yuchen Ding, Jing Liu, Kai Liu, Ruiyang Ren, Wayne Xin Zhao, Daxiang Dong, Hua Wu, and Haifeng Wang. 2021. [RocketQA: An optimized training approach to dense passage retrieval for open-domain question answering](https://md.research.elamur.ai/free/en/papers/qu-2021-rocketqa-optimized-training-approach-dense/). In Proc. NAACL, pages 5835–5847.
27. Jimmy Lin, Xueguang Ma, Sheng-Chieh Lin, Jheng-Hong Yang, Ronak Pradeep, and Rodrigo Nogueira. 2021a. Pyserini: A python toolkit for reproducible information retrieval research with sparse and dense representations. In Proc. SIGIR, page 2356–2362. [find](https://scholar.google.com/scholar?q=Jimmy+Lin%2C+Xueguang+Ma%2C+Sheng%2DChieh+Lin%2C+Jheng%2DHong+Yang%2C+Ronak+Pradeep%2C+and+Rodrigo+Nogueira%2E+2021a%2E+Pyserini%3A+A+python+toolkit+for+reproducible+information+retrieval+research+with+sparse+and+dense+r)
28. Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. 2020. [Exploring the limits of transfer learning with a unified text-to-text transformer](https://md.research.elamur.ai/free/en/papers/raffel-2019-exploring-limits-transfer-learning-unified/). J. Mach. Learn. Res., 21(140):1–67.
29. Sheng-Chieh Lin, Jheng-Hong Yang, and Jimmy Lin. 2021b. In-batch negatives for knowledge distillation with tightly-coupled teachers for dense retrieval. In Proc. RepL4NLP, pages 163–173. [find](https://scholar.google.com/scholar?q=Sheng%2DChieh+Lin%2C+Jheng%2DHong+Yang%2C+and+Jimmy+Lin%2E+2021b%2E+In%2Dbatch+negatives+for+knowledge+distillation+with+tightly%2Dcoupled+teachers+for+dense+retrieval%2E+In+Proc%2E+RepL4NLP%2C+pages+163%E2%80%93173%2E)
30. Nils Reimers and Iryna Gurevych. 2019. [Sentence-BERT: Sentence embeddings using Siamese BERT-networks](https://md.research.elamur.ai/free/en/papers/reimers-2019-sentence-bert-sentence-embeddings-siamese/). In Proc. EMNLP, pages 3982–3992.
31. Zhenghao Lin, Yeyun Gong, Xiao Liu, Hang Zhang, Chen Lin, Anlei Dong, Jian Jiao, Jingwen Lu, Daxin Jiang, Rangan Majumder, and Nan Duan. 2022. Prod: Progressive distillation for dense retrieval. arXiv:2209.13335. — [arXiv:2209.13335](https://arxiv.org/abs/2209.13335) [find](https://scholar.google.com/scholar?q=Zhenghao+Lin%2C+Yeyun+Gong%2C+Xiao+Liu%2C+Hang+Zhang%2C+Chen+Lin%2C+Anlei+Dong%2C+Jian+Jiao%2C+Jingwen+Lu%2C+Daxin+Jiang%2C+Rangan+Majumder%2C+and+Nan+Duan%2E+2022%2E+Prod%3A+Progressive+distillation+for+dense+retrieval%2E+arXiv)
32. Shuqi Lu, Di He, Chenyan Xiong, Guolin Ke, Waleed Malik, Zhicheng Dou, Paul Bennett, Tie-Yan Liu, and Arnold Overwijk. 2021. Less is more: Pretrain a strong Siamese encoder for dense text retrieval using a weak decoder. In Proc. EMNLP, pages 2780–2791. [find](https://scholar.google.com/scholar?q=Shuqi+Lu%2C+Di+He%2C+Chenyan+Xiong%2C+Guolin+Ke%2C+Waleed+Malik%2C+Zhicheng+Dou%2C+Paul+Bennett%2C+Tie%2DYan+Liu%2C+and+Arnold+Overwijk%2E+2021%2E+Less+is+more%3A+Pretrain+a+strong+Siamese+encoder+for+dense+text+retrieval+us)
33. Xueguang Ma, Kai Sun, Ronak Pradeep, and Jimmy Lin. 2021. A replication study of dense passage retriever. arXiv:2104.05740. — [arXiv:2104.05740](https://arxiv.org/abs/2104.05740) [find](https://scholar.google.com/scholar?q=Xueguang+Ma%2C+Kai+Sun%2C+Ronak+Pradeep%2C+and+Jimmy+Lin%2E+2021%2E+A+replication+study+of+dense+passage+retriever%2E+arXiv%3A2104%2E05740%2E+%E2%80%94+%5BarXiv%3A2104%2E05740%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2104%2E05740%29)
34. Joel Mackenzie, Andrew Trotman, and Jimmy Lin. 2021. Wacky weights in learned sparse representations and the revenge of score-at-a-time query evaluation. arXiv:2110.11540. — [arXiv:2110.11540](https://arxiv.org/abs/2110.11540) [find](https://scholar.google.com/scholar?q=Joel+Mackenzie%2C+Andrew+Trotman%2C+and+Jimmy+Lin%2E+2021%2E+Wacky+weights+in+learned+sparse+representations+and+the+revenge+of+score%2Dat%2Da%2Dtime+query+evaluation%2E+arXiv%3A2110%2E11540%2E+%E2%80%94+%5BarXiv%3A2110%2E11540%5D%28https%3A%2F)
35. Alex Mallen, Akari Asai, Victor Zhong, Rajarshi Das, Daniel Khashabi, and Hannaneh Hajishirzi. 2023. When not to trust language models: Investigating effectiveness of parametric and non-parametric memories. In Proc. ACL, pages 9802–9822. [find](https://scholar.google.com/scholar?q=Alex+Mallen%2C+Akari+Asai%2C+Victor+Zhong%2C+Rajarshi+Das%2C+Daniel+Khashabi%2C+and+Hannaneh+Hajishirzi%2E+2023%2E+When+not+to+trust+language+models%3A+Investigating+effectiveness+of+parametric+and+non%2Dparametric+mem)
36. Rui Meng, Ye Liu, Semih Yavuz, Divyansh Agarwal, Lifu Tu, Ning Yu, Jianguo Zhang, Meghana Bhat, and Yingbo Zhou. 2023. AugTriever: Unsupervised dense retrieval by scalable data augmentation. arXiv:2212.08841. — [arXiv:2212.08841](https://arxiv.org/abs/2212.08841) [find](https://scholar.google.com/scholar?q=Rui+Meng%2C+Ye+Liu%2C+Semih+Yavuz%2C+Divyansh+Agarwal%2C+Lifu+Tu%2C+Ning+Yu%2C+Jianguo+Zhang%2C+Meghana+Bhat%2C+and+Yingbo+Zhou%2E+2023%2E+AugTriever%3A+Unsupervised+dense+retrieval+by+scalable+data+augmentation%2E+arXiv%3A221)
37. Jianmo Ni, Chen Qu, Jing Lu, Zhuyun Dai, Gustavo Hernandez Abrego, Ji Ma, Vincent Zhao, Yi Luan, Keith Hall, Ming-Wei Chang, and Yinfei Yang. 2022. [Large dual encoders are generalizable retrievers](https://md.research.elamur.ai/free/en/papers/ni-2021-large-dual-encoders-generalizable-retrievers/). In Proc. EMNLP, pages 9844–9855.
38. Rodrigo Nogueira and Jimmy Lin. 2019. doc2query to docTTTTTquery. [find](https://scholar.google.com/scholar?q=Rodrigo+Nogueira+and+Jimmy+Lin%2E+2019%2E+doc2query+to+docTTTTTquery%2E)
39. From Ruiyang Ren, Yingqi Qu, Jing Liu, Wayne Xin Zhao, QiaoQiao She, Hua Wu, Haifeng Wang, and Ji-Rong Wen. 2021. [RocketQAv2: A joint training method for dense passage retrieval and passage re-ranking](https://md.research.elamur.ai/free/en/papers/ren-2021-rocketqav2-joint-training-method-dense/). In Proc. EMNLP, pages 2825–2835.
40. Keshav Santhanam, Omar Khattab, Christopher Potts, and Matei Zaharia. 2022a. Plaid: An efficient engine for late interaction retrieval. arXiv:2205.09707. [find](https://scholar.google.com/scholar?q=Keshav+Santhanam%2C+Omar+Khattab%2C+Christopher+Potts%2C+and+Matei+Zaharia%2E+2022a%2E+Plaid%3A+An+efficient+engine+for+late+interaction+retrieval%2E+arXiv%3A2205%2E09707%2E)
41. Keshav Santhanam, Omar Khattab, Jon Saad-Falcon, Christopher Potts, and Matei Zaharia. 2022b. Col-BERTv2: Effective and efficient retrieval via lightweight late interaction. In Proc. NAACL, pages 3715–3734. [find](https://scholar.google.com/scholar?q=Keshav+Santhanam%2C+Omar+Khattab%2C+Jon+Saad%2DFalcon%2C+Christopher+Potts%2C+and+Matei+Zaharia%2E+2022b%2E+Col%2DBERTv2%3A+Effective+and+efficient+retrieval+via+lightweight+late+interaction%2E+In+Proc%2E+NAACL%2C+pages+3715)
42. Weijia Shi, Sewon Min, Michihiro Yasunaga, Minjoon Seo, Rich James, Mike Lewis, Luke Zettlemoyer, and Wen-tau Yih. 2023. [Replug: Retrieval-augmented black-box language models](https://md.research.elamur.ai/free/en/papers/shi-2023-replug-retrieval-augmented-black-box/). arXiv:2301.12652.
43. Nandan Thakur, Nils Reimers, Johannes Daxenberger, and Iryna Gurevych. 2021a. [Augmented SBERT: Data augmentation method for improving bi-encoders for pairwise sentence scoring tasks](https://md.research.elamur.ai/free/en/papers/thakur-2021-augmented-sbert-data-augmentation-method/). In Proc. NAACL, pages 296–310.
44. Nandan Thakur, Nils Reimers, Andreas Rücklé, Abhishek Srivastava, and Iryna Gurevych. 2021b. BEIR: A heterogeneous benchmark for zero-shot evaluation of information retrieval models. In Proc. NIPS. [find](https://scholar.google.com/scholar?q=Nandan+Thakur%2C+Nils+Reimers%2C+Andreas+R%C3%BCckl%C3%A9%2C+Abhishek+Srivastava%2C+and+Iryna+Gurevych%2E+2021b%2E+BEIR%3A+A+heterogeneous+benchmark+for+zero%2Dshot+evaluation+of+information+retrieval+models%2E+In+Proc%2E+NIPS%2E)
45. Kexin Wang, Nandan Thakur, Nils Reimers, and Iryna Gurevych. 2022. [GPL: Generative pseudo labeling for unsupervised domain adaptation of dense retrieval](https://md.research.elamur.ai/free/en/papers/wang-2022-gpl-generative-pseudo-labeling-unsupervised/). In Proc. NAACL, pages 2345–2360.
46. Xing Wu, Guangyuan Ma, Meng Lin, Zijia Lin, Zhongyuan Wang, and Songlin Hu. 2022. Contextual masked auto-encoder for dense passage retrieval. arXiv:2208.07670. — [arXiv:2208.07670](https://arxiv.org/abs/2208.07670) [find](https://scholar.google.com/scholar?q=Xing+Wu%2C+Guangyuan+Ma%2C+Meng+Lin%2C+Zijia+Lin%2C+Zhongyuan+Wang%2C+and+Songlin+Hu%2E+2022%2E+Contextual+masked+auto%2Dencoder+for+dense+passage+retrieval%2E+arXiv%3A2208%2E07670%2E+%E2%80%94+%5BarXiv%3A2208%2E07670%5D%28https%3A%2F%2Farxiv%2Eorg%2Fa)
47. Shitao Xiao, Zheng Liu, Yingxia Shao, and Zhao Cao. 2022. [RetroMAE: Pre-training retrieval-oriented language models via masked auto-encoder](https://md.research.elamur.ai/free/en/papers/xiao-2022-retromae-pre-training-retrieval-oriented/). In Proc. EMNLP, pages 538–548.
48. Ji Xin, Chenyan Xiong, Ashwin Srinivasan, Ankita Sharma, Damien Jose, and Paul Bennett. 2022. Zero-shot dense retrieval with momentum adversarial domain invariant representations. In Proc. ACL, pages 4008–4020. [find](https://scholar.google.com/scholar?q=Ji+Xin%2C+Chenyan+Xiong%2C+Ashwin+Srinivasan%2C+Ankita+Sharma%2C+Damien+Jose%2C+and+Paul+Bennett%2E+2022%2E+Zero%2Dshot+dense+retrieval+with+momentum+adversarial+domain+invariant+representations%2E+In+Proc%2E+ACL%2C+pages+)
49. Yue Yu, Chenyan Xiong, Si Sun, Chao Zhang, and Arnold Overwijk. 2022. COCO-DR: Combating distribution shift in zero-shot dense retrieval with contrastive and distributionally robust learning. In Proc. EMNLP, pages 1462–1479. [find](https://scholar.google.com/scholar?q=Yue+Yu%2C+Chenyan+Xiong%2C+Si+Sun%2C+Chao+Zhang%2C+and+Arnold+Overwijk%2E+2022%2E+COCO%2DDR%3A+Combating+distribution+shift+in+zero%2Dshot+dense+retrieval+with+contrastive+and+distributionally+robust+learning%2E+In+Proc%2E)
50. Hansi Zeng, Hamed Zamani, and Vishwa Vinay. 2022. Curriculum learning for dense retrieval distillation. In Proc. SIGIR, page 1979–1983. [find](https://scholar.google.com/scholar?q=Hansi+Zeng%2C+Hamed+Zamani%2C+and+Vishwa+Vinay%2E+2022%2E+Curriculum+learning+for+dense+retrieval+distillation%2E+In+Proc%2E+SIGIR%2C+page+1979%E2%80%931983%2E)

*Updated 24 October 2026*