---
title: "RocketQAv2: A Joint Training Method for Dense — Ren 2021"
description: "Retrieve-then-rerank pipelines pair a fast dual-encoder retriever with a stronger cross-encoder re-ranker."
authors: ["Ruiyang Ren", "Yingqi Qu", "Jing Liu", "Wayne Xin Zhao", "QiaoQiao She", "Hua Wu", "Haifeng Wang", "Ji-Rong Wen"]
year: 2021
venue: "Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing"
source: "https://aclanthology.org/2021.emnlp-main.224/"
doi: "10.18653/v1/2021.emnlp-main.224"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/ren-2021-rocketqav2-joint-training-method-dense/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/ren-2021-rocketqav2-joint-training-method-dense/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2021](https://md.research.elamur.ai/free/en/papers/2021/page/4/) / RocketQAv2: A Joint Training Method for Dense Passage Retrieval and Passage Re-ranking

# RocketQAv2: A Joint Training Method for Dense Passage Retrieval and Passage Re-ranking

**Authors:** [Ruiyang Ren](https://md.research.elamur.ai/free/en/authors/ruiyang-ren/), [Yingqi Qu](https://md.research.elamur.ai/free/en/authors/yingqi-qu/), [Jing Liu](https://md.research.elamur.ai/free/en/authors/jing-liu/), [Wayne Xin Zhao](https://md.research.elamur.ai/free/en/authors/wayne-xin-zhao/), [QiaoQiao She](https://md.research.elamur.ai/free/en/authors/qiaoqiao-she/), [Hua Wu](https://md.research.elamur.ai/free/en/authors/hua-wu/), [Haifeng Wang](https://md.research.elamur.ai/free/en/authors/haifeng-wang/), [Ji-Rong Wen](https://md.research.elamur.ai/free/en/authors/ji-rong-wen/) · **Year:** 2021 · **Venue:** Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2021.emnlp-main.224/) · [PDF](https://aclanthology.org/2021.emnlp-main.224.pdf) · [DOI 10.18653/v1/2021.emnlp-main.224](https://doi.org/10.18653/v1/2021.emnlp-main.224) · [OpenAlex](https://openalex.org/W3206455169)

**Citations:** 149 citations (openalex-snapshot · 2026-10-14) · [107](#cited-by) in our library, of the 140884 papers whose bibliographies we read

## Summary

Retrieve-then-rerank pipelines pair a fast dual-encoder retriever with a stronger cross-encoder re-ranker. The two are usually trained separately or in alternation, with one model frozen while the other learns. A key obstacle to joint training is that retrievers learn from whole candidate lists, while re-rankers are typically trained pointwise or pairwise. RocketQAv2 trains both modules together.

Its core idea, dynamic listwise distillation, gives both models one listwise objective. Each model scores a query's candidate list, and the scores are softmax-normalized into relevance distributions. A KL-divergence term then aligns the retriever's distribution with the re-ranker's while the parameters of both keep updating. The re-ranker also receives a supervised cross-entropy loss on labeled positives.

To supply varied candidate lists, hybrid data augmentation mixes two sources. One is hard negatives randomly sampled from passages a RocketQA retriever returns. The other is positives and negatives filtered by a RocketQA re-ranker.

Experiments on MSMARCO and Natural Questions show gains for both components. The authors present this as the first joint training of retriever and re-ranker, a step toward end-to-end retrieval systems.

## Key points

- In the MSMARCO ablation (eight instances per query), the full RocketQAv2 retriever reaches MRR@10 37.4 and R@50 84.9. Variants score lower on MRR@10:
  - static distillation from a frozen re-ranker: 36.0 / 84.5;
  - a pointwise re-ranker: 36.3 / 83.9;
  - no denoised instances: 36.3 / 84.9.
- Longer instance lists (one positive plus more hard negatives) improved both retrieval and re-ranking MRR@10. Adding in-batch negatives brought no further gain.
- Applied to BM25's top-1000 candidates, the RocketQAv2 re-ranker clearly beats other base-size models and is comparable to large ones.
- Baselines compared include RocketQA, TFR-BERT and HYBRID.
- Code is released at github.com/PaddlePaddle/RocketQA (Baidu and Renmin University of China, EMNLP 2021).

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

Fields: [Natural language processing](https://md.research.elamur.ai/free/en/shelves/nlp/)

## Cited by

Cited by

107 papers of our library cite it (of the 140884 whose bibliographies we read); 7 of them have a page here, 100 not yet.

- [M3-Embedding: Multi-Linguality, Multi-Functionality, Multi-Granularity Text Embeddings Through Self-Knowledge Distillation](https://md.research.elamur.ai/free/en/papers/chen-2024-m3-embedding-multi-linguality/)
- [Promptagator: Few-shot Dense Retrieval From 8 Examples](https://md.research.elamur.ai/free/en/papers/dai-2022-promptagator-few-shot-dense-retrieval/)
- [Evaluating Open-Domain Question Answering in the Era of Large Language Models](https://md.research.elamur.ai/free/en/papers/kamalloo-2023-evaluating-open-domain-question-answering/)
- [GPL: Generative Pseudo Labeling for Unsupervised Domain Adaptation of Dense Retrieval](https://md.research.elamur.ai/free/en/papers/wang-2022-gpl-generative-pseudo-labeling-unsupervised/)
- [Text Embeddings by Weakly-Supervised Contrastive Pre-training](https://md.research.elamur.ai/free/en/papers/wang-2022-text-embeddings-weakly-supervised-contrastive/)
- [Query2doc: Query Expansion with Large Language Models](https://md.research.elamur.ai/free/en/papers/wang-2023-query2doc-query-expansion-large-language/)
- [A Survey of Large Language Models](https://md.research.elamur.ai/free/en/papers/zhao-2023-survey-large-language-models/)

## Cite

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{ren2021rocketqav2,
  title = {{RocketQAv2: A Joint Training Method for Dense Passage Retrieval and Passage Re-ranking}},
  author = {Ren, Ruiyang and Qu, Yingqi and Liu, Jing and Zhao, Wayne Xin and She, QiaoQiao and Wu, Hua and Wang, Haifeng and Wen, Ji-Rong},
  booktitle = {Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing},
  year = {2021},
  doi = {10.18653/v1/2021.emnlp-main.224},
  url = {https://doi.org/10.18653/v1/2021.emnlp-main.224}
}
```

- [← previous RocketQA: An Optimized Training Approach to Dense Passage Retrieval for Open-Domain Question Answering](https://md.research.elamur.ai/free/en/papers/qu-2021-rocketqa-optimized-training-approach-dense/)
- [next → Sarcasm and Sentiment Detection in Arabic language A Hybrid Approach Combining Embeddings and Rule-based Features](https://md.research.elamur.ai/free/en/papers/gaanoun-2021-sarcasm-sentiment-detection-arabic-language/)
- [↑ back to papers of 2021](https://md.research.elamur.ai/free/en/papers/2021/page/4/)

## References

49 references, 16 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Zhe Cao, Tao Qin, Tie-Yan Liu, Ming-Feng Tsai, and Hang Li. 2007. Learning to rank: from pairwise approach to listwise approach. In Machine Learning, Proceedings of the Twenty-Fourth International Conference (ICML 2007), Corvallis, Oregon, USA, June 20-24, 2007, volume 227 of ACM International Confe… [find](https://scholar.google.com/scholar?q=Zhe+Cao%2C+Tao+Qin%2C+Tie%2DYan+Liu%2C+Ming%2DFeng+Tsai%2C+and+Hang+Li%2E+2007%2E+Learning+to+rank%3A+from+pairwise+approach+to+listwise+approach%2E+In+Machine+Learning%2C+Proceedings+of+the+Twenty%2DFourth+International+Con)
2. Wei-Cheng Chang, Felix X. Yu, Yin-Wen Chang, Yiming Yang, and Sanjiv Kumar. 2020. Pre-training tasks for embedding-based large-scale retrieval. In 8th International Conference on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia, April 26-30, 2020. [find](https://scholar.google.com/scholar?q=Wei%2DCheng+Chang%2C+Felix+X%2E+Yu%2C+Yin%2DWen+Chang%2C+Yiming+Yang%2C+and+Sanjiv+Kumar%2E+2020%2E+Pre%2Dtraining+tasks+for+embedding%2Dbased+large%2Dscale+retrieval%2E+In+8th+International+Conference+on+Learning+Representati)
3. Zhuyun Dai and Jamie Callan. 2019. Deeper text understanding for IR with contextual neural language modeling. In Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR 2019, Paris, France, July 21-25, 2019, pages 985–988. [find](https://scholar.google.com/scholar?q=Zhuyun+Dai+and+Jamie+Callan%2E+2019%2E+Deeper+text+understanding+for+IR+with+contextual+neural+language+modeling%2E+In+Proceedings+of+the+42nd+International+ACM+SIGIR+Conference+on+Research+and+Development+)
4. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies,… [find](https://scholar.google.com/scholar?q=Jacob+Devlin%2C+Ming%2DWei+Chang%2C+Kenton+Lee%2C+and+Kristina+Toutanova%2E+2019%2E+BERT%3A+pre%2Dtraining+of+deep+bidirectional+transformers+for+language+understanding%2E+In+Proceedings+of+the+2019+Conference+of+the+N)
5. Luyu Gao, Zhuyun Dai, and Jamie Callan. 2020. Modularized transfomer-based ranking framework. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, EMNLP 2020, Online, November 16-20, 2020, pages 4180– 4190. [find](https://scholar.google.com/scholar?q=Luyu+Gao%2C+Zhuyun+Dai%2C+and+Jamie+Callan%2E+2020%2E+Modularized+transfomer%2Dbased+ranking+framework%2E+In+Proceedings+of+the+2020+Conference+on+Empirical+Methods+in+Natural+Language+Processing%2C+EMNLP+2020%2C+Onl)
6. Luyu Gao, Zhuyun Dai, and Jamie Callan. 2021a. COIL: Revisit exact lexical match in information retrieval with contextualized inverted list. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 3030–3… [find](https://scholar.google.com/scholar?q=Luyu+Gao%2C+Zhuyun+Dai%2C+and+Jamie+Callan%2E+2021a%2E+COIL%3A+Revisit+exact+lexical+match+in+information+retrieval+with+contextualized+inverted+list%2E+In+Proceedings+of+the+2021+Conference+of+the+North+American)
7. Luyu Gao, Zhuyun Dai, and Jamie Callan. 2021b. Rethink training of BERT rerankers in multi-stage retrieval pipeline. In Advances in Information Retrieval - 43rd European Conference on IR Research, ECIR 2021, Virtual Event, March 28 - April 1, 2021, Proceedings, Part II, volume 12657, pages 280–286. [find](https://scholar.google.com/scholar?q=Luyu+Gao%2C+Zhuyun+Dai%2C+and+Jamie+Callan%2E+2021b%2E+Rethink+training+of+BERT+rerankers+in+multi%2Dstage+retrieval+pipeline%2E+In+Advances+in+Information+Retrieval+%2D+43rd+European+Conference+on+IR+Research%2C+ECI)
8. Daniel Gillick, Sayali Kulkarni, Larry Lansing, Alessandro Presta, Jason Baldridge, Eugene Ie, and Diego García-Olano. 2019. Learning dense representations for entity retrieval. In Proceedings of the 23rd Conference on Computational Natural Language Learning, CoNLL 2019, Hong Kong, China, November 3… [find](https://scholar.google.com/scholar?q=Daniel+Gillick%2C+Sayali+Kulkarni%2C+Larry+Lansing%2C+Alessandro+Presta%2C+Jason+Baldridge%2C+Eugene+Ie%2C+and+Diego+Garc%C3%ADa%2DOlano%2E+2019%2E+Learning+dense+representations+for+entity+retrieval%2E+In+Proceedings+of+the+) [our card](https://md.research.elamur.ai/free/en/papers/gillick-2019-learning-dense-representations-entity-retrieval/)
9. Kelvin Guu, Kenton Lee, Zora Tung, Panupong Pasupat, and Ming-Wei Chang. 2020. REALM: retrieval-augmented language model pre-training. CoRR, abs/2002.08909. [find](https://scholar.google.com/scholar?q=Kelvin+Guu%2C+Kenton+Lee%2C+Zora+Tung%2C+Panupong+Pasupat%2C+and+Ming%2DWei+Chang%2E+2020%2E+REALM%3A+retrieval%2Daugmented+language+model+pre%2Dtraining%2E+CoRR%2C+abs%2F2002%2E08909%2E)
10. Shuguang Han, Xuanhui Wang, Mike Bendersky, and Marc Najork. 2020. Learning-to-rank with BERT in tf-ranking. CoRR, abs/2004.08476. — [arXiv:2004.08476](https://arxiv.org/abs/2004.08476) [find](https://scholar.google.com/scholar?q=Shuguang+Han%2C+Xuanhui+Wang%2C+Mike+Bendersky%2C+and+Marc+Najork%2E+2020%2E+Learning%2Dto%2Drank+with+BERT+in+tf%2Dranking%2E+CoRR%2C+abs%2F2004%2E08476%2E+%E2%80%94+%5BarXiv%3A2004%2E08476%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2004%2E08476%29)
11. Matthew L. Henderson, Rami Al-Rfou, Brian Strope, Yun-Hsuan Sung, László Lukács, Ruiqi Guo, Sanjiv Kumar, Balint Miklos, and Ray Kurzweil. 2017. [find](https://scholar.google.com/scholar?q=Matthew+L%2E+Henderson%2C+Rami+Al%2DRfou%2C+Brian+Strope%2C+Yun%2DHsuan+Sung%2C+L%C3%A1szl%C3%B3+Luk%C3%A1cs%2C+Ruiqi+Guo%2C+Sanjiv+Kumar%2C+Balint+Miklos%2C+and+Ray+Kurzweil%2E+2017%2E)
12. Natural questions: a benchmark for question answering research. Trans. Assoc. Comput. Linguistics, 7:452–466. Efficient natural language response suggestion for smart reply. CoRR, abs/1705.00652. [find](https://scholar.google.com/scholar?q=Natural+questions%3A+a+benchmark+for+question+answering+research%2E+Trans%2E+Assoc%2E+Comput%2E+Linguistics%2C+7%3A452%E2%80%93466%2E+Efficient+natural+language+response+suggestion+for+smart+reply%2E+CoRR%2C+abs%2F1705%2E00652%2E) [our card](https://md.research.elamur.ai/free/en/papers/kwiatkowski-2019-natural-questions-benchmark-question-answering/)
13. Sebastian Hofstätter, Sophia Althammer, Michael Schröder, Mete Sertkan, and Allan Hanbury. 2020. Improving efficient neural ranking models with cross-architecture knowledge distillation. CoRR, abs/2010.02666. — [arXiv:2010.02666](https://arxiv.org/abs/2010.02666) [find](https://scholar.google.com/scholar?q=Sebastian+Hofst%C3%A4tter%2C+Sophia+Althammer%2C+Michael+Schr%C3%B6der%2C+Mete+Sertkan%2C+and+Allan+Hanbury%2E+2020%2E+Improving+efficient+neural+ranking+models+with+cross%2Darchitecture+knowledge+distillation%2E+CoRR%2C+abs%2F201)
14. Sebastian Hofstätter, Sheng-Chieh Lin, Jheng-Hong Yang, Jimmy Lin, and Allan Hanbury. 2021. Efficiently teaching an effective dense retriever with balanced topic aware sampling. CoRR, abs/2104.06967. [find](https://scholar.google.com/scholar?q=Sebastian+Hofst%C3%A4tter%2C+Sheng%2DChieh+Lin%2C+Jheng%2DHong+Yang%2C+Jimmy+Lin%2C+and+Allan+Hanbury%2E+2021%2E+Efficiently+teaching+an+effective+dense+retriever+with+balanced+topic+aware+sampling%2E+CoRR%2C+abs%2F2104%2E06967%2E) [our card](https://md.research.elamur.ai/free/en/papers/hofstatter-2021-efficiently-teaching-effective-dense-retriever/)
15. Jui-Ting Huang, Ashish Sharma, Shuying Sun, Li Xia, David Zhang, Philip Pronin, Janani Padmanabhan, Giuseppe Ottaviano, and Linjun Yang. 2020. Embedding-based retrieval in facebook search. In KDD ’20: The 26th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, Virtual Event, CA, USA, Augu… [find](https://scholar.google.com/scholar?q=Jui%2DTing+Huang%2C+Ashish+Sharma%2C+Shuying+Sun%2C+Li+Xia%2C+David+Zhang%2C+Philip+Pronin%2C+Janani+Padmanabhan%2C+Giuseppe+Ottaviano%2C+and+Linjun+Yang%2E+2020%2E+Embedding%2Dbased+retrieval+in+facebook+search%2E+In+KDD+%E2%80%9920%3A)
16. Kenton Lee, Ming-Wei Chang, and Kristina Toutanova. 2019. Latent retrieval for weakly supervised open domain question answering. In Proceedings of the 57th Conference of the Association for Computational Linguistics, ACL 2019, Florence, Italy, July 28- August 2, 2019, Volume 1: Long Papers, pages 60… [find](https://scholar.google.com/scholar?q=Kenton+Lee%2C+Ming%2DWei+Chang%2C+and+Kristina+Toutanova%2E+2019%2E+Latent+retrieval+for+weakly+supervised+open+domain+question+answering%2E+In+Proceedings+of+the+57th+Conference+of+the+Association+for+Computatio) [our card](https://md.research.elamur.ai/free/en/papers/lee-2019-latent-retrieval-weakly-supervised-open/)
17. Jing Lu, Gustavo Hernández Ábrego, Ji Ma, Jianmo Ni, and Yinfei Yang. 2020. Neural passage retrieval with improved negative contrast. CoRR, abs/2010.12523. — [arXiv:2010.12523](https://arxiv.org/abs/2010.12523) [find](https://scholar.google.com/scholar?q=Jing+Lu%2C+Gustavo+Hern%C3%A1ndez+%C3%81brego%2C+Ji+Ma%2C+Jianmo+Ni%2C+and+Yinfei+Yang%2E+2020%2E+Neural+passage+retrieval+with+improved+negative+contrast%2E+CoRR%2C+abs%2F2010%2E12523%2E+%E2%80%94+%5BarXiv%3A2010%2E12523%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2)
18. Yi Luan, Jacob Eisenstein, Kristina Toutanova, and Michael Collins. 2021. Sparse, dense, and attentional representations for text retrieval. Transactions of the Association for Computational Linguistics, 9:329–345. [find](https://scholar.google.com/scholar?q=Yi+Luan%2C+Jacob+Eisenstein%2C+Kristina+Toutanova%2C+and+Michael+Collins%2E+2021%2E+Sparse%2C+dense%2C+and+attentional+representations+for+text+retrieval%2E+Transactions+of+the+Association+for+Computational+Linguisti) [our card](https://md.research.elamur.ai/free/en/papers/luan-2021-sparse-dense-attentional-representations-text/)
19. Gautier Izacard and Edouard Grave. 2020. Distilling knowledge from reader to retriever for question answering. CoRR, abs/2012.04584. — [arXiv:2012.04584](https://arxiv.org/abs/2012.04584) [find](https://scholar.google.com/scholar?q=Gautier+Izacard+and+Edouard+Grave%2E+2020%2E+Distilling+knowledge+from+reader+to+retriever+for+question+answering%2E+CoRR%2C+abs%2F2012%2E04584%2E+%E2%80%94+%5BarXiv%3A2012%2E04584%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2012%2E04584%29)
20. Yanjun Ma, Dianhai Yu, Tian Wu, and Haifeng Wang. 2019. Paddlepaddle: An open-source deep learning platform from industrial practice. Frontiers of Data and Domputing, 1(1):105–115. [find](https://scholar.google.com/scholar?q=Yanjun+Ma%2C+Dianhai+Yu%2C+Tian+Wu%2C+and+Haifeng+Wang%2E+2019%2E+Paddlepaddle%3A+An+open%2Dsource+deep+learning+platform+from+industrial+practice%2E+Frontiers+of+Data+and+Domputing%2C+1%281%29%3A105%E2%80%93115%2E)
21. Kyoungrok Jang, Junmo Kang, Giwon Hong, Sung-Hyon Myaeng, Joohee Park, Taewon Yoon, and Hee-Cheol Seo. 2021. UHD-BERT: bucketed ultra-high dimensional sparse representations for full ranking. CoRR, abs/2104.07198. — [arXiv:2104.07198](https://arxiv.org/abs/2104.07198) [find](https://scholar.google.com/scholar?q=Kyoungrok+Jang%2C+Junmo+Kang%2C+Giwon+Hong%2C+Sung%2DHyon+Myaeng%2C+Joohee+Park%2C+Taewon+Yoon%2C+and+Hee%2DCheol+Seo%2E+2021%2E+UHD%2DBERT%3A+bucketed+ultra%2Dhigh+dimensional+sparse+representations+for+full+ranking%2E+CoRR%2C+ab)
22. Yuning Mao, Pengcheng He, Xiaodong Liu, Yelong Shen, Jianfeng Gao, Jiawei Han, and Weizhu Chen. 2020. Generation-augmented retrieval for open-domain question answering. CoRR, abs/2009.08553. [find](https://scholar.google.com/scholar?q=Yuning+Mao%2C+Pengcheng+He%2C+Xiaodong+Liu%2C+Yelong+Shen%2C+Jianfeng+Gao%2C+Jiawei+Han%2C+and+Weizhu+Chen%2E+2020%2E+Generation%2Daugmented+retrieval+for+open%2Ddomain+question+answering%2E+CoRR%2C+abs%2F2009%2E08553%2E) [our card](https://md.research.elamur.ai/free/en/papers/mao-2020-generation-augmented-retrieval-open-domain/)
23. Zongcheng Ji, Zhengdong Lu, and Hang Li. 2014. An information retrieval approach to short text conversation. CoRR, abs/1408.6988. — [arXiv:1408.6988](https://arxiv.org/abs/1408.6988) [find](https://scholar.google.com/scholar?q=Zongcheng+Ji%2C+Zhengdong+Lu%2C+and+Hang+Li%2E+2014%2E+An+information+retrieval+approach+to+short+text+conversation%2E+CoRR%2C+abs%2F1408%2E6988%2E+%E2%80%94+%5BarXiv%3A1408%2E6988%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F1408%2E6988%29)
24. Yuning Mao, Pengcheng He, Xiaodong Liu, Yelong Shen, Jianfeng Gao, Jiawei Han, and Weizhu Chen. 2021. Reader-guided passage reranking for open-domain question answering. CoRR, abs/2101.00294. [find](https://scholar.google.com/scholar?q=Yuning+Mao%2C+Pengcheng+He%2C+Xiaodong+Liu%2C+Yelong+Shen%2C+Jianfeng+Gao%2C+Jiawei+Han%2C+and+Weizhu+Chen%2E+2021%2E+Reader%2Dguided+passage+reranking+for+open%2Ddomain+question+answering%2E+CoRR%2C+abs%2F2101%2E00294%2E)
25. Vladimir Karpukhin, Barlas Oguz, Sewon Min, Patrick S. H. Lewis, Ledell Wu, Sergey Edunov, Danqi Chen, and Wen-tau Yih. 2020. Dense passage retrieval for open-domain question answering. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, EMNLP 2020, Online, Nov… [find](https://scholar.google.com/scholar?q=Vladimir+Karpukhin%2C+Barlas+Oguz%2C+Sewon+Min%2C+Patrick+S%2E+H%2E+Lewis%2C+Ledell+Wu%2C+Sergey+Edunov%2C+Danqi+Chen%2C+and+Wen%2Dtau+Yih%2E+2020%2E+Dense+passage+retrieval+for+open%2Ddomain+question+answering%2E+In+Proceedings) [our card](https://md.research.elamur.ai/free/en/papers/karpukhin-2020-dense-passage-retrieval-open-domain/)
26. Omar Khattab and Matei Zaharia. 2020. Colbert: Efficient and effective passage search via contextualized late interaction over BERT. In Proceedings of the 43rd International ACM SIGIR conference on research and development in Information Retrieval, SIGIR 2020, Virtual Event, China, July 25-30, 2020,… [find](https://scholar.google.com/scholar?q=Omar+Khattab+and+Matei+Zaharia%2E+2020%2E+Colbert%3A+Efficient+and+effective+passage+search+via+contextualized+late+interaction+over+BERT%2E+In+Proceedings+of+the+43rd+International+ACM+SIGIR+conference+on+re) [our card](https://md.research.elamur.ai/free/en/papers/khattab-2020-colbert-efficient-effective-passage/)
27. Diederik P. Kingma and Jimmy Ba. 2015. Adam: A method for stochastic optimization. In 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings. [find](https://scholar.google.com/scholar?q=Diederik+P%2E+Kingma+and+Jimmy+Ba%2E+2015%2E+Adam%3A+A+method+for+stochastic+optimization%2E+In+3rd+International+Conference+on+Learning+Representations%2C+ICLR+2015%2C+San+Diego%2C+CA%2C+USA%2C+May+7%2D9%2C+2015%2C+Conference)
28. Tom Kwiatkowski, Jennimaria Palomaki, Olivia Redfield, Michael Collins, Ankur P. Parikh, Chris Alberti, Danielle Epstein, Illia Polosukhin, Jacob Devlin, Kenton Lee, Kristina Toutanova, Llion Jones, Matthew Kelcey, Ming-Wei Chang, Andrew M. Dai, Jakob Uszkoreit, Quoc Le, and Slav Petrov. 2019. [find](https://scholar.google.com/scholar?q=Tom+Kwiatkowski%2C+Jennimaria+Palomaki%2C+Olivia+Redfield%2C+Michael+Collins%2C+Ankur+P%2E+Parikh%2C+Chris+Alberti%2C+Danielle+Epstein%2C+Illia+Polosukhin%2C+Jacob+Devlin%2C+Kenton+Lee%2C+Kristina+Toutanova%2C+Llion+Jones%2C+M)
29. Donald Metzler, Yi Tay, Dara Bahri, and Marc Najork. 2021. Rethinking search: Making experts out of dilettantes. CoRR, abs/2105.02274. — [arXiv:2105.02274](https://arxiv.org/abs/2105.02274) [find](https://scholar.google.com/scholar?q=Donald+Metzler%2C+Yi+Tay%2C+Dara+Bahri%2C+and+Marc+Najork%2E+2021%2E+Rethinking+search%3A+Making+experts+out+of+dilettantes%2E+CoRR%2C+abs%2F2105%2E02274%2E+%E2%80%94+%5BarXiv%3A2105%2E02274%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2105%2E02274%29)
30. Tri Nguyen, Mir Rosenberg, Xia Song, Jianfeng Gao, Saurabh Tiwary, Rangan Majumder, and Li Deng. 2016. Ms marco: A human-generated machine reading comprehension dataset. [find](https://scholar.google.com/scholar?q=Tri+Nguyen%2C+Mir+Rosenberg%2C+Xia+Song%2C+Jianfeng+Gao%2C+Saurabh+Tiwary%2C+Rangan+Majumder%2C+and+Li+Deng%2E+2016%2E+Ms+marco%3A+A+human%2Dgenerated+machine+reading+comprehension+dataset%2E) [our card](https://md.research.elamur.ai/free/en/papers/bajaj-2016-ms-marco-human-generated/)
31. Rodrigo Nogueira and Kyunghyun Cho. 2019. Passage re-ranking with BERT. CoRR, abs/1901.04085. [find](https://scholar.google.com/scholar?q=Rodrigo+Nogueira+and+Kyunghyun+Cho%2E+2019%2E+Passage+re%2Dranking+with+BERT%2E+CoRR%2C+abs%2F1901%2E04085%2E) [our card](https://md.research.elamur.ai/free/en/papers/nogueira-2019-passage-re-ranking-bert/)
32. Rodrigo Nogueira, Jimmy Lin, and AI Epistemic. 2019a. From doc2query to doctttttquery. Online preprint. [find](https://scholar.google.com/scholar?q=Rodrigo+Nogueira%2C+Jimmy+Lin%2C+and+AI+Epistemic%2E+2019a%2E+From+doc2query+to+doctttttquery%2E+Online+preprint%2E)
33. Rodrigo Nogueira, Wei Yang, Kyunghyun Cho, and Jimmy Lin. 2019b. Multi-stage document ranking with BERT. CoRR, abs/1910.14424. — [arXiv:1910.14424](https://arxiv.org/abs/1910.14424) [find](https://scholar.google.com/scholar?q=Rodrigo+Nogueira%2C+Wei+Yang%2C+Kyunghyun+Cho%2C+and+Jimmy+Lin%2E+2019b%2E+Multi%2Dstage+document+ranking+with+BERT%2E+CoRR%2C+abs%2F1910%2E14424%2E+%E2%80%94+%5BarXiv%3A1910%2E14424%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F1910%2E14424%29)
34. Rodrigo Nogueira, Wei Yang, Jimmy Lin, and Kyunghyun Cho. 2019c. Document expansion by query prediction. CoRR, abs/1904.08375. [find](https://scholar.google.com/scholar?q=Rodrigo+Nogueira%2C+Wei+Yang%2C+Jimmy+Lin%2C+and+Kyunghyun+Cho%2E+2019c%2E+Document+expansion+by+query+prediction%2E+CoRR%2C+abs%2F1904%2E08375%2E) [our card](https://md.research.elamur.ai/free/en/papers/nogueira-2019-document-expansion-query-prediction/)
35. Yifan Qiao, Chenyan Xiong, Zhenghao Liu, and Zhiyuan Liu. 2019. Understanding the behaviors of BERT in ranking. CoRR, abs/1904.07531. — [arXiv:1904.07531](https://arxiv.org/abs/1904.07531) [find](https://scholar.google.com/scholar?q=Yifan+Qiao%2C+Chenyan+Xiong%2C+Zhenghao+Liu%2C+and+Zhiyuan+Liu%2E+2019%2E+Understanding+the+behaviors+of+BERT+in+ranking%2E+CoRR%2C+abs%2F1904%2E07531%2E+%E2%80%94+%5BarXiv%3A1904%2E07531%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F1904%2E07531%29)
36. Yingqi Qu, Yuchen Ding, Jing Liu, Kai Liu, Ruiyang Ren, Wayne Xin Zhao, Daxiang Dong, Hua Wu, and Haifeng Wang. 2021. Rocketqa: An optimized training approach to dense passage retrieval for open-domain question answering. In Proceedings of the 2021 Conference of the North American Chapter of the Ass… [find](https://scholar.google.com/scholar?q=Yingqi+Qu%2C+Yuchen+Ding%2C+Jing+Liu%2C+Kai+Liu%2C+Ruiyang+Ren%2C+Wayne+Xin+Zhao%2C+Daxiang+Dong%2C+Hua+Wu%2C+and+Haifeng+Wang%2E+2021%2E+Rocketqa%3A+An+optimized+training+approach+to+dense+passage+retrieval+for+open%2Ddomai) [our card](https://md.research.elamur.ai/free/en/papers/qu-2021-rocketqa-optimized-training-approach-dense/)
37. Ruiyang Ren, Shangwen Lv, Yingqi Qu, Jing Liu, Wayne Xin Zhao, QiaoQiao She, Hua Wu, Haifeng Wang, and Ji-Rong Wen. 2021. PAIR: Leveraging passage-centric similarity relation for improving dense passage retrieval. In Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021, pages 2… [find](https://scholar.google.com/scholar?q=Ruiyang+Ren%2C+Shangwen+Lv%2C+Yingqi+Qu%2C+Jing+Liu%2C+Wayne+Xin+Zhao%2C+QiaoQiao+She%2C+Hua+Wu%2C+Haifeng+Wang%2C+and+Ji%2DRong+Wen%2E+2021%2E+PAIR%3A+Leveraging+passage%2Dcentric+similarity+relation+for+improving+dense+passa)
38. Devendra Singh Sachan, Mostofa Patwary, Mohammad Shoeybi, Neel Kant, Wei Ping, William L. Hamilton, and Bryan Catanzaro. 2021. End-to-end training of neural retrievers for open-domain question answering. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and t… [find](https://scholar.google.com/scholar?q=Devendra+Singh+Sachan%2C+Mostofa+Patwary%2C+Mohammad+Shoeybi%2C+Neel+Kant%2C+Wei+Ping%2C+William+L%2E+Hamilton%2C+and+Bryan+Catanzaro%2E+2021%2E+End%2Dto%2Dend+training+of+neural+retrievers+for+open%2Ddomain+question+answeri)
39. Yu Sun, Shuohuan Wang, Yu-Kun Li, Shikun Feng, Hao Tian, Hua Wu, and Haifeng Wang. 2020. ERNIE 2.0: A continual pre-training framework for language understanding. In The Thirty-Fourth AAAI Conference on Artificial Intelligence, AAAI 2020, The Thirty-Second Innovative Applications of Artificial Intel… [find](https://scholar.google.com/scholar?q=Yu+Sun%2C+Shuohuan+Wang%2C+Yu%2DKun+Li%2C+Shikun+Feng%2C+Hao+Tian%2C+Hua+Wu%2C+and+Haifeng+Wang%2E+2020%2E+ERNIE+2%2E0%3A+A+continual+pre%2Dtraining+framework+for+language+understanding%2E+In+The+Thirty%2DFourth+AAAI+Conference+) [our card](https://md.research.elamur.ai/free/en/papers/sun-2019-ernie-2-0-continual/)
40. Wenhan Xiong, Xiang Lorraine Li, Srinivasan Iyer, Jingfei Du, Patrick S. H. Lewis, William Yang Wang, Yashar Mehdad, Wen-tau Yih, Sebastian Riedel, Douwe Kiela, and Barlas Oguz. 2020b. Answering complex open-domain questions with multi-hop dense retrieval. CoRR, abs/2009.12756. — [arXiv:2009.12756](https://arxiv.org/abs/2009.12756) [find](https://scholar.google.com/scholar?q=Wenhan+Xiong%2C+Xiang+Lorraine+Li%2C+Srinivasan+Iyer%2C+Jingfei+Du%2C+Patrick+S%2E+H%2E+Lewis%2C+William+Yang+Wang%2C+Yashar+Mehdad%2C+Wen%2Dtau+Yih%2C+Sebastian+Riedel%2C+Douwe+Kiela%2C+and+Barlas+Oguz%2E+2020b%2E+Answering+compl)
41. Ming Yan, Chenliang Li, Chen Wu, Bin Bi, Wei Wang, Jiangnan Xia, and Luo Si. 2019. IDST at TREC 2019 deep learning track: Deep cascade ranking with generation-based document expansion and pre-trained language modeling. In Proceedings of the Twenty-Eighth Text REtrieval Conference, TREC 2019, Gaither… [find](https://scholar.google.com/scholar?q=Ming+Yan%2C+Chenliang+Li%2C+Chen+Wu%2C+Bin+Bi%2C+Wei+Wang%2C+Jiangnan+Xia%2C+and+Luo+Si%2E+2019%2E+IDST+at+TREC+2019+deep+learning+track%3A+Deep+cascade+ranking+with+generation%2Dbased+document+expansion+and+pre%2Dtrained+)
42. Peilin Yang, Hui Fang, and Jimmy Lin. 2017. Anserini: Enabling the use of lucene for information retrieval research. In Proceedings of the 40th International ACM SIGIR Conference on Research and Development in Information Retrieval, Shinjuku, Tokyo, Japan, August 7-11, 2017, pages 1253–1256. [find](https://scholar.google.com/scholar?q=Peilin+Yang%2C+Hui+Fang%2C+and+Jimmy+Lin%2E+2017%2E+Anserini%3A+Enabling+the+use+of+lucene+for+information+retrieval+research%2E+In+Proceedings+of+the+40th+International+ACM+SIGIR+Conference+on+Research+and+Devel)
43. Sohee Yang and Minjoon Seo. 2020. Is retriever merely an approximator of reader? arXiv preprint arXiv:2010.10999. — [arXiv:2010.10999](https://arxiv.org/abs/2010.10999) [find](https://scholar.google.com/scholar?q=Sohee+Yang+and+Minjoon+Seo%2E+2020%2E+Is+retriever+merely+an+approximator+of+reader%3F+arXiv+preprint+arXiv%3A2010%2E10999%2E+%E2%80%94+%5BarXiv%3A2010%2E10999%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2010%2E10999%29)
44. Yinfei Yang, Ning Jin, Kuo Lin, Mandy Guo, and Daniel Cer. 2020. Neural retrieval for question answering with cross-attention supervised data augmentation. CoRR, abs/2009.13815. [find](https://scholar.google.com/scholar?q=Yinfei+Yang%2C+Ning+Jin%2C+Kuo+Lin%2C+Mandy+Guo%2C+and+Daniel+Cer%2E+2020%2E+Neural+retrieval+for+question+answering+with+cross%2Dattention+supervised+data+augmentation%2E+CoRR%2C+abs%2F2009%2E13815%2E)
45. Jingtao Zhan, Jiaxin Mao, Yiqun Liu, Jiafeng Guo, Min Zhang, and Shaoping Ma. 2021. Optimizing dense retrieval model training with hard negatives. CoRR, abs/2104.08051. — [arXiv:2104.08051](https://arxiv.org/abs/2104.08051) [find](https://scholar.google.com/scholar?q=Jingtao+Zhan%2C+Jiaxin+Mao%2C+Yiqun+Liu%2C+Jiafeng+Guo%2C+Min+Zhang%2C+and+Shaoping+Ma%2E+2021%2E+Optimizing+dense+retrieval+model+training+with+hard+negatives%2E+CoRR%2C+abs%2F2104%2E08051%2E+%E2%80%94+%5BarXiv%3A2104%2E08051%5D%28https%3A%2F%2Far)
46. Jingtao Zhan, Jiaxin Mao, Yiqun Liu, Min Zhang, and Shaoping Ma. 2020. Repbert: Contextualized text embeddings for first-stage retrieval. CoRR, abs/2006.15498. — [arXiv:2006.15498](https://arxiv.org/abs/2006.15498) [find](https://scholar.google.com/scholar?q=Jingtao+Zhan%2C+Jiaxin+Mao%2C+Yiqun+Liu%2C+Min+Zhang%2C+and+Shaoping+Ma%2E+2020%2E+Repbert%3A+Contextualized+text+embeddings+for+first%2Dstage+retrieval%2E+CoRR%2C+abs%2F2006%2E15498%2E+%E2%80%94+%5BarXiv%3A2006%2E15498%5D%28https%3A%2F%2Farxiv%2Eorg%2Fa)
47. Zhiguo Wang, Patrick Ng, Xiaofei Ma, Ramesh Nallapati, and Bing Xiang. 2019. Multi-passage BERT: A globally normalized BERT model for open-domain question answering. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference o… [find](https://scholar.google.com/scholar?q=Zhiguo+Wang%2C+Patrick+Ng%2C+Xiaofei+Ma%2C+Ramesh+Nallapati%2C+and+Bing+Xiang%2E+2019%2E+Multi%2Dpassage+BERT%3A+A+globally+normalized+BERT+model+for+open%2Ddomain+question+answering%2E+In+Proceedings+of+the+2019+Confere) [our card](https://md.research.elamur.ai/free/en/papers/wang-2019-multi-passage-bert-globally-normalized/)
48. Ledell Wu, Fabio Petroni, Martin Josifoski, Sebastian Riedel, and Luke Zettlemoyer. 2020. Scalable zero-shot entity linking with dense entity retrieval. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, EMNLP 2020, Online, November 16-20, 2020, pages 6397– 64… [find](https://scholar.google.com/scholar?q=Ledell+Wu%2C+Fabio+Petroni%2C+Martin+Josifoski%2C+Sebastian+Riedel%2C+and+Luke+Zettlemoyer%2E+2020%2E+Scalable+zero%2Dshot+entity+linking+with+dense+entity+retrieval%2E+In+Proceedings+of+the+2020+Conference+on+Empiri) [our card](https://md.research.elamur.ai/free/en/papers/wu-2020-scalable-zero-shot-entity-linking/)
49. Lee Xiong, Chenyan Xiong, Ye Li, Kwok-Fung Tang, Jialin Liu, Paul Bennett, Junaid Ahmed, and Arnold Overwijk. 2020a. Approximate nearest neighbor negative contrastive learning for dense text retrieval. CoRR, abs/2007.00808. [find](https://scholar.google.com/scholar?q=Lee+Xiong%2C+Chenyan+Xiong%2C+Ye+Li%2C+Kwok%2DFung+Tang%2C+Jialin+Liu%2C+Paul+Bennett%2C+Junaid+Ahmed%2C+and+Arnold+Overwijk%2E+2020a%2E+Approximate+nearest+neighbor+negative+contrastive+learning+for+dense+text+retrieval) [our card](https://md.research.elamur.ai/free/en/papers/xiong-2020-approximate-nearest-neighbor-negative-contrastive/)