---
title: "ColBERT: Efficient and Effective Passage — Khattab 2020"
description: "BERT-based rankers greatly improved passage search, but they are very expensive to run."
authors: ["Omar Khattab", "Matei Zaharia"]
year: 2020
venue: "SIGIR 2020"
source: "https://arxiv.org/abs/2004.12832"
doi: "10.1145/3397271.3401075"
arxiv: "2004.12832"
licence: "arXiv — non-exclusive licence to distribute"
licence_url: "http://arxiv.org/licenses/nonexclusive-distrib/1.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/khattab-2020-colbert-efficient-effective-passage/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/khattab-2020-colbert-efficient-effective-passage/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2020](https://md.research.elamur.ai/free/en/papers/2020/page/2/) / ColBERT: Efficient and Effective Passage Search via Contextualized Late Interaction over BERT

# ColBERT: Efficient and Effective Passage Search via Contextualized Late Interaction over BERT

**Authors:** [Omar Khattab](https://md.research.elamur.ai/free/en/authors/omar-khattab/), [Matei Zaharia](https://md.research.elamur.ai/free/en/authors/matei-zaharia/) · **Year:** 2020 · **Venue:** SIGIR 2020 · **Licence:** [arXiv — non-exclusive licence to distribute](http://arxiv.org/licenses/nonexclusive-distrib/1.0/)

**Links:** [Publisher's page](https://doi.org/10.1145/3397271.3401075) · [PDF](https://arxiv.org/pdf/2004.12832) · [DOI 10.1145/3397271.3401075](https://doi.org/10.1145/3397271.3401075) · [arXiv 2004.12832](https://arxiv.org/abs/2004.12832) · [OpenAlex](https://openalex.org/W3021397474)

**Citations:** 1248 citations (openalex-snapshot · 2026-10-14) · [526](#cited-by) in our library, of the 140673 papers whose bibliographies we read

## Summary
BERT-based rankers greatly improved passage search, but they are very expensive to run. Each query–document pair has to go through the full transformer, so a single query can take seconds on a GPU. ColBERT addresses this with "late interaction". BERT encodes the query and each document separately into sets of per-token embeddings. Relevance is then scored with a light MaxSim step: every query token finds its most similar document token, and these maxima are added up. Because documents are encoded without the query, their embeddings can be computed and stored offline. The query is encoded only once per search. The same scoring also works with vector-similarity indexes, so ColBERT can either re-rank BM25 candidates or retrieve end-to-end from the whole collection. The authors test it on MS MARCO and TREC CAR. Its accuracy is close to existing BERT rankers and better than every non-BERT baseline, while it is about two orders of magnitude faster. This makes neural ranking with pretrained LMs practical at production latencies.

## Key points
- In re-ranking mode, ColBERT is over 170× faster than earlier BERT rankers and needs about 14,000× fewer FLOPs per query. Its main model reaches MRR@10 of 34.9% on the MS MARCO dev set.
- End-to-end retrieval improves both MRR@10 and recall over re-ranking BM25 output. ColBERT's Recall@50 is higher than the official BM25 Recall@1000.
- Ablations on a 5-layer BERT variant show that each part matters: a single-vector [CLS] dot-product model, swapping MaxSim for average similarity, and removing query augmentation all lower MRR@10.
- Indexing the roughly 9M-passage MS MARCO collection takes about 3 hours on one server with four GPUs, because each document goes through BERT only once.
- Storage can drop from 286 GiB to 27 GiB by using 24-dimensional embeddings stored as 2-byte floats, at a cost of about 1 point of MRR@10.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

Fields: [Computing & GPUs](https://md.research.elamur.ai/free/en/shelves/computing/)

## Cited by

Cited by

526 papers of our library cite it (of the 140673 whose bibliographies we read); 28 of them have a page here, 498 not yet.

- [Task-aware Retrieval with Instructions](https://md.research.elamur.ai/free/en/papers/asai-2023-task-aware-retrieval-instructions/)
- [LLM2Vec: Large Language Models Are Secretly Powerful Text Encoders](https://md.research.elamur.ai/free/en/papers/behnamghader-2024-llm2vec-large-language-models-secretly/)
- [Enhanced Arabic Text Retrieval with Attentive Relevance Scoring](https://md.research.elamur.ai/free/en/papers/bekhouche-2025-enhanced-arabic-text-retrieval-attentive/)
- [M3-Embedding: Multi-Linguality, Multi-Functionality, Multi-Granularity Text Embeddings Through Self-Knowledge Distillation](https://md.research.elamur.ai/free/en/papers/chen-2024-m3-embedding-multi-linguality/)
- [Promptagator: Few-shot Dense Retrieval From 8 Examples](https://md.research.elamur.ai/free/en/papers/dai-2022-promptagator-few-shot-dense-retrieval/)
- [Cross-document Event Coreference Search: Task, Dataset and Modeling](https://md.research.elamur.ai/free/en/papers/eirew-2022-cross-document-event-coreference-search/)
- [A Survey on RAG Meeting LLMs: Towards Retrieval-Augmented Large Language Models](https://md.research.elamur.ai/free/en/papers/fan-2024-survey-rag-meeting-llms-towards/)
- [SPLADE: Sparse Lexical and Expansion Model for First Stage Ranking](https://md.research.elamur.ai/free/en/papers/formal-2021-splade-sparse-lexical-expansion/)
- [Efficiently Teaching an Effective Dense Retriever with Balanced Topic Aware Sampling](https://md.research.elamur.ai/free/en/papers/hofstatter-2021-efficiently-teaching-effective-dense-retriever/)
- [Bridging Cultures in the Kitchen: A Framework and Benchmark for Cross-Cultural Recipe Retrieval](https://md.research.elamur.ai/free/en/papers/hu-2024-bridging-cultures-kitchen-framework-benchmark/)
- [Dense Passage Retrieval for Open-Domain Question Answering](https://md.research.elamur.ai/free/en/papers/karpukhin-2020-dense-passage-retrieval-open-domain/)
- [Demonstrate-Search-Predict: Composing retrieval and language models for knowledge-intensive NLP](https://md.research.elamur.ai/free/en/papers/khattab-2022-demonstrate-search-predict-composing-retrieval/)
- [Paraphrasing evades detectors of AI-generated text, but retrieval is an effective defense](https://md.research.elamur.ai/free/en/papers/krishna-2023-paraphrasing-evades-detectors-ai-generated/)
- [RA-DIT: Retrieval-Augmented Dual Instruction Tuning](https://md.research.elamur.ai/free/en/papers/lin-2023-ra-dit-retrieval-augmented-dual/)
- [SGPT: GPT Sentence Embeddings for Semantic Search](https://md.research.elamur.ai/free/en/papers/muennighoff-2022-sgpt-gpt-sentence-embeddings-semantic/)
- [Large Dual Encoders Are Generalizable Retrievers](https://md.research.elamur.ai/free/en/papers/ni-2021-large-dual-encoders-generalizable-retrievers/)
- [RocketQA: An Optimized Training Approach to Dense Passage Retrieval for Open-Domain Question Answering](https://md.research.elamur.ai/free/en/papers/qu-2021-rocketqa-optimized-training-approach-dense/)
- [RocketQAv2: A Joint Training Method for Dense Passage Retrieval and Passage Re-ranking](https://md.research.elamur.ai/free/en/papers/ren-2021-rocketqav2-joint-training-method-dense/)
- [Learning To Retrieve Prompts for In-Context Learning](https://md.research.elamur.ai/free/en/papers/rubin-2021-learning-retrieve-prompts-context-learning/)
- [ColBERTv2: Effective and Efficient Retrieval via Lightweight Late Interaction](https://md.research.elamur.ai/free/en/papers/santhanam-2022-colbertv2-effective-efficient-retrieval-lightweight/)
- [RAPTOR: Recursive Abstractive Processing for Tree-Organized Retrieval](https://md.research.elamur.ai/free/en/papers/sarthi-2024-raptor-recursive-abstractive-processing-tree/)
- [BEIR: A Heterogenous Benchmark for Zero-shot Evaluation of Information Retrieval Models](https://md.research.elamur.ai/free/en/papers/thakur-2021-beir-heterogenous-benchmark-zero/)
- [A Study on Building Efficient Zero-Shot Relation Extraction Models](https://md.research.elamur.ai/free/en/papers/thomas-2026-study-building-efficient-zero-shot/)
- [Text Embeddings by Weakly-Supervised Contrastive Pre-training](https://md.research.elamur.ai/free/en/papers/wang-2022-text-embeddings-weakly-supervised-contrastive/)
- [Smarter, Better, Faster, Longer: A Modern Bidirectional Encoder for Fast, Memory Efficient, and Long Context Finetuning and Inference](https://md.research.elamur.ai/free/en/papers/warner-2024-smarter-better-faster-longer-modern/)
- [Approximate Nearest Neighbor Negative Contrastive Learning for Dense Text Retrieval](https://md.research.elamur.ai/free/en/papers/xiong-2020-approximate-nearest-neighbor-negative-contrastive/)
- [Making Retrieval-Augmented Language Models Robust to Irrelevant Context](https://md.research.elamur.ai/free/en/papers/yoran-2023-making-retrieval-augmented-language-models/)
- [MIRACL: A Multilingual Retrieval Dataset Covering 18 Diverse Languages](https://md.research.elamur.ai/free/en/papers/zhang-2023-miracl-multilingual-retrieval-dataset-covering/)

## Cite

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{khattab2020colbert,
  title = {{ColBERT: Efficient and Effective Passage Search via Contextualized Late Interaction over BERT}},
  author = {Khattab, Omar and Zaharia, Matei},
  booktitle = {SIGIR 2020},
  year = {2020},
  doi = {10.1145/3397271.3401075},
  eprint = {2004.12832},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.1145/3397271.3401075}
}
```

- [← previous CodeSearchNet Challenge: Evaluating the State of Semantic Code Search](https://md.research.elamur.ai/free/en/papers/husain-2020-codesearchnet-challenge-evaluating-state-semantic/)
- [next → Combining Automatic Labelers and Expert Annotations for Accurate Radiology Report Labeling Using BERT](https://md.research.elamur.ai/free/en/papers/smit-2020-combining-automatic-labelers-expert-annotations/)
- [↑ back to papers of 2020](https://md.research.elamur.ai/free/en/papers/2020/page/2/)

## References

42 references, 6 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Firas Abuzaid, Geet Sethi, Peter Bailis, and Matei Zaharia. 2019. To Index or Not to Index: Optimizing Exact Maximum Inner Product Search. In 2019 IEEE 35th International Conference on Data Engineering (ICDE). IEEE, 1250–1261. [find](https://scholar.google.com/scholar?q=Firas+Abuzaid%2C+Geet+Sethi%2C+Peter+Bailis%2C+and+Matei+Zaharia%2E+2019%2E+To+Index+or+Not+to+Index%3A+Optimizing+Exact+Maximum+Inner+Product+Search%2E+In+2019+IEEE+35th+International+Conference+on+Data+Engineerin)
2. Zhuyun Dai and Jamie Callan. 2019. Context-Aware Sentence/Passage Term Importance Estimation For First Stage Retrieval. arXiv preprint arXiv:1910.10687 (2019). — [arXiv:1910.10687](https://arxiv.org/abs/1910.10687) [find](https://scholar.google.com/scholar?q=Zhuyun+Dai+and+Jamie+Callan%2E+2019%2E+Context%2DAware+Sentence%2FPassage+Term+Importance+Estimation+For+First+Stage+Retrieval%2E+arXiv+preprint+arXiv%3A1910%2E10687+%282019%29%2E+%E2%80%94+%5BarXiv%3A1910%2E10687%5D%28https%3A%2F%2Farxiv%2Eorg%2Fa)
3. Zhuyun Dai and Jamie Callan. 2019. Deeper Text Understanding for IR with Contextual Neural Language Modeling. arXiv preprint arXiv:1905.09217 (2019). — [arXiv:1905.09217](https://arxiv.org/abs/1905.09217) [find](https://scholar.google.com/scholar?q=Zhuyun+Dai+and+Jamie+Callan%2E+2019%2E+Deeper+Text+Understanding+for+IR+with+Contextual+Neural+Language+Modeling%2E+arXiv+preprint+arXiv%3A1905%2E09217+%282019%29%2E+%E2%80%94+%5BarXiv%3A1905%2E09217%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F1905%2E09)
4. Zhuyun Dai, Chenyan Xiong, Jamie Callan, and Zhiyuan Liu. 2018. Convolutional neural networks for soft-matching n-grams in ad-hoc search. In Proceedings of the eleventh ACM international conference on web search and data mining. 126–134. [find](https://scholar.google.com/scholar?q=Zhuyun+Dai%2C+Chenyan+Xiong%2C+Jamie+Callan%2C+and+Zhiyuan+Liu%2E+2018%2E+Convolutional+neural+networks+for+soft%2Dmatching+n%2Dgrams+in+ad%2Dhoc+search%2E+In+Proceedings+of+the+eleventh+ACM+international+conference+on)
5. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805 (2018). [find](https://scholar.google.com/scholar?q=Jacob+Devlin%2C+Ming%2DWei+Chang%2C+Kenton+Lee%2C+and+Kristina+Toutanova%2E+2018%2E+Bert%3A+Pre%2Dtraining+of+deep+bidirectional+transformers+for+language+understanding%2E+arXiv+preprint+arXiv%3A1810%2E04805+%282018%29%2E)
6. Laura Dietz, Manisha Verma, Filip Radlinski, and Nick Craswell. 2017. TREC Complex Answer Retrieval Overview.. In TREC. [find](https://scholar.google.com/scholar?q=Laura+Dietz%2C+Manisha+Verma%2C+Filip+Radlinski%2C+and+Nick+Craswell%2E+2017%2E+TREC+Complex+Answer+Retrieval+Overview%2E%2E+In+TREC%2E)
7. Jiafeng Guo, Yixing Fan, Qingyao Ai, and W Bruce Croft. 2016. A deep relevance matching model for ad-hoc retrieval. In Proceedings of the 25th ACM International on Conference on Information and Knowledge Management. ACM, 55–64. [find](https://scholar.google.com/scholar?q=Jiafeng+Guo%2C+Yixing+Fan%2C+Qingyao+Ai%2C+and+W+Bruce+Croft%2E+2016%2E+A+deep+relevance+matching+model+for+ad%2Dhoc+retrieval%2E+In+Proceedings+of+the+25th+ACM+International+on+Conference+on+Information+and+Knowle)
8. Jiafeng Guo, Yixing Fan, Liang Pang, Liu Yang, Qingyao Ai, Hamed Zamani, Chen Wu, W Bruce Croft, and Xueqi Cheng. 2019. A deep look into neural ranking models for information retrieval. arXiv preprint arXiv:1903.06902 (2019). — [arXiv:1903.06902](https://arxiv.org/abs/1903.06902) [find](https://scholar.google.com/scholar?q=Jiafeng+Guo%2C+Yixing+Fan%2C+Liang+Pang%2C+Liu+Yang%2C+Qingyao+Ai%2C+Hamed+Zamani%2C+Chen+Wu%2C+W+Bruce+Croft%2C+and+Xueqi+Cheng%2E+2019%2E+A+deep+look+into+neural+ranking+models+for+information+retrieval%2E+arXiv+preprint)
9. Sebastian Hofstätter and Allan Hanbury. 2019. Let’s measure run time! Extending the IR replicability infrastructure to include performance aspects. arXiv preprint arXiv:1907.04614 (2019). — [arXiv:1907.04614](https://arxiv.org/abs/1907.04614) [find](https://scholar.google.com/scholar?q=Sebastian+Hofsta%CC%88tter+and+Allan+Hanbury%2E+2019%2E+Let%E2%80%99s+measure+run+time%21+Extending+the+IR+replicability+infrastructure+to+include+performance+aspects%2E+arXiv+preprint+arXiv%3A1907%2E04614+%282019%29%2E+%E2%80%94+%5BarXiv%3A19)
10. Sebastian Hofstätter, Navid Rekabsaz, Carsten Eickhoff, and Allan Hanbury. 2019. On the effect of low-frequency terms on neural-IR models. In Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval. 1137–1140. [find](https://scholar.google.com/scholar?q=Sebastian+Hofsta%CC%88tter%2C+Navid+Rekabsaz%2C+Carsten+Eickhoff%2C+and+Allan+Hanbury%2E+2019%2E+On+the+effect+of+low%2Dfrequency+terms+on+neural%2DIR+models%2E+In+Proceedings+of+the+42nd+International+ACM+SIGIR+Conferenc)
11. Sebastian Hofstätter, Markus Zlabinger, and Allan Hanbury. 2019. TU Wien@ TREC Deep Learning’19–Simple Contextualization for Re-ranking. arXiv preprint arXiv:1912.01385 (2019). — [arXiv:1912.01385](https://arxiv.org/abs/1912.01385) [find](https://scholar.google.com/scholar?q=Sebastian+Hofsta%CC%88tter%2C+Markus+Zlabinger%2C+and+Allan+Hanbury%2E+2019%2E+TU+Wien%40+TREC+Deep+Learning%E2%80%9919%E2%80%93Simple+Contextualization+for+Re%2Dranking%2E+arXiv+preprint+arXiv%3A1912%2E01385+%282019%29%2E+%E2%80%94+%5BarXiv%3A1912%2E01385%5D%28h)
12. Po-Sen Huang, Xiaodong He, Jianfeng Gao, Li Deng, Alex Acero, and Larry Heck. 2013. Learning deep structured semantic models for web search using clickthrough data. In Proceedings of the 22nd ACM international conference on Information & Knowledge Management. 2333–2338. [find](https://scholar.google.com/scholar?q=Po%2DSen+Huang%2C+Xiaodong+He%2C+Jianfeng+Gao%2C+Li+Deng%2C+Alex+Acero%2C+and+Larry+Heck%2E+2013%2E+Learning+deep+structured+semantic+models+for+web+search+using+clickthrough+data%2E+In+Proceedings+of+the+22nd+ACM+inte)
13. Shiyu Ji, Jinjin Shao, and Tao Yang. 2019. Efficient Interaction-based Neural Ranking with Locality Sensitive Hashing. In The World Wide Web Conference. ACM, 2858–2864. [find](https://scholar.google.com/scholar?q=Shiyu+Ji%2C+Jinjin+Shao%2C+and+Tao+Yang%2E+2019%2E+Efficient+Interaction%2Dbased+Neural+Ranking+with+Locality+Sensitive+Hashing%2E+In+The+World+Wide+Web+Conference%2E+ACM%2C+2858%E2%80%932864%2E)
14. Xiaoqi Jiao, Yichun Yin, Lifeng Shang, Xin Jiang, Xiao Chen, Linlin Li, Fang Wang, and Qun Liu. 2019. Tinybert: Distilling bert for natural language understanding. arXiv preprint arXiv:1909.10351 (2019). [find](https://scholar.google.com/scholar?q=Xiaoqi+Jiao%2C+Yichun+Yin%2C+Lifeng+Shang%2C+Xin+Jiang%2C+Xiao+Chen%2C+Linlin+Li%2C+Fang+Wang%2C+and+Qun+Liu%2E+2019%2E+Tinybert%3A+Distilling+bert+for+natural+language+understanding%2E+arXiv+preprint+arXiv%3A1909%2E10351+%28201)
15. Jeff Johnson, Matthijs Douze, and Hervé Jégou. 2017. Billion-scale similarity search with GPUs. arXiv preprint arXiv:1702.08734 (2017). [find](https://scholar.google.com/scholar?q=Jeff+Johnson%2C+Matthijs+Douze%2C+and+Herve%CC%81+Je%CC%81gou%2E+2017%2E+Billion%2Dscale+similarity+search+with+GPUs%2E+arXiv+preprint+arXiv%3A1702%2E08734+%282017%29%2E) [our card](https://md.research.elamur.ai/free/en/papers/johnson-2017-billion-scale-similarity-search-gpus/)
16. Diederik P Kingma and Jimmy Ba. 2014. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014). [find](https://scholar.google.com/scholar?q=Diederik+P+Kingma+and+Jimmy+Ba%2E+2014%2E+Adam%3A+A+method+for+stochastic+optimization%2E+arXiv+preprint+arXiv%3A1412%2E6980+%282014%29%2E)
17. Ron Kohavi, Alex Deng, Brian Frasca, Toby Walker, Ya Xu, and Nils Pohlmann. 2013. Online controlled experiments at large scale. In SIGKDD. [find](https://scholar.google.com/scholar?q=Ron+Kohavi%2C+Alex+Deng%2C+Brian+Frasca%2C+Toby+Walker%2C+Ya+Xu%2C+and+Nils+Pohlmann%2E+2013%2E+Online+controlled+experiments+at+large+scale%2E+In+SIGKDD%2E)
18. Sean MacAvaney, Andrew Yates, Arman Cohan, and Nazli Goharian. 2019. Cedr: Contextualized embeddings for document ranking. In Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval. ACM, 1101–1104. [find](https://scholar.google.com/scholar?q=Sean+MacAvaney%2C+Andrew+Yates%2C+Arman+Cohan%2C+and+Nazli+Goharian%2E+2019%2E+Cedr%3A+Contextualized+embeddings+for+document+ranking%2E+In+Proceedings+of+the+42nd+International+ACM+SIGIR+Conference+on+Research+and)
19. Paul Michel, Omer Levy, and Graham Neubig. 2019. Are Sixteen Heads Really Better than One?. In Advances in Neural Information Processing Systems. 14014– 14024. [find](https://scholar.google.com/scholar?q=Paul+Michel%2C+Omer+Levy%2C+and+Graham+Neubig%2E+2019%2E+Are+Sixteen+Heads+Really+Better+than+One%3F%2E+In+Advances+in+Neural+Information+Processing+Systems%2E+14014%E2%80%93+14024%2E) [our card](https://md.research.elamur.ai/free/en/papers/michel-2019-sixteen-heads-really-better-than/)
20. Bhaskar Mitra and Nick Craswell. 2019. An Updated Duet Model for Passage Re-ranking. arXiv preprint arXiv:1903.07666 (2019). — [arXiv:1903.07666](https://arxiv.org/abs/1903.07666) [find](https://scholar.google.com/scholar?q=Bhaskar+Mitra+and+Nick+Craswell%2E+2019%2E+An+Updated+Duet+Model+for+Passage+Re%2Dranking%2E+arXiv+preprint+arXiv%3A1903%2E07666+%282019%29%2E+%E2%80%94+%5BarXiv%3A1903%2E07666%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F1903%2E07666%29)
21. Bhaskar Mitra, Nick Craswell, et al. 2018. An introduction to neural information retrieval. Foundations and Trends® in Information Retrieval 13, 1 (2018), 1–126. [find](https://scholar.google.com/scholar?q=Bhaskar+Mitra%2C+Nick+Craswell%2C+et+al%2E+2018%2E+An+introduction+to+neural+information+retrieval%2E+Foundations+and+Trends%C2%AE+in+Information+Retrieval+13%2C+1+%282018%29%2C+1%E2%80%93126%2E)
22. Bhaskar Mitra, Fernando Diaz, and Nick Craswell. 2017. Learning to match using local and distributed representations of text for web search. In Proceedings of the 26th International Conference on World Wide Web. International World Wide Web Conferences Steering Committee, 1291–1299. [find](https://scholar.google.com/scholar?q=Bhaskar+Mitra%2C+Fernando+Diaz%2C+and+Nick+Craswell%2E+2017%2E+Learning+to+match+using+local+and+distributed+representations+of+text+for+web+search%2E+In+Proceedings+of+the+26th+International+Conference+on+Worl)
23. Bhaskar Mitra, Corby Rosset, David Hawking, Nick Craswell, Fernando Diaz, and Emine Yilmaz. 2019. Incorporating query term independence assumption for efficient retrieval and ranking using deep neural networks. arXiv preprint arXiv:1907.03693 (2019). — [arXiv:1907.03693](https://arxiv.org/abs/1907.03693) [find](https://scholar.google.com/scholar?q=Bhaskar+Mitra%2C+Corby+Rosset%2C+David+Hawking%2C+Nick+Craswell%2C+Fernando+Diaz%2C+and+Emine+Yilmaz%2E+2019%2E+Incorporating+query+term+independence+assumption+for+efficient+retrieval+and+ranking+using+deep+neural)
24. Tri Nguyen, Mir Rosenberg, Xia Song, Jianfeng Gao, Saurabh Tiwary, Rangan Majumder, and Li Deng. 2016. MS MARCO: A Human-Generated MAchine Reading COmprehension Dataset. (2016). [find](https://scholar.google.com/scholar?q=Tri+Nguyen%2C+Mir+Rosenberg%2C+Xia+Song%2C+Jianfeng+Gao%2C+Saurabh+Tiwary%2C+Rangan+Majumder%2C+and+Li+Deng%2E+2016%2E+MS+MARCO%3A+A+Human%2DGenerated+MAchine+Reading+COmprehension+Dataset%2E+%282016%29%2E) [our card](https://md.research.elamur.ai/free/en/papers/bajaj-2016-ms-marco-human-generated/)
25. Rodrigo Nogueira and Kyunghyun Cho. 2019. Passage Re-ranking with BERT. arXiv preprint arXiv:1901.04085 (2019). [find](https://scholar.google.com/scholar?q=Rodrigo+Nogueira+and+Kyunghyun+Cho%2E+2019%2E+Passage+Re%2Dranking+with+BERT%2E+arXiv+preprint+arXiv%3A1901%2E04085+%282019%29%2E) [our card](https://md.research.elamur.ai/free/en/papers/nogueira-2019-passage-re-ranking-bert/)
26. Rodrigo Nogueira, Jimmy Lin, and AI Epistemic. 2019. From doc2query to docTTTTTquery. (2019). [find](https://scholar.google.com/scholar?q=Rodrigo+Nogueira%2C+Jimmy+Lin%2C+and+AI+Epistemic%2E+2019%2E+From+doc2query+to+docTTTTTquery%2E+%282019%29%2E)
27. Rodrigo Nogueira, Wei Yang, Kyunghyun Cho, and Jimmy Lin. 2019. Multi-Stage Document Ranking with BERT. arXiv preprint arXiv:1910.14424 (2019). — [arXiv:1910.14424](https://arxiv.org/abs/1910.14424) [find](https://scholar.google.com/scholar?q=Rodrigo+Nogueira%2C+Wei+Yang%2C+Kyunghyun+Cho%2C+and+Jimmy+Lin%2E+2019%2E+Multi%2DStage+Document+Ranking+with+BERT%2E+arXiv+preprint+arXiv%3A1910%2E14424+%282019%29%2E+%E2%80%94+%5BarXiv%3A1910%2E14424%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F1910%2E14424%29)
28. Rodrigo Nogueira, Wei Yang, Jimmy Lin, and Kyunghyun Cho. 2019. Document Expansion by Query Prediction. arXiv preprint arXiv:1904.08375 (2019). [find](https://scholar.google.com/scholar?q=Rodrigo+Nogueira%2C+Wei+Yang%2C+Jimmy+Lin%2C+and+Kyunghyun+Cho%2E+2019%2E+Document+Expansion+by+Query+Prediction%2E+arXiv+preprint+arXiv%3A1904%2E08375+%282019%29%2E) [our card](https://md.research.elamur.ai/free/en/papers/nogueira-2019-document-expansion-query-prediction/)
29. Matthew E Peters, Mark Neumann, Mohit Iyyer, Matt Gardner, Christopher Clark, Kenton Lee, and Luke Zettlemoyer. 2018. Deep contextualized word representations. arXiv preprint arXiv:1802.05365 (2018). [find](https://scholar.google.com/scholar?q=Matthew+E+Peters%2C+Mark+Neumann%2C+Mohit+Iyyer%2C+Matt+Gardner%2C+Christopher+Clark%2C+Kenton+Lee%2C+and+Luke+Zettlemoyer%2E+2018%2E+Deep+contextualized+word+representations%2E+arXiv+preprint+arXiv%3A1802%2E05365+%282018%29%2E)
30. Yifan Qiao, Chenyan Xiong, Zhenghao Liu, and Zhiyuan Liu. 2019. Understanding the Behaviors of BERT in Ranking. arXiv preprint arXiv:1904.07531 (2019). — [arXiv:1904.07531](https://arxiv.org/abs/1904.07531) [find](https://scholar.google.com/scholar?q=Yifan+Qiao%2C+Chenyan+Xiong%2C+Zhenghao+Liu%2C+and+Zhiyuan+Liu%2E+2019%2E+Understanding+the+Behaviors+of+BERT+in+Ranking%2E+arXiv+preprint+arXiv%3A1904%2E07531+%282019%29%2E+%E2%80%94+%5BarXiv%3A1904%2E07531%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F1904%2E)
31. Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J Liu. 2019. Exploring the limits of transfer learning with a unified text-to-text transformer. arXiv preprint arXiv:1910.10683 (2019). [find](https://scholar.google.com/scholar?q=Colin+Raffel%2C+Noam+Shazeer%2C+Adam+Roberts%2C+Katherine+Lee%2C+Sharan+Narang%2C+Michael+Matena%2C+Yanqi+Zhou%2C+Wei+Li%2C+and+Peter+J+Liu%2E+2019%2E+Exploring+the+limits+of+transfer+learning+with+a+unified+text%2Dto%2Dtext)
32. Stephen E Robertson, Steve Walker, Susan Jones, Micheline M Hancock-Beaulieu, Mike Gatford, et al. 1995. Okapi at TREC-3. NIST Special Publication (1995). [find](https://scholar.google.com/scholar?q=Stephen+E+Robertson%2C+Steve+Walker%2C+Susan+Jones%2C+Micheline+M+Hancock%2DBeaulieu%2C+Mike+Gatford%2C+et+al%2E+1995%2E+Okapi+at+TREC%2D3%2E+NIST+Special+Publication+%281995%29%2E) [our card](https://md.research.elamur.ai/free/en/papers/robertson-1994-okapi-trec-3/)
33. Raphael Tang, Yao Lu, Linqing Liu, Lili Mou, Olga Vechtomova, and Jimmy Lin. 2019. Distilling task-specific knowledge from BERT into simple neural networks. arXiv preprint arXiv:1903.12136 (2019). — [arXiv:1903.12136](https://arxiv.org/abs/1903.12136) [find](https://scholar.google.com/scholar?q=Raphael+Tang%2C+Yao+Lu%2C+Linqing+Liu%2C+Lili+Mou%2C+Olga+Vechtomova%2C+and+Jimmy+Lin%2E+2019%2E+Distilling+task%2Dspecific+knowledge+from+BERT+into+simple+neural+networks%2E+arXiv+preprint+arXiv%3A1903%2E12136+%282019%29%2E+%E2%80%94+%5B)
34. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In Advances in neural information processing systems. 5998–6008. [find](https://scholar.google.com/scholar?q=Ashish+Vaswani%2C+Noam+Shazeer%2C+Niki+Parmar%2C+Jakob+Uszkoreit%2C+Llion+Jones%2C+Aidan+N+Gomez%2C+Lukasz+Kaiser%2C+and+Illia+Polosukhin%2E+2017%2E+Attention+is+all+you+need%2E+In+Advances+in+neural+information+processi)
35. Yonghui Wu, Mike Schuster, Zhifeng Chen, Quoc V Le, Mohammad Norouzi, Wolfgang Macherey, Maxim Krikun, Yuan Cao, Qin Gao, Klaus Macherey, et al. 2016. Google’s neural machine translation system: Bridging the gap between human and machine translation. arXiv preprint arXiv:1609.08144 (2016). [find](https://scholar.google.com/scholar?q=Yonghui+Wu%2C+Mike+Schuster%2C+Zhifeng+Chen%2C+Quoc+V+Le%2C+Mohammad+Norouzi%2C+Wolfgang+Macherey%2C+Maxim+Krikun%2C+Yuan+Cao%2C+Qin+Gao%2C+Klaus+Macherey%2C+et+al%2E+2016%2E+Google%E2%80%99s+neural+machine+translation+system%3A+Bridg)
36. Chenyan Xiong, Zhuyun Dai, Jamie Callan, Zhiyuan Liu, and Russell Power. 2017. End-to-end neural ad-hoc ranking with kernel pooling. In Proceedings of the 40th International ACM SIGIR conference on research and development in information retrieval. 55–64. [find](https://scholar.google.com/scholar?q=Chenyan+Xiong%2C+Zhuyun+Dai%2C+Jamie+Callan%2C+Zhiyuan+Liu%2C+and+Russell+Power%2E+2017%2E+End%2Dto%2Dend+neural+ad%2Dhoc+ranking+with+kernel+pooling%2E+In+Proceedings+of+the+40th+International+ACM+SIGIR+conference+on+re)
37. Peilin Yang, Hui Fang, and Jimmy Lin. 2018. Anserini: Reproducible ranking baselines using Lucene. Journal of Data and Information Quality (JDIQ) 10, 4 (2018), 1–20. [find](https://scholar.google.com/scholar?q=Peilin+Yang%2C+Hui+Fang%2C+and+Jimmy+Lin%2E+2018%2E+Anserini%3A+Reproducible+ranking+baselines+using+Lucene%2E+Journal+of+Data+and+Information+Quality+%28JDIQ%29+10%2C+4+%282018%29%2C+1%E2%80%9320%2E)
38. Wei Yang, Kuang Lu, Peilin Yang, and Jimmy Lin. 2019. Critically Examining the” Neural Hype” Weak Baselines and the Additivity of Effectiveness Gains from Neural Ranking Models. In Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval. 1129–1… [find](https://scholar.google.com/scholar?q=Wei+Yang%2C+Kuang+Lu%2C+Peilin+Yang%2C+and+Jimmy+Lin%2E+2019%2E+Critically+Examining+the%E2%80%9D+Neural+Hype%E2%80%9D+Weak+Baselines+and+the+Additivity+of+Effectiveness+Gains+from+Neural+Ranking+Models%2E+In+Proceedings+of+the+)
39. Zeynep Akkalyoncu Yilmaz, Wei Yang, Haotian Zhang, and Jimmy Lin. 2019. Cross-domain modeling of sentence-level evidence for document retrieval. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Pr… [find](https://scholar.google.com/scholar?q=Zeynep+Akkalyoncu+Yilmaz%2C+Wei+Yang%2C+Haotian+Zhang%2C+and+Jimmy+Lin%2E+2019%2E+Cross%2Ddomain+modeling+of+sentence%2Dlevel+evidence+for+document+retrieval%2E+In+Proceedings+of+the+2019+Conference+on+Empirical+Meth)
40. Ofir Zafrir, Guy Boudoukh, Peter Izsak, and Moshe Wasserblat. 2019. Q8bert: Quantized 8bit bert. arXiv preprint arXiv:1910.06188 (2019). — [arXiv:1910.06188](https://arxiv.org/abs/1910.06188) [find](https://scholar.google.com/scholar?q=Ofir+Zafrir%2C+Guy+Boudoukh%2C+Peter+Izsak%2C+and+Moshe+Wasserblat%2E+2019%2E+Q8bert%3A+Quantized+8bit+bert%2E+arXiv+preprint+arXiv%3A1910%2E06188+%282019%29%2E+%E2%80%94+%5BarXiv%3A1910%2E06188%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F1910%2E06188%29)
41. Hamed Zamani, Mostafa Dehghani, W Bruce Croft, Erik Learned-Miller, and Jaap Kamps. 2018. From neural re-ranking to neural ranking: Learning a sparse representation for inverted indexing. In Proceedings of the 27th ACM International Conference on Information and Knowledge Management. ACM, 497–506. [find](https://scholar.google.com/scholar?q=Hamed+Zamani%2C+Mostafa+Dehghani%2C+W+Bruce+Croft%2C+Erik+Learned%2DMiller%2C+and+Jaap+Kamps%2E+2018%2E+From+neural+re%2Dranking+to+neural+ranking%3A+Learning+a+sparse+representation+for+inverted+indexing%2E+In+Proceedin)
42. Le Zhao. 2012. Modeling and solving term mismatch for full-text retrieval. Ph.D. Dissertation. Carnegie Mellon University. [find](https://scholar.google.com/scholar?q=Le+Zhao%2E+2012%2E+Modeling+and+solving+term+mismatch+for+full%2Dtext+retrieval%2E+Ph%2ED%2E+Dissertation%2E+Carnegie+Mellon+University%2E)