---
title: "FlashRAG — paper by Jin et al. (2024) — summary and ideas"
description: "Comparing retrieval-augmented generation (RAG) methods fairly is difficult. 117 references, cited by 68 papers here."
authors: ["Jiajie Jin", "Yutao Zhu", "Guanting Dong", "Yuyao Zhang", "Xinyu Yang", "Chenghao Zhang", "Tong Zhao", "Zhao Yang", "Zhicheng Dou", "Ji-Rong Wen"]
year: 2024
venue: "arXiv:2405.13576"
source: "https://arxiv.org/abs/2405.13576"
doi: "10.1145/3701716.3715313."
arxiv: "2405.13576"
licence: "CC BY-NC-SA 4.0 (arXiv)"
licence_url: "http://creativecommons.org/licenses/by-nc-sa/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/jin-2024-flashrag-modular-toolkit-efficient-retrieval/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/jin-2024-flashrag-modular-toolkit-efficient-retrieval/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2024](https://md.research.elamur.ai/free/en/papers/2024/page/6/) / FlashRAG: A Modular Toolkit for Efficient Retrieval-Augmented Generation Research

# FlashRAG: A Modular Toolkit for Efficient Retrieval-Augmented Generation Research

**Authors:** [Jiajie Jin](https://md.research.elamur.ai/free/en/authors/jiajie-jin/), [Yutao Zhu](https://md.research.elamur.ai/free/en/authors/yutao-zhu/), [Guanting Dong](https://md.research.elamur.ai/free/en/authors/guanting-dong/), [Yuyao Zhang](https://md.research.elamur.ai/free/en/authors/yuyao-zhang/), [Xinyu Yang](https://md.research.elamur.ai/free/en/authors/xinyu-yang/), [Chenghao Zhang](https://md.research.elamur.ai/free/en/authors/chenghao-zhang/), [Tong Zhao](https://md.research.elamur.ai/free/en/authors/tong-zhao/), [Zhao Yang](https://md.research.elamur.ai/free/en/authors/zhao-yang/), [Zhicheng Dou](https://md.research.elamur.ai/free/en/authors/zhicheng-dou/) and [Ji-Rong Wen](https://md.research.elamur.ai/free/en/authors/ji-rong-wen/) · **Year:** 2024 · **Venue:** arXiv:2405.13576 · **Licence:** [CC BY-NC-SA 4.0 (arXiv)](http://creativecommons.org/licenses/by-nc-sa/4.0/)

**Links:** [Publisher's page](https://doi.org/10.1145/3701716.3715313.) · [PDF](https://arxiv.org/pdf/2405.13576) · [DOI 10.1145/3701716.3715313.](https://doi.org/10.1145/3701716.3715313.) · [arXiv 2405.13576](https://arxiv.org/abs/2405.13576) · [OpenAlex](https://openalex.org/W4398795713)

**Citations:** 9 citations (OpenAlex, 2026-10-14) · [68](https://md.research.elamur.ai/free/en/papers/jin-2024-flashrag-modular-toolkit-efficient-retrieval/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
Comparing retrieval-augmented generation (RAG) methods fairly is difficult. Many implementations are closed or tied to specific setups, and datasets and retrieval corpora are scattered and inconsistently formatted. General-purpose frameworks such as LangChain and LlamaIndex hide too much of the internals for research use. The authors respond with FlashRAG, an open-source Python toolkit built around three layers:
- an environment layer for data, configuration and metrics;
- a component layer for the individual RAG building blocks;
- a pipeline layer that wires the blocks into complete workflows.

On top of this, the toolkit re-implements 16 published RAG algorithms, collects 38 benchmark datasets in a unified format on HuggingFace, and adds corpus-preparation scripts and a web UI for inspecting each pipeline step. It also covers multimodal RAG with MLLMs and CLIP-style retrievers. Multimodal experiments show that image-text retrieval clearly helps on knowledge-heavy visual QA but can hurt multimodal math reasoning. The resource matters because it lowers engineering overhead and makes RAG results reproducible and comparable.

## Key points
- **Five core components:** a Judger (SKR-based retrieval-necessity check), a Retriever, a Reranker, a Refiner and a Generator.
  - The Retriever supports BM25 via Pyserini, plus DPR, E5, BGE and ANCE indexed with FAISS, and includes a retrieval cache.
  - The Reranker uses cross-encoders such as bge-reranker and jina-reranker.
  - The Refiner offers extractive, abstractive and perplexity-based variants, including RECOMP and LLMLingua.
  - The Generator runs on vLLM, FastChat or Transformers, with Flan-T5 fusion-in-decoder.
- **Pipelines and methods:** 9 standardized pipelines span sequential, conditional, branching and loop designs. Pre-implemented methods include Self-RAG and FLARE.
- **Comparison with other toolkits:** 16 supported methods and 38 datasets, versus 6 and 10 for RAGLab and 2 methods each for LangChain and LlamaIndex.
- **Multimodal gains and losses:**
  - Pairing Qwen2-VL with OpenAI-Clip raises EM by more than 10 points on knowledge-intensive multimodal QA.
  - On MathVista, Clip with Qwen2-VL-7B lowers accuracy by about 10 points.
- **Retriever choice, top-K and model size:**
  - A retriever's language match (e.g., Chinese-Clip) is not decisive for its usefulness.
  - Clip retrieval improves up to top-3 documents, after which results become unstable.
  - Larger Qwen2-VL and InternVL2.5 models make better use of retrieved knowledge.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Renmin University of China](https://md.research.elamur.ai/free/en/universities/renmin-university-of-china/) — University, Beijing, China · 139 works · 17,752 citations

📚 Shelves (6): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 18,614 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 17,965 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 17,450 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 17,296 works · [👁️ Computer Vision and Pattern Recognition](https://md.research.elamur.ai/free/en/shelves/computer-science/computer-vision-and-pattern-recognition/) · 3,486 works · [🗄️ Information Systems](https://md.research.elamur.ai/free/en/shelves/computer-science/information-systems/) · 1,730 works

🧭 Topics (1): [🔍 Information retrieval & search](https://md.research.elamur.ai/free/en/topics/information-retrieval/) · 552 works

## Cite

Cite as: Jiajie Jin, Yutao Zhu, Guanting Dong, Yuyao Zhang, Xinyu Yang, Chenghao Zhang, Tong Zhao, Zhao Yang, Zhicheng Dou and Ji-Rong Wen. 2024. FlashRAG: A Modular Toolkit for Efficient Retrieval-Augmented Generation Research. arXiv:2405.13576.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@misc{jin2024flashrag,
  title = {{FlashRAG: A Modular Toolkit for Efficient Retrieval-Augmented Generation Research}},
  author = {Jiajie Jin, Yutao Zhu, Guanting Dong, Yuyao Zhang, Xinyu Yang, Chenghao Zhang, Tong Zhao, Zhao Yang, Zhicheng Dou and Ji-Rong Wen},
  year = {2024},
  doi = {10.1145/3701716.3715313.},
  eprint = {2405.13576},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.1145/3701716.3715313.}
}
```

- [← previous Factuality of Large Language Models: A Survey](https://md.research.elamur.ai/free/en/papers/wang-2024-factuality-large-language-models-survey/)
- [next → LAraBench: Benchmarking Arabic AI with Large Language Models](https://md.research.elamur.ai/free/en/papers/abdelali-2024-larabench-benchmarking-arabic-ai-large/)
- [↑ back to papers of 2024](https://md.research.elamur.ai/free/en/papers/2024/page/6/)

## References

117 references, 69 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Sebastian Borgeaud, Arthur Mensch, Jordan Hoffmann, Trevor Cai, Eliza Rutherford, Katie Millican, George van den Driessche, Jean-Baptiste Lespiau, Bogdan Damoc, Aidan Clark, Diego de Las Casas, Aurelia Guy, Jacob Menick, Roman Ring, Tom Hennigan, Saffron Huang, Loren Maggiore, Chris Jones, Albin Cas… [find](https://scholar.google.com/scholar?q=Sebastian+Borgeaud%2C+Arthur+Mensch%2C+Jordan+Hoffmann%2C+Trevor+Cai%2C+Eliza+Rutherford%2C+Katie+Millican%2C+George+van+den+Driessche%2C+Jean%2DBaptiste+Lespiau%2C+Bogdan+Damoc%2C+Aidan+Clark%2C+Diego+de+Las+Casas%2C+Aureli)
2. Kelvin Guu, Kenton Lee, Zora Tung, Panupong Pasupat, and Ming-Wei Chang. [REALM: retrieval-augmented language model pre-training](https://md.research.elamur.ai/free/en/papers/guu-2020-realm-retrieval-augmented-language-model/). CoRR, abs/2002.08909, 2020. InternVL-2.5 0.6 2B 8B 0.5 0.4 Performance 0.3 0.2 0.1 0.0 GaoKaoMM MMQA Datasets MathVista
3. Y. Bang, S. Cahyawijaya, N. Lee, W. Dai, D. Su, B. Wilie, H. Lovenia, Z. Ji, T. Yu, W. Chung, Q. V. Do, Y. Xu, and P. Fung. [A multitask, multilingual, multimodal evaluation of ChatGPT on reasoning, hallucination, and interactivity](https://md.research.elamur.ai/free/en/papers/bang-2023-multitask-multilingual-multimodal-evaluation-chatgpt/). In Proc. of IJCNLP, 2023.
4. Haoran Luo, Haihong E, Zichen Tang, Shiyao Peng, Yikai Guo, Wentai Zhang, Chenghao Ma, Guanting Dong, Meina Song, Wei Lin, Yifan Zhu, and Anh Tuan Luu. Chatkbqa: A generate-then-retrieve framework for knowledge base question answering with fine-tuned large language models. In Lun-Wei Ku, Andre Marti… [find](https://scholar.google.com/scholar?q=Haoran+Luo%2C+Haihong+E%2C+Zichen+Tang%2C+Shiyao+Peng%2C+Yikai+Guo%2C+Wentai+Zhang%2C+Chenghao+Ma%2C+Guanting+Dong%2C+Meina+Song%2C+Wei+Lin%2C+Yifan+Zhu%2C+and+Anh+Tuan+Luu%2E+Chatkbqa%3A+A+generate%2Dthen%2Dretrieve+framework+for)
5. Guanting Dong, Rumei Li, Sirui Wang, Yupeng Zhang, Yunsen Xian, and Weiran Xu. Bridging the kb-text gap: Leveraging structured knowledge-aware pre-training for KBQA. In Ingo Frommholz, Frank Hopfgartner, Mark Lee, Michael Oakes, Mounia Lalmas, Min Zhang, and Rodrygo L. T. Santos, editors, Proceeding… [find](https://scholar.google.com/scholar?q=Guanting+Dong%2C+Rumei+Li%2C+Sirui+Wang%2C+Yupeng+Zhang%2C+Yunsen+Xian%2C+and+Weiran+Xu%2E+Bridging+the+kb%2Dtext+gap%3A+Leveraging+structured+knowledge%2Daware+pre%2Dtraining+for+KBQA%2E+In+Ingo+Frommholz%2C+Frank+Hopfgartn)
6. Y. Gao, Y. Xiong, X. Gao, K. Jia, J. Pan, Y. Bi, Y. Dai, J. Sun, M. Wang, and H. Wang. [Retrieval-augmented generation for large language models: A survey](https://md.research.elamur.ai/free/en/papers/gao-2023-retrieval-augmented-generation-large-language/). ArXiv preprint, 2023.
7. Yiruo Cheng, Kelong Mao, Ziliang Zhao, Guanting Dong, Hongjin Qian, Yongkang Wu, Tetsuya Sakai, Ji-Rong Wen, and Zhicheng Dou. Coral: Benchmarking multi-turn conversational retrieval-augmentation generation, 2024. [find](https://scholar.google.com/scholar?q=Yiruo+Cheng%2C+Kelong+Mao%2C+Ziliang+Zhao%2C+Guanting+Dong%2C+Hongjin+Qian%2C+Yongkang+Wu%2C+Tetsuya+Sakai%2C+Ji%2DRong+Wen%2C+and+Zhicheng+Dou%2E+Coral%3A+Benchmarking+multi%2Dturn+conversational+retrieval%2Daugmentation+gene)
8. Guanting Dong, Yutao Zhu, Chenghao Zhang, Zechen Wang, Zhicheng Dou, and Ji-Rong Wen. Understand what LLM needs: Dual preference alignment for retrieval-augmented generation. CoRR, abs/2406.18676, 2024. — [arXiv:2406.18676](https://arxiv.org/abs/2406.18676) [find](https://scholar.google.com/scholar?q=Guanting+Dong%2C+Yutao+Zhu%2C+Chenghao+Zhang%2C+Zechen+Wang%2C+Zhicheng+Dou%2C+and+Ji%2DRong+Wen%2E+Understand+what+LLM+needs%3A+Dual+preference+alignment+for+retrieval%2Daugmented+generation%2E+CoRR%2C+abs%2F2406%2E18676%2C+202)
9. Harrison Chase. LangChain, October 2022. [find](https://scholar.google.com/scholar?q=Harrison+Chase%2E+LangChain%2C+October+2022%2E)
10. Jerry Liu. LlamaIndex, November 2022. [find](https://scholar.google.com/scholar?q=Jerry+Liu%2E+LlamaIndex%2C+November+2022%2E)
11. A. Asai, Z. Wu, Y. Wang, A. Sil, and H. Hajishirzi. [Self-rag: Learning to retrieve, generate, and critique through self-reflection](https://md.research.elamur.ai/free/en/papers/asai-2023-self-rag-learning-retrieve-generate/). ArXiv preprint, 2023.
12. Zhengbao Jiang, Frank F. Xu, Luyu Gao, Zhiqing Sun, Qian Liu, Jane Dwivedi-Yu, Yiming Yang, Jamie Callan, and Graham Neubig. [Active retrieval augmented generation](https://md.research.elamur.ai/free/en/papers/jiang-2023-active-retrieval-augmented-generation/). In Houda Bouamor, Juan Pino, and Kalika Bali, editors, Proceedings of the 2023 Conference on Empirical Methods in Natural Language Proce…
13. P. Wang, S. Bai, S. Tan, S. Wang, Z. Fan, J. Bai, K. Chen, X. Liu, J. Wang, W. Ge, Y. Fan, K. Dang, M. Du, X. Ren, R. Men, D. Liu, C. Zhou, J. Zhou, and J. Lin. Qwen2-vl: Enhancing vision-language model’s perception of the world at any resolution. ArXiv preprint, 2024. — [Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolution](https://md.research.elamur.ai/free/en/papers/wang-2024-qwen2-vl-enhancing-vision/)
14. Zhe Chen, Jiannan Wu, Wenhai Wang, Weijie Su, Guo Chen, Sen Xing, Muyan Zhong, Qinglong Zhang, Xizhou Zhu, Lewei Lu, Bin Li, Ping Luo, Tong Lu, Yu Qiao, and Jifeng Dai. [Internvl: Scaling up vision foundation models and aligning for generic visual-linguistic tasks](https://md.research.elamur.ai/free/en/papers/chen-2023-internvl-scaling-up-vision/). CoRR, abs/2312.14238, 2023.
15. Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. [Visual instruction tuning](https://md.research.elamur.ai/free/en/papers/liu-2023-visual-instruction-tuning/), 2023.
16. Haotian Liu, Chunyuan Li, Yuheng Li, Bo Li, Yuanhan Zhang, Sheng Shen, and Yong Jae Lee. Llava-next: Improved reasoning, ocr, and world knowledge, January 2024. [find](https://scholar.google.com/scholar?q=Haotian+Liu%2C+Chunyuan+Li%2C+Yuheng+Li%2C+Bo+Li%2C+Yuanhan+Zhang%2C+Sheng+Shen%2C+and+Yong+Jae+Lee%2E+Llava%2Dnext%3A+Improved+reasoning%2C+ocr%2C+and+world+knowledge%2C+January+2024%2E)
17. A. Radford, J. Wook Kim, C. Hallacy, A. Ramesh, G. Goh, S. Agarwal, G. Sastry, A. Askell, P. Mishkin, J. Clark, G. Krueger, and I. Sutskever. [Learning transferable visual models from natural language supervision](https://md.research.elamur.ai/free/en/papers/radford-2021-learning-transferable-visual-models-natural/). In Proc. of ICML, Proceedings of Machine Learning Research, 2021.
18. Malte Pietsch, Timo Möller, Bogdan Kostic, Julian Risch, Massimiliano Pippi, Mayank Jobanputra, Sara Zanzottera, Silvano Cerza, Vladimir Blagojevic, Thomas Stadelmann, Tanay Soni, and Sebastian Lee. Haystack: the end-to-end NLP framework for pragmatic builders, November 2019. [find](https://scholar.google.com/scholar?q=Malte+Pietsch%2C+Timo+M%C3%B6ller%2C+Bogdan+Kostic%2C+Julian+Risch%2C+Massimiliano+Pippi%2C+Mayank+Jobanputra%2C+Sara+Zanzottera%2C+Silvano+Cerza%2C+Vladimir+Blagojevic%2C+Thomas+Stadelmann%2C+Tanay+Soni%2C+and+Sebastian+Lee%2E+H)
19. Peter Izsak, Moshe Berchansky, Daniel Fleischer, and Ronen Laperdon. fastRAG: Efficient Retrieval Augmentation and Generation Framework, February 2023. [find](https://scholar.google.com/scholar?q=Peter+Izsak%2C+Moshe+Berchansky%2C+Daniel+Fleischer%2C+and+Ronen+Laperdon%2E+fastRAG%3A+Efficient+Retrieval+Augmentation+and+Generation+Framework%2C+February+2023%2E)
20. Xiao Yu, Yunan Lu, and Zhou Yu. Localrqa: From generating data to locally training, testing, and deploying retrieval-augmented QA systems. CoRR, abs/2403.00982, 2024. [find](https://scholar.google.com/scholar?q=Xiao+Yu%2C+Yunan+Lu%2C+and+Zhou+Yu%2E+Localrqa%3A+From+generating+data+to+locally+training%2C+testing%2C+and+deploying+retrieval%2Daugmented+QA+systems%2E+CoRR%2C+abs%2F2403%2E00982%2C+2024%2E)
21. D. Kim, B. Kim, D. Han, and M. Eibich. Autorag: Automated framework for optimization of retrieval augmented generation pipeline. ArXiv preprint, 2024. [find](https://scholar.google.com/scholar?q=D%2E+Kim%2C+B%2E+Kim%2C+D%2E+Han%2C+and+M%2E+Eibich%2E+Autorag%3A+Automated+framework+for+optimization+of+retrieval+augmented+generation+pipeline%2E+ArXiv+preprint%2C+2024%2E)
22. X. Zhang, Y.-Z. Song, Y. Wang, Shuyun Tang, Xinfeng Li, Zhengran Zeng, Zhen Wu, Wei Ye, Wenyuan Xu, Yue Zhang, Xinyu Dai, Shikun Zhang, and Qingsong Wen. RAGLAB: A modular and research-oriented unified framework for retrieval-augmented generation. In Delia Irazu Hernandez Farias, Tom Hope, and Manli… [find](https://scholar.google.com/scholar?q=X%2E+Zhang%2C+Y%2E%2DZ%2E+Song%2C+Y%2E+Wang%2C+Shuyun+Tang%2C+Xinfeng+Li%2C+Zhengran+Zeng%2C+Zhen+Wu%2C+Wei+Ye%2C+Wenyuan+Xu%2C+Yue+Zhang%2C+Xinyu+Dai%2C+Shikun+Zhang%2C+and+Qingsong+Wen%2E+RAGLAB%3A+A+modular+and+research%2Doriented+unifie)
23. Hiroaki Hayashi, Prashant Budania, Peng Wang, Chris Ackerson, Raj Neervannan, and Graham Neubig. [Wikiasp: A dataset for multi-domain aspect-based summarization](https://md.research.elamur.ai/free/en/papers/hayashi-2021-wikiasp-dataset-multi-domain-aspect/). Trans. Assoc. Comput. Linguistics, 9:211–225, 2021.
24. Tomás Kociský, Jonathan Schwarz, Phil Blunsom, Chris Dyer, Karl Moritz Hermann, Gábor Melis, and Edward Grefenstette. [The narrativeqa reading comprehension challenge](https://md.research.elamur.ai/free/en/papers/kocisky-2018-narrativeqa-reading-comprehension-challenge/). Trans. Assoc. Comput. Linguistics, 6:317–328, 2018.
25. Xiaoxi Li, Guanting Dong, Jiajie Jin, Yuyao Zhang, Yujia Zhou, Yutao Zhu, Peitian Zhang, and Zhicheng Dou. [Search-o1: Agentic search-enhanced large reasoning models](https://md.research.elamur.ai/free/en/papers/li-2025-search-o1-agentic-search-enhanced/). arXiv preprint arXiv:2501.05366, 2025.
26. Yujia Zhou, Zheng Liu, Jiajie Jin, Jian-Yun Nie, and Zhicheng Dou. Metacognitive retrieval-augmented large language models. In Tat-Seng Chua, Chong-Wah Ngo, Ravi Kumar, Hady W. Lauw, and Roy Ka-Wei Lee, editors, Proceedings of the ACM on Web Conference 2024, WWW 2024, Singapore, May 13-17, 2024, pag… [find](https://scholar.google.com/scholar?q=Yujia+Zhou%2C+Zheng+Liu%2C+Jiajie+Jin%2C+Jian%2DYun+Nie%2C+and+Zhicheng+Dou%2E+Metacognitive+retrieval%2Daugmented+large+language+models%2E+In+Tat%2DSeng+Chua%2C+Chong%2DWah+Ngo%2C+Ravi+Kumar%2C+Hady+W%2E+Lauw%2C+and+Roy+Ka%2DWei+Le)
27. Guanting Dong, Keming Lu, Chengpeng Li, Tingyu Xia, Bowen Yu, Chang Zhou, and Jingren Zhou. Self-play with execution feedback: Improving instruction-following capabilities of large language models. CoRR, abs/2406.13542, 2024. [find](https://scholar.google.com/scholar?q=Guanting+Dong%2C+Keming+Lu%2C+Chengpeng+Li%2C+Tingyu+Xia%2C+Bowen+Yu%2C+Chang+Zhou%2C+and+Jingren+Zhou%2E+Self%2Dplay+with+execution+feedback%3A+Improving+instruction%2Dfollowing+capabilities+of+large+language+models%2E+Co)
28. Xiaoxi Li, Yujia Zhou, and Zhicheng Dou. Unigen: A unified generative framework for retrieval and question answering with large language models. In Michael J. Wooldridge, Jennifer G. Dy, and Sriraam Natarajan, editors, Thirty-Eighth AAAI Conference on Artificial Intelligence, AAAI 2024, Thirty-Sixth… [find](https://scholar.google.com/scholar?q=Xiaoxi+Li%2C+Yujia+Zhou%2C+and+Zhicheng+Dou%2E+Unigen%3A+A+unified+generative+framework+for+retrieval+and+question+answering+with+large+language+models%2E+In+Michael+J%2E+Wooldridge%2C+Jennifer+G%2E+Dy%2C+and+Sriraam+N)
29. Guanting Dong, Xiaoshuai Song, Yutao Zhu, Runqi Qiao, Zhicheng Dou, and Ji-Rong Wen. Toward general instruction-following alignment for retrieval-augmented generation. arXiv preprint arXiv:2410.09584, 2024. — [arXiv:2410.09584](https://arxiv.org/abs/2410.09584) [find](https://scholar.google.com/scholar?q=Guanting+Dong%2C+Xiaoshuai+Song%2C+Yutao+Zhu%2C+Runqi+Qiao%2C+Zhicheng+Dou%2C+and+Ji%2DRong+Wen%2E+Toward+general+instruction%2Dfollowing+alignment+for+retrieval%2Daugmented+generation%2E+arXiv+preprint+arXiv%3A2410%2E09584%2C)
30. Yiruo Cheng, Kelong Mao, and Zhicheng Dou. Interpreting conversational dense retrieval by rewriting-enhanced inversion of session embedding. In Lun-Wei Ku, Andre Martins, and Vivek Srikumar, editors, Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: L… [find](https://scholar.google.com/scholar?q=Yiruo+Cheng%2C+Kelong+Mao%2C+and+Zhicheng+Dou%2E+Interpreting+conversational+dense+retrieval+by+rewriting%2Denhanced+inversion+of+session+embedding%2E+In+Lun%2DWei+Ku%2C+Andre+Martins%2C+and+Vivek+Srikumar%2C+editors%2C+)
31. Yujia Zhou, Yan Liu, Xiaoxi Li, Jiajie Jin, Hongjin Qian, Zheng Liu, Chaozhuo Li, Zhicheng Dou, Tsung-Yi Ho, and Philip S. Yu. Trustworthiness in retrieval-augmented generation systems: A survey. CoRR, abs/2409.10102, 2024. — [arXiv:2409.10102](https://arxiv.org/abs/2409.10102) [find](https://scholar.google.com/scholar?q=Yujia+Zhou%2C+Yan+Liu%2C+Xiaoxi+Li%2C+Jiajie+Jin%2C+Hongjin+Qian%2C+Zheng+Liu%2C+Chaozhuo+Li%2C+Zhicheng+Dou%2C+Tsung%2DYi+Ho%2C+and+Philip+S%2E+Yu%2E+Trustworthiness+in+retrieval%2Daugmented+generation+systems%3A+A+survey%2E+CoRR)
32. Xiaoxi Li, Jiajie Jin, Yujia Zhou, Yongkang Wu, Zhonghua Li, Qi Ye, and Zhicheng Dou. [Retrollm: Empowering large language models to retrieve fine-grained evidence within generation](https://md.research.elamur.ai/free/en/papers/li-2025-retrollm-empowering-large-language-models/). CoRR, abs/2412.11919, 2024.
33. Jiejun Tan, Zhicheng Dou, Wen Wang, Mang Wang, Weipeng Chen, and Ji-Rong Wen. Htmlrag: HTML is better than plain text for modeling retrieved knowledge in RAG systems. CoRR, abs/2411.02959, 2024. — [arXiv:2411.02959](https://arxiv.org/abs/2411.02959) [find](https://scholar.google.com/scholar?q=Jiejun+Tan%2C+Zhicheng+Dou%2C+Wen+Wang%2C+Mang+Wang%2C+Weipeng+Chen%2C+and+Ji%2DRong+Wen%2E+Htmlrag%3A+HTML+is+better+than+plain+text+for+modeling+retrieved+knowledge+in+RAG+systems%2E+CoRR%2C+abs%2F2411%2E02959%2C+2024%2E+%E2%80%94+%5Bar)
34. Fengran Mo, Kelong Mao, Ziliang Zhao, Hongjin Qian, Haonan Chen, Yiruo Cheng, Xiaoxi Li, Yutao Zhu, Zhicheng Dou, and Jian-Yun Nie. A survey of conversational search, 2024. [find](https://scholar.google.com/scholar?q=Fengran+Mo%2C+Kelong+Mao%2C+Ziliang+Zhao%2C+Hongjin+Qian%2C+Haonan+Chen%2C+Yiruo+Cheng%2C+Xiaoxi+Li%2C+Yutao+Zhu%2C+Zhicheng+Dou%2C+and+Jian%2DYun+Nie%2E+A+survey+of+conversational+search%2C+2024%2E)
35. Xiaoxi Li, Jiajie Jin, Yujia Zhou, Yuyao Zhang, Peitian Zhang, Yutao Zhu, and Zhicheng Dou. From matching to generation: A survey on generative information retrieval. CoRR, abs/2404.14851, 2024. — [arXiv:2404.14851](https://arxiv.org/abs/2404.14851) [find](https://scholar.google.com/scholar?q=Xiaoxi+Li%2C+Jiajie+Jin%2C+Yujia+Zhou%2C+Yuyao+Zhang%2C+Peitian+Zhang%2C+Yutao+Zhu%2C+and+Zhicheng+Dou%2E+From+matching+to+generation%3A+A+survey+on+generative+information+retrieval%2E+CoRR%2C+abs%2F2404%2E14851%2C+2024%2E+%E2%80%94+%5Bar)
36. Jiejun Tan, Zhicheng Dou, Yutao Zhu, Peidong Guo, Kun Fang, and Ji-Rong Wen. Small models, big insights: Leveraging slim proxy models to decide when and what to retrieve for llms. In Lun-Wei Ku, Andre Martins, and Vivek Srikumar, editors, Proceedings of the 62nd Annual Meeting of the Association for… [find](https://scholar.google.com/scholar?q=Jiejun+Tan%2C+Zhicheng+Dou%2C+Yutao+Zhu%2C+Peidong+Guo%2C+Kun+Fang%2C+and+Ji%2DRong+Wen%2E+Small+models%2C+big+insights%3A+Leveraging+slim+proxy+models+to+decide+when+and+what+to+retrieve+for+llms%2E+In+Lun%2DWei+Ku%2C+Andre)
37. Y. Wang, P. Li, M. Sun, and Y. Liu. [Self-knowledge guided retrieval augmentation for large language models](https://md.research.elamur.ai/free/en/papers/wang-2023-self-knowledge-guided-retrieval-augmentation/). In Proc. of EMNLP 2023 Findings, 2023.
38. S. Jeong, J. Baek, S. Cho, S. Hwang, and J. Park. [Adaptive-RAG: Learning to adapt retrieval-augmented large language models through question complexity](https://md.research.elamur.ai/free/en/papers/jeong-2024-adaptive-rag-learning-adapt-retrieval/). In Proc. of NAACL-HLT, 2024.
39. Guanting Dong, Yutao Zhu, Chenghao Zhang, Zechen Wang, Zhicheng Dou, and Ji-Rong Wen. Understand what LLM needs: Dual preference alignment for retrieval-augmented generation. CoRR, abs/2406.18676, 2024. — [arXiv:2406.18676](https://arxiv.org/abs/2406.18676) [find](https://scholar.google.com/scholar?q=Guanting+Dong%2C+Yutao+Zhu%2C+Chenghao+Zhang%2C+Zechen+Wang%2C+Zhicheng+Dou%2C+and+Ji%2DRong+Wen%2E+Understand+what+LLM+needs%3A+Dual+preference+alignment+for+retrieval%2Daugmented+generation%2E+CoRR%2C+abs%2F2406%2E18676%2C+202)
40. Jimmy Lin, Xueguang Ma, Sheng-Chieh Lin, Jheng-Hong Yang, Ronak Pradeep, and Rodrigo Frassetto Nogueira. Pyserini: A python toolkit for reproducible information retrieval research with sparse and dense representations. In Fernando Diaz, Chirag Shah, Torsten Suel, Pablo Castells, Rosie Jones, and Tet… [find](https://scholar.google.com/scholar?q=Jimmy+Lin%2C+Xueguang+Ma%2C+Sheng%2DChieh+Lin%2C+Jheng%2DHong+Yang%2C+Ronak+Pradeep%2C+and+Rodrigo+Frassetto+Nogueira%2E+Pyserini%3A+A+python+toolkit+for+reproducible+information+retrieval+research+with+sparse+and+dens)
41. S. E. Robertson and H. Zaragoza. The probabilistic relevance framework: BM25 and beyond. Found. Trends Inf. Retr., (4), 2009. [find](https://scholar.google.com/scholar?q=S%2E+E%2E+Robertson+and+H%2E+Zaragoza%2E+The+probabilistic+relevance+framework%3A+BM25+and+beyond%2E+Found%2E+Trends+Inf%2E+Retr%2E%2C+%284%29%2C+2009%2E)
42. Vladimir Karpukhin, Barlas Oguz, Sewon Min, Patrick S. H. Lewis, Ledell Wu, Sergey Edunov, Danqi Chen, and Wen-tau Yih. [Dense passage retrieval for open-domain question answering](https://md.research.elamur.ai/free/en/papers/karpukhin-2020-dense-passage-retrieval-open-domain/). In Bonnie Webber, Trevor Cohn, Yulan He, and Yang Liu, editors, Proceedings of the 2020 Conference on Empirical Methods…
43. Liang Wang, Nan Yang, Xiaolong Huang, Binxing Jiao, Linjun Yang, Daxin Jiang, Rangan Majumder, and Furu Wei. [Text embeddings by weakly-supervised contrastive pre-training](https://md.research.elamur.ai/free/en/papers/wang-2022-text-embeddings-weakly-supervised-contrastive/). CoRR, abs/2212.03533, 2022.
44. Shitao Xiao, Zheng Liu, Peitian Zhang, and Niklas Muennighof. C-pack: Packaged resources to advance general chinese embedding. CoRR, abs/2309.[07597, 2023](https://md.research.elamur.ai/free/en/papers/xiao-2023-c-pack-packed-resources/).
45. Lee Xiong, Chenyan Xiong, Ye Li, Kwok-Fung Tang, Jialin Liu, Paul N. Bennett, Junaid Ahmed, and Arnold Overwijk. [Approximate nearest neighbor negative contrastive learning for dense text retrieval](https://md.research.elamur.ai/free/en/papers/xiong-2020-approximate-nearest-neighbor-negative-contrastive/). In 9th International Conference on Learning Representations, ICLR 2021, Virtual Event, Austria, May 3-…
46. Matthijs Douze, Alexandr Guzhva, Chengqi Deng, Jeff Johnson, Gergely Szilvasy, Pierre-Emmanuel Mazaré, Maria Lomeli, Lucas Hosseini, and Hervé Jégou. [The faiss library](https://md.research.elamur.ai/free/en/papers/douze-2024-faiss-library/). CoRR, abs/2401.08281, 2024.
47. Jeff Johnson, Matthijs Douze, and Hervé Jégou. Billion-scale similarity search with gpus. IEEE Trans. Big Data, 7(3):535–547, 2021. [find](https://scholar.google.com/scholar?q=Jeff+Johnson%2C+Matthijs+Douze%2C+and+Herv%C3%A9+J%C3%A9gou%2E+Billion%2Dscale+similarity+search+with+gpus%2E+IEEE+Trans%2E+Big+Data%2C+7%283%29%3A535%E2%80%93547%2C+2021%2E)
48. Huiqiang Jiang, Qianhui Wu, Chin-Yew Lin, Yuqing Yang, and Lili Qiu. [Llmlingua: Compressing prompts for accelerated inference of large language models](https://md.research.elamur.ai/free/en/papers/jiang-2023-llmlingua-compressing-prompts-accelerated-inference/). In Houda Bouamor, Juan Pino, and Kalika Bali, editors, Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, EMNLP…
49. Huiqiang Jiang, Qianhui Wu, Xufang Luo, Dongsheng Li, Chin-Yew Lin, Yuqing Yang, and Lili Qiu. [Longllmlingua: Accelerating and enhancing llms in long context scenarios via prompt compression](https://md.research.elamur.ai/free/en/papers/jiang-2023-longllmlingua-accelerating-enhancing-llms-long/). CoRR, abs/2310.06839, 2023.
50. Yucheng Li. [Unlocking context constraints of llms: Enhancing context efficiency of llms with self-information-based content filtering](https://md.research.elamur.ai/free/en/papers/li-2023-unlocking-context-constraints-llms-enhancing/). CoRR, abs/2304.12102, 2023.
51. Jiajie Jin, Yutao Zhu, Yujia Zhou, and Zhicheng Dou. [BIDER: bridging knowledge inconsistency for efficient retrieval-augmented llms via key supporting evidence](https://md.research.elamur.ai/free/en/papers/jin-2024-bider-bridging-knowledge-inconsistency-efficient/). In Lun-Wei Ku, Andre Martins, and Vivek Srikumar, editors, Findings of the Association for Computational Linguistics, ACL 2024, Bangkok, Th…
52. F. Xu, W. Shi, and E. Choi. [RECOMP: improving retrieval-augmented lms with compression and selective augmentation](https://md.research.elamur.ai/free/en/papers/xu-2023-recomp-improving-retrieval-augmented-lms/). ArXiv preprint, 2023.
53. W. Kwon, Z. Li, S. Zhuang, Y. Sheng, L. Zheng, C. H. Yu, J. Gonzalez, H. Zhang, and I. Stoica. [Efficient memory management for large language model serving with pagedattention](https://md.research.elamur.ai/free/en/papers/kwon-2023-efficient-memory-management-large-language/). In Proc. of SOSP, 2023.
54. Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric P. Xing, Hao Zhang, Joseph E. Gonzalez, and Ion Stoica. [Judging llm-as-a-judge with mt-bench and chatbot arena](https://md.research.elamur.ai/free/en/papers/zheng-2023-judging-llm-judge-mt-bench/). In Alice Oh, Tristan Naumann, Amir Globerson, Kate Saenko, Morit…
55. T. Wolf, L. Debut, V. Sanh, J. Chaumond, C. Delangue, A. Moi, P. Cistac, T. Rault, R. Louf, M. Funtowicz, J. Davison, S. Shleifer, P. von Platen, C. Ma, Y. Jernite, J. Plu, C. Xu, T. Le Scao, S. Gugger, M. Drame, Q. Lhoest, and A. Rush. [Transformers: State-of-the-art natural language processing](https://md.research.elamur.ai/free/en/papers/wolf-2020-transformers-state-art-natural-language/). In…
56. Jason Wei, Maarten Bosma, Vincent Y. Zhao, Kelvin Guu, Adams Wei Yu, Brian Lester, Nan Du, Andrew M. Dai, and Quoc V. Le. [Finetuned language models are zero-shot learners](https://md.research.elamur.ai/free/en/papers/wei-2021-finetuned-language-models-zero-shot/). In The Tenth International Conference on Learning Representations, ICLR 2022, Virtual Event, April 25-29, 2022. OpenReview.net,…
57. Gautier Izacard and Edouard Grave. [Leveraging passage retrieval with generative models for open domain question answering](https://md.research.elamur.ai/free/en/papers/izacard-2021-leveraging-passage-retrieval-generative-models/). In Paola Merlo, Jörg Tiedemann, and Reut Tsarfaty, editors, Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Vol…
58. Tom Kwiatkowski, Jennimaria Palomaki, Olivia Redfield, Michael Collins, Ankur P. Parikh, Chris Alberti, Danielle Epstein, Illia Polosukhin, Jacob Devlin, Kenton Lee, Kristina Toutanova, Llion Jones, Matthew Kelcey, Ming-Wei Chang, Andrew M. Dai, Jakob Uszkoreit, Quoc Le, and Slav Petrov. Natural que… — [Natural Questions: A Benchmark for Question Answering Research](https://md.research.elamur.ai/free/en/papers/kwiatkowski-2019-natural-questions-benchmark-question-answering/)
59. Mandar Joshi, Eunsol Choi, Daniel S. Weld, and Luke Zettlemoyer. [Triviaqa: A large scale distantly supervised challenge dataset for reading comprehension](https://md.research.elamur.ai/free/en/papers/joshi-2017-triviaqa-large-scale-distantly-supervised/). In Regina Barzilay and Min-Yen Kan, editors, Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, ACL 2017, V…
60. Alex Mallen, Akari Asai, Victor Zhong, Rajarshi Das, Hannaneh Hajishirzi, and Daniel Khashabi. When not to trust language models: Investigating effectiveness and limitations of parametric and non-parametric memories. CoRR, abs/2212.10511, 2022. [find](https://scholar.google.com/scholar?q=Alex+Mallen%2C+Akari+Asai%2C+Victor+Zhong%2C+Rajarshi+Das%2C+Hannaneh+Hajishirzi%2C+and+Daniel+Khashabi%2E+When+not+to+trust+language+models%3A+Investigating+effectiveness+and+limitations+of+parametric+and+non%2Dpara)
61. Pranav Rajpurkar, Jian Zhang, Konstantin Lopyrev, and Percy Liang. Squad: 100, 000+ questions for machine comprehension of text. In Jian Su, Xavier Carreras, and Kevin Duh, editors, Proceedings of the 2016 [Conference on Empirical Methods in Natural Language Processing, EMNLP 2016, Austin, Texas, USA…](https://md.research.elamur.ai/free/en/papers/rajpurkar-2016-squad-100-000-questions-machine/)
62. Tri Nguyen, Mir Rosenberg, Xia Song, Jianfeng Gao, Saurabh Tiwary, Rangan Majumder, and Li Deng. [MS MARCO: A human generated machine reading comprehension dataset](https://md.research.elamur.ai/free/en/papers/bajaj-2016-ms-marco-human-generated/). In Tarek Richard Besold, Antoine Bordes, Artur S. d’Avila Garcez, and Greg Wayne, editors, Proceedings of the Workshop on Cognitive Comp…
63. Yi Yang, Wen-tau Yih, and Christopher Meek. [Wikiqa: A challenge dataset for open-domain question answering](https://md.research.elamur.ai/free/en/papers/yang-2015-wikiqa-challenge-dataset-open-domain/). In Lluís Màrquez, Chris Callison-Burch, Jian Su, Daniele Pighin, and Yuval Marton, editors, Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, EMNLP 2015,…
64. Jonathan Berant, Andrew Chou, Roy Frostig, and Percy Liang. [Semantic parsing on freebase from question-answer pairs](https://md.research.elamur.ai/free/en/papers/berant-2013-semantic-parsing-freebase-question-answer/). In Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, EMNLP 2013, 18-21 October 2013, Grand Hyatt Seattle, Seattle, Washington, USA, A meeting of…
65. Sewon Min, Julian Michael, Hannaneh Hajishirzi, and Luke Zettlemoyer. Ambigqa: Answering ambiguous open-domain questions. In Bonnie Webber, Trevor Cohn, Yulan He, and Yang Liu, editors, Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, EMNLP 2020, Online, Novemb… [find](https://scholar.google.com/scholar?q=Sewon+Min%2C+Julian+Michael%2C+Hannaneh+Hajishirzi%2C+and+Luke+Zettlemoyer%2E+Ambigqa%3A+Answering+ambiguous+open%2Ddomain+questions%2E+In+Bonnie+Webber%2C+Trevor+Cohn%2C+Yulan+He%2C+and+Yang+Liu%2C+editors%2C+Proceedings+of)
66. Maarten Sap, Hannah Rashkin, Derek Chen, Ronan Le Bras, and Yejin Choi. Social iqa: Commonsense reasoning about social interactions. In Kentaro Inui, Jing Jiang, Vincent Ng, and Xiaojun Wan, editors, Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th I… [find](https://scholar.google.com/scholar?q=Maarten+Sap%2C+Hannah+Rashkin%2C+Derek+Chen%2C+Ronan+Le+Bras%2C+and+Yejin+Choi%2E+Social+iqa%3A+Commonsense+reasoning+about+social+interactions%2E+In+Kentaro+Inui%2C+Jing+Jiang%2C+Vincent+Ng%2C+and+Xiaojun+Wan%2C+editors%2C+)
67. Alon Talmor, Jonathan Herzig, Nicholas Lourie, and Jonathan Berant. [Commonsenseqa: A question answering challenge targeting commonsense knowledge](https://md.research.elamur.ai/free/en/papers/talmor-2019-commonsenseqa-question-answering-challenge-targeting/). In Jill Burstein, Christy Doran, and Thamar Solorio, editors, Proceedings of the 2019 Conference of the North American Chapter of the Association for Com…
68. Christopher Clark, Kenton Lee, Ming-Wei Chang, Tom Kwiatkowski, Michael Collins, and Kristina Toutanova. [Boolq: Exploring the surprising difficulty of natural yes/no questions](https://md.research.elamur.ai/free/en/papers/clark-2019-boolq-exploring-surprising-difficulty-natural/). In Jill Burstein, Christy Doran, and Thamar Solorio, editors, Proceedings of the 2019 Conference of the North American Chap…
69. Yonatan Bisk, Rowan Zellers, Ronan Le Bras, Jianfeng Gao, and Yejin Choi. [PIQA: reasoning about physical commonsense in natural language](https://md.research.elamur.ai/free/en/papers/bisk-2019-piqa-reasoning-about-physical-commonsense/). In The Thirty-Fourth AAAI Conference on Artificial Intelligence, AAAI 2020, The Thirty-Second Innovative Applications of Artificial Intelligence Conference, IAAI…
70. Ashwin Kalyan, Abhinav Kumar, Arjun Chandrasekaran, Ashish Sabharwal, and Peter Clark. How much coffee was consumed during EMNLP 2019? fermi problems: A new reasoning challenge for AI. In Marie-Francine Moens, Xuanjing Huang, Lucia Specia, and Scott Wen-tau Yih, editors, Proceedings of the 2021 Conf… [find](https://scholar.google.com/scholar?q=Ashwin+Kalyan%2C+Abhinav+Kumar%2C+Arjun+Chandrasekaran%2C+Ashish+Sabharwal%2C+and+Peter+Clark%2E+How+much+coffee+was+consumed+during+EMNLP+2019%3F+fermi+problems%3A+A+new+reasoning+challenge+for+AI%2E+In+Marie%2DFranci)
71. Zhilin Yang, Peng Qi, Saizheng Zhang, Yoshua Bengio, William W. Cohen, Ruslan Salakhutdinov, and Christopher D. Manning. [Hotpotqa: A dataset for diverse, explainable multi-hop question answering](https://md.research.elamur.ai/free/en/papers/yang-2018-hotpotqa-dataset-diverse-explainable-multi/). In Ellen Riloff, David Chiang, Julia Hockenmaier, and Jun’ichi Tsujii, editors, Proceedings of the 2018…
72. Xanh Ho, Anh-Khoa Duong Nguyen, Saku Sugawara, and Akiko Aizawa. [Constructing A multi-hop QA dataset for comprehensive evaluation of reasoning steps](https://md.research.elamur.ai/free/en/papers/ho-2020-constructing-multi-hop-qa-dataset/). In Donia Scott, Núria Bel, and Chengqing Zong, editors, Proceedings of the 28th International Conference on Computational Linguistics, COLING 2020, Ba…
73. Harsh Trivedi, Niranjan Balasubramanian, Tushar Khot, and Ashish Sabharwal. Musique: Multihop questions via single-hop question composition. Trans. Assoc. Comput. Linguistics, 10:539–554, 2022. — [♫ MuSiQue: Multihop Questions via Single-hop Question Composition](https://md.research.elamur.ai/free/en/papers/trivedi-2022-musique-multihop-questions-single-hop/)
74. Ofir Press, Muru Zhang, Sewon Min, Ludwig Schmidt, Noah A. Smith, and Mike Lewis. Measuring and narrowing the compositionality gap in language models. In Houda Bouamor, Juan Pino, and Kalika Bali, editors, Findings of the Association for Computational Linguistics: EMNLP 2023, Singapore, December 6-1… [find](https://scholar.google.com/scholar?q=Ofir+Press%2C+Muru+Zhang%2C+Sewon+Min%2C+Ludwig+Schmidt%2C+Noah+A%2E+Smith%2C+and+Mike+Lewis%2E+Measuring+and+narrowing+the+compositionality+gap+in+language+models%2E+In+Houda+Bouamor%2C+Juan+Pino%2C+and+Kalika+Bali%2C+edi)
75. Mor Geva, Daniel Khashabi, Elad Segal, Tushar Khot, Dan Roth, and Jonathan Berant. Did aristotle use a laptop? A question answering benchmark with implicit reasoning strategies. Trans. Assoc. Comput. Linguistics, 9:346–361, 2021. [find](https://scholar.google.com/scholar?q=Mor+Geva%2C+Daniel+Khashabi%2C+Elad+Segal%2C+Tushar+Khot%2C+Dan+Roth%2C+and+Jonathan+Berant%2E+Did+aristotle+use+a+laptop%3F+A+question+answering+benchmark+with+implicit+reasoning+strategies%2E+Trans%2E+Assoc%2E+Comput%2E+)
76. Ivan Stelmakh, Yi Luan, Bhuwan Dhingra, and Ming-Wei Chang. [ASQA: factoid questions meet long-form answers](https://md.research.elamur.ai/free/en/papers/stelmakh-2022-asqa-factoid-questions-meet-long/). In Yoav Goldberg, Zornitsa Kozareva, and Yue Zhang, editors, Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, EMNLP 2022, Abu Dhabi, United Arab Emirates…
77. Angela Fan, Yacine Jernite, Ethan Perez, David Grangier, Jason Weston, and Michael Auli. [ELI5: long form question answering](https://md.research.elamur.ai/free/en/papers/fan-2019-eli5-long-form-question-answering/). In Anna Korhonen, David R. Traum, and Lluís Màrquez, editors, Proceedings of the 57th Conference of the Association for Computational Linguistics, ACL 2019, Florence, Italy, J…
78. Daniel Cohen, Liu Yang, and W. Bruce Croft. Wikipassageqa: A benchmark collection for research on non-factoid answer passage retrieval. In Kevyn Collins-Thompson, Qiaozhu Mei, Brian D. Davison, Yiqun Liu, and Emine Yilmaz, editors, The 41st International ACM SIGIR Conference on Research & Developmen… [find](https://scholar.google.com/scholar?q=Daniel+Cohen%2C+Liu+Yang%2C+and+W%2E+Bruce+Croft%2E+Wikipassageqa%3A+A+benchmark+collection+for+research+on+non%2Dfactoid+answer+passage+retrieval%2E+In+Kevyn+Collins%2DThompson%2C+Qiaozhu+Mei%2C+Brian+D%2E+Davison%2C+Yiqun+)
79. Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. [Measuring massive multitask language understanding](https://md.research.elamur.ai/free/en/papers/hendrycks-2020-measuring-massive-multitask-language-understanding/). In 9th International Conference on Learning Representations, ICLR 2021, Virtual Event, Austria, May 3-7, 2021. OpenReview.net, 2021.
80. Dan Hendrycks, Collin Burns, Steven Basart, Andrew Critch, Jerry Li, Dawn Song, and Jacob Steinhardt. [Aligning AI with shared human values](https://md.research.elamur.ai/free/en/papers/hendrycks-2020-aligning-ai-shared-human-values/). In 9th International Conference on Learning Representations, ICLR 2021, Virtual Event, Austria, May 3-7, 2021. OpenReview.net, 2021.
81. Stephanie Lin, Jacob Hilton, and Owain Evans. [Truthfulqa: Measuring how models mimic human falsehoods](https://md.research.elamur.ai/free/en/papers/lin-2022-truthfulqa-measuring-how-models-mimic/). In Smaranda Muresan, Preslav Nakov, and Aline Villavicencio, editors, Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), ACL 2022, Dubl…
82. Rowan Zellers, Ari Holtzman, Yonatan Bisk, Ali Farhadi, and Yejin Choi. Hellaswag: Can a machine really finish your sentence? In Anna Korhonen, David R. Traum, and Lluís Màrquez, editors, Proceedings of the 57th Conference of the Association for Computational Linguistics, ACL 2019, Florence, Italy,… [find](https://scholar.google.com/scholar?q=Rowan+Zellers%2C+Ari+Holtzman%2C+Yonatan+Bisk%2C+Ali+Farhadi%2C+and+Yejin+Choi%2E+Hellaswag%3A+Can+a+machine+really+finish+your+sentence%3F+In+Anna+Korhonen%2C+David+R%2E+Traum%2C+and+Llu%C3%ADs+M%C3%A0rquez%2C+editors%2C+Proceedings+)
83. Peter Clark, Isaac Cowhey, Oren Etzioni, Tushar Khot, Ashish Sabharwal, Carissa Schoenick, and Oyvind Tafjord. [Think you have solved question answering? try arc, the AI2 reasoning challenge](https://md.research.elamur.ai/free/en/papers/clark-2018-think-have-solved-question-answering/). CoRR, abs/1803.05457, 2018.
84. Todor Mihaylov, Peter Clark, Tushar Khot, and Ashish Sabharwal. Can a suit of armor conduct electricity? A new dataset for open book question answering. In Ellen Riloff, David Chiang, Julia Hockenmaier, and Jun’ichi Tsujii, editors, Proceedings of the 2018 Conference on Empirical Methods in Natural… [find](https://scholar.google.com/scholar?q=Todor+Mihaylov%2C+Peter+Clark%2C+Tushar+Khot%2C+and+Ashish+Sabharwal%2E+Can+a+suit+of+armor+conduct+electricity%3F+A+new+dataset+for+open+book+question+answering%2E+In+Ellen+Riloff%2C+David+Chiang%2C+Julia+Hockenmaie)
85. Oyvind Tafjord, Matt Gardner, Kevin Lin, and Peter Clark. [Quartz: An open-domain dataset of qualitative relationship questions](https://md.research.elamur.ai/free/en/papers/tafjord-2019-quartz-open-domain-dataset-qualitative/). In Kentaro Inui, Jing Jiang, Vincent Ng, and Xiaojun Wan, editors, Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th Intern…
86. Johannes Hoffart, Mohamed Amir Yosef, Ilaria Bordino, Hagen Fürstenau, Manfred Pinkal, Marc Spaniol, Bilyana Taneva, Stefan Thater, and Gerhard Weikum. Robust disambiguation of named entities in text. In Proceedings of the 2011 Conference on Empirical Methods in Natural Language Processing, EMNLP 20… [find](https://scholar.google.com/scholar?q=Johannes+Hoffart%2C+Mohamed+Amir+Yosef%2C+Ilaria+Bordino%2C+Hagen+F%C3%BCrstenau%2C+Manfred+Pinkal%2C+Marc+Spaniol%2C+Bilyana+Taneva%2C+Stefan+Thater%2C+and+Gerhard+Weikum%2E+Robust+disambiguation+of+named+entities+in+text%2E)
87. Fabio Petroni, Aleksandra Piktus, Angela Fan, Patrick S. H. Lewis, Majid Yazdani, Nicola De Cao, James Thorne, Yacine Jernite, Vladimir Karpukhin, Jean Maillard, Vassilis Plachouras, Tim Rocktäschel, and Sebastian Riedel. [KILT: a benchmark for knowledge intensive language tasks](https://md.research.elamur.ai/free/en/papers/petroni-2020-kilt-benchmark-knowledge-intensive-language/). In Kristina Toutanov…
88. Simone Tedeschi, Simone Conia, Francesco Cecconi, and Roberto Navigli. Named entity recognition for entity linking: What works and what’s next. In Marie-Francine Moens, Xuanjing Huang, Lucia Specia, and Scott Wen-tau Yih, editors, Findings of the Association for Computational Linguistics: EMNLP 2021… [find](https://scholar.google.com/scholar?q=Simone+Tedeschi%2C+Simone+Conia%2C+Francesco+Cecconi%2C+and+Roberto+Navigli%2E+Named+entity+recognition+for+entity+linking%3A+What+works+and+what%E2%80%99s+next%2E+In+Marie%2DFrancine+Moens%2C+Xuanjing+Huang%2C+Lucia+Specia%2C+a)
89. Hady ElSahar, Pavlos Vougiouklis, Arslen Remaci, Christophe Gravier, Jonathon S. Hare, Frédérique Laforest, and Elena Simperl. [T-rex: A large scale alignment of natural language with knowledge base triples](https://md.research.elamur.ai/free/en/papers/elsahar-2018-t-rex-large-scale-alignment/). In Proceedings of the Eleventh International Conference on Language Resources and Evaluation,…
90. Omer Levy, Minjoon Seo, Eunsol Choi, and Luke Zettlemoyer. [Zero-shot relation extraction via reading comprehension](https://md.research.elamur.ai/free/en/papers/levy-2017-zero-shot-relation-extraction-reading/). In Roger Levy and Lucia Specia, editors, Proceedings of the 21st Conference on Computational Natural Language Learning (CoNLL 2017), Vancouver, Canada, August 3-4, 2017, pages 333–342.…
91. James Thorne, Andreas Vlachos, Christos Christodoulopoulos, and Arpit Mittal. [FEVER: a large-scale dataset for fact extraction and verification](https://md.research.elamur.ai/free/en/papers/thorne-2018-fever-large-scale-dataset-fact/). In Marilyn A. Walker, Heng Ji, and Amanda Stent, editors, Proceedings of the 2018 Conference of the North American Chapter of the Association for Computati…
92. Emily Dinan, Stephen Roller, Kurt Shuster, Angela Fan, Michael Auli, and Jason Weston. [Wizard of wikipedia: Knowledge-powered conversational agents](https://md.research.elamur.ai/free/en/papers/dinan-2018-wizard-wikipedia-knowledge-powered-conversational/). In 7th International Conference on Learning Representations, ICLR 2019, New Orleans, LA, USA, May 6-9, 2019. OpenReview.net, 2019.
93. Shuting Wang, Jiongnan Liu, Shiren Song, Jiehan Cheng, Yuqi Fu, Peidong Guo, Kun Fang, Yutao Zhu, and Zhicheng Dou. Domainrag: A chinese benchmark for evaluating domain-specific retrieval-augmented generation. CoRR, abs/2406.05654, 2024. — [arXiv:2406.05654](https://arxiv.org/abs/2406.05654) [find](https://scholar.google.com/scholar?q=Shuting+Wang%2C+Jiongnan+Liu%2C+Shiren+Song%2C+Jiehan+Cheng%2C+Yuqi+Fu%2C+Peidong+Guo%2C+Kun+Fang%2C+Yutao+Zhu%2C+and+Zhicheng+Dou%2E+Domainrag%3A+A+chinese+benchmark+for+evaluating+domain%2Dspecific+retrieval%2Daugmented+ge)
94. Yi Zong and Xipeng Qiu. [GAOKAO-MM: A chinese human-level benchmark for multimodal models evaluation](https://md.research.elamur.ai/free/en/papers/zong-2024-gaokao-mm-chinese-human/). In Lun-Wei Ku, Andre Martins, and Vivek Srikumar, editors, Findings of the Association for Computational Linguistics, ACL 2024, Bangkok, Thailand and virtual meeting, August 11-16, 2024, pages 8817–8…
95. Alon Talmor, Ori Yoran, Amnon Catav, Dan Lahav, Yizhong Wang, Akari Asai, Gabriel Ilharco, Hannaneh Hajishirzi, and Jonathan Berant. Multimodalqa: complex question answering over text, tables and images. In 9th International Conference on Learning Representations, ICLR 2021, Virtual Event, Austria,… [find](https://scholar.google.com/scholar?q=Alon+Talmor%2C+Ori+Yoran%2C+Amnon+Catav%2C+Dan+Lahav%2C+Yizhong+Wang%2C+Akari+Asai%2C+Gabriel+Ilharco%2C+Hannaneh+Hajishirzi%2C+and+Jonathan+Berant%2E+Multimodalqa%3A+complex+question+answering+over+text%2C+tables+and+imag)
96. Pan Lu, Hritik Bansal, Tony Xia, Jiacheng Liu, Chunyuan Li, Hannaneh Hajishirzi, Hao Cheng, Kai-Wei Chang, Michel Galley, and Jianfeng Gao. [Mathvista: Evaluating mathematical reasoning of foundation models in visual contexts](https://md.research.elamur.ai/free/en/papers/lu-2023-mathvista-evaluating-mathematical-reasoning-foundation/). In The Twelfth International Conference on Learning Representations, ICLR…
97. Weijia Shi, Sewon Min, Michihiro Yasunaga, Minjoon Seo, Richard James, Mike Lewis, Luke Zettlemoyer, and Wen-tau Yih. [REPLUG: retrieval-augmented black-box language models](https://md.research.elamur.ai/free/en/papers/shi-2023-replug-retrieval-augmented-black-box/). In Kevin Duh, Helena Gómez-Adorno, and Steven Bethard, editors, Proceedings of the 2024 Conference of the North American Chapte…
98. J. Kim, J. Nam, S. Mo, J. Park, S.-W. Lee, M. Seo, J.-W. Ha, and J. Shin. Sure: Summarizing retrievals using answer candidates for open-domain QA of LLMs. In Proc. of ICLR, 2024. [find](https://scholar.google.com/scholar?q=J%2E+Kim%2C+J%2E+Nam%2C+S%2E+Mo%2C+J%2E+Park%2C+S%2E%2DW%2E+Lee%2C+M%2E+Seo%2C+J%2E%2DW%2E+Ha%2C+and+J%2E+Shin%2E+Sure%3A+Summarizing+retrievals+using+answer+candidates+for+open%2Ddomain+QA+of+LLMs%2E+In+Proc%2E+of+ICLR%2C+2024%2E)
99. Zhihong Shao, Yeyun Gong, Yelong Shen, Minlie Huang, Nan Duan, and Weizhu Chen. [Enhancing retrieval-augmented large language models with iterative retrieval-generation synergy](https://md.research.elamur.ai/free/en/papers/shao-2023-enhancing-retrieval-augmented-large-language/). In Houda Bouamor, Juan Pino, and Kalika Bali, editors, Findings of the Association for Computational Linguistics: EMNLP 202…
100. Zhangyin Feng, Xiaocheng Feng, Dezhi Zhao, Maojin Yang, and Bing Qin. Retrieval-generation synergy augmented large language models. CoRR, abs/2310.05149, 2023. — [arXiv:2310.05149](https://arxiv.org/abs/2310.05149) [find](https://scholar.google.com/scholar?q=Zhangyin+Feng%2C+Xiaocheng+Feng%2C+Dezhi+Zhao%2C+Maojin+Yang%2C+and+Bing+Qin%2E+Retrieval%2Dgeneration+synergy+augmented+large+language+models%2E+CoRR%2C+abs%2F2310%2E05149%2C+2023%2E+%E2%80%94+%5BarXiv%3A2310%2E05149%5D%28https%3A%2F%2Farxiv%2Eorg%2Fa)
101. Zichun Yu, Chenyan Xiong, Shi Yu, and Zhiyuan Liu. Augmentation-adapted retriever improves generalization of language models as generic plug-in. In Anna Rogers, Jordan L. Boyd-Graber, and Naoaki Okazaki, editors, Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics… [find](https://scholar.google.com/scholar?q=Zichun+Yu%2C+Chenyan+Xiong%2C+Shi+Yu%2C+and+Zhiyuan+Liu%2E+Augmentation%2Dadapted+retriever+improves+generalization+of+language+models+as+generic+plug%2Din%2E+In+Anna+Rogers%2C+Jordan+L%2E+Boyd%2DGraber%2C+and+Naoaki+Okaza)
102. J. Fang, Z. Meng, and C. Macdonald. TRACE the evidence: Constructing knowledge-grounded reasoning chains for retrieval-augmented generation. ArXiv preprint, 2024. [find](https://scholar.google.com/scholar?q=J%2E+Fang%2C+Z%2E+Meng%2C+and+C%2E+Macdonald%2E+TRACE+the+evidence%3A+Constructing+knowledge%2Dgrounded+reasoning+chains+for+retrieval%2Daugmented+generation%2E+ArXiv+preprint%2C+2024%2E)
103. Y. Zhu, Z. Huang, Z. Dou, and J.-R. Wen. One token can help! learning scalable and pluggable virtual token. ArXiv preprint, 2024. [find](https://scholar.google.com/scholar?q=Y%2E+Zhu%2C+Z%2E+Huang%2C+Z%2E+Dou%2C+and+J%2E%2DR%2E+Wen%2E+One+token+can+help%21+learning+scalable+and+pluggable+virtual+token%2E+ArXiv+preprint%2C+2024%2E)
104. O. Yoran, T. Wolfson, O. Ram, and J. Berant. [Making retrieval-augmented language models robust to irrelevant context](https://md.research.elamur.ai/free/en/papers/yoran-2023-making-retrieval-augmented-language-models/). ArXiv preprint, 2023.
105. H. Trivedi, N. Balasubramanian, T. Khot, and A. Sabharwal. [Interleaving retrieval with chain-of-thought reasoning for knowledge-intensive multi-step questions](https://md.research.elamur.ai/free/en/papers/trivedi-2022-interleaving-retrieval-chain-thought-reasoning/). In Proc. of ACL, 2023.
106. Chi-Min Chan, Chunpu Xu, Ruibin Yuan, Hongyin Luo, Wei Xue, Yike Guo, and Jie Fu. [RQ-RAG: learning to refine queries for retrieval augmented generation](https://md.research.elamur.ai/free/en/papers/chan-2024-rq-rag-learning-refine-queries/). CoRR, abs/2404.00610, 2024.
107. Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. [Bleu: a method for automatic evaluation of machine translation](https://md.research.elamur.ai/free/en/papers/papineni-2002-bleu-method-automatic-evaluation-machine/). In Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics, July 6-12, 2002, Philadelphia, PA, USA, pages 311–318. ACL, 2002.
108. Chin-Yew Lin. [ROUGE: A package for automatic evaluation of summaries](https://md.research.elamur.ai/free/en/papers/lin-2004-rouge-package-automatic-evaluation-summaries/). In Text Summarization Branches Out, pages 74–81, Barcelona, Spain, July 2004. Association for Computational Linguistics.
109. AI@Meta. Llama 3 model card. 2024. [find](https://scholar.google.com/scholar?q=AI%40Meta%2E+Llama+3+model+card%2E+2024%2E)
110. E. J. Hu, Y. Shen, P. Wallis, Z. Allen-Zhu, Y. Li, S. Wang, L. Wang, and W. Chen. [Lora: Low-rank adaptation of large language models](https://md.research.elamur.ai/free/en/papers/hu-2021-lora-low-rank-adaptation-large/). In Proc. of ICLR, 2022.
111. Manveer Singh Tamber, Ronak Pradeep, and Jimmy Lin. Pre-processing matters! improved wikipedia corpora for open-domain question answering. In Jaap Kamps, Lorraine Goeuriot, Fabio Crestani, Maria Maistro, Hideo Joho, Brian Davis, Cathal Gurrin, Udo Kruschwitz, and Annalina Caputo, editors, Advances i… [find](https://scholar.google.com/scholar?q=Manveer+Singh+Tamber%2C+Ronak+Pradeep%2C+and+Jimmy+Lin%2E+Pre%2Dprocessing+matters%21+improved+wikipedia+corpora+for+open%2Ddomain+question+answering%2E+In+Jaap+Kamps%2C+Lorraine+Goeuriot%2C+Fabio+Crestani%2C+Maria+Maist)
112. Danqi Chen, Adam Fisch, Jason Weston, and Antoine Bordes. [Reading wikipedia to answer open-domain questions](https://md.research.elamur.ai/free/en/papers/chen-2017-reading-wikipedia-answer-open-domain/). In Regina Barzilay and Min-Yen Kan, editors, Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, ACL 2017, Vancouver, Canada, July 30 - August 4, Volume 1…
113. Guanting Dong, Chenghao Zhang, Mengjie Deng, Yutao Zhu, Zhicheng Dou, and Ji-Rong Wen. [Progressive multimodal reasoning via active retrieval](https://md.research.elamur.ai/free/en/papers/dong-2025-progressive-multimodal-reasoning-active-retrieval/). CoRR, abs/2412.14835, 2024.
114. Xiaoxi Li, Guanting Dong, Jiajie Jin, Yuyao Zhang, Yujia Zhou, Yutao Zhu, Peitian Zhang, and Zhicheng Dou. [Search-o1: Agentic search-enhanced large reasoning models](https://md.research.elamur.ai/free/en/papers/li-2025-search-o1-agentic-search-enhanced/). CoRR, abs/2501.05366, 2025.
115. Guanting Dong, Hongyi Yuan, Keming Lu, Chengpeng Li, Mingfeng Xue, Dayiheng Liu, Wei Wang, Zheng Yuan, Chang Zhou, and Jingren Zhou. [How abilities in large language models are affected by supervised fine-tuning data composition](https://md.research.elamur.ai/free/en/papers/dong-2024-how-abilities-large-language-models/). In Lun-Wei Ku, Andre Martins, and Vivek Srikumar, editors, Proceedings…
116. J. Lin, X. Ma, S.-C. Lin, J.-H. Yang, R. Pradeep, and R. F. Nogueira. Pyserini: A python toolkit for reproducible information retrieval research with sparse and dense representations. In Proc. of SIGIR, 2021. [find](https://scholar.google.com/scholar?q=J%2E+Lin%2C+X%2E+Ma%2C+S%2E%2DC%2E+Lin%2C+J%2E%2DH%2E+Yang%2C+R%2E+Pradeep%2C+and+R%2E+F%2E+Nogueira%2E+Pyserini%3A+A+python+toolkit+for+reproducible+information+retrieval+research+with+sparse+and+dense+representations%2E+In+Proc%2E+of+SIGI)
117. Kenton Lee, Ming-Wei Chang, and Kristina Toutanova. [Latent retrieval for weakly supervised open domain question answering](https://md.research.elamur.ai/free/en/papers/lee-2019-latent-retrieval-weakly-supervised-open/). In Anna Korhonen, David R. Traum, and Lluís Màrquez, editors, Proceedings of the 57th Conference of the Association for Computational Linguistics, ACL 2019, Florence, Italy, Jul…

*Updated 2 November 2026*