---
title: "RQ-RAG — paper by Chan et al. (2024) — summary and ideas"
description: "Retrieval-augmented LLMs usually send the user's raw input straight to the search step. 40 references, cited by 74 papers here."
authors: ["Chi-Min Chan", "Chunpu Xu", "Ruibin Yuan", "Hongyin Luo", "Wei Xue", "Yike Guo", "Jie Fu"]
year: 2024
venue: "arXiv:2404.00610"
source: "https://arxiv.org/abs/2404.00610"
doi: "10.48550/arxiv.2404.00610"
arxiv: "2404.00610"
licence: "arXiv — non-exclusive license to distribute"
licence_url: "http://arxiv.org/licenses/nonexclusive-distrib/1.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/chan-2024-rq-rag-learning-refine-queries/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/chan-2024-rq-rag-learning-refine-queries/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2024](https://md.research.elamur.ai/free/en/papers/2024/page/5/) / RQ-RAG: Learning to Refine Queries for Retrieval Augmented Generation

# RQ-RAG: Learning to Refine Queries for Retrieval Augmented Generation

**Authors:** [Chi-Min Chan](https://md.research.elamur.ai/free/en/authors/chi-min-chan/), [Chunpu Xu](https://md.research.elamur.ai/free/en/authors/chunpu-xu/), [Ruibin Yuan](https://md.research.elamur.ai/free/en/authors/ruibin-yuan/), [Hongyin Luo](https://md.research.elamur.ai/free/en/authors/hongyin-luo/), [Wei Xue](https://md.research.elamur.ai/free/en/authors/wei-xue/), [Yike Guo](https://md.research.elamur.ai/free/en/authors/yike-guo/) and [Jie Fu](https://md.research.elamur.ai/free/en/authors/jie-fu/) · **Year:** 2024 · **Venue:** arXiv:2404.00610 · **Licence:** [arXiv — non-exclusive license to distribute](http://arxiv.org/licenses/nonexclusive-distrib/1.0/)

**Links:** [Publisher's page](https://arxiv.org/abs/2404.00610) · [PDF](https://arxiv.org/pdf/2404.00610) · [DOI 10.48550/arxiv.2404.00610](https://doi.org/10.48550/arxiv.2404.00610) · [arXiv 2404.00610](https://arxiv.org/abs/2404.00610) · [OpenAlex](https://openalex.org/W4393905017)

**Citations:** 16 citations (OpenAlex, 2026-10-14) · [74](https://md.research.elamur.ai/free/en/papers/chan-2024-rq-rag-learning-refine-queries/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
Retrieval-augmented LLMs usually send the user's raw input straight to the search step. This works poorly in three cases. Simple inputs like greetings need no retrieval at all. Multi-step questions need splitting into parts. Vague questions have several valid readings. RQ-RAG addresses this by teaching a 7B Llama2 model to improve its own search queries before answering. The authors assembled about 40k training examples from multi-turn dialogue, decomposition and disambiguation datasets. ChatGPT wrote refined queries, DuckDuckGo supplied documents for them, and ChatGPT then rewrote the target answers so they match the retrieved evidence. Special control tokens let the model decide at each step whether to rewrite, decompose, disambiguate, or stop and respond. Because this produces several possible reasoning paths, the final answer is chosen by perplexity, confidence, or an ensemble of the two, with no external LLM judge. The model beats Self-RAG by 1.9% on average on single-hop QA and also performs strongly on multi-hop QA. This shows that refining queries, plus answers grounded in retrieved context, makes RAG more reliable.

## Key points
- Base model: Llama2 7B, fine-tuned end-to-end on about 40k ChatGPT-annotated instances (gpt-3.5-turbo-0125) built with DuckDuckGo search. Code is released at github.com/chanchimin/RQ-RAG.
- Benchmarks: ARC_C, PopQA and OBQA (single-hop), and HotpotQA, 2Wiki and MuSiQue (multi-hop). RQ-RAG exceeds the previous SOTA, Self-RAG, by 1.9% on average on the single-hop sets.
- Trajectory selection uses PPL, confidence or an ensemble. An oracle "upper bound" checks whether any generated path contains the right answer, and it is notably high, so better selection could bring large further gains.
- Ablation on HotpotQA: the model does best when 0% of the original dataset answers are kept. Performance falls as the retained share rises toward 100%.
- Robustness to the retrieval source (DuckDuckGo, Wikipedia, Bing) on single-hop tasks: RQ-RAG averages 67.6 with variance 0.7, while Self-RAG averages 65.5 with variance 1.8. Self-RAG loses 3–5% with Bing.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Massachusetts Institute of Technology](https://md.research.elamur.ai/free/en/universities/massachusetts-institute-of-technology/) — University, Cambridge, United States · 243 works · 76,506 citations
- [Hong Kong University of Science and Technology](https://md.research.elamur.ai/free/en/universities/hong-kong-university-of-science-and-technology/) — University, Hong Kong, China · 170 works · 26,341 citations
- [Hong Kong Polytechnic University](https://md.research.elamur.ai/free/en/universities/hong-kong-polytechnic-university/) — University, Hong Kong, China · 106 works · 5,190 citations

📚 Shelves (6): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 18,614 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 17,965 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 17,450 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 17,296 works · [👁️ Computer Vision and Pattern Recognition](https://md.research.elamur.ai/free/en/shelves/computer-science/computer-vision-and-pattern-recognition/) · 3,486 works · [🗄️ Information Systems](https://md.research.elamur.ai/free/en/shelves/computer-science/information-systems/) · 1,730 works

🧭 Topics (1): [🔍 Information retrieval & search](https://md.research.elamur.ai/free/en/topics/information-retrieval/) · 552 works

## Cite

Cite as: Chi-Min Chan, Chunpu Xu, Ruibin Yuan, Hongyin Luo, Wei Xue, Yike Guo and Jie Fu. 2024. RQ-RAG: Learning to Refine Queries for Retrieval Augmented Generation. arXiv:2404.00610.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@misc{chan2024rq,
  title = {{RQ-RAG: Learning to Refine Queries for Retrieval Augmented Generation}},
  author = {Chi-Min Chan, Chunpu Xu, Ruibin Yuan, Hongyin Luo, Wei Xue, Yike Guo and Jie Fu},
  year = {2024},
  doi = {10.48550/arxiv.2404.00610},
  eprint = {2404.00610},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2404.00610}
}
```

- [← previous Roleplay-doh: Enabling Domain-Experts to Create LLM-simulated Patients via Eliciting and Adhering to Principles](https://md.research.elamur.ai/free/en/papers/louie-2024-roleplay-doh-enabling-domain-experts/)
- [next → Simplified and Generalized Masked Diffusion for Discrete Data](https://md.research.elamur.ai/free/en/papers/shi-2024-simplified-generalized-masked-diffusion-discrete/)
- [↑ back to papers of 2024](https://md.research.elamur.ai/free/en/papers/2024/page/5/)

## References

40 references, 29 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Asai, A., Wu, Z., Wang, Y., Sil, A., and Hajishirzi, H. (2024). [Self-rag: Learning to retrieve, generate, and critique through self-reflection](https://md.research.elamur.ai/free/en/papers/asai-2023-self-rag-learning-retrieve-generate/). In The Twelfth International Conference on Learning Representations.
2. Brown, T., Mann, B., Ryder, N., Subbiah, M., Kaplan, J. D., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020). [Language models are few-shot learners](https://md.research.elamur.ai/free/en/papers/brown-2020-language-models-few-shot-learners/). Advances in neural information processing systems, 33:1877–1901.
3. Clark, P., Cowhey, I., Etzioni, O., Khot, T., Sabharwal, A., Schoenick, C., and Tafjord, O. (2018). [Think you have solved question answering? try arc, the ai2 reasoning challenge](https://md.research.elamur.ai/free/en/papers/clark-2018-think-have-solved-question-answering/). arXiv preprint arXiv:1803.05457.
4. Ho, X., Nguyen, A.-K. D., Sugawara, S., and Aizawa, A. (2020). [Constructing a multi-hop qa dataset for comprehensive evaluation of reasoning steps](https://md.research.elamur.ai/free/en/papers/ho-2020-constructing-multi-hop-qa-dataset/). arXiv preprint arXiv:2011.01060.
5. Ji, Z., Lee, N., Frieske, R., Yu, T., Su, D., Xu, Y., Ishii, E., Bang, Y. J., Madotto, A., and Fung, P. (2023). [Survey of hallucination in natural language generation](https://md.research.elamur.ai/free/en/papers/ji-2022-survey-hallucination-natural-language-generation/). ACM Computing Surveys, 55(12):1–38.
6. Karpukhin, V., Oğuz, B., Min, S., Lewis, P., Wu, L., Edunov, S., Chen, D., and Yih, W.-t. (2020). [Dense passage retrieval for open-domain question answering](https://md.research.elamur.ai/free/en/papers/karpukhin-2020-dense-passage-retrieval-open-domain/). arXiv preprint arXiv:2004.04906.
7. Keskar, N. S., McCann, B., Varshney, L. R., Xiong, C., and Socher, R. (2019). [Ctrl: A conditional transformer language model for controllable generation](https://md.research.elamur.ai/free/en/papers/keskar-2019-ctrl-conditional-transformer-language-model/). arXiv preprint arXiv:1909.05858.
8. Köpf, A., Kilcher, Y., von Rütte, D., Anagnostidis, S., Tam, Z. R., Stevens, K., Barhoum, A., Nguyen, D., Stanley, O., Nagyfi, R., et al. (2024). [Openassistant conversations-democratizing large language model alignment](https://md.research.elamur.ai/free/en/papers/kopf-2023-openassistant-conversations-democratizing-large-language/). Advances in Neural Information Processing Systems, 36.
9. Lewis, P., Perez, E., Piktus, A., Petroni, F., Karpukhin, V., Goyal, N., Küttler, H., Lewis, M., Yih, W.-t., Rocktäschel, T., et al. (2020). [Retrieval-augmented generation for knowledge-intensive nlp tasks](https://md.research.elamur.ai/free/en/papers/lewis-2020-retrieval-augmented-generation-knowledge-intensive/). Advances in Neural Information Processing Systems, 33:9459–9474.
10. Lu, X., Welleck, S., Hessel, J., Jiang, L., Qin, L., West, P., Ammanabrolu, P., and Choi, Y. (2022). [Quark: Controllable text generation with reinforced unlearning](https://md.research.elamur.ai/free/en/papers/lu-2022-quark-controllable-text-generation-reinforced/). Advances in neural information processing systems, 35:27591–27609.
11. Luo, H., Zhang, T., Chuang, Y.-S., Gong, Y., Kim, Y., Wu, X., Meng, H. M., and Glass, J. R. (2023). Search augmented instruction learning. In The 2023 Conference on Empirical Methods in Natural Language Processing. [find](https://scholar.google.com/scholar?q=Luo%2C+H%2E%2C+Zhang%2C+T%2E%2C+Chuang%2C+Y%2E%2DS%2E%2C+Gong%2C+Y%2E%2C+Kim%2C+Y%2E%2C+Wu%2C+X%2E%2C+Meng%2C+H%2E+M%2E%2C+and+Glass%2C+J%2E+R%2E+%282023%29%2E+Search+augmented+instruction+learning%2E+In+The+2023+Conference+on+Empirical+Methods+in+Natural+Langua)
12. Ma, X., Gong, Y., He, P., Zhao, H., and Duan, N. (2023). Query rewriting for retrieval-augmented large language models. arXiv preprint arXiv:2305.14283. — [arXiv:2305.14283](https://arxiv.org/abs/2305.14283) [find](https://scholar.google.com/scholar?q=Ma%2C+X%2E%2C+Gong%2C+Y%2E%2C+He%2C+P%2E%2C+Zhao%2C+H%2E%2C+and+Duan%2C+N%2E+%282023%29%2E+Query+rewriting+for+retrieval%2Daugmented+large+language+models%2E+arXiv+preprint+arXiv%3A2305%2E14283%2E+%E2%80%94+%5BarXiv%3A2305%2E14283%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2305)
13. Mallen, A., Asai, A., Zhong, V., Das, R., Khashabi, D., and Hajishirzi, H. (2022). When not to trust language models: Investigating effectiveness of parametric and non-parametric memories. arXiv preprint arXiv:2212.10511. [find](https://scholar.google.com/scholar?q=Mallen%2C+A%2E%2C+Asai%2C+A%2E%2C+Zhong%2C+V%2E%2C+Das%2C+R%2E%2C+Khashabi%2C+D%2E%2C+and+Hajishirzi%2C+H%2E+%282022%29%2E+When+not+to+trust+language+models%3A+Investigating+effectiveness+of+parametric+and+non%2Dparametric+memories%2E+arXiv+prepr)
14. Mihaylov, T., Clark, P., Khot, T., and Sabharwal, A. (2018). Can a suit of armor conduct electricity? a new dataset for open book question answering. arXiv preprint arXiv:1809.02789. [find](https://scholar.google.com/scholar?q=Mihaylov%2C+T%2E%2C+Clark%2C+P%2E%2C+Khot%2C+T%2E%2C+and+Sabharwal%2C+A%2E+%282018%29%2E+Can+a+suit+of+armor+conduct+electricity%3F+a+new+dataset+for+open+book+question+answering%2E+arXiv+preprint+arXiv%3A1809%2E02789%2E)
15. Mukherjee, S., Mitra, A., Jawahar, G., Agarwal, S., Palangi, H., and Awadallah, A. (2023). [Orca: Progressive learning from complex explanation traces of gpt-4](https://md.research.elamur.ai/free/en/papers/mukherjee-2023-orca-progressive-learning-complex-explanation/). arXiv preprint arXiv:2306.02707.
16. OpenAI (2023). [Gpt-4 technical report](https://md.research.elamur.ai/free/en/papers/openai-2023-gpt-4-technical-report/). arXiv preprint arXiv:2303.08774.
17. Ouyang, L., Wu, J., Jiang, X., Almeida, D., Wainwright, C., Mishkin, P., Zhang, C., Agarwal, S., Slama, K., Ray, A., et al. (2022). [Training language models to follow instructions with human feedback](https://md.research.elamur.ai/free/en/papers/ouyang-2022-training-language-models-follow-instructions/). Advances in neural information processing systems, 35:27730–27744.
18. Raffel, C., Shazeer, N., Roberts, A., Lee, K., Narang, S., Matena, M., Zhou, Y., Li, W., and Liu, P. J. (2020). [Exploring the limits of transfer learning with a unified text-to-text transformer](https://md.research.elamur.ai/free/en/papers/raffel-2019-exploring-limits-transfer-learning-unified/). Journal of machine learning research, 21(140):1–67.
19. Robertson, S., Zaragoza, H., et al. (2009). The probabilistic relevance framework: Bm25 and beyond. Foundations and Trends® in Information Retrieval, 3(4):333–389. [find](https://scholar.google.com/scholar?q=Robertson%2C+S%2E%2C+Zaragoza%2C+H%2E%2C+et+al%2E+%282009%29%2E+The+probabilistic+relevance+framework%3A+Bm25+and+beyond%2E+Foundations+and+Trends%C2%AE+in+Information+Retrieval%2C+3%284%29%3A333%E2%80%93389%2E)
20. Shi, F., Chen, X., Misra, K., Scales, N., Dohan, D., Chi, E. H., Schärli, N., and Zhou, D. (2023a). [Large language models can be easily distracted by irrelevant context](https://md.research.elamur.ai/free/en/papers/shi-2023-large-language-models-easily-distracted/). In International Conference on Machine Learning, pages 31210–31227. PMLR.
21. Shi, W., Min, S., Yasunaga, M., Seo, M., James, R., Lewis, M., Zettlemoyer, L., and Yih, W.-t. (2023b). [Replug: Retrieval-augmented black-box language models](https://md.research.elamur.ai/free/en/papers/shi-2023-replug-retrieval-augmented-black-box/). arXiv preprint arXiv:2301.12652.
22. Shuster, K., Poff, S., Chen, M., Kiela, D., and Weston, J. (2021). [Retrieval augmentation reduces hallucination in conversation](https://md.research.elamur.ai/free/en/papers/shuster-2021-retrieval-augmentation-reduces-hallucination-conversation/). arXiv preprint arXiv:2104.07567.
23. Stelmakh, I., Luan, Y., Dhingra, B., and Chang, M.-W. (2022). [Asqa: Factoid questions meet long-form answers](https://md.research.elamur.ai/free/en/papers/stelmakh-2022-asqa-factoid-questions-meet-long/). arXiv preprint arXiv:2204.06092.
24. Sun, Z., Wang, X., Tay, Y., Yang, Y., and Zhou, D. (2022). Recitation-augmented language models. arXiv preprint arXiv:2210.01296. [find](https://scholar.google.com/scholar?q=Sun%2C+Z%2E%2C+Wang%2C+X%2E%2C+Tay%2C+Y%2E%2C+Yang%2C+Y%2E%2C+and+Zhou%2C+D%2E+%282022%29%2E+Recitation%2Daugmented+language+models%2E+arXiv+preprint+arXiv%3A2210%2E01296%2E)
25. Taori, R., Gulrajani, I., Zhang, T., Dubois, Y., Li, X., Guestrin, C., Liang, P., and Hashimoto, T. B. (2023). Stanford alpaca: An instruction-following llama model. [find](https://scholar.google.com/scholar?q=Taori%2C+R%2E%2C+Gulrajani%2C+I%2E%2C+Zhang%2C+T%2E%2C+Dubois%2C+Y%2E%2C+Li%2C+X%2E%2C+Guestrin%2C+C%2E%2C+Liang%2C+P%2E%2C+and+Hashimoto%2C+T%2E+B%2E+%282023%29%2E+Stanford+alpaca%3A+An+instruction%2Dfollowing+llama+model%2E)
26. Touvron, H., Martin, L., Stone, K., Albert, P., Almahairi, A., Babaei, Y., Bashlykov, N., Batra, S., Bhargava, P., Bhosale, S., et al. (2023). [Llama 2: Open foundation and fine-tuned chat models](https://md.research.elamur.ai/free/en/papers/touvron-2023-llama-2-open-foundation-fine/). arXiv preprint arXiv:2307.09288.
27. Trivedi, H., Balasubramanian, N., Khot, T., and Sabharwal, A. (2022). [Musique: Multihop questions via single-hop question composition](https://md.research.elamur.ai/free/en/papers/trivedi-2022-musique-multihop-questions-single-hop/). Transactions of the Association for Computational Linguistics, 10:539–554.
28. Trivedi, H., Balasubramanian, N., Khot, T., and Sabharwal, A. (2023). [Interleaving retrieval with chain-of-thought reasoning for knowledge-intensive multi-step questions](https://md.research.elamur.ai/free/en/papers/trivedi-2022-interleaving-retrieval-chain-thought-reasoning/). The 61st Annual Meeting of the Association for Computational Linguistics.
29. Vu, T., Iyyer, M., Wang, X., Constant, N., Wei, J., Wei, J., Tar, C., Sung, Y.-H., Zhou, D., Le, Q., et al. (2023). [Freshllms: Refreshing large language models with search engine augmentation](https://md.research.elamur.ai/free/en/papers/vu-2023-freshllms-refreshing-large-language-models/). arXiv preprint arXiv:2310.03214.
30. Wang, X., Wei, J., Schuurmans, D., Le, Q., Chi, E., Narang, S., Chowdhery, A., and Zhou, D. (2022). [Self-consistency improves chain of thought reasoning in language models](https://md.research.elamur.ai/free/en/papers/wang-2022-self-consistency-improves-chain-thought/). arXiv preprint arXiv:2203.11171.
31. Wei, J., Wang, X., Schuurmans, D., Bosma, M., Xia, F., Chi, E., Le, Q. V., Zhou, D., et al. (2022). [Chain-of-thought prompting elicits reasoning in large language models](https://md.research.elamur.ai/free/en/papers/wei-2022-chain-thought-prompting-elicits-reasoning/). Advances in neural information processing systems, 35:24824–24837.
32. Xu, C., Chern, S., Chern, E., Zhang, G., Wang, Z., Liu, R., Li, J., Fu, J., and Liu, P. (2023a). Align on the fly: Adapting chatbot behavior to established norms. arXiv preprint arXiv:2312.15907. — [arXiv:2312.15907](https://arxiv.org/abs/2312.15907) [find](https://scholar.google.com/scholar?q=Xu%2C+C%2E%2C+Chern%2C+S%2E%2C+Chern%2C+E%2E%2C+Zhang%2C+G%2E%2C+Wang%2C+Z%2E%2C+Liu%2C+R%2E%2C+Li%2C+J%2E%2C+Fu%2C+J%2E%2C+and+Liu%2C+P%2E+%282023a%29%2E+Align+on+the+fly%3A+Adapting+chatbot+behavior+to+established+norms%2E+arXiv+preprint+arXiv%3A2312%2E15907%2E+%E2%80%94+%5Ba)
33. Xu, C., Sun, Q., Zheng, K., Geng, X., Zhao, P., Feng, J., Tao, C., and Jiang, D. (2023b). [Wizardlm: Empowering large language models to follow complex instructions](https://md.research.elamur.ai/free/en/papers/xu-2023-wizardlm-empowering-large-pre-trained/). arXiv preprint arXiv:2304.12244.
34. Xu, F., Shi, W., and Choi, E. (2023c). [Recomp: Improving retrieval-augmented lms with compression and selective augmentation](https://md.research.elamur.ai/free/en/papers/xu-2023-recomp-improving-retrieval-augmented-lms/). arXiv preprint arXiv:2310.04408.
35. Yang, Z., Qi, P., Zhang, S., Bengio, Y., Cohen, W. W., Salakhutdinov, R., and Manning, C. D. (2018). [Hotpotqa: A dataset for diverse, explainable multi-hop question answering](https://md.research.elamur.ai/free/en/papers/yang-2018-hotpotqa-dataset-diverse-explainable-multi/). arXiv preprint arXiv:1809.09600.
36. Yao, S., Yu, D., Zhao, J., Shafran, I., Griffiths, T., Cao, Y., and Narasimhan, K. (2024). Tree of thoughts: Deliberate problem solving with large language models. Advances in Neural Information Processing Systems, 36. [find](https://scholar.google.com/scholar?q=Yao%2C+S%2E%2C+Yu%2C+D%2E%2C+Zhao%2C+J%2E%2C+Shafran%2C+I%2E%2C+Griffiths%2C+T%2E%2C+Cao%2C+Y%2E%2C+and+Narasimhan%2C+K%2E+%282024%29%2E+Tree+of+thoughts%3A+Deliberate+problem+solving+with+large+language+models%2E+Advances+in+Neural+Information+Proce)
37. Yoran, O., Wolfson, T., Ram, O., and Berant, J. (2023). [Making retrieval-augmented language models robust to irrelevant context](https://md.research.elamur.ai/free/en/papers/yoran-2023-making-retrieval-augmented-language-models/). arXiv preprint arXiv:2310.01558.
38. Yu, W., Zhang, H., Pan, X., Ma, K., Wang, H., and Yu, D. (2023). [Chain-of-note: Enhancing robustness in retrieval-augmented language models](https://md.research.elamur.ai/free/en/papers/yu-2024-chain-note-enhancing-robustness-retrieval/). arXiv preprint arXiv:2311.09210.
39. Zhou, C., Liu, P., Xu, P., Iyer, S., Sun, J., Mao, Y., Ma, X., Efrat, A., Yu, P., Yu, L., et al. (2024). Lima: Less is more for alignment. Advances in Neural Information Processing Systems, 36. [find](https://scholar.google.com/scholar?q=Zhou%2C+C%2E%2C+Liu%2C+P%2E%2C+Xu%2C+P%2E%2C+Iyer%2C+S%2E%2C+Sun%2C+J%2E%2C+Mao%2C+Y%2E%2C+Ma%2C+X%2E%2C+Efrat%2C+A%2E%2C+Yu%2C+P%2E%2C+Yu%2C+L%2E%2C+et+al%2E+%282024%29%2E+Lima%3A+Less+is+more+for+alignment%2E+Advances+in+Neural+Information+Processing+Systems%2C+36%2E)
40. Zhou, S., Alon, U., Xu, F. F., Wang, Z., Jiang, Z., and Neubig, G. (2022). Docprompting: Generating code by retrieving the docs. arXiv preprint arXiv:2207.05987. [find](https://scholar.google.com/scholar?q=Zhou%2C+S%2E%2C+Alon%2C+U%2E%2C+Xu%2C+F%2E+F%2E%2C+Wang%2C+Z%2E%2C+Jiang%2C+Z%2E%2C+and+Neubig%2C+G%2E+%282022%29%2E+Docprompting%3A+Generating+code+by+retrieving+the+docs%2E+arXiv+preprint+arXiv%3A2207%2E05987%2E)

*Updated 2 November 2026*