---
title: "Augmented Large Language Models with Parametric Knowledge"
description: "Luo et al. propose Parametric Knowledge Guiding (PKG). 56 references, cited by 24 papers here."
authors: ["Ziyang Luo", "Can Xu", "Pu Zhao", "Xiubo Geng", "Chongyang Tao", "Jing Ma", "Qingwei Lin", "Daxin Jiang"]
year: 2023
venue: "arXiv:2305.04757"
source: "https://arxiv.org/abs/2305.04757"
doi: "10.48550/arxiv.2305.04757"
arxiv: "2305.04757"
licence: "CC BY 4.0 (arXiv)"
licence_url: "http://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/luo-2023-augmented-large-language-models-parametric/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/luo-2023-augmented-large-language-models-parametric/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2023](https://md.research.elamur.ai/free/en/papers/2023/page/9/) / Augmented Large Language Models with Parametric Knowledge Guiding

# Augmented Large Language Models with Parametric Knowledge Guiding

**Authors:** [Ziyang Luo](https://md.research.elamur.ai/free/en/authors/ziyang-luo/), [Can Xu](https://md.research.elamur.ai/free/en/authors/can-xu/), [Pu Zhao](https://md.research.elamur.ai/free/en/authors/pu-zhao/), [Xiubo Geng](https://md.research.elamur.ai/free/en/authors/xiubo-geng/), [Chongyang Tao](https://md.research.elamur.ai/free/en/authors/chongyang-tao/), [Jing Ma](https://md.research.elamur.ai/free/en/authors/jing-ma/), [Qingwei Lin](https://md.research.elamur.ai/free/en/authors/qingwei-lin/) and [Daxin Jiang](https://md.research.elamur.ai/free/en/authors/daxin-jiang/) · **Year:** 2023 · **Venue:** arXiv:2305.04757 · **Licence:** [CC BY 4.0 (arXiv)](http://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://arxiv.org/abs/2305.04757) · [PDF](https://arxiv.org/pdf/2305.04757) · [DOI 10.48550/arxiv.2305.04757](https://doi.org/10.48550/arxiv.2305.04757) · [arXiv 2305.04757](https://arxiv.org/abs/2305.04757) · [OpenAlex](https://openalex.org/W4375959372)

**Citations:** 8 citations (OpenAlex, 2026-10-14) · [24](https://md.research.elamur.ai/free/en/papers/luo-2023-augmented-large-language-models-parametric/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
Commercial LLMs that are reachable only through APIs often perform poorly on tasks that need specialised domain knowledge. Users also cannot easily fine-tune them, and sending private data to the provider creates privacy risks. Luo et al. propose Parametric Knowledge Guiding (PKG). It leaves the black-box model untouched and adds a separate knowledge module that is trained offline. An open-source LLaMa-7B is instruction-tuned on raw task or domain data, which is rewritten as (instruction, input, output) triples. This teaches it to produce background material for a given input. At inference, the module writes a relevant background document, such as facts, a table, a medical explanation or an image-aware description. That document is then inserted into the prompt of the black-box LLM, following a generate-then-read scheme. The authors position this as an alternative to retrieval with dense retrievers like DPR, where query and document interact only shallowly. On four knowledge-heavy benchmarks the method improves over plain prompting. It offers a way to adapt API-only models to a domain without changing their weights or exposing data, although the generated knowledge can be hallucinated.

## Key points
- Reported gains: FM2 (fact checking) +7.9%, NQ-Table (open-domain QA over tables) +11.9%, MedMC-QA (medical) +3.0%, ScienceQA (multimodal) +8.1%.
- In one reported evaluation, PKG on top of InstructGPT models reached an average of 80.76%. This beat systems built on gpt-3.5-turbo (78.31%). The guiding baselines CoT (−1.44%) and GenRead (+0.42%) brought little or no benefit.
- Effect of the black-box model's size (text-ada-001, text-babbage-001, text-curie-001, text-davinci-002): larger models profit more from the generated knowledge. The smallest ones gain almost nothing on the tabular task.
- Effect of the guide's size, tested with the OPT family: moving from 1.3B to 2.7B parameters added 1.4 on FM2, 1.4 on NQ-Table, 0.5 on MedMC-QA and 2.5 on ScienceQA.
- Main limitation: the guide can hallucinate wrong background facts. The authors suggest combining PKG with retrieval as future work.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Microsoft](https://md.research.elamur.ai/free/en/companies/microsoft/) — Company, Redmond, United States · 325 works · 142,931 citations
- [Hong Kong Baptist University](https://md.research.elamur.ai/free/en/universities/hong-kong-baptist-university/) — University, Hong Kong, China · 7 works · 246 citations

📚 Shelves (5): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 16,039 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 15,891 works · [👁️ Computer Vision and Pattern Recognition](https://md.research.elamur.ai/free/en/shelves/computer-science/computer-vision-and-pattern-recognition/) · 2,869 works

🧭 Topics (1): [🧠 Language models & pretraining](https://md.research.elamur.ai/free/en/topics/language-models/) · 3,320 works

## Cite

Cite as: Ziyang Luo, Can Xu, Pu Zhao, Xiubo Geng, Chongyang Tao, Jing Ma, Qingwei Lin and Daxin Jiang. 2023. Augmented Large Language Models with Parametric Knowledge Guiding. arXiv:2305.04757.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@misc{luo2023augmented,
  title = {{Augmented Large Language Models with Parametric Knowledge Guiding}},
  author = {Ziyang Luo, Can Xu, Pu Zhao, Xiubo Geng, Chongyang Tao, Jing Ma, Qingwei Lin and Daxin Jiang},
  year = {2023},
  doi = {10.48550/arxiv.2305.04757},
  eprint = {2305.04757},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2305.04757}
}
```

- [← previous TransNormerLLM: A Faster and Better Large Language Model with Improved TransNormer](https://md.research.elamur.ai/free/en/papers/qin-2023-transnormerllm-faster-better-large/)
- [next → BLSP: Bootstrapping Language-Speech Pre-training via Behavior Alignment of Continuation Writing](https://md.research.elamur.ai/free/en/papers/wang-2023-blsp-bootstrapping-language-speech-pre/)
- [↑ back to papers of 2023](https://md.research.elamur.ai/free/en/papers/2023/page/9/)

## References

56 references, 38 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. BBC, 2023. Chatgpt banned in italy over privacy concerns. Webpage. URL: [https://www.bbc.com/news/technology-65139406](https://www.bbc.com/news/technology-65139406). accessed on May 8, 2023. [↗](https://www.bbc.com/news/technology-65139406)
2. Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Ches… — [Language Models are Few-Shot Learners](https://md.research.elamur.ai/free/en/papers/brown-2020-language-models-few-shot-learners/)
3. Chalkidis, I., 2023. Chatgpt may pass the bar exam soon, but has a long way to go for the lexglue benchmark. arXiv:2304.12202. [find](https://scholar.google.com/scholar?q=Chalkidis%2C+I%2E%2C+2023%2E+Chatgpt+may+pass+the+bar+exam+soon%2C+but+has+a+long+way+to+go+for+the+lexglue+benchmark%2E+arXiv%3A2304%2E12202%2E)
4. Chen, D., Fisch, A., Weston, J., Bordes, A., 2017. [Reading wikipedia to answer open-domain questions](https://md.research.elamur.ai/free/en/papers/chen-2017-reading-wikipedia-answer-open-domain/), in: Barzilay, R., Kan, M. (Eds.), Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, ACL 2017, Vancouver, Canada, July 30 - August 4, Volume 1: Long Papers, Ass…
5. Chen, M., Tworek, J., Jun, H., Yuan, Q., de Oliveira Pinto, H.P., Kaplan, J., Edwards, H., Burda, Y., Joseph, N., Brockman, G., Ray, A., Puri, R., Krueger, G., Petrov, M., Khlaaf, H., Sastry, G., Mishkin, P., Chan, B., Gray, S., Ryder, N., Pavlov, M., Power, A., Kaiser, L., Bavarian, M., Winter, C.,… — [Evaluating Large Language Models Trained on Code](https://md.research.elamur.ai/free/en/papers/chen-2021-evaluating-large-language-models-trained/)
6. Chowdhery, A., Narang, S., Devlin, J., Bosma, M., Mishra, G., Roberts, A., Barham, P., Chung, H.W., Sutton, C., Gehrmann, S., Schuh, P., Shi, K., Tsvyashchenko, S., Maynez, J., Rao, A., Barnes, P., Tay, Y., Shazeer, N., Prabhakaran, V., Reif, E., Du, N., Hutchinson, B., Pope, R., Bradbury, J., Austi… — [PaLM: Scaling Language Modeling with Pathways](https://md.research.elamur.ai/free/en/papers/chowdhery-2022-palm-scaling-language-modeling-pathways/)
7. Devlin, J., Chang, M., Lee, K., Toutanova, K., 2019. [BERT: pre-training of deep bidirectional transformers for language understanding](https://md.research.elamur.ai/free/en/papers/devlin-2019-bert-pre-training-deep-bidirectional/), in: Burstein, J., Doran, C., Solorio, T. (Eds.), Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: H…
8. Eisenschlos, J., Dhingra, B., Bulian, J., Börschinger, B., Boyd-Graber, J.L., 2021. Fool me twice: Entailment from wikipedia gamification, in: Toutanova, K., Rumshisky, A., Zettlemoyer, L., Hakkani-Tür, D., Beltagy, I., Bethard, S., Cotterell, R., Chakraborty, T., Zhou, Y. (Eds.), Proceedings of the… [find](https://scholar.google.com/scholar?q=Eisenschlos%2C+J%2E%2C+Dhingra%2C+B%2E%2C+Bulian%2C+J%2E%2C+B%C3%B6rschinger%2C+B%2E%2C+Boyd%2DGraber%2C+J%2EL%2E%2C+2021%2E+Fool+me+twice%3A+Entailment+from+wikipedia+gamification%2C+in%3A+Toutanova%2C+K%2E%2C+Rumshisky%2C+A%2E%2C+Zettlemoyer%2C+L%2E%2C+Hakkani%2DT%C3%BC)
9. Gao, Y., Sheng, T., Xiang, Y., Xiong, Y., Wang, H., Zhang, J., 2023. Chat-rec: Towards interactive and explainable llms-augmented recommender system. CoRR abs/2303.14524. URL: https://doi.org/10.48550/arXiv.2303.14524, doi:10.48550/ arXiv.2303.14524, arXiv:2303.14524. — [doi:10.48550/arxiv.2303.14524](https://doi.org/10.48550/arxiv.2303.14524) [↗](https://doi.org/10.48550/arXiv.2303.14524)
10. Geng, X., Gudibande, A., Liu, H., Wallace, E., Abbeel, P., Levine, S., Song, D., 2023. Koala: A dialogue model for academic research. Blog post. URL: [https://bair.berkeley.edu/blog/](https://bair.berkeley.edu/blog/) 2023/04/03/koala/. [↗](https://bair.berkeley.edu/blog/)
11. Guu, K., Lee, K., Tung, Z., Pasupat, P., Chang, M., 2020. Retrieval augmented language model pre-training, in: Proceedings of the 37th International Conference on Machine Learning, ICML 2020, 13-18 July 2020, Virtual Event, PMLR. pp. 3929–3938. URL: [http://proceedings.mlr.press/v119/guu20a.html](http://proceedings.mlr.press/v119/guu20a.html). [↗](http://proceedings.mlr.press/v119/guu20a.html)
12. Herzig, J., Müller, T., Krichene, S., Eisenschlos, J., 2021. [Open domain question answering over tables via dense retrieval](https://md.research.elamur.ai/free/en/papers/herzig-2021-open-domain-question-answering-over/), in: Toutanova, K., Rumshisky, A., Zettlemoyer, L., Hakkani-Tür, D., Beltagy, I., Bethard, S., Cotterell, R., Chakraborty, T., Zhou, Y. (Eds.), Proceedings of the 2021 Conferen…
13. Huang, R., Li, M., Yang, D., Shi, J., Chang, X., Ye, Z., Wu, Y., Hong, Z., Huang, J., Liu, J., Ren, Y., Zhao, Z., Watanabe, S., 2023. Audiogpt: Understanding and generating speech, music, sound, and talking head. CoRR abs/2304.12995. URL: https://doi.org/10.48550/arXiv.2304.12995, doi:10.48550/arXiv… — [doi:10.48550/arxiv.2304.12995](https://doi.org/10.48550/arxiv.2304.12995) [↗](https://doi.org/10.48550/arXiv.2304.12995)
14. Izacard, G., Lewis, P.S.H., Lomeli, M., Hosseini, L., Petroni, F., Schick, T., Dwivedi-Yu, J., Joulin, A., Riedel, S., Grave, E., 2022. [Few-shot learning with retrieval augmented language models](https://md.research.elamur.ai/free/en/papers/izacard-2022-atlas-few-shot-learning-retrieval/). CoRR abs/2208.03299. URL: https://doi.org/10.48550/arXiv.2208.03299, doi:10.48550/arXiv.2208.03299, arXi… [↗](https://doi.org/10.48550/arXiv.2208.03299)
15. Jiao, W., Wang, W., Huang, J., Wang, X., Tu, Z., 2023. [Is chatgpt A good translator?](https://md.research.elamur.ai/free/en/papers/jiao-2023-chatgpt-good-translator-yes-gpt/) A preliminary study. CoRR abs/2301.08745. URL: https://doi.org/10.48550/arXiv.2301.08745, doi:10.48550/arXiv.2301.08745, arXiv:2301.08745. [↗](https://doi.org/10.48550/arXiv.2301.08745)
16. Kaplan, J., McCandlish, S., Henighan, T., Brown, T.B., Chess, B., Child, R., Gray, S., Radford, A., Wu, J., Amodei, D., 2020. [Scaling laws for neural language models](https://md.research.elamur.ai/free/en/papers/kaplan-2020-scaling-laws-neural-language-models/). CoRR abs/2001.08361. URL: https://arxiv.org/abs/2001.08361, arXiv:2001.08361. [↗](https://arxiv.org/abs/2001.08361)
17. Karpukhin, V., Oguz, B., Min, S., Lewis, P.S.H., Wu, L., Edunov, S., Chen, D., Yih, W., 2020. [Dense passage retrieval for open-domain question answering](https://md.research.elamur.ai/free/en/papers/karpukhin-2020-dense-passage-retrieval-open-domain/), in: Webber, B., Cohn, T., He, Y., Liu, Y. (Eds.), Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, EMNLP 20…
18. Kasai, J., Kasai, Y., Sakaguchi, K., Yamada, Y., Radev, D., 2023. [Evaluating GPT-4 and chatgpt on japanese medical licensing examinations](https://md.research.elamur.ai/free/en/papers/kasai-2023-evaluating-gpt-4-chatgpt-japanese/). CoRR abs/2303.18027. URL: https://doi.org/10. 48550/arXiv.2303.18027, doi:10.48550/arXiv.2303.18027, arXiv:2303.18027.
19. Kojima, T., Gu, S.S., Reid, M., Matsuo, Y., Iwasawa, Y., 2022. [Large language models are zero-shot reasoners](https://md.research.elamur.ai/free/en/papers/kojima-2022-large-language-models-zero-shot/), in: Oh, A.H., Agarwal, A., Belgrave, D., Cho, K. (Eds.), Advances in Neural Information Processing Systems. URL: [https://openreview.net/forum?id=e2TBb5y0yFf](https://openreview.net/forum?id=e2TBb5y0yFf). [↗](https://openreview.net/forum?id=e2TBb5y0yFf)
20. Lazaridou, A., Gribovskaya, E., Stokowiec, W., Grigorev, N., 2022. [Internet-augmented language models through few-shot prompting for open-domain question answering](https://md.research.elamur.ai/free/en/papers/lazaridou-2022-internet-augmented-language-models-through/). CoRR abs/2203.05115. URL: https://doi.org/10.48550/arXiv.2203.05115, doi:10.48550/ arXiv.2203.05115, arXiv:2203.05115. [↗](https://doi.org/10.48550/arXiv.2203.05115)
21. Liu, J., 2022. LlamaIndex. URL: [https://github.com/jerryjliu/llama_index](https://github.com/jerryjliu/llama_index), doi:10. 5281/zenodo.1234. [↗](https://github.com/jerryjliu/llama_index)
22. Lu, P., Mishra, S., Xia, T., Qiu, L., Chang, K., Zhu, S., Tafjord, O., Clark, P., Kalyan, A., 2022. Learn to explain: Multimodal reasoning via thought chains for science question answering. CoRR abs/2209.09513. URL: https://doi.org/10.48550/arXiv.2209.09513, doi:10.48550/arXiv.2209.09513, arXiv:2209… [↗](https://doi.org/10.48550/arXiv.2209.09513)
23. Lu, P., Peng, B., Cheng, H., Galley, M., Chang, K., Wu, Y.N., Zhu, S., Gao, J., 2023. [Chameleon: Plug-and-play compositional reasoning with large language models](https://md.research.elamur.ai/free/en/papers/lu-2023-chameleon-plug-play-compositional-reasoning/). CoRR abs/2304.09842. URL: https://doi.org/10.48550/arXiv.2304.09842, doi:10.48550/ arXiv.2304.09842, arXiv:2304.09842. [↗](https://doi.org/10.48550/arXiv.2304.09842)
24. Mialon, G., Dessì, R., Lomeli, M., Nalmpantis, C., Pasunuru, R., Raileanu, R., Rozière, B., Schick, T., Dwivedi-Yu, J., Celikyilmaz, A., Grave, E., LeCun, Y., Scialom, T., 2023. [Augmented language models: a survey](https://md.research.elamur.ai/free/en/papers/mialon-2023-augmented-language-models-survey/). CoRR abs/2302.07842. URL: https://doi.org/10.48550/arXiv.2302. 07842, doi:10.48550/ar… [↗](https://doi.org/10.48550/arXiv.2302)
25. Microsoft, 2023. New bing. Webpage. URL: [https://www.bing.com/new](https://www.bing.com/new). accessed on May 8, 2023. [↗](https://www.bing.com/new)
26. Nascimento, C., Monteiro, C., Pimentel, Silva, A., 2023. Do large language models understand chemistry? a conversation with chatgpt. Journal of Chemical Information and Modeling 63, 1649–1655. URL: https://doi.org/10.1021/acs.jcim.3c00285, doi:10.1021/acs.jcim.3c00285, arXiv:https://doi.org/10.1021/… — [doi:10.1021/acs.jcim.3c00285](https://doi.org/10.1021/acs.jcim.3c00285) [↗](https://doi.org/10.1021/acs.jcim.3c00285)
27. OpenAI, 2023a. Chatgpt plugins. Webpage. chatgpt-plugins. accessed on May 8, 2023. [find](https://scholar.google.com/scholar?q=OpenAI%2C+2023a%2E+Chatgpt+plugins%2E+Webpage%2E+chatgpt%2Dplugins%2E+accessed+on+May+8%2C+2023%2E)
28. URL: [https://openai.com/blog/](https://openai.com/blog/) [↗](https://openai.com/blog/)
29. OpenAI, 2023b. [GPT-4 technical report](https://md.research.elamur.ai/free/en/papers/openai-2023-gpt-4-technical-report/). CoRR abs/2303.08774. URL: https://doi.org/10. 48550/arXiv.2303.08774, doi:10.48550/arXiv.2303.08774, arXiv:2303.08774.
30. Ouyang, L., Wu, J., Jiang, X., Almeida, D., Wainwright, C.L., Mishkin, P., Zhang, C., Agarwal, S., Slama, K., Ray, A., Schulman, J., Hilton, J., Kelton, F., Miller, L., Simens, M., Askell, A., Welinder, P., Christiano, P.F., Leike, J., Lowe, R., 2022. [Training language models to follow instructions…](https://md.research.elamur.ai/free/en/papers/ouyang-2022-training-language-models-follow-instructions/)
31. Pal, A., Umapathi, L.K., Sankarasubbu, M., 2022. [Medmcqa: A large-scale multi-subject multi-choice dataset for medical domain question answering, in: Flores, G., Chen, G.H., Pollard, T.J., Ho, J.C., Naumann, T](https://md.research.elamur.ai/free/en/papers/pal-2022-medmcqa-large-scale-multi-subject/). (Eds.), Conference on Health, Inference, and Learning, CHIL 2022, 7-8 April 2022, Virtual…
32. Peng, B., Galley, M., He, P., Cheng, H., Xie, Y., Hu, Y., Huang, Q., Liden, L., Yu, Z., Chen, W., Gao, J., 2023a. [Check your facts and try again: Improving large language models with external knowledge and automated feedback](https://md.research.elamur.ai/free/en/papers/peng-2023-check-facts-try-again-improving/). CoRR abs/2302.12813. URL: https://doi.org/10. 48550/arXiv.2302.12813, doi:…
33. Peng, B., Li, C., He, P., Galley, M., Gao, J., 2023b. [Instruction tuning with GPT-4](https://md.research.elamur.ai/free/en/papers/peng-2023-instruction-tuning-gpt-4/). CoRR abs/2304.03277. URL: https://doi.org/10.48550/arXiv.2304.03277, doi:10.48550/ arXiv.2304.03277, arXiv:2304.03277. [↗](https://doi.org/10.48550/arXiv.2304.03277)
34. Radford, A., Kim, J.W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., Krueger, G., Sutskever, I., 2021. [Learning transferable visual models from natural language supervision](https://md.research.elamur.ai/free/en/papers/radford-2021-learning-transferable-visual-models-natural/), in: Meila, M., Zhang, T. (Eds.), Proceedings of the 38th International Confe…
35. Ram, O., Levine, Y., Dalmedigos, I., Muhlgay, D., Shashua, A., Leyton-Brown, K., Shoham, Y., 2023. [In-context retrieval-augmented language models](https://md.research.elamur.ai/free/en/papers/ram-2023-context-retrieval-augmented-language-models/). CoRR abs/2302.00083. URL: https://doi.org/10.48550/arXiv.2302.00083, doi:10.48550/ arXiv.2302.00083, arXiv:2302.00083. [↗](https://doi.org/10.48550/arXiv.2302.00083)
36. Robertson, S.E., Zaragoza, H., 2009. The probabilistic relevance framework: BM25 and beyond. Found. Trends Inf. Retr. 3, 333–389. URL: https://doi.org/10.1561/1500000019, doi:10. 1561/1500000019. [↗](https://doi.org/10.1561/1500000019)
37. Sanh, V., Webson, A., Raffel, C., Bach, S.H., Sutawika, L., Alyafeai, Z., Chaffin, A., Stiegler, A., Raja, A., Dey, M., Bari, M.S., Xu, C., Thakker, U., Sharma, S.S., Szczechla, E., Kim, T., Chhablani, G., Nayak, N.V., Datta, D., Chang, J., Jiang, M.T., Wang, H., Manica, M., Shen, S., Yong, Z.X., Pa… — [Multitask Prompted Training Enables Zero-Shot Task Generalization](https://md.research.elamur.ai/free/en/papers/sanh-2021-multitask-prompted-training-enables-zero/)
38. Scao, T.L., Fan, A., Akiki, C., Pavlick, E., Ilic, S., Hesslow, D., Castagné, R., Luccioni, A.S., Yvon, F., Gallé, M., Tow, J., Rush, A.M., Biderman, S., Webson, A., Ammanamanchi, P.S., Wang, T., Sagot, B., Muennighoff, N., del Moral, A.V., Ruwase, O., Bawden, R., Bekman, S., McMillan-Major, A., Bel… [find](https://scholar.google.com/scholar?q=Scao%2C+T%2EL%2E%2C+Fan%2C+A%2E%2C+Akiki%2C+C%2E%2C+Pavlick%2C+E%2E%2C+Ilic%2C+S%2E%2C+Hesslow%2C+D%2E%2C+Castagn%C3%A9%2C+R%2E%2C+Luccioni%2C+A%2ES%2E%2C+Yvon%2C+F%2E%2C+Gall%C3%A9%2C+M%2E%2C+Tow%2C+J%2E%2C+Rush%2C+A%2EM%2E%2C+Biderman%2C+S%2E%2C+Webson%2C+A%2E%2C+Ammanamanchi%2C+P%2ES%2E%2C+Wang%2C+T%2E%2C+Sago)
39. Shen, Y., Song, K., Tan, X., Li, D., Lu, W., Zhuang, Y., 2023. [Hugginggpt: Solving AI tasks with chatgpt and its friends in huggingface](https://md.research.elamur.ai/free/en/papers/shen-2023-hugginggpt-solving-ai-tasks-chatgpt/). CoRR abs/2303.17580. URL: https://doi.org/10. 48550/arXiv.2303.17580, doi:10.48550/arXiv.2303.17580, arXiv:2303.17580.
40. Shi, F., Suzgun, M., Freitag, M., Wang, X., Srivats, S., Vosoughi, S., Chung, H.W., Tay, Y., Ruder, S., Zhou, D., Das, D., Wei, J., 2022. [Language models are multilingual chain-of-thought reasoners](https://md.research.elamur.ai/free/en/papers/shi-2022-language-models-multilingual-chain-thought/). CoRR abs/2210.03057. URL: https://doi.org/10.48550/arXiv.2210.03057, doi:10.48550/arXiv.2210.03057, a… [↗](https://doi.org/10.48550/arXiv.2210.03057)
41. Shi, W., Min, S., Yasunaga, M., Seo, M., James, R., Lewis, M., Zettlemoyer, L., Yih, W., 2023. [REPLUG: retrieval-augmented black-box language models](https://md.research.elamur.ai/free/en/papers/shi-2023-replug-retrieval-augmented-black-box/). CoRR abs/2301.12652. URL: https://doi.org/10.48550/arXiv.2301.12652, doi:10.48550/ arXiv.2301.12652, arXiv:2301.12652. [↗](https://doi.org/10.48550/arXiv.2301.12652)
42. Taori, R., Gulrajani, I., Zhang, T., Dubois, Y., Li, X., Guestrin, C., Liang, P., Hashimoto, T.B., 2023. Stanford alpaca: An instruction-following llama model. [https://github.com/tatsu-lab/](https://github.com/tatsu-lab/) stanford_alpaca. [↗](https://github.com/tatsu-lab/)
43. Touvron, H., Lavril, T., Izacard, G., Martinet, X., Lachaux, M., Lacroix, T., Rozière, B., Goyal, N., Hambro, E., Azhar, F., Rodriguez, A., Joulin, A., Grave, E., Lample, G., 2023. [Llama: Open and efficient foundation language models](https://md.research.elamur.ai/free/en/papers/touvron-2023-llama-open-efficient-foundation-language/). CoRR abs/2302.13971. URL: https://doi.org/10. 48550/arXiv.2302.13…
44. Wei, J., Bosma, M., Zhao, V.Y., Guu, K., Yu, A.W., Lester, B., Du, N., Dai, A.M., Le, Q.V., 2022a. [Finetuned language models are zero-shot learners](https://md.research.elamur.ai/free/en/papers/wei-2021-finetuned-language-models-zero-shot/), in: The Tenth International Conference on Learning Representations, ICLR 2022, Virtual Event, April 25-29, 2022, OpenReview.net. URL: https://openrevie…
45. Wei, J., Tay, Y., Bommasani, R., Raffel, C., Zoph, B., Borgeaud, S., Yogatama, D., Bosma, M., Zhou, D., Metzler, D., Chi, E.H., Hashimoto, T., Vinyals, O., Liang, P., Dean, J., Fedus, W., 2022b. [Emergent abilities of large language models](https://md.research.elamur.ai/free/en/papers/wei-2022-emergent-abilities-large-language-models/). CoRR abs/2206.07682. URL: https://doi.org/ 10.48550/arXiv.22…
46. Wei, J., Wang, X., Schuurmans, D., Bosma, M., Ichter, B., Xia, F., Chi, E.H., Le, Q.V., Zhou, D., 2022c. [Chain-of-thought prompting elicits reasoning in large language models](https://md.research.elamur.ai/free/en/papers/wei-2022-chain-thought-prompting-elicits-reasoning/), in: NeurIPS. URL: [http://papers.nips.cc/paper_files/paper/2022/hash/](http://papers.nips.cc/paper_files/paper/2022/hash/) 9d5609613524ecf4f15af0f7b31abca4-Abstract-Conference.ht… [↗](http://papers.nips.cc/paper_files/paper/2022/hash/)
47. West, C.G., 2023. AI and the FCI: can chatgpt project an understanding of introductory physics? CoRR abs/2303.01067. URL: https://doi.org/10.48550/arXiv.2303.01067, doi:10.48550/arXiv.2303.01067, arXiv:2303.01067. — [doi:10.48550/arxiv.2303.01067](https://doi.org/10.48550/arxiv.2303.01067) [↗](https://doi.org/10.48550/arXiv.2303.01067)
48. Wu, C., Yin, S., Qi, W., Wang, X., Tang, Z., Duan, N., 2023. [Visual chatgpt: Talking, drawing and editing with visual foundation models](https://md.research.elamur.ai/free/en/papers/wu-2023-visual-chatgpt-talking-drawing-editing/). CoRR abs/2303.04671. URL: https://doi.org/10. 48550/arXiv.2303.04671, doi:10.48550/arXiv.2303.04671, arXiv:2303.04671.
49. Xie, T., Wu, C.H., Shi, P., Zhong, R., Scholak, T., Yasunaga, M., Wu, C., Zhong, M., Yin, P., Wang, S.I., Zhong, V., Wang, B., Li, C., Boyle, C., Ni, A., Yao, Z., Radev, D., Xiong, C., Kong, L., Zhang, R., Smith, N.A., Zettlemoyer, L., Yu, T., 2022. [Unifiedskg: Unifying and multi-tasking structured…](https://md.research.elamur.ai/free/en/papers/xie-2022-unifiedskg-unifying-multi-tasking-structured/)
50. Xu, C., Sun, Q., Zheng, K., Geng, X., Zhao, P., Feng, J., Tao, C., Jiang, D., 2023. [Wizardlm: Empowering large language models to follow complex instructions](https://md.research.elamur.ai/free/en/papers/xu-2023-wizardlm-empowering-large-pre-trained/). arXiv:2304.12244.
51. Xu, H., Chen, Y., Du, Y., Shao, N., Wang, Y., Li, H., Yang, Z., 2022. Zeroprompt: Scaling prompt-based pretraining to 1, 000 tasks improves zero-shot generalization, in: Goldberg, Y., Kozareva, Z., Zhang, Y. (Eds.), Findings of the Association for Computational Linguistics: EMNLP 2022, Abu Dhabi, Un… [find](https://scholar.google.com/scholar?q=Xu%2C+H%2E%2C+Chen%2C+Y%2E%2C+Du%2C+Y%2E%2C+Shao%2C+N%2E%2C+Wang%2C+Y%2E%2C+Li%2C+H%2E%2C+Yang%2C+Z%2E%2C+2022%2E+Zeroprompt%3A+Scaling+prompt%2Dbased+pretraining+to+1%2C+000+tasks+improves+zero%2Dshot+generalization%2C+in%3A+Goldberg%2C+Y%2E%2C+Kozareva%2C+Z%2E%2C+Zh)
52. Yang, W., Xie, Y., Lin, A., Li, X., Tan, L., Xiong, K., Li, M., Lin, J., 2019. [End-to-end open-domain question answering with bertserini](https://md.research.elamur.ai/free/en/papers/yang-2019-end-end-open-domain-question/), in: Ammar, W., Louis, A., Mostafazadeh, N. (Eds.), Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguisti…
53. Yang, X., Li, Y., Zhang, X., Chen, H., Cheng, W., 2023. [Exploring the limits of chatgpt for query or aspect-based text summarization](https://md.research.elamur.ai/free/en/papers/yang-2023-exploring-limits-chatgpt-query-aspect/). CoRR abs/2302.08081. URL: https://doi.org/10.48550/ arXiv.2302.08081, doi:10.48550/arXiv.2302.08081, arXiv:2302.08081.
54. Yu, W., Iter, D., Wang, S., Xu, Y., Ju, M., Sanyal, S., Zhu, C., Zeng, M., Jiang, M., 2023. [Generate rather than retrieve: Large language models are strong context generators](https://md.research.elamur.ai/free/en/papers/yu-2022-generate-rather-than-retrieve-large/), in: The Eleventh International Conference on Learning Representations. URL: [https://openreview.net/forum?](https://openreview.net/forum?) id=fB0hRu9GZUS. [↗](https://openreview.net/forum?)
55. Zhang, S., Roller, S., Goyal, N., Artetxe, M., Chen, M., Chen, S., Dewan, C., Diab, M.T., Li, X., Lin, X.V., Mihaylov, T., Ott, M., Shleifer, S., Shuster, K., Simig, D., Koura, P.S., Sridhar, A., Wang, T., Zettlemoyer, L., 2022. [OPT: open pre-trained transformer language models](https://md.research.elamur.ai/free/en/papers/zhang-2022-opt-open-pre-trained-transformer/). CoRR abs/2205.01068.…
56. Zhang, Z., Zhang, A., Li, M., Zhao, H., Karypis, G., Smola, A., 2023. [Multimodal chain-of-thought reasoning in language models](https://md.research.elamur.ai/free/en/papers/zhang-2023-multimodal-chain-thought-reasoning-language/). CoRR abs/2302.00923. URL: https://doi.org/10.48550/ arXiv.2302.00923, doi:10.48550/arXiv.2302.00923, arXiv:2302.00923.

*Updated 30 October 2026*