---
title: "PoisonedRAG — paper by Zou et al. (2024) — summary and ideas"
description: "The authors present PoisonedRAG, which they describe as the first knowledge corruption attack on… 107 references, cited by 67 papers here."
authors: ["Wei Zou", "Runpeng Geng", "Binghui Wang", "Jinyuan Jia"]
year: 2024
venue: "arXiv:2402.07867"
source: "https://arxiv.org/abs/2402.07867"
doi: "10.48550/arxiv.2402.07867"
arxiv: "2402.07867"
licence: "arXiv — non-exclusive license to distribute"
licence_url: "http://arxiv.org/licenses/nonexclusive-distrib/1.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/zou-2024-poisonedrag-knowledge-corruption-attacks-retrieval/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/zou-2024-poisonedrag-knowledge-corruption-attacks-retrieval/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2024](https://md.research.elamur.ai/free/en/papers/2024/page/6/) / PoisonedRAG: Knowledge Corruption Attacks to Retrieval-Augmented Generation of Large Language Models

# PoisonedRAG: Knowledge Corruption Attacks to Retrieval-Augmented Generation of Large Language Models

**Authors:** [Wei Zou](https://md.research.elamur.ai/free/en/authors/wei-zou/), [Runpeng Geng](https://md.research.elamur.ai/free/en/authors/runpeng-geng/), [Binghui Wang](https://md.research.elamur.ai/free/en/authors/binghui-wang/) and [Jinyuan Jia](https://md.research.elamur.ai/free/en/authors/jinyuan-jia/) · **Year:** 2024 · **Venue:** arXiv:2402.07867 · **Licence:** [arXiv — non-exclusive license to distribute](http://arxiv.org/licenses/nonexclusive-distrib/1.0/)

**Links:** [Publisher's page](https://arxiv.org/abs/2402.07867) · [PDF](https://arxiv.org/pdf/2402.07867) · [DOI 10.48550/arxiv.2402.07867](https://doi.org/10.48550/arxiv.2402.07867) · [arXiv 2402.07867](https://arxiv.org/abs/2402.07867) · [OpenAlex](https://openalex.org/W4391801218)

**Citations:** 13 citations (OpenAlex, 2026-10-14) · [67](https://md.research.elamur.ai/free/en/papers/zou-2024-poisonedrag-knowledge-corruption-attacks-retrieval/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
Retrieval-Augmented Generation (RAG) lets an LLM answer questions using texts pulled from an external knowledge database. Most prior work on RAG has targeted accuracy and efficiency, not security. This paper treats the knowledge database as an attack surface: an adversary who can add content, for example by editing Wikipedia, posting web pages, or acting as an insider, can steer the model's output. The authors present PoisonedRAG, which they describe as the first knowledge corruption attack on RAG. The attacker picks target questions and arbitrary target answers, then plants a few crafted passages. Each passage combines two parts. One part makes it rank highly for the target question (the retrieval condition). The other part pushes the LLM toward the chosen answer (the generation condition). The attack has variants for white-box access to the retriever and for a fully black-box setting. Across benchmarks, retrievers and LLMs, a handful of poisoned texts hidden among millions of clean ones succeeded most of the time. Tested defenses did not stop it, which shows that RAG deployments need new protections.

## Key points
- Evaluation covered NQ, HotpotQA and MS-MARCO, 8 LLMs (including GPT-4 and LLaMA-2), and real-world settings such as advanced RAG schemes, a Wikipedia-based chatbot and an LLM agent. The metric was Attack Success Rate (ASR).
- On NQ, inserting 5 malicious texts per target question into a database of 2,681,468 clean texts gave 97% ASR in the black-box setting. Five baselines, including Zhong et al.'s adversarial passages, stayed below 70%.
- Paraphrasing, perplexity-based detection and SHA-256 duplicate filtering all failed as defenses. Duplicate filtering left ASR unchanged because each GPT-4-generated sub-text is unique. With knowledge expansion at k = 50, ASR on HotpotQA was still 41% (black-box) and 43% (white-box).
- The poisoned texts had little effect on unrelated questions: 0.3% (black-box) and 0.9% (white-box) of non-target questions retrieved them.
- The attack also worked for fact verification on FEVER, reaching 0.97 (black-box) and 0.88 (white-box) ASR. The code is released on GitHub (sleeepeer/PoisonedRAG).

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Pennsylvania State University](https://md.research.elamur.ai/free/en/universities/pennsylvania-state-university/) — University, State College, United States · 67 works · 5,865 citations
- [Illinois Institute of Technology](https://md.research.elamur.ai/free/en/universities/illinois-institute-of-technology/) — University, Chicago, United States · 18 works · 2,097 citations

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 18,614 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 17,965 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 17,450 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 17,296 works

🧭 Topics (2): [🧠 Language models & pretraining](https://md.research.elamur.ai/free/en/topics/language-models/) · 3,894 works · [🔍 Information retrieval & search](https://md.research.elamur.ai/free/en/topics/information-retrieval/) · 552 works

## Cite

Cite as: Wei Zou, Runpeng Geng, Binghui Wang and Jinyuan Jia. 2024. PoisonedRAG: Knowledge Corruption Attacks to Retrieval-Augmented Generation of Large Language Models. arXiv:2402.07867.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@misc{zou2024poisonedrag,
  title = {{PoisonedRAG: Knowledge Corruption Attacks to Retrieval-Augmented Generation of Large Language Models}},
  author = {Wei Zou, Runpeng Geng, Binghui Wang and Jinyuan Jia},
  year = {2024},
  doi = {10.48550/arxiv.2402.07867},
  eprint = {2402.07867},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2402.07867}
}
```

- [← previous Paths-over-Graph: Knowledge Graph Empowered Large Language Model Reasoning](https://md.research.elamur.ai/free/en/papers/tan-2024-paths-over-graph-knowledge-graph/)
- [next → Toward Self-Improvement of LLMs via Imagination, Searching, and Criticizing](https://md.research.elamur.ai/free/en/papers/tian-2024-toward-self-improvement-llms-imagination/)
- [↑ back to papers of 2024](https://md.research.elamur.ai/free/en/papers/2024/page/6/)

## References

107 references, 46 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. T. Brown, B. Mann, N. Ryder, M. Subbiah, J. D. Kaplan, P. Dhariwal, A. Neelakantan, P. Shyam, G. Sastry, A. Askell et al., “[Language models are few-shot learners](https://md.research.elamur.ai/free/en/papers/brown-2020-language-models-few-shot-learners/),” NeurIPS, 2020.
2. J. Achiam, S. Adler, S. Agarwal, L. Ahmad, I. Akkaya, F. L. Aleman, D. Almeida, J. Altenschmidt, S. Altman, S. Anadkat et al., “[Gpt-4 technical report](https://md.research.elamur.ai/free/en/papers/openai-2023-gpt-4-technical-report/),” arXiv, 2023.
3. R. Anil, A. M. Dai, O. Firat, M. Johnson, D. Lepikhin, A. Passos, S. Shakeri, E. Taropa, P. Bailey, Z. Chen et al., “[Palm 2 technical report](https://md.research.elamur.ai/free/en/papers/anil-2023-palm-2-technical-report/),” arXiv, 2023.
4. Z. Ji, N. Lee, R. Frieske, T. Yu, D. Su, Y. Xu, E. Ishii, Y. J. Bang, A. Madotto, and P. Fung, “[Survey of hallucination in natural language generation](https://md.research.elamur.ai/free/en/papers/ji-2022-survey-hallucination-natural-language-generation/),” ACM Computing Surveys, vol. 55, no. 12, pp. 1–38, 2023.
5. Y. Al Ghadban, H. Y. Lu, U. Adavi, A. Sharma, S. Gara, N. Das, B. Kumar, R. John, P. Devarsetty, and J. E. Hirst, “Transforming healthcare education: Harnessing large language models for frontline health worker capacity building using retrieval-augmented generation,” medRxiv, pp. 2023–12, 2023. [find](https://scholar.google.com/scholar?q=Y%2E+Al+Ghadban%2C+H%2E+Y%2E+Lu%2C+U%2E+Adavi%2C+A%2E+Sharma%2C+S%2E+Gara%2C+N%2E+Das%2C+B%2E+Kumar%2C+R%2E+John%2C+P%2E+Devarsetty%2C+and+J%2E+E%2E+Hirst%2C+%E2%80%9CTransforming+healthcare+education%3A+Harnessing+large+language+models+for+frontline+hea)
6. C. Wang, J. Ong, C. Wang, H. Ong, R. Cheng, and D. Ong, “Potential for gpt technology to optimize future clinical decision-making using retrieval-augmented generation,” Annals of Biomedical Engineering, pp. 1–4, 2023. [find](https://scholar.google.com/scholar?q=C%2E+Wang%2C+J%2E+Ong%2C+C%2E+Wang%2C+H%2E+Ong%2C+R%2E+Cheng%2C+and+D%2E+Ong%2C+%E2%80%9CPotential+for+gpt+technology+to+optimize+future+clinical+decision%2Dmaking+using+retrieval%2Daugmented+generation%2C%E2%80%9D+Annals+of+Biomedical+Engineerin)
7. L. Loukas, I. Stogiannidis, O. Diamantopoulos, P. Malakasiotis, and S. Vassos, “Making llms worth every penny: Resource-limited text classification in banking,” in ICAIF, 2023. [find](https://scholar.google.com/scholar?q=L%2E+Loukas%2C+I%2E+Stogiannidis%2C+O%2E+Diamantopoulos%2C+P%2E+Malakasiotis%2C+and+S%2E+Vassos%2C+%E2%80%9CMaking+llms+worth+every+penny%3A+Resource%2Dlimited+text+classification+in+banking%2C%E2%80%9D+in+ICAIF%2C+2023%2E)
8. A. Kuppa, N. Rasumov-Rahe, and M. Voses, “Chain of reference prompting helps llm to think like a lawyer.” [find](https://scholar.google.com/scholar?q=A%2E+Kuppa%2C+N%2E+Rasumov%2DRahe%2C+and+M%2E+Voses%2C+%E2%80%9CChain+of+reference+prompting+helps+llm+to+think+like+a+lawyer%2E%E2%80%9D)
9. R. Z. Mahari, “Autolaw: Augmented legal reasoning through legal precedent prediction,” arXiv, 2021. [find](https://scholar.google.com/scholar?q=R%2E+Z%2E+Mahari%2C+%E2%80%9CAutolaw%3A+Augmented+legal+reasoning+through+legal+precedent+prediction%2C%E2%80%9D+arXiv%2C+2021%2E)
10. V. Kumar, L. Gleyzer, A. Kahana, K. Shukla, and G. E. Karniadakis, “Mycrunchgpt: A llm assisted framework for scientific machine learning,” Journal of Machine Learning for Modeling and Computing, vol. 4, no. 4, 2023. [find](https://scholar.google.com/scholar?q=V%2E+Kumar%2C+L%2E+Gleyzer%2C+A%2E+Kahana%2C+K%2E+Shukla%2C+and+G%2E+E%2E+Karniadakis%2C+%E2%80%9CMycrunchgpt%3A+A+llm+assisted+framework+for+scientific+machine+learning%2C%E2%80%9D+Journal+of+Machine+Learning+for+Modeling+and+Computing%2C+vol%2E)
11. J. Boyko, J. Cohen, N. Fox, M. H. Veiga, J. I. Li, J. Liu, B. Modenesi, A. H. Rauch, K. N. Reid, S. Tribedi et al., “An interdisciplinary outlook on large language models for scientific research,” arXiv, 2023. [find](https://scholar.google.com/scholar?q=J%2E+Boyko%2C+J%2E+Cohen%2C+N%2E+Fox%2C+M%2E+H%2E+Veiga%2C+J%2E+I%2E+Li%2C+J%2E+Liu%2C+B%2E+Modenesi%2C+A%2E+H%2E+Rauch%2C+K%2E+N%2E+Reid%2C+S%2E+Tribedi+et+al%2E%2C+%E2%80%9CAn+interdisciplinary+outlook+on+large+language+models+for+scientific+research%2C%E2%80%9D+arX)
12. M. H. Prince, H. Chan, A. Vriza, T. Zhou, V. K. Sastry, M. T. Dearing, R. J. Harder, R. K. Vasudevan, and M. J. Cherukara, “Opportunities for retrieval and tool augmented large language models in scientific facilities,” arXiv, 2023. [find](https://scholar.google.com/scholar?q=M%2E+H%2E+Prince%2C+H%2E+Chan%2C+A%2E+Vriza%2C+T%2E+Zhou%2C+V%2E+K%2E+Sastry%2C+M%2E+T%2E+Dearing%2C+R%2E+J%2E+Harder%2C+R%2E+K%2E+Vasudevan%2C+and+M%2E+J%2E+Cherukara%2C+%E2%80%9COpportunities+for+retrieval+and+tool+augmented+large+language+models+in+scie)
13. V. Karpukhin, B. Oguz, S. Min, P. Lewis, L. Wu, S. Edunov, D. Chen, and W.-t. Yih, “[Dense passage retrieval for open-domain question answering](https://md.research.elamur.ai/free/en/papers/karpukhin-2020-dense-passage-retrieval-open-domain/),” in EMNLP, 2020.
14. P. Lewis, E. Perez, A. Piktus, F. Petroni, V. Karpukhin, N. Goyal, H. Küttler, M. Lewis, W.-t. Yih, T. Rocktäschel et al., “[Retrieval-augmented generation for knowledge-intensive nlp tasks](https://md.research.elamur.ai/free/en/papers/lewis-2020-retrieval-augmented-generation-knowledge-intensive/),” NeurIPS, 2020.
15. S. Borgeaud, A. Mensch, J. Hoffmann, T. Cai, E. Rutherford, K. Millican, G. B. Van Den Driessche, J.-B. Lespiau, B. Damoc, A. Clark et al., “Improving language models by retrieving from trillions of tokens,” in ICML, 2022. [find](https://scholar.google.com/scholar?q=S%2E+Borgeaud%2C+A%2E+Mensch%2C+J%2E+Hoffmann%2C+T%2E+Cai%2C+E%2E+Rutherford%2C+K%2E+Millican%2C+G%2E+B%2E+Van+Den+Driessche%2C+J%2E%2DB%2E+Lespiau%2C+B%2E+Damoc%2C+A%2E+Clark+et+al%2E%2C+%E2%80%9CImproving+language+models+by+retrieving+from+trillions+of+t)
16. R. Thoppilan, D. De Freitas, J. Hall, N. Shazeer, A. Kulshreshtha, H.-T. Cheng, A. Jin, T. Bos, L. Baker, Y. Du et al., “[Lamda: Language models for dialog applications](https://md.research.elamur.ai/free/en/papers/thoppilan-2022-lamda-language-models-dialog-applications/),” arXiv, 2022.
17. N. Thakur, N. Reimers, A. Rücklé, A. Srivastava, and I. Gurevych, “Beir: A heterogeneous benchmark for zero-shot evaluation of information retrieval models,” in NeurIPS, 2021. [find](https://scholar.google.com/scholar?q=N%2E+Thakur%2C+N%2E+Reimers%2C+A%2E+R%C3%BCckl%C3%A9%2C+A%2E+Srivastava%2C+and+I%2E+Gurevych%2C+%E2%80%9CBeir%3A+A+heterogeneous+benchmark+for+zero%2Dshot+evaluation+of+information+retrieval+models%2C%E2%80%9D+in+NeurIPS%2C+2021%2E)
18. I. Soboroff, S. Huang, and D. Harman, “Trec 2019 news track overview.” in TREC, 2019. [find](https://scholar.google.com/scholar?q=I%2E+Soboroff%2C+S%2E+Huang%2C+and+D%2E+Harman%2C+%E2%80%9CTrec+2019+news+track+overview%2E%E2%80%9D+in+TREC%2C+2019%2E)
19. E. Voorhees, T. Alam, S. Bedrick, D. Demner-Fushman, W. R. Hersh, K. Lo, K. Roberts, I. Soboroff, and L. L. Wang, “Trec-covid: constructing a pandemic information retrieval test collection,” in ACM SIGIR Forum, vol. 54, no. 1, 2021, pp. 1–12. [find](https://scholar.google.com/scholar?q=E%2E+Voorhees%2C+T%2E+Alam%2C+S%2E+Bedrick%2C+D%2E+Demner%2DFushman%2C+W%2E+R%2E+Hersh%2C+K%2E+Lo%2C+K%2E+Roberts%2C+I%2E+Soboroff%2C+and+L%2E+L%2E+Wang%2C+%E2%80%9CTrec%2Dcovid%3A+constructing+a+pandemic+information+retrieval+test+collection%2C%E2%80%9D+in+ACM+SI)
20. “Chatgpt knowledge retrieval,” [https://platform.openai.com/docs/assistants/tools/knowledge-retrieval](https://platform.openai.com/docs/assistants/tools/knowledge-retrieval), 2023. [↗](https://platform.openai.com/docs/assistants/tools/knowledge-retrieval)
21. J. Liu, “LlamaIndex,” 11 2022. [Online]. Available: [https://github.com/jerryjliu/llama_index](https://github.com/jerryjliu/llama_index) [↗](https://github.com/jerryjliu/llama_index)
22. “Chatrtx,” [https://www.nvidia.com/en-us/ai-on-rtx/chatrtx/](https://www.nvidia.com/en-us/ai-on-rtx/chatrtx/). [↗](https://www.nvidia.com/en-us/ai-on-rtx/chatrtx/)
23. “LangChain,” [https://www.langchain.com/](https://www.langchain.com/). [↗](https://www.langchain.com/)
24. S. Semnani, V. Yao, H. Zhang, and M. Lam, “WikiChat: Stopping the hallucination of large language model chatbots by few-shot grounding on Wikipedia,” in EMNLP, 2023. [find](https://scholar.google.com/scholar?q=S%2E+Semnani%2C+V%2E+Yao%2C+H%2E+Zhang%2C+and+M%2E+Lam%2C+%E2%80%9CWikiChat%3A+Stopping+the+hallucination+of+large+language+model+chatbots+by+few%2Dshot+grounding+on+Wikipedia%2C%E2%80%9D+in+EMNLP%2C+2023%2E)
25. “Bing search,” [https://www.microsoft.com/en-us/bing?form=MG0AUO&OCID=MG0AUO\#faq](https://www.microsoft.com/en-us/bing?form=MG0AUO&OCID=MG0AUO\#faq), 2024. [↗](https://www.microsoft.com/en-us/bing?form=MG0AUO&OCID=MG0AUO\#faq)
26. A. Lozano, S. L. Fleming, C.-C. Chiang, and N. Shah, “Clinfo. ai: An open-source retrieval-augmented large language model system for answering medical questions using scientific literature,” in PACIFIC SYMPO-SIUM ON BIOCOMPUTING 2024, 2023. [find](https://scholar.google.com/scholar?q=A%2E+Lozano%2C+S%2E+L%2E+Fleming%2C+C%2E%2DC%2E+Chiang%2C+and+N%2E+Shah%2C+%E2%80%9CClinfo%2E+ai%3A+An+open%2Dsource+retrieval%2Daugmented+large+language+model+system+for+answering+medical+questions+using+scientific+literature%2C%E2%80%9D+in+PACIFI)
27. “Generative ai in search: Let google do the searching for you,” [https://blog.google/products/search/generative-ai-google-search-may-2024/](https://blog.google/products/search/generative-ai-google-search-may-2024/). [↗](https://blog.google/products/search/generative-ai-google-search-may-2024/)
28. “Perplexity ai,” [https://www.perplexity.ai//](https://www.perplexity.ai//). [↗](https://www.perplexity.ai//)
29. N. Shinn, B. Labash, and A. Gopinath, “Reflexion: an autonomous agent with dynamic memory and self-reflection,” arXiv preprint arXiv:2303.[11366, 2023](https://md.research.elamur.ai/free/en/papers/shinn-2023-reflexion-language-agents-verbal-reinforcement/).
30. S. Yao, J. Zhao, D. Yu, N. Du, I. Shafran, K. Narasimhan, and Y. Cao, “[React: Synergizing reasoning and acting in language models](https://md.research.elamur.ai/free/en/papers/yao-2022-react-synergizing-reasoning-acting-language/),” arXiv, 2022.
31. A. Asai, Z. Wu, Y. Wang, A. Sil, and H. Hajishirzi, “[Self-rag: Learning to retrieve, generate, and critique through self-reflection](https://md.research.elamur.ai/free/en/papers/asai-2023-self-rag-learning-retrieve-generate/),” in ICLR, 2024.
32. G. Izacard, M. Caron, L. Hosseini, S. Riedel, P. Bojanowski, A. Joulin, and E. Grave, “[Unsupervised dense information retrieval with contrastive learning](https://md.research.elamur.ai/free/en/papers/izacard-2021-unsupervised-dense-information-retrieval-contrastive/),” Transactions on Machine Learning Research, 2022.
33. L. Xiong, C. Xiong, Y. Li, K.-F. Tang, J. Liu, P. N. Bennett, J. Ahmed, and A. Overwijk, “[Approximate nearest neighbor negative contrastive learning for dense text retrieval](https://md.research.elamur.ai/free/en/papers/xiong-2020-approximate-nearest-neighbor-negative-contrastive/),” in ICLR, 2021.
34. Z. Peng, X. Wu, and Y. Fang, “Soft prompt tuning for augmenting dense retrieval with large language models,” arXiv, 2023. [find](https://scholar.google.com/scholar?q=Z%2E+Peng%2C+X%2E+Wu%2C+and+Y%2E+Fang%2C+%E2%80%9CSoft+prompt+tuning+for+augmenting+dense+retrieval+with+large+language+models%2C%E2%80%9D+arXiv%2C+2023%2E)
35. N. Kassner and H. Schütze, “Bert-knn: Adding a knn search component to pretrained language models for better qa,” in Findings of ACL: EMNLP, 2020. [find](https://scholar.google.com/scholar?q=N%2E+Kassner+and+H%2E+Sch%C3%BCtze%2C+%E2%80%9CBert%2Dknn%3A+Adding+a+knn+search+component+to+pretrained+language+models+for+better+qa%2C%E2%80%9D+in+Findings+of+ACL%3A+EMNLP%2C+2020%2E)
36. V. Karpukhin, B. Oguz, S. Min, P. Lewis, L. Wu, S. Edunov, D. Chen, and W.-t. Yih, “[Dense passage retrieval for open-domain question answering](https://md.research.elamur.ai/free/en/papers/karpukhin-2020-dense-passage-retrieval-open-domain/),” in EMNLP, 2020.
37. N. Carlini, M. Jagielski, C. A. Choquette-Choo, D. Paleka, W. Pearce, H. Anderson, A. Terzis, K. Thomas, and F. Tramèr, “Poisoning web-scale training datasets is practical,” arXiv, 2023. [find](https://scholar.google.com/scholar?q=N%2E+Carlini%2C+M%2E+Jagielski%2C+C%2E+A%2E+Choquette%2DChoo%2C+D%2E+Paleka%2C+W%2E+Pearce%2C+H%2E+Anderson%2C+A%2E+Terzis%2C+K%2E+Thomas%2C+and+F%2E+Tram%C3%A8r%2C+%E2%80%9CPoisoning+web%2Dscale+training+datasets+is+practical%2C%E2%80%9D+arXiv%2C+2023%2E)
38. T. Kwiatkowski, J. Palomaki, O. Redfield, M. Collins, A. Parikh, C. Alberti, D. Epstein, I. Polosukhin, J. Devlin, K. Lee et al., “[Natural questions: a benchmark for question answering research](https://md.research.elamur.ai/free/en/papers/kwiatkowski-2019-natural-questions-benchmark-question-answering/),” TACL, vol. 7, pp. 452– 466, 2019.
39. Z. Yang, P. Qi, S. Zhang, Y. Bengio, W. Cohen, R. Salakhutdinov, and C. D. Manning, “[Hotpotqa: A dataset for diverse, explainable multi-hop question answering](https://md.research.elamur.ai/free/en/papers/yang-2018-hotpotqa-dataset-diverse-explainable-multi/),” in EMNLP, 2018.
40. T. Nguyen, M. Rosenberg, X. Song, J. Gao, S. Tiwary, R. Majumder, and L. Deng, “[Ms marco: A human generated machine reading comprehension dataset](https://md.research.elamur.ai/free/en/papers/bajaj-2016-ms-marco-human-generated/),” choice, vol. 2640, p. 660, 2016.
41. H. Touvron, L. Martin, K. Stone, P. Albert, A. Almahairi, Y. Babaei, N. Bashlykov, S. Batra, P. Bhargava, S. Bhosale et al., “[Llama 2: Open foundation and fine-tuned chat models](https://md.research.elamur.ai/free/en/papers/touvron-2023-llama-2-open-foundation-fine/),” arXiv, 2023.
42. Y. Liu, Y. Jia, R. Geng, J. Jia, and N. Z. Gong, “[Formalizing and benchmarking prompt injection attacks and defenses](https://md.research.elamur.ai/free/en/papers/liu-2023-formalizing-benchmarking-prompt-injection-attacks/),” arXiv, 2024.
43. Z. Zhong, Z. Huang, A. Wettig, and D. Chen, “Poisoning retrieval corpora by injecting adversarial passages,” in EMNLP, 2023. [find](https://scholar.google.com/scholar?q=Z%2E+Zhong%2C+Z%2E+Huang%2C+A%2E+Wettig%2C+and+D%2E+Chen%2C+%E2%80%9CPoisoning+retrieval+corpora+by+injecting+adversarial+passages%2C%E2%80%9D+in+EMNLP%2C+2023%2E)
44. N. Jain, A. Schwarzschild, Y. Wen, G. Somepalli, J. Kirchenbauer, P.-y. Chiang, M. Goldblum, A. Saha, J. Geiping, and T. Goldstein, “[Baseline defenses for adversarial attacks against aligned language models](https://md.research.elamur.ai/free/en/papers/jain-2023-baseline-defenses-adversarial-attacks-against/),” arXiv, 2023.
45. G. Alon and M. Kamfonas, “[Detecting language model attacks with perplexity](https://md.research.elamur.ai/free/en/papers/alon-2023-detecting-language-model-attacks-perplexity/),” arXiv, 2023.
46. H. Gonen, S. Iyer, T. Blevins, N. A. Smith, and L. Zettlemoyer, “[Demystifying prompts in language models via perplexity estimation](https://md.research.elamur.ai/free/en/papers/gonen-2022-demystifying-prompts-language-models-perplexity/),” arXiv, 2022.
47. F. Perez and I. Ribeiro, “[Ignore previous prompt: Attack techniques for language models](https://md.research.elamur.ai/free/en/papers/perez-2022-ignore-previous-prompt-attack-techniques/),” in NeurIPS ML Safety Workshop, 2022.
48. Y. Liu, G. Deng, Y. Li, K. Wang, T. Zhang, Y. Liu, H. Wang, Y. Zheng, and Y. Liu, “[Prompt injection attack against llm-integrated applications](https://md.research.elamur.ai/free/en/papers/liu-2023-prompt-injection-attack-against-llm/),” arXiv, 2023.
49. K. Greshake, S. Abdelnabi, S. Mishra, C. Endres, T. Holz, and M. Fritz, “Not what you’ve signed up for: Compromising real-world llm-integrated applications with indirect prompt injection,” in AISec, 2023. — [Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injection](https://md.research.elamur.ai/free/en/papers/greshake-2023-ve-signed-up-compromising-real/)
50. R. Pedro, D. Castro, P. Carreira, and N. Santos, “From prompt injections to sql injection attacks: How protected is your llm-integrated web application?” arXiv, 2023. [find](https://scholar.google.com/scholar?q=R%2E+Pedro%2C+D%2E+Castro%2C+P%2E+Carreira%2C+and+N%2E+Santos%2C+%E2%80%9CFrom+prompt+injections+to+sql+injection+attacks%3A+How+protected+is+your+llm%2Dintegrated+web+application%3F%E2%80%9D+arXiv%2C+2023%2E)
51. H. J. Branch, J. R. Cefalu, J. McHugh, L. Hujer, A. Bahl, D. d. C. Iglesias, R. Heichman, and R. Darwishi, “Evaluating the susceptibility of pre-trained language models via handcrafted adversarial examples,” arXiv, 2022. [find](https://scholar.google.com/scholar?q=H%2E+J%2E+Branch%2C+J%2E+R%2E+Cefalu%2C+J%2E+McHugh%2C+L%2E+Hujer%2C+A%2E+Bahl%2C+D%2E+d%2E+C%2E+Iglesias%2C+R%2E+Heichman%2C+and+R%2E+Darwishi%2C+%E2%80%9CEvaluating+the+susceptibility+of+pre%2Dtrained+language+models+via+handcrafted+adversarial+exa)
52. A. Wei, N. Haghtalab, and J. Steinhardt, “[Jailbroken: How does llm safety training fail?](https://md.research.elamur.ai/free/en/papers/wei-2023-jailbroken-llm-safety-training-fail/)” in NeurIPS, 2023.
53. A. Zou, Z. Wang, J. Z. Kolter, and M. Fredrikson, “[Universal and transferable adversarial attacks on aligned language models](https://md.research.elamur.ai/free/en/papers/zou-2023-universal-transferable-adversarial-attacks-aligned/),” arXiv, 2023.
54. G. Deng, Y. Liu, Y. Li, K. Wang, Y. Zhang, Z. Li, H. Wang, T. Zhang, and Y. Liu, “Masterkey: Automated jailbreaking of large language model chatbots,” in NDSS, 2024. [find](https://scholar.google.com/scholar?q=G%2E+Deng%2C+Y%2E+Liu%2C+Y%2E+Li%2C+K%2E+Wang%2C+Y%2E+Zhang%2C+Z%2E+Li%2C+H%2E+Wang%2C+T%2E+Zhang%2C+and+Y%2E+Liu%2C+%E2%80%9CMasterkey%3A+Automated+jailbreaking+of+large+language+model+chatbots%2C%E2%80%9D+in+NDSS%2C+2024%2E)
55. X. Qi, K. Huang, A. Panda, P. Henderson, M. Wang, and P. Mittal, “[Visual adversarial examples jailbreak aligned large language models](https://md.research.elamur.ai/free/en/papers/qi-2023-visual-adversarial-examples-jailbreak-aligned/),” arXiv, 2023.
56. H. Li, D. Guo, W. Fan, M. Xu, J. Huang, F. Meng, and Y. Song, “[Multi-step jailbreaking privacy attacks on chatgpt](https://md.research.elamur.ai/free/en/papers/li-2023-multi-step-jailbreaking-privacy-attacks/),” arXiv, 2023.
57. X. Shen, Z. Chen, M. Backes, Y. Shen, and Y. Zhang, “["do anything now": Characterizing and evaluating in-the-wild jailbreak prompts on large language models](https://md.research.elamur.ai/free/en/papers/shen-2023-anything-now-characterizing-evaluating-wild/),” arXiv, 2023.
58. N. Carlini, F. Tramer, E. Wallace, M. Jagielski, A. Herbert-Voss, K. Lee, A. Roberts, T. Brown, D. Song, U. Erlingsson et al., “[Extracting training data from large language models](https://md.research.elamur.ai/free/en/papers/carlini-2020-extracting-training-data-large-language/),” in Usenix Security, 2021.
59. N. Kandpal, M. Jagielski, F. Tramèr, and N. Carlini, “Backdoor attacks for in-context learning with language models,” in ICML Workshop, 2023. [find](https://scholar.google.com/scholar?q=N%2E+Kandpal%2C+M%2E+Jagielski%2C+F%2E+Tram%C3%A8r%2C+and+N%2E+Carlini%2C+%E2%80%9CBackdoor+attacks+for+in%2Dcontext+learning+with+language+models%2C%E2%80%9D+in+ICML+Workshop%2C+2023%2E)
60. A. Wan, E. Wallace, S. Shen, and D. Klein, “Poisoning language models during instruction tuning,” in ICML, 2023. [find](https://scholar.google.com/scholar?q=A%2E+Wan%2C+E%2E+Wallace%2C+S%2E+Shen%2C+and+D%2E+Klein%2C+%E2%80%9CPoisoning+language+models+during+instruction+tuning%2C%E2%80%9D+in+ICML%2C+2023%2E)
61. N. Carlini, D. Ippolito, M. Jagielski, K. Lee, F. Tramer, and C. Zhang, “Quantifying memorization across neural language models,” in ICLR, 2022. [find](https://scholar.google.com/scholar?q=N%2E+Carlini%2C+D%2E+Ippolito%2C+M%2E+Jagielski%2C+K%2E+Lee%2C+F%2E+Tramer%2C+and+C%2E+Zhang%2C+%E2%80%9CQuantifying+memorization+across+neural+language+models%2C%E2%80%9D+in+ICLR%2C+2022%2E)
62. N. Carlini, F. Tramèr, E. Wallace, M. Jagielski, A. Herbert-Voss, K. Lee, A. Roberts, T. Brown, D. Song, Ú. Erlingsson, A. Oprea, and C. Raffel, “[Extracting training data from large language models](https://md.research.elamur.ai/free/en/papers/carlini-2020-extracting-training-data-large-language/),” in Usenix Security, 2021.
63. J. Mattern, F. Mireshghallah, Z. Jin, B. Schoelkopf, M. Sachan, and T. Berg-Kirkpatrick, “[Membership inference attacks against language models via neighbourhood comparison](https://md.research.elamur.ai/free/en/papers/mattern-2023-membership-inference-attacks-against-language/),” in Findings of ACL: EMNLP, 2023.
64. X. Pan, M. Zhang, S. Ji, and M. Yang, “Privacy risks of general-purpose language models,” in IEEE S & P, 2020. [find](https://scholar.google.com/scholar?q=X%2E+Pan%2C+M%2E+Zhang%2C+S%2E+Ji%2C+and+M%2E+Yang%2C+%E2%80%9CPrivacy+risks+of+general%2Dpurpose+language+models%2C%E2%80%9D+in+IEEE+S+%26+P%2C+2020%2E)
65. “Exploring prompt injection attacks,” [https://research.nccgroup.com/2022/12/05/exploring-prompt-injection-attacks/](https://research.nccgroup.com/2022/12/05/exploring-prompt-injection-attacks/). [↗](https://research.nccgroup.com/2022/12/05/exploring-prompt-injection-attacks/)
66. B. Biggio, B. Nelson, and P. Laskov, “Poisoning attacks against support vector machines,” in ICML, 2012. [find](https://scholar.google.com/scholar?q=B%2E+Biggio%2C+B%2E+Nelson%2C+and+P%2E+Laskov%2C+%E2%80%9CPoisoning+attacks+against+support+vector+machines%2C%E2%80%9D+in+ICML%2C+2012%2E)
67. T. Gu, B. Dolan-Gavitt, and S. Garg, “[Badnets: Identifying vulnerabilities in the machine learning model supply chain](https://md.research.elamur.ai/free/en/papers/gu-2017-badnets-identifying-vulnerabilities-machine-learning/),” IEEE Access, 2017.
68. Y. Liu, S. Ma, Y. Aafer, W.-C. Lee, J. Zhai, W. Wang, and X. Zhang, “Trojaning attack on neural networks,” in NDSS, 2018. [find](https://scholar.google.com/scholar?q=Y%2E+Liu%2C+S%2E+Ma%2C+Y%2E+Aafer%2C+W%2E%2DC%2E+Lee%2C+J%2E+Zhai%2C+W%2E+Wang%2C+and+X%2E+Zhang%2C+%E2%80%9CTrojaning+attack+on+neural+networks%2C%E2%80%9D+in+NDSS%2C+2018%2E)
69. X. Chen, C. Liu, B. Li, K. Lu, and D. Song, “Targeted backdoor attacks on deep learning systems using data poisoning,” arXiv, 2017. [find](https://scholar.google.com/scholar?q=X%2E+Chen%2C+C%2E+Liu%2C+B%2E+Li%2C+K%2E+Lu%2C+and+D%2E+Song%2C+%E2%80%9CTargeted+backdoor+attacks+on+deep+learning+systems+using+data+poisoning%2C%E2%80%9D+arXiv%2C+2017%2E)
70. A. Shafahi, W. R. Huang, M. Najibi, O. Suciu, C. Studer, T. Dumitras, and T. Goldstein, “Poison frogs! targeted clean-label poisoning attacks on neural networks,” in NeurIPS, 2018. [find](https://scholar.google.com/scholar?q=A%2E+Shafahi%2C+W%2E+R%2E+Huang%2C+M%2E+Najibi%2C+O%2E+Suciu%2C+C%2E+Studer%2C+T%2E+Dumitras%2C+and+T%2E+Goldstein%2C+%E2%80%9CPoison+frogs%21+targeted+clean%2Dlabel+poisoning+attacks+on+neural+networks%2C%E2%80%9D+in+NeurIPS%2C+2018%2E)
71. E. Bagdasaryan, A. Veit, Y. Hua, D. Estrin, and V. Shmatikov, “How to backdoor federated learning,” in AISTATS, 2020. [find](https://scholar.google.com/scholar?q=E%2E+Bagdasaryan%2C+A%2E+Veit%2C+Y%2E+Hua%2C+D%2E+Estrin%2C+and+V%2E+Shmatikov%2C+%E2%80%9CHow+to+backdoor+federated+learning%2C%E2%80%9D+in+AISTATS%2C+2020%2E)
72. M. Fang, X. Cao, J. Jia, and N. Gong, “Local model poisoning attacks to byzantine-robust federated learning,” in USENIX Security Symposium, 2020. [find](https://scholar.google.com/scholar?q=M%2E+Fang%2C+X%2E+Cao%2C+J%2E+Jia%2C+and+N%2E+Gong%2C+%E2%80%9CLocal+model+poisoning+attacks+to+byzantine%2Drobust+federated+learning%2C%E2%80%9D+in+USENIX+Security+Symposium%2C+2020%2E)
73. Z. Zhang, J. Jia, B. Wang, and N. Z. Gong, “Backdoor attacks to graph neural networks,” in SACMAT, 2021. [find](https://scholar.google.com/scholar?q=Z%2E+Zhang%2C+J%2E+Jia%2C+B%2E+Wang%2C+and+N%2E+Z%2E+Gong%2C+%E2%80%9CBackdoor+attacks+to+graph+neural+networks%2C%E2%80%9D+in+SACMAT%2C+2021%2E)
74. J. Jia, Y. Liu, and N. Z. Gong, “Badencoder: Backdoor attacks to pre-trained encoders in self-supervised learning,” in IEEE S & P, 2022. [find](https://scholar.google.com/scholar?q=J%2E+Jia%2C+Y%2E+Liu%2C+and+N%2E+Z%2E+Gong%2C+%E2%80%9CBadencoder%3A+Backdoor+attacks+to+pre%2Dtrained+encoders+in+self%2Dsupervised+learning%2C%E2%80%9D+in+IEEE+S+%26+P%2C+2022%2E)
75. X. Li and J. Li, “[Angle-optimized text embeddings](https://md.research.elamur.ai/free/en/papers/li-2023-angle-optimized-text-embeddings/),” arXiv preprint arXiv:2309.12871, 2023.
76. “Whereisai/uae-large-v1,” [https://huggingface.co/WhereIsAI/UAE-Large-V1](https://huggingface.co/WhereIsAI/UAE-Large-V1). [↗](https://huggingface.co/WhereIsAI/UAE-Large-V1)
77. A. Kerckhoffs, “La cryptographie militaire,” Journal des sciences militaires, 1883. [find](https://scholar.google.com/scholar?q=A%2E+Kerckhoffs%2C+%E2%80%9CLa+cryptographie+militaire%2C%E2%80%9D+Journal+des+sciences+militaires%2C+1883%2E)
78. J. Ebrahimi, A. Rao, D. Lowd, and D. Dou, “[Hotflip: White-box adversarial examples for text classification](https://md.research.elamur.ai/free/en/papers/ebrahimi-2018-hotflip-white-box-adversarial-examples/),” in ACL, 2018.
79. J. Morris, E. Lifland, J. Y. Yoo, J. Grigsby, D. Jin, and Y. Qi, “[Textattack: A framework for adversarial attacks, data augmentation, and adversarial training in nlp](https://md.research.elamur.ai/free/en/papers/morris-2020-textattack-framework-adversarial-attacks-data/),” in EMNLP, 2020.
80. D. Jin, Z. Jin, J. T. Zhou, and P. Szolovits, “[Is bert really robust? a strong baseline for natural language attack on text classification and entailment](https://md.research.elamur.ai/free/en/papers/jin-2019-bert-really-robust-strong-baseline/),” in AAAI, 2020.
81. J. Li, S. Ji, T. Du, B. Li, and T. Wang, “[Textbugger: Generating adversarial text against real-world applications](https://md.research.elamur.ai/free/en/papers/li-2018-textbugger-generating-adversarial-text-against/),” in NDSS, 2019.
82. L. Li, R. Ma, Q. Guo, X. Xue, and X. Qiu, “[Bert-attack: Adversarial attack against bert using bert](https://md.research.elamur.ai/free/en/papers/li-2020-bert-attack-adversarial-attack-against/),” in EMNLP, 2020.
83. J. Gao, J. Lanchantin, M. L. Soffa, and Y. Qi, “[Black-box generation of adversarial text sequences to evade deep learning classifiers](https://md.research.elamur.ai/free/en/papers/gao-2018-black-box-generation-adversarial-text/),” in SPW, 2018.
84. “Ms-marco microsoft bing,” [https://microsoft.github.io/msmarco/](https://microsoft.github.io/msmarco/). [↗](https://microsoft.github.io/msmarco/)
85. W.-L. Chiang, Z. Li, Z. Lin, Y. Sheng, Z. Wu, H. Zhang, L. Zheng, S. Zhuang, Y. Zhuang, J. E. Gonzalez, I. Stoica, and E. P. Xing, “Vicuna: An open-source chatbot impressing gpt-4 with 90%* chatgpt quality,” 2023. [find](https://scholar.google.com/scholar?q=W%2E%2DL%2E+Chiang%2C+Z%2E+Li%2C+Z%2E+Lin%2C+Y%2E+Sheng%2C+Z%2E+Wu%2C+H%2E+Zhang%2C+L%2E+Zheng%2C+S%2E+Zhuang%2C+Y%2E+Zhuang%2C+J%2E+E%2E+Gonzalez%2C+I%2E+Stoica%2C+and+E%2E+P%2E+Xing%2C+%E2%80%9CVicuna%3A+An+open%2Dsource+chatbot+impressing+gpt%2D4+with+90%25%2A+chatgpt+qu)
86. N. Carlini and A. Terzis, “Poisoning and backdooring contrastive learning,” in ICLR, 2022. [find](https://scholar.google.com/scholar?q=N%2E+Carlini+and+A%2E+Terzis%2C+%E2%80%9CPoisoning+and+backdooring+contrastive+learning%2C%E2%80%9D+in+ICLR%2C+2022%2E)
87. N. Carlini, “Poisoning the unlabeled dataset of semi-supervised learning,” in USENIX Security, 2021. [find](https://scholar.google.com/scholar?q=N%2E+Carlini%2C+%E2%80%9CPoisoning+the+unlabeled+dataset+of+semi%2Dsupervised+learning%2C%E2%80%9D+in+USENIX+Security%2C+2021%2E)
88. H. Liu, J. Jia, and N. Z. Gong, “Poisonedencoder: Poisoning the unlabeled pre-training data in contrastive learning,” in USENIX Security Symposium, 2022. [find](https://scholar.google.com/scholar?q=H%2E+Liu%2C+J%2E+Jia%2C+and+N%2E+Z%2E+Gong%2C+%E2%80%9CPoisonedencoder%3A+Poisoning+the+unlabeled+pre%2Dtraining+data+in+contrastive+learning%2C%E2%80%9D+in+USENIX+Security+Symposium%2C+2022%2E)
89. M. R. Rizqullah, A. Purwarianti, and A. F. Aji, “Qasina: Religious domain question answering using sirah nabawiyah,” in ICAICTA, 2023. [find](https://scholar.google.com/scholar?q=M%2E+R%2E+Rizqullah%2C+A%2E+Purwarianti%2C+and+A%2E+F%2E+Aji%2C+%E2%80%9CQasina%3A+Religious+domain+question+answering+using+sirah+nabawiyah%2C%E2%80%9D+in+ICAICTA%2C+2023%2E)
90. Y. Huang, S. Gupta, M. Xia, K. Li, and D. Chen, “[Catastrophic jailbreak of open-source llms via exploiting generation](https://md.research.elamur.ai/free/en/papers/huang-2023-catastrophic-jailbreak-open-source-llms/),” arXiv, 2023.
91. Y. Du, A. Bosselut, and C. D. Manning, “Synthetic disinformation attacks on automated fact verification systems,” in AAAI, 2022. [find](https://scholar.google.com/scholar?q=Y%2E+Du%2C+A%2E+Bosselut%2C+and+C%2E+D%2E+Manning%2C+%E2%80%9CSynthetic+disinformation+attacks+on+automated+fact+verification+systems%2C%E2%80%9D+in+AAAI%2C+2022%2E)
92. Y. Pan, L. Pan, W. Chen, P. Nakov, M.-Y. Kan, and W. Y. Wang, “[On the risk of misinformation pollution with large language models](https://md.research.elamur.ai/free/en/papers/pan-2023-risk-misinformation-pollution-large-language/),” in EMNLP, 2023.
93. S.-Q. Yan, J.-C. Gu, Y. Zhu, and Z.-H. Ling, “[Corrective retrieval augmented generation](https://md.research.elamur.ai/free/en/papers/yan-2024-corrective-retrieval-augmented-generation/),” arXiv, 2024.
94. O. Yoran, T. Wolfson, O. Ram, and J. Berant, “[Making retrieval-augmented language models robust to irrelevant context](https://md.research.elamur.ai/free/en/papers/yoran-2023-making-retrieval-augmented-language-models/),” in ICLR, 2023.
95. H. Luo, T. Zhang, Y.-S. Chuang, Y. Gong, Y. Kim, X. Wu, H. Meng, and J. Glass, “Search augmented instruction learning,” in EMNLP, 2023, pp. 3717–3729. [find](https://scholar.google.com/scholar?q=H%2E+Luo%2C+T%2E+Zhang%2C+Y%2E%2DS%2E+Chuang%2C+Y%2E+Gong%2C+Y%2E+Kim%2C+X%2E+Wu%2C+H%2E+Meng%2C+and+J%2E+Glass%2C+%E2%80%9CSearch+augmented+instruction+learning%2C%E2%80%9D+in+EMNLP%2C+2023%2C+pp%2E+3717%E2%80%933729%2E)
96. T. Zhang, S. G. Patil, N. Jain, S. Shen, M. Zaharia, I. Stoica, and J. E. Gonzalez, “[Raft: Adapting language model to domain specific rag](https://md.research.elamur.ai/free/en/papers/zhang-2024-raft-adapting-language-model-domain/),” arXiv, 2024.
97. J. Steinhardt, P. W. W. Koh, and P. S. Liang, “Certified defenses for data poisoning attacks,” NeurIPS, 2017. [find](https://scholar.google.com/scholar?q=J%2E+Steinhardt%2C+P%2E+W%2E+W%2E+Koh%2C+and+P%2E+S%2E+Liang%2C+%E2%80%9CCertified+defenses+for+data+poisoning+attacks%2C%E2%80%9D+NeurIPS%2C+2017%2E)
98. B. Wang, Y. Yao, S. Shan, H. Li, B. Viswanath, H. Zheng, and B. Y. Zhao, “Neural cleanse: Identifying and mitigating backdoor attacks in neural networks,” in IEEE S& P, 2019. [find](https://scholar.google.com/scholar?q=B%2E+Wang%2C+Y%2E+Yao%2C+S%2E+Shan%2C+H%2E+Li%2C+B%2E+Viswanath%2C+H%2E+Zheng%2C+and+B%2E+Y%2E+Zhao%2C+%E2%80%9CNeural+cleanse%3A+Identifying+and+mitigating+backdoor+attacks+in+neural+networks%2C%E2%80%9D+in+IEEE+S%26+P%2C+2019%2E)
99. J. Jia, X. Cao, and N. Z. Gong, “Intrinsic certified robustness of bagging against data poisoning attacks,” in AAAI, 2021. [find](https://scholar.google.com/scholar?q=J%2E+Jia%2C+X%2E+Cao%2C+and+N%2E+Z%2E+Gong%2C+%E2%80%9CIntrinsic+certified+robustness+of+bagging+against+data+poisoning+attacks%2C%E2%80%9D+in+AAAI%2C+2021%2E)
100. J. Jia, Y. Liu, X. Cao, and N. Z. Gong, “Certified robustness of nearest neighbors against data poisoning and backdoor attacks,” in AAAI, 2022. [find](https://scholar.google.com/scholar?q=J%2E+Jia%2C+Y%2E+Liu%2C+X%2E+Cao%2C+and+N%2E+Z%2E+Gong%2C+%E2%80%9CCertified+robustness+of+nearest+neighbors+against+data+poisoning+and+backdoor+attacks%2C%E2%80%9D+in+AAAI%2C+2022%2E)
101. J. Jia, Y. Liu, Y. Hu, and N. Z. Gong, “Pore: Provably robust recommender systems against data poisoning attacks,” in USENIX Security Symposium, 2023. [find](https://scholar.google.com/scholar?q=J%2E+Jia%2C+Y%2E+Liu%2C+Y%2E+Hu%2C+and+N%2E+Z%2E+Gong%2C+%E2%80%9CPore%3A+Provably+robust+recommender+systems+against+data+poisoning+attacks%2C%E2%80%9D+in+USENIX+Security+Symposium%2C+2023%2E)
102. Y. Wang, W. Zou, and J. Jia, “Fcert: Certifiably robust few-shot classification in the era of foundation models,” in IEEE S & P, 2024. [find](https://scholar.google.com/scholar?q=Y%2E+Wang%2C+W%2E+Zou%2C+and+J%2E+Jia%2C+%E2%80%9CFcert%3A+Certifiably+robust+few%2Dshot+classification+in+the+era+of+foundation+models%2C%E2%80%9D+in+IEEE+S+%26+P%2C+2024%2E)
103. S. Shan, A. N. Bhagoji, H. Zheng, and B. Y. Zhao, “Poison forensics: Traceback of data poisoning attacks in neural networks,” in USENIX Security Symposium, 2022. [find](https://scholar.google.com/scholar?q=S%2E+Shan%2C+A%2E+N%2E+Bhagoji%2C+H%2E+Zheng%2C+and+B%2E+Y%2E+Zhao%2C+%E2%80%9CPoison+forensics%3A+Traceback+of+data+poisoning+attacks+in+neural+networks%2C%E2%80%9D+in+USENIX+Security+Symposium%2C+2022%2E)
104. F. Jelinek, “Interpolated estimation of markov source parameters from sparse data,” in Proc. Workshop on Pattern Recognition in Practice, 1980. [find](https://scholar.google.com/scholar?q=F%2E+Jelinek%2C+%E2%80%9CInterpolated+estimation+of+markov+source+parameters+from+sparse+data%2C%E2%80%9D+in+Proc%2E+Workshop+on+Pattern+Recognition+in+Practice%2C+1980%2E)
105. “Tiktoken,” [https://github.com/openai/tiktoken](https://github.com/openai/tiktoken). [↗](https://github.com/openai/tiktoken)
106. J. Thorne, A. Vlachos, C. Christodoulopoulos, and A. Mittal, “[Fever: a large-scale dataset for fact extraction and verification](https://md.research.elamur.ai/free/en/papers/thorne-2018-fever-large-scale-dataset-fact/),” arXiv, 2018.
107. E. Kortukov, A. Rubinstein, E. Nguyen, and S. J. Oh, “Studying large language model behaviors under realistic knowledge conflicts,” arXiv, 2024. Algorithm 2: PoisonedRAG (white-box) Input: M target questions Q 1 , Q 2 , · · · , Q M , target answers R 1 , R 2 , · · · , R M , hyperparameters N, L, V ,… [find](https://scholar.google.com/scholar?q=E%2E+Kortukov%2C+A%2E+Rubinstein%2C+E%2E+Nguyen%2C+and+S%2E+J%2E+Oh%2C+%E2%80%9CStudying+large+language+model+behaviors+under+realistic+knowledge+conflicts%2C%E2%80%9D+arXiv%2C+2024%2E+Algorithm+2%3A+PoisonedRAG+%28white%2Dbox%29+Input%3A+M+target+qu)

*Updated 2 November 2026*