---
title: "Can Indirect Prompt Injection Attacks Be Detected and Removed?"
description: "Indirect prompt injection happens when an attacker plants malicious instructions in external content… 42 references, cited by 17 papers here."
authors: ["Yulin Chen", "Haoran Li", "Yuan Sui", "Yufei He", "Yue Liu", "Yangqiu Song", "Bryan Hooi"]
year: 2025
venue: "Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)"
source: "https://aclanthology.org/2025.acl-long.890/"
doi: "10.18653/v1/2025.acl-long.890"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/chen-2025-can-indirect-prompt-injection-attacks/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/chen-2025-can-indirect-prompt-injection-attacks/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2025](https://md.research.elamur.ai/free/en/papers/2025/page/3/) / Can Indirect Prompt Injection Attacks Be Detected and Removed?

# Can Indirect Prompt Injection Attacks Be Detected and Removed?

**Authors:** [Yulin Chen](https://md.research.elamur.ai/free/en/authors/yulin-chen/), [Haoran Li](https://md.research.elamur.ai/free/en/authors/haoran-li/), [Yuan Sui](https://md.research.elamur.ai/free/en/authors/yuan-sui/), [Yufei He](https://md.research.elamur.ai/free/en/authors/yufei-he/), [Yue Liu](https://md.research.elamur.ai/free/en/authors/yue-liu/), [Yangqiu Song](https://md.research.elamur.ai/free/en/authors/yangqiu-song/) and [Bryan Hooi](https://md.research.elamur.ai/free/en/authors/bryan-hooi/) · **Year:** 2025 · **Venue:** [Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)](https://md.research.elamur.ai/free/en/venues/acl/) ([ACL 2025](https://md.research.elamur.ai/free/en/venues/acl/2025/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2025.acl-long.890/) · [PDF](https://aclanthology.org/2025.acl-long.890.pdf) · [DOI 10.18653/v1/2025.acl-long.890](https://doi.org/10.18653/v1/2025.acl-long.890) · [OpenAlex](https://openalex.org/W4412944992)

**Citations:** 13 citations (OpenAlex, 2026-10-14) · [17](https://md.research.elamur.ai/free/en/papers/chen-2025-can-indirect-prompt-injection-attacks/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
Indirect prompt injection happens when an attacker plants malicious instructions in external content, such as web pages a tool retrieves, and the LLM then follows them instead of the user's request. Earlier filtering work mostly targeted direct injection and only flagged attacks without cleaning the data. This paper asks whether indirect injections can be both detected and stripped out.

The authors built two benchmarks from SQuAD and TriviaQA validation documents. They inserted GPT-4o-generated instructions aimed at phishing, advertising and propaganda. Each instruction has a probe string, and an attack counts as successful if that string appears in the model's output.

They tested prompted LLMs and off-the-shelf detectors, then trained their own detectors on purpose-built data. They also compared two cleanup strategies: classifying document segments and dropping the flagged ones, or training a generative model to extract the injected text.

Existing detectors performed poorly, while the trained ones worked well on in-domain data but over-flagged clean out-of-domain documents. Pairing detection with removal sharply reduced attack success. This makes lightweight filtering a practical alternative to costly fine-tuning of the LLM itself.

## Key points
- **Benchmarks:** "Inj-SQuAD" and "Inj-TriviaQA" contain 900 samples each. Every sample is a tuple of instruction, clean document, answer, injected instruction and probe.
- **Detection:** Prompted LLMs and open-source detectors (ProtectAI, Meta) struggle with indirect injections, whereas specially trained models such as DeBERTa perform well. Over-defense shows up mainly on out-of-domain text and is less common with stronger models and more fluent documents.
- **Removal methods:** The segmentation method (Segment-DeBERTa) is better overall. The extraction models (Qwen2-1.5B, Llama3.2-3B) are strongest at removing instructions placed at the document's end, which is the most effective attack position.
- **Defense results:** Without protection, attack success rate (ASR) reaches up to 96.44% (Qwen2-7b-Instruct, Fakecom attack, tail position). The filtering pipelines push most ASR values far below the Sandwich and Instructional prompting baselines and generally below the fine-tuned StruQ. Removal rarely strips useful content from clean documents.
- **Code:** Available at github.com/LukeChen-go/indirect-pia-detection.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [National University of Singapore](https://md.research.elamur.ai/free/en/universities/national-university-of-singapore/) — University, Singapore, Singapore · 180 works · 34,304 citations
- [Hong Kong University of Science and Technology](https://md.research.elamur.ai/free/en/universities/hong-kong-university-of-science-and-technology/) — University, Hong Kong, China · 142 works · 25,136 citations

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 16,039 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 15,891 works

## Cite

Cite as: Yulin Chen, Haoran Li, Yuan Sui, Yufei He, Yue Liu, Yangqiu Song and Bryan Hooi. 2025. Can Indirect Prompt Injection Attacks Be Detected and Removed?. Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers).

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{chen2025can,
  title = {{Can Indirect Prompt Injection Attacks Be Detected and Removed?}},
  author = {Yulin Chen, Haoran Li, Yuan Sui, Yufei He, Yue Liu, Yangqiu Song and Bryan Hooi},
  booktitle = {Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2025},
  doi = {10.18653/v1/2025.acl-long.890},
  url = {https://doi.org/10.18653/v1/2025.acl-long.890}
}
```

- [← previous Benchmark Self-Evolving: A Multi-Agent Framework for Dynamic LLM Evaluation](https://md.research.elamur.ai/free/en/papers/wang-2025-benchmark-self-evolving-multi-agent/)
- [next → Can Large Language Models Detect Errors in Long Chain-of-Thought Reasoning?](https://md.research.elamur.ai/free/en/papers/he-2025-can-large-language-models-detect/)
- [↑ back to papers of 2025](https://md.research.elamur.ai/free/en/papers/2025/page/3/)

## References

42 references, 15 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. 2023. Instruction defense. [https://learnprompting.org/docs/prompt_hacking/defensive_](https://learnprompting.org/docs/prompt_hacking/defensive_) measures/instruction. 2023. Sandwich defense. https://learnprompting. org/docs/prompt_hacking/defensive_ measures/sandwich_defense. [↗](https://learnprompting.org/docs/prompt_hacking/defensive_)
2. Mark Breitenbach, Adrian Wood, Win Suen, and Po-Ning Tseng. 2023. [Don’t you (forget nlp): Prompt injection with control characters in chatgpt](https://md.research.elamur.ai/free/en/papers/openai-2024-gpt-4o-system-card/). [https://dropbox.tech/machine-learning/prompt-injection-with-control-characters_](https://dropbox.tech/machine-learning/prompt-injection-with-control-characters_) openai-chatgpt-llm. et al. 2024. Gpt-4o system card. arXiv preprint arXiv:241… [↗](https://dropbox.tech/machine-learning/prompt-injection-with-control-characters_)
3. Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde, Jared Kaplan, Harrison Edwards, Yura Burda, Nicholas Joseph, Greg Brockman, Alex Ray, Raul Puri, Gretchen Krueger, Michael Petrov, Heidy Khlaaf, Girish Sastry, Pamela Mishkin, Brooke Chan, Scott Gray, Nick Ryder, Mikhail Pavlov, Aleth… — [Evaluating Large Language Models Trained on Code](https://md.research.elamur.ai/free/en/papers/chen-2021-evaluating-large-language-models-trained/)
4. Mandar Joshi, Eunsol Choi, Daniel Weld, and Luke Zettlemoyer. 2017. [triviaqa: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehension](https://md.research.elamur.ai/free/en/papers/joshi-2017-triviaqa-large-scale-distantly-supervised/). arXiv e-prints, arXiv:1705.03551.
5. Sizhe Chen, Julien Piet, Chawin Sitawarin, and David Wagner. 2024a. Struq: Defending against prompt injection with structured queries. arXiv preprint arXiv:2402.06363. — [arXiv:2402.06363](https://arxiv.org/abs/2402.06363) [find](https://scholar.google.com/scholar?q=Sizhe+Chen%2C+Julien+Piet%2C+Chawin+Sitawarin%2C+and+David+Wagner%2E+2024a%2E+Struq%3A+Defending+against+prompt+injection+with+structured+queries%2E+arXiv+preprint+arXiv%3A2402%2E06363%2E+%E2%80%94+%5BarXiv%3A2402%2E06363%5D%28https%3A%2F%2Farx)
6. Yulin Chen, Haoran Li, Zihao Zheng, Yangqiu Song, Dekai Wu, and Bryan Hooi. 2024b. [Defense against prompt injection attack by leveraging attack techniques](https://md.research.elamur.ai/free/en/papers/chen-2025-defense-against-prompt-injection-attack/). arXiv preprint arXiv:2411.00459.
7. Kai Greshake, Sahar Abdelnabi, Shailesh Mishra, Christoph Endres, Thorsten Holz, and Mario Fritz. 2023. [Not what you’ve signed up for: Compromising real-world llm-integrated applications with indirect prompt injection](https://md.research.elamur.ai/free/en/papers/greshake-2023-ve-signed-up-compromising-real/). In Proceedings of the 16th ACM Workshop on Artificial Intelligence and Security,…
8. Yufei He, Yuexin Li, Jiaying Wu, Yuan Sui, Yulin Chen, and Bryan Hooi. 2025. Evaluating the paperclip maximizer: Are rl-based language models more likely to pursue instrumental goals? arXiv preprint arXiv:2502.12206. — [arXiv:2502.12206](https://arxiv.org/abs/2502.12206) [find](https://scholar.google.com/scholar?q=Yufei+He%2C+Yuexin+Li%2C+Jiaying+Wu%2C+Yuan+Sui%2C+Yulin+Chen%2C+and+Bryan+Hooi%2E+2025%2E+Evaluating+the+paperclip+maximizer%3A+Are+rl%2Dbased+language+models+more+likely+to+pursue+instrumental+goals%3F+arXiv+preprint+a)
9. Yufei He, Yuan Sui, Xiaoxin He, and Bryan Hooi. 2024. Unigraph: Learning a unified cross-domain foundation model for text-attributed graphs. arXiv preprint arXiv:2402.13630. — [arXiv:2402.13630](https://arxiv.org/abs/2402.13630) [find](https://scholar.google.com/scholar?q=Yufei+He%2C+Yuan+Sui%2C+Xiaoxin+He%2C+and+Bryan+Hooi%2E+2024%2E+Unigraph%3A+Learning+a+unified+cross%2Ddomain+foundation+model+for+text%2Dattributed+graphs%2E+arXiv+preprint+arXiv%3A2402%2E13630%2E+%E2%80%94+%5BarXiv%3A2402%2E13630%5D%28https)
10. Keegan Hines, Gary Lopez, Matthew Hall, Federico Zarfati, Yonatan Zunger, and Emre Kiciman. 2024. Defending against indirect prompt injection attacks with spotlighting. arXiv preprint arXiv:2403.14720. — [arXiv:2403.14720](https://arxiv.org/abs/2403.14720) [find](https://scholar.google.com/scholar?q=Keegan+Hines%2C+Gary+Lopez%2C+Matthew+Hall%2C+Federico+Zarfati%2C+Yonatan+Zunger%2C+and+Emre+Kiciman%2E+2024%2E+Defending+against+indirect+prompt+injection+attacks+with+spotlighting%2E+arXiv+preprint+arXiv%3A2403%2E14720)
11. Yihao Huang, Chong Wang, Xiaojun Jia, Qing Guo, Felix Juefei-Xu, Jian Zhang, Geguang Pu, and Yang Liu. 2024. Semantic-guided prompt organization for universal goal hijacking against llms. arXiv preprint arXiv:2405.14189. — [arXiv:2405.14189](https://arxiv.org/abs/2405.14189) [find](https://scholar.google.com/scholar?q=Yihao+Huang%2C+Chong+Wang%2C+Xiaojun+Jia%2C+Qing+Guo%2C+Felix+Juefei%2DXu%2C+Jian+Zhang%2C+Geguang+Pu%2C+and+Yang+Liu%2E+2024%2E+Semantic%2Dguided+prompt+organization+for+universal+goal+hijacking+against+llms%2E+arXiv+prepri)
12. Aaron Hurst, Adam Lerer, Adam P Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, AJ Ostrow, Akila Welihinda, Alan Hayes, Alec Radford, Takeshi Kojima, Shixiang (Shane) Gu, Machel Reid, Yutaka Matsuo, and Yusuke Iwasawa. 2022. [Large language models are zero-shot reasoners](https://md.research.elamur.ai/free/en/papers/kojima-2022-large-language-models-zero-shot/). In Advances in Neural In…
13. Haoran Li, Yulin Chen, Jinglong Luo, Jiecong Wang, Hao Peng, Yan Kang, Xiaojin Zhang, Qi Hu, Chunkit Chan, Zenglin Xu, et al. 2023a. Privacy in large language models: Attacks, defenses and future directions. arXiv preprint arXiv:2310.10383. — [arXiv:2310.10383](https://arxiv.org/abs/2310.10383) [find](https://scholar.google.com/scholar?q=Haoran+Li%2C+Yulin+Chen%2C+Jinglong+Luo%2C+Jiecong+Wang%2C+Hao+Peng%2C+Yan+Kang%2C+Xiaojin+Zhang%2C+Qi+Hu%2C+Chunkit+Chan%2C+Zenglin+Xu%2C+et+al%2E+2023a%2E+Privacy+in+large+language+models%3A+Attacks%2C+defenses+and+future+dire)
14. Yuexin Li, Chengyu Huang, Shumin Deng, Mei Lin Lock, Tri Cao, Nay Oo, Hoon Wei Lim, and Bryan Hooi. 2024. KnowPhish: Large language models meet multimodal knowledge graphs for enhancing Reference-Based phishing detection. In 33rd USENIX Security Symposium (USENIX Security 24), pages 793–810, Philade… [find](https://scholar.google.com/scholar?q=Yuexin+Li%2C+Chengyu+Huang%2C+Shumin+Deng%2C+Mei+Lin+Lock%2C+Tri+Cao%2C+Nay+Oo%2C+Hoon+Wei+Lim%2C+and+Bryan+Hooi%2E+2024%2E+KnowPhish%3A+Large+language+models+meet+multimodal+knowledge+graphs+for+enhancing+Reference%2DBase)
15. Yunxin Li, Zhenyu Liu, Zitao Li, Xuanyu Zhang, Zhenran Xu, Xinyu Chen, Haoyuan Shi, Shenyuan Jiang, Xintong Wang, Jifang Wang, Shouzheng Huang, Xinping Zhao, Borui Jiang, Lanqing Hong, Longyue Wang, Zhuotao Tian, Baoxing Huai, Wenhan Luo, Weihua Luo, Zheng Zhang, Baotian Hu, and Min Zhang. 2025. Per… — [arXiv:2505.04921](https://arxiv.org/abs/2505.04921) [find](https://scholar.google.com/scholar?q=Yunxin+Li%2C+Zhenyu+Liu%2C+Zitao+Li%2C+Xuanyu+Zhang%2C+Zhenran+Xu%2C+Xinyu+Chen%2C+Haoyuan+Shi%2C+Shenyuan+Jiang%2C+Xintong+Wang%2C+Jifang+Wang%2C+Shouzheng+Huang%2C+Xinping+Zhao%2C+Borui+Jiang%2C+Lanqing+Hong%2C+Longyue+Wang%2C+Z)
16. Zekun Li, Baolin Peng, Pengcheng He, and Xifeng Yan. 2023b. Evaluating the instruction-following robustness of large language models to prompt injection. [find](https://scholar.google.com/scholar?q=Zekun+Li%2C+Baolin+Peng%2C+Pengcheng+He%2C+and+Xifeng+Yan%2E+2023b%2E+Evaluating+the+instruction%2Dfollowing+robustness+of+large+language+models+to+prompt+injection%2E)
17. Xiaogeng Liu, Zhiyuan Yu, Yizhe Zhang, Ning Zhang, and Chaowei Xiao. 2024a. Automatic and universal prompt injection attacks against large language models. arXiv preprint arXiv:2403.04957. — [arXiv:2403.04957](https://arxiv.org/abs/2403.04957) [find](https://scholar.google.com/scholar?q=Xiaogeng+Liu%2C+Zhiyuan+Yu%2C+Yizhe+Zhang%2C+Ning+Zhang%2C+and+Chaowei+Xiao%2E+2024a%2E+Automatic+and+universal+prompt+injection+attacks+against+large+language+models%2E+arXiv+preprint+arXiv%3A2403%2E04957%2E+%E2%80%94+%5BarXiv%3A24)
18. Yi Liu, Gelei Deng, Yuekang Li, Kailong Wang, Zihao Wang, Xiaofeng Wang, Tianwei Zhang, Yepang Liu, Haoyu Wang, Yan Zheng, et al. 2023. [Prompt injection attack against llm-integrated applications](https://md.research.elamur.ai/free/en/papers/liu-2023-prompt-injection-attack-against-llm/). arXiv preprint arXiv:2306.05499.
19. Yue Liu, Jiaying Wu, Yufei He, Hongcheng Gao, Hongyu Chen, Baolong Bi, Jiaheng Zhang, Zhiqi Huang, and Bryan Hooi. 2025. Efficient inference for large reasoning models: A survey. arXiv preprint arXiv:2503.23077. — [arXiv:2503.23077](https://arxiv.org/abs/2503.23077) [find](https://scholar.google.com/scholar?q=Yue+Liu%2C+Jiaying+Wu%2C+Yufei+He%2C+Hongcheng+Gao%2C+Hongyu+Chen%2C+Baolong+Bi%2C+Jiaheng+Zhang%2C+Zhiqi+Huang%2C+and+Bryan+Hooi%2E+2025%2E+Efficient+inference+for+large+reasoning+models%3A+A+survey%2E+arXiv+preprint+arXiv%3A)
20. Yupei Liu, Yuqi Jia, Runpeng Geng, Jinyuan Jia, and Neil Zhenqiang Gong. 2024b. Formalizing and benchmarking prompt injection attacks and defenses. In USENIX Security Symposium. [find](https://scholar.google.com/scholar?q=Yupei+Liu%2C+Yuqi+Jia%2C+Runpeng+Geng%2C+Jinyuan+Jia%2C+and+Neil+Zhenqiang+Gong%2E+2024b%2E+Formalizing+and+benchmarking+prompt+injection+attacks+and+defenses%2E+In+USENIX+Security+Symposium%2E)
21. Aleksander M adry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. 2017. [Towards deep learning models resistant to adversarial attacks](https://md.research.elamur.ai/free/en/papers/madry-2017-towards-deep-learning-models-resistant/). stat, 1050(9).
22. Meta. 2024. Prompt guard-86m \| model cards and prompt formats. [https://www.llama.com/](https://www.llama.com/) docs/model-cards-and-prompt-formats/ prompt-guard/. Accessed: 2024-11-18. [↗](https://www.llama.com/)
23. Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al. 2019. [Pytorch: An imperative style, high-performance deep learning library](https://md.research.elamur.ai/free/en/papers/paszke-2019-pytorch-imperative-style-high-performance/). Advances in neural information processing systems, 32.
24. Fábio Perez and Ian Ribeiro. 2022. [Ignore previous prompt: Attack techniques for language models](https://md.research.elamur.ai/free/en/papers/perez-2022-ignore-previous-prompt-attack-techniques/). arXiv preprint arXiv:2211.09527.
25. Julien Piet, Maha Alrashed, Chawin Sitawarin, Sizhe Chen, Zeming Wei, Elizabeth Sun, Basel Alomair, and David Wagner. 2023. Jatmo: Prompt injection defense by task-specific finetuning. arXiv preprint arXiv:2312.17673. — [arXiv:2312.17673](https://arxiv.org/abs/2312.17673) [find](https://scholar.google.com/scholar?q=Julien+Piet%2C+Maha+Alrashed%2C+Chawin+Sitawarin%2C+Sizhe+Chen%2C+Zeming+Wei%2C+Elizabeth+Sun%2C+Basel+Alomair%2C+and+David+Wagner%2E+2023%2E+Jatmo%3A+Prompt+injection+defense+by+task%2Dspecific+finetuning%2E+arXiv+preprint+)
26. ProtectAI.com. 2024. Fine-tuned deberta-v3-base for prompt injection detection. [find](https://scholar.google.com/scholar?q=ProtectAI%2Ecom%2E+2024%2E+Fine%2Dtuned+deberta%2Dv3%2Dbase+for+prompt+injection+detection%2E)
27. Samyam Rajbhandari, Jeff Rasley, Olatunji Ruwase, and Yuxiong He. 2020. Zero: Memory optimizations toward training trillion parameter models. In SC20: International Conference for High Performance Computing, Networking, Storage and Analysis, pages 1– 16. IEEE. [find](https://scholar.google.com/scholar?q=Samyam+Rajbhandari%2C+Jeff+Rasley%2C+Olatunji+Ruwase%2C+and+Yuxiong+He%2E+2020%2E+Zero%3A+Memory+optimizations+toward+training+trillion+parameter+models%2E+In+SC20%3A+International+Conference+for+High+Performance+Com)
28. Pranav Rajpurkar, Jian Zhang, Konstantin Lopyrev, and Percy Liang. 2016. [SQuAD: 100,000+ questions for machine comprehension of text](https://md.research.elamur.ai/free/en/papers/rajpurkar-2016-squad-100-000-questions-machine/). In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 2383–2392, Austin, Texas. Association for Computational Linguistics.
29. Avital Shafran, Roei Schuster, and Vitaly Shmatikov. 2024. Machine against the rag: Jamming retrieval-augmented generation with blocker documents. arXiv preprint arXiv:2406.05870. — [arXiv:2406.05870](https://arxiv.org/abs/2406.05870) [find](https://scholar.google.com/scholar?q=Avital+Shafran%2C+Roei+Schuster%2C+and+Vitaly+Shmatikov%2E+2024%2E+Machine+against+the+rag%3A+Jamming+retrieval%2Daugmented+generation+with+blocker+documents%2E+arXiv+preprint+arXiv%3A2406%2E05870%2E+%E2%80%94+%5BarXiv%3A2406%2E05870%5D)
30. Jiawen Shi, Zenghui Yuan, Yinuo Liu, Yue Huang, Pan Zhou, Lichao Sun, and Neil Zhenqiang Gong. 2024. Optimization-based prompt injection attack to llm-as-a-judge. arXiv preprint arXiv:2403.17710. — [arXiv:2403.17710](https://arxiv.org/abs/2403.17710) [find](https://scholar.google.com/scholar?q=Jiawen+Shi%2C+Zenghui+Yuan%2C+Yinuo+Liu%2C+Yue+Huang%2C+Pan+Zhou%2C+Lichao+Sun%2C+and+Neil+Zhenqiang+Gong%2E+2024%2E+Optimization%2Dbased+prompt+injection+attack+to+llm%2Das%2Da%2Djudge%2E+arXiv+preprint+arXiv%3A2403%2E17710%2E+%E2%80%94+%5Ba)
31. Manli Shu, Jiongxiao Wang, Chen Zhu, Jonas Geiping, Chaowei Xiao, and Tom Goldstein. 2023. On the exploitability of instruction tuning. Advances in Neural Information Processing Systems, 36:61836–61856. rgorman Stuart Armstrong. 2022. Using gpt-eliezer against chatgpt jailbreaking. LessWrong. [Acces… [find](https://scholar.google.com/scholar?q=Manli+Shu%2C+Jiongxiao+Wang%2C+Chen+Zhu%2C+Jonas+Geiping%2C+Chaowei+Xiao%2C+and+Tom+Goldstein%2E+2023%2E+On+the+exploitability+of+instruction+tuning%2E+Advances+in+Neural+Information+Processing+Systems%2C+36%3A61836%E2%80%936185)
32. Yuan Sui, Yufei He, Tri Cao, Simeng Han, and Bryan Hooi. 2025. [Meta-reasoner: Dynamic guidance for optimized inference-time reasoning in large language models](https://md.research.elamur.ai/free/en/papers/sui-2026-meta-reasoner-dynamic-guidance-optimized/). arXiv preprint arXiv:2502.19918.
33. Yuan Sui, Yufei He, Zifeng Ding, and Bryan Hooi. 2024. [Can knowledge graphs make large language models more trustworthy? an empirical study over open-ended question answering](https://md.research.elamur.ai/free/en/papers/sui-2025-can-knowledge-graphs-make-large/). arXiv preprint arXiv:2410.08085.
34. Xuchen Suo. 2024. Signed-prompt: A new approach to prevent prompt injection attacks against llm-integrated applications. arXiv preprint arXiv:2401.07612. — [arXiv:2401.07612](https://arxiv.org/abs/2401.07612) [find](https://scholar.google.com/scholar?q=Xuchen+Suo%2E+2024%2E+Signed%2Dprompt%3A+A+new+approach+to+prevent+prompt+injection+attacks+against+llm%2Dintegrated+applications%2E+arXiv+preprint+arXiv%3A2401%2E07612%2E+%E2%80%94+%5BarXiv%3A2401%2E07612%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F240)
35. Rohan Taori, Ishaan Gulrajani, Tianyi Zhang, Yann Dubois, Xuechen Li, Carlos Guestrin, Percy Liang, and Tatsunori B. Hashimoto. 2023. Stanford alpaca: An instruction-following llama model. [https://github.com/tatsu-lab/stanford_alpaca](https://github.com/tatsu-lab/stanford_alpaca). [↗](https://github.com/tatsu-lab/stanford_alpaca)
36. Eric Wallace, Kai Xiao, Reimar Leike, Lilian Weng, Johannes Heidecke, and Alex Beutel. 2024. The instruction hierarchy: Training llms to prioritize privileged instructions. arXiv preprint arXiv:2404.13208. [find](https://scholar.google.com/scholar?q=Eric+Wallace%2C+Kai+Xiao%2C+Reimar+Leike%2C+Lilian+Weng%2C+Johannes+Heidecke%2C+and+Alex+Beutel%2E+2024%2E+The+instruction+hierarchy%3A+Training+llms+to+prioritize+privileged+instructions%2E+arXiv+preprint+arXiv%3A2404%2E1)
37. Simon Willison. 2023. Delimiters won’t save you from prompt injection. [https://simonwillison.net/](https://simonwillison.net/) 2023/May/11/delimiters-wont-save-you. [↗](https://simonwillison.net/)
38. Jingwei Yi, Yueqi Xie, Bin Zhu, Keegan Hines, Emre Kiciman, Guangzhong Sun, Xing Xie, and Fangzhao Wu. 2023. Benchmarking and defending against indirect prompt injection attacks on large language models. arXiv preprint arXiv:2312.14197. — [arXiv:2312.14197](https://arxiv.org/abs/2312.14197) [find](https://scholar.google.com/scholar?q=Jingwei+Yi%2C+Yueqi+Xie%2C+Bin+Zhu%2C+Keegan+Hines%2C+Emre+Kiciman%2C+Guangzhong+Sun%2C+Xing+Xie%2C+and+Fangzhao+Wu%2E+2023%2E+Benchmarking+and+defending+against+indirect+prompt+injection+attacks+on+large+language+mode)
39. Qiusi Zhan, Zhixiang Liang, Zifan Ying, and Daniel Kang. 2024. [Injecagent: Benchmarking indirect prompt injections in tool-integrated large language model agents](https://md.research.elamur.ai/free/en/papers/zhan-2024-injecagent-benchmarking-indirect-prompt-injections/). arXiv preprint arXiv:2403.02691.
40. Denny Zhou, Nathanael Schärli, Le Hou, Jason Wei, Nathan Scales, Xuezhi Wang, Dale Schuurmans, Claire Cui, Olivier Bousquet, Quoc V Le, and Ed H. Chi. 2023. [Least-to-most prompting enables complex reasoning in large language models](https://md.research.elamur.ai/free/en/papers/al-2022-least-most-prompting-enables-complex/). In The Eleventh International Conference on Learning Representation…
41. Qing Zong, Zhaowei Wang, Tianshi Zheng, Xiyu Ren, and Yangqiu Song. 2024. Comparisonqa: Evaluating factuality robustness of llms through knowledge frequency control and uncertainty. [find](https://scholar.google.com/scholar?q=Qing+Zong%2C+Zhaowei+Wang%2C+Tianshi+Zheng%2C+Xiyu+Ren%2C+and+Yangqiu+Song%2E+2024%2E+Comparisonqa%3A+Evaluating+factuality+robustness+of+llms+through+knowledge+frequency+control+and+uncertainty%2E)
42. Egor Zverev, Sahar Abdelnabi, Soroush Tabesh, Mario Fritz, and Christoph H Lampert. 2024. Can llms separate instructions from data? and what do we even mean by that? arXiv preprint arXiv:2403.06833. — [arXiv:2403.06833](https://arxiv.org/abs/2403.06833) [find](https://scholar.google.com/scholar?q=Egor+Zverev%2C+Sahar+Abdelnabi%2C+Soroush+Tabesh%2C+Mario+Fritz%2C+and+Christoph+H+Lampert%2E+2024%2E+Can+llms+separate+instructions+from+data%3F+and+what+do+we+even+mean+by+that%3F+arXiv+preprint+arXiv%3A2403%2E06833%2E+%E2%80%94)

*Updated 30 October 2026*