---
title: "Agentless: Demystifying LLM-based Software Engineering Agents"
description: "Autonomous LLM agents for resolving GitHub issues have become elaborate. 110 references, cited by 96 papers here."
authors: ["Chunqiu Steven Xia", "Yinlin Deng", "Soren Dunn", "Lingming Zhang"]
year: 2024
venue: "arXiv:2407.01489"
source: "https://arxiv.org/abs/2407.01489"
doi: "10.48550/arxiv.2407.01489"
arxiv: "2407.01489"
licence: "CC BY 4.0 (arXiv)"
licence_url: "http://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/xia-2024-agentless-demystifying-llm-based/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/xia-2024-agentless-demystifying-llm-based/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2024](https://md.research.elamur.ai/free/en/papers/2024/page/3/) / Agentless: Demystifying LLM-based Software Engineering Agents

# Agentless: Demystifying LLM-based Software Engineering Agents

**Authors:** [Chunqiu Steven Xia](https://md.research.elamur.ai/free/en/authors/chunqiu-steven-xia/), Yinlin Deng, Soren Dunn and [Lingming Zhang](https://md.research.elamur.ai/free/en/authors/lingming-zhang/) · **Year:** 2024 · **Venue:** arXiv:2407.01489 · **Licence:** [CC BY 4.0 (arXiv)](http://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://arxiv.org/abs/2407.01489) · [PDF](https://arxiv.org/pdf/2407.01489) · [DOI 10.48550/arxiv.2407.01489](https://doi.org/10.48550/arxiv.2407.01489) · [arXiv 2407.01489](https://arxiv.org/abs/2407.01489) · [OpenAlex](https://openalex.org/W4400337928)

**Citations:** 21 citations (OpenAlex, 2026-10-14) · [96](https://md.research.elamur.ai/free/en/papers/xia-2024-agentless-demystifying-llm-based/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
Autonomous LLM agents for resolving GitHub issues have become elaborate. They plan their own steps, call custom tools and iterate over dozens of turns, which makes them costly, hard to debug and prone to compounding mistakes. The authors ask whether this machinery is actually needed and answer with Agentless, a fixed pipeline in which the LLM never chooses its next action. It narrows down the fault in stages (first files, then classes/functions, then exact edit spots) by combining prompting with embedding-based retrieval. It then samples several candidate patches in a lightweight diff format, writes tests that reproduce the reported bug, and chooses the final patch using those tests together with the project's existing regression tests. On SWE-bench Lite, Agentless with GPT-4o beat every open-source agent while costing well under a dollar per issue. A manual audit of the benchmark also exposed flawed tasks, which led to the cleaner SWE-bench Lite-S subset. The work matters because it gives complex agents a strong, cheap and transparent baseline that they must outperform to justify their complexity.

## Key points
- On SWE-bench Lite (300 bug-fixing issues), Agentless resolved 96 issues (32.00%) at an average cost of about $0.70, the best result among open-source approaches at the time.
- The manual audit found three kinds of flawed tasks: 4.3% include the exact fix in the issue text, 10.0% lack information needed to solve them, and 5.0% suggest misleading solutions. Removing these produced SWE-bench Lite-S.
- On OpenAI's human-validated SWE-bench Verified, Agentless solved 194 of 500 problems (38.80%). This was second among open-source tools and the top result of any GPT-4o-based system.
- Agent-based tools still did better on issues that give no hint about where the bug lives, which the authors identify as a direction for future work.
- OpenAI adopted Agentless to showcase the SWE-bench performance of GPT-4o and the o1 models. The code is released at github.com/OpenAutoCoder/Agentless.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [University of Illinois Urbana-Champaign](https://md.research.elamur.ai/free/en/universities/university-of-illinois-urbana-champaign/) — University, Urbana, United States · 209 works · 71,394 citations

📚 Shelves (3): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 18,614 works · [🗄️ Information Systems](https://md.research.elamur.ai/free/en/shelves/computer-science/information-systems/) · 1,730 works · [💻 Software](https://md.research.elamur.ai/free/en/shelves/computer-science/software/) · 119 works

🧭 Topics (1): [💻 Code & programming languages](https://md.research.elamur.ai/free/en/topics/code-and-programming/) · 168 works

## Cite

Cite as: Chunqiu Steven Xia, Yinlin Deng, Soren Dunn and Lingming Zhang. 2024. Agentless: Demystifying LLM-based Software Engineering Agents. arXiv:2407.01489.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@misc{xia2024agentless,
  title = {{Agentless: Demystifying LLM-based Software Engineering Agents}},
  author = {Chunqiu Steven Xia, Yinlin Deng, Soren Dunn and Lingming Zhang},
  year = {2024},
  doi = {10.48550/arxiv.2407.01489},
  eprint = {2407.01489},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2407.01489}
}
```

- [← previous VideoLLaMA 2: Advancing Spatial-Temporal Modeling and Audio Understanding in Video-LLMs](https://md.research.elamur.ai/free/en/papers/cheng-2024-videollama-2-advancing-spatial-temporal/)
- [next → ArabicMMLU: Assessing Massive Multitask Language Understanding in Arabic](https://md.research.elamur.ai/free/en/papers/koto-2024-arabicmmlu-assessing-massive-multitask-language/)
- [↑ back to papers of 2024](https://md.research.elamur.ai/free/en/papers/2024/page/3/)

## References

110 references, 12 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. 2024. Agent-101: A Software Engineering Agent for Code Assistance developed by IBM Research. [https://github.com/swe-bench/experiments/blob/main/](https://github.com/swe-bench/experiments/blob/main/) evaluation/lite/20240612_IBM_Research_Agent101/README.md/. [↗](https://github.com/swe-bench/experiments/blob/main/)
2. 2024. Aide by Codestory. [https://github.com/swe-bench/experiments/tree/](https://github.com/swe-bench/experiments/tree/) main/evaluation/lite/20240702_codestory_aide_mixed. [↗](https://github.com/swe-bench/experiments/tree/)
3. 2024. Alex SIMA. [https://github.com/swe-bench/experiments/tree/main/](https://github.com/swe-bench/experiments/tree/main/) evaluation/lite/20240706_sima_gpt4o. [↗](https://github.com/swe-bench/experiments/tree/main/)
4. 2024. Amazon Q Developer The most capable generative AI–powered assistant for software development. [https://aws.amazon.com/q/developer//](https://aws.amazon.com/q/developer//). [↗](https://aws.amazon.com/q/developer//)
5. 2024. AppMap speedruns to the top of the SWE Bench Leaderboard. [https://appmap.io/blog/2024/06/20/appmap-navie-swe-bench-leader/](https://appmap.io/blog/2024/06/20/appmap-navie-swe-bench-leader/). [↗](https://appmap.io/blog/2024/06/20/appmap-navie-swe-bench-leader/)
6. 2024. AutoCodeRover Autonomous Software Engineering. [https://autocoderover.dev/](https://autocoderover.dev/). Devin, AI [↗](https://autocoderover.dev/)
7. 2024. introducing-devin. software engineer. [https://www.cognition.ai/](https://www.cognition.ai/) [↗](https://www.cognition.ai/)
8. 2024. Empower your AI agents with Composio - a platform for managing and integrating tools with LLMs and AI agents using Function Calling. [https://docs.composio.dev/introduction/intro/overview](https://docs.composio.dev/introduction/intro/overview). [↗](https://docs.composio.dev/introduction/intro/overview)
9. 2024. Factory Bringing Autonomy to Software Engineering. [https://www.factory.ai/](https://www.factory.ai/). [↗](https://www.factory.ai/)
10. 2024. Honeycomb. [https://honeycomb.sh](https://honeycomb.sh). [↗](https://honeycomb.sh)
11. 2024. Introducing computer use, a new Claude 3.5 Sonnet, and Claude 3.5 Haiku. [https://www.anthropic.com/news/3-5-models-and-computer-use](https://www.anthropic.com/news/3-5-models-and-computer-use). [↗](https://www.anthropic.com/news/3-5-models-and-computer-use)
12. 2024. Isoform. [https://github.com/swe-bench/experiments/tree/main/](https://github.com/swe-bench/experiments/tree/main/) evaluation/lite/20240829_Isoform. [↗](https://github.com/swe-bench/experiments/tree/main/)
13. 2024. Lingma Agent. [https://github.com/swe-bench/experiments/tree/main/](https://github.com/swe-bench/experiments/tree/main/) evaluation/lite/20240622_Lingma_Agent. \#\# 4 Sadly the bike is currently broken. [↗](https://github.com/swe-bench/experiments/tree/main/)
14. 2024. MentatBot: New SOTA Coding Agent, Available Now. [https://mentat.ai/](https://mentat.ai/) blog/mentatbot-sota-coding-agent. [↗](https://mentat.ai/)
15. 2024. Moatless Tools. [https://github.com/aorwall/moatless-tools](https://github.com/aorwall/moatless-tools). [↗](https://github.com/aorwall/moatless-tools)
16. 2024. OpenCSG StarShip. [https://opencsg.com/product?class=StarShip/](https://opencsg.com/product?class=StarShip/). [↗](https://opencsg.com/product?class=StarShip/)
17. 2024. OpenDevin: Code Less, Make More. OpenDevin/. [https://github.com/OpenDevin/](https://github.com/OpenDevin/) [↗](https://github.com/OpenDevin/)
18. 2024. Python ast — Abstract Syntax Trees. [https://docs.python.org/3/library/](https://docs.python.org/3/library/) ast.html/. [↗](https://docs.python.org/3/library/)
19. 2024. RepoGraph: Enhancing AI Software Engineering with Repository-level Code Graph. [https://github.com/ozyyshr/RepoGraph](https://github.com/ozyyshr/RepoGraph). [↗](https://github.com/ozyyshr/RepoGraph)
20. 2024. The Road to Ultimate Pull Request Machine. road-to-ultimate-pull-request-machine/. [https://gru.ai/blog/](https://gru.ai/blog/) [↗](https://gru.ai/blog/)
21. 2024. Solver. [https://solverai.com](https://solverai.com). [↗](https://solverai.com)
22. 2024. SuperCoder. [https://superagi.com/supercoder/](https://superagi.com/supercoder/). [↗](https://superagi.com/supercoder/)
23. 2024. SWE-bench Lite. [https://www.swebench.com/lite.html](https://www.swebench.com/lite.html). [↗](https://www.swebench.com/lite.html)
24. Rui Abreu, Peter Zoeteweij, Rob Golsteijn, and Arjan JC Van Gemund. 2009. A practical evaluation of spectrum-based fault localization. Journal of Systems and Software 82, 11 (2009), 1780–1792. [find](https://scholar.google.com/scholar?q=Rui+Abreu%2C+Peter+Zoeteweij%2C+Rob+Golsteijn%2C+and+Arjan+JC+Van+Gemund%2E+2009%2E+A+practical+evaluation+of+spectrum%2Dbased+fault+localization%2E+Journal+of+Systems+and+Software+82%2C+11+%282009%29%2C+1780%E2%80%931792%2E)
25. Rui Abreu, Peter Zoeteweij, and Arjan JC Van Gemund. 2007. On the accuracy of spectrum-based fault localization. In Testing: Academic and industrial conference practice and research techniques-MUTATION (TAICPART-MUTATION 2007). IEEE, 89–98. [find](https://scholar.google.com/scholar?q=Rui+Abreu%2C+Peter+Zoeteweij%2C+and+Arjan+JC+Van+Gemund%2E+2007%2E+On+the+accuracy+of+spectrum%2Dbased+fault+localization%2E+In+Testing%3A+Academic+and+industrial+conference+practice+and+research+techniques%2DMUTATIO)
26. Anthropic. 2024. Introducing Claude 3.5 Sonnet. [https://www.anthropic.com/news/](https://www.anthropic.com/news/) claude-3-5-sonnet/. [↗](https://www.anthropic.com/news/)
27. Daman Arora, Atharv Sonwane, Nalin Wadhwa, Abhav Mehrotra, Saiteja Utpala, Ramakrishna Bairi, Aditya Kanade, and Nagarajan Natarajan. 2024. MASAI: Modular Architecture for Software-engineering AI Agents. arXiv preprint arXiv:2406.11638 (2024). — [arXiv:2406.11638](https://arxiv.org/abs/2406.11638) [find](https://scholar.google.com/scholar?q=Daman+Arora%2C+Atharv+Sonwane%2C+Nalin+Wadhwa%2C+Abhav+Mehrotra%2C+Saiteja+Utpala%2C+Ramakrishna+Bairi%2C+Aditya+Kanade%2C+and+Nagarajan+Natarajan%2E+2024%2E+MASAI%3A+Modular+Architecture+for+Software%2Dengineering+AI+Agen)
28. Jacob Austin, Augustus Odena, Maxwell Nye, Maarten Bosma, Henryk Michalewski, David Dohan, Ellen Jiang, Carrie Cai, Michael Terry, Quoc Le, and Charles Sutton. 2021. [Program Synthesis with Large Language Models](https://md.research.elamur.ai/free/en/papers/austin-2021-program-synthesis-large-language-models/). arXiv:2108.07732 [cs.PL]
29. Islem Bouzenia, Premkumar Devanbu, and Michael Pradel. 2024. Repairagent: An autonomous, llm-based agent for program repair. arXiv preprint arXiv:2403.17134 (2024). — [arXiv:2403.17134](https://arxiv.org/abs/2403.17134) [find](https://scholar.google.com/scholar?q=Islem+Bouzenia%2C+Premkumar+Devanbu%2C+and+Michael+Pradel%2E+2024%2E+Repairagent%3A+An+autonomous%2C+llm%2Dbased+agent+for+program+repair%2E+arXiv+preprint+arXiv%3A2403%2E17134+%282024%29%2E+%E2%80%94+%5BarXiv%3A2403%2E17134%5D%28https%3A%2F%2Farxiv%2E)
30. Dong Chen, Shaoxin Lin, Muhan Zeng, Daoguang Zan, Jian-Gang Wang, Anton Cheshkov, Jun Sun, Hao Yu, Guoliang Dong, Artem Aliev, et al. 2024. [CodeR: Issue Resolving with Multi-Agent and Task Graphs](https://md.research.elamur.ai/free/en/papers/chen-2024-coder-issue-resolving-multi-agent/). arXiv preprint arXiv:2406.01304 (2024).
31. Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde de Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, et al. 2021. [Evaluating large language models trained on code](https://md.research.elamur.ai/free/en/papers/chen-2021-evaluating-large-language-models-trained/). arXiv preprint arXiv:2107.03374 (2021).
32. Yang Chen. 2024. Flakiness Repair in the Era of Large Language Models. In Proceedings of the 2024 IEEE/ACM 46th International Conference on Software Engineering: Companion Proceedings. 441–443. [find](https://scholar.google.com/scholar?q=Yang+Chen%2E+2024%2E+Flakiness+Repair+in+the+Era+of+Large+Language+Models%2E+In+Proceedings+of+the+2024+IEEE%2FACM+46th+International+Conference+on+Software+Engineering%3A+Companion+Proceedings%2E+441%E2%80%93443%2E)
33. Zimin Chen, Steve Kommrusch, Michele Tufano, Louis-Noël Pouchet, Denys Poshyvanyk, and Martin Monperrus. 2019. SequenceR: Sequence-to-Sequence Learning for End-to-End Program Repair. IEEE Transaction on Software Engineering (2019). [find](https://scholar.google.com/scholar?q=Zimin+Chen%2C+Steve+Kommrusch%2C+Michele+Tufano%2C+Louis%2DNoe%CC%88l+Pouchet%2C+Denys+Poshyvanyk%2C+and+Martin+Monperrus%2E+2019%2E+SequenceR%3A+Sequence%2Dto%2DSequence+Learning+for+End%2Dto%2DEnd+Program+Repair%2E+IEEE+Transaction)
34. Neil Chowdhury, James Aung, Chan Jun Shern, Oliver Jaffe, Dane Sherburn, Giulio Starace, Evan Mays, Rachel Dias, Marwan Aljubeh, Mia Glaese, Carlos E. Jimenez, John Yang, Kevin Liu, and Aleksander Madry. 2024. Introducing SWE-bench Verified. OpenAI Blog (2024). [https://openai.com/index/](https://openai.com/index/) introducing-… [↗](https://openai.com/index/)
35. Yinlin Deng, Chunqiu Steven Xia, Haoran Peng, Chenyuan Yang, and Lingming Zhang. 2023. Large Language Models are Zero-Shot Fuzzers: Fuzzing Deep-Learning Libraries via Large Language Models. In 32nd International Symposium on Software Testing and Analysis (ISSTA). [find](https://scholar.google.com/scholar?q=Yinlin+Deng%2C+Chunqiu+Steven+Xia%2C+Haoran+Peng%2C+Chenyuan+Yang%2C+and+Lingming+Zhang%2E+2023%2E+Large+Language+Models+are+Zero%2DShot+Fuzzers%3A+Fuzzing+Deep%2DLearning+Libraries+via+Large+Language+Models%2E+In+32nd+I)
36. Yinlin Deng, Chunqiu Steven Xia, Chenyuan Yang, Shizhuo Dylan Zhang, Shujing Yang, and Lingming Zhang. 2024. Large Language Models are Edge-Case Fuzzers: Testing Deep Learning Libraries via FuzzGPT. In 46th International Conference on Software Engineering (ICSE). [find](https://scholar.google.com/scholar?q=Yinlin+Deng%2C+Chunqiu+Steven+Xia%2C+Chenyuan+Yang%2C+Shizhuo+Dylan+Zhang%2C+Shujing+Yang%2C+and+Lingming+Zhang%2E+2024%2E+Large+Language+Models+are+Edge%2DCase+Fuzzers%3A+Testing+Deep+Learning+Libraries+via+FuzzGPT%2E+I)
37. Paul Gauthier. 2024. Aider is AI pair programming in your terminal. [https://aider.chat/](https://aider.chat/). [↗](https://aider.chat/)
38. Luca Gazzola, Daniela Micucci, and Leonardo Mariani. 2019. Automatic Software Repair: A Survey. IEEE Transactions on Software Engineering 45, 1 (2019), 34–67. [find](https://scholar.google.com/scholar?q=Luca+Gazzola%2C+Daniela+Micucci%2C+and+Leonardo+Mariani%2E+2019%2E+Automatic+Software+Repair%3A+A+Survey%2E+IEEE+Transactions+on+Software+Engineering+45%2C+1+%282019%29%2C+34%E2%80%9367%2E)
39. Ali Ghanbari, Samuel Benton, and Lingming Zhang. 2019. Practical Program Repair via Bytecode Mutation. In Proceedings of the 28th ACM SIGSOFT International Symposium on Software Testing and Analysis (Beijing, China) (ISSTA 2019). ACM, 19–30. [find](https://scholar.google.com/scholar?q=Ali+Ghanbari%2C+Samuel+Benton%2C+and+Lingming+Zhang%2E+2019%2E+Practical+Program+Repair+via+Bytecode+Mutation%2E+In+Proceedings+of+the+28th+ACM+SIGSOFT+International+Symposium+on+Software+Testing+and+Analysis+%28)
40. Dávid Hidvégi, Khashayar Etemadi, Sofia Bobadilla, and Martin Monperrus. 2024. Cigar: Cost-efficient program repair with llms. arXiv preprint arXiv:2402.06598 (2024). — [arXiv:2402.06598](https://arxiv.org/abs/2402.06598) [find](https://scholar.google.com/scholar?q=Da%CC%81vid+Hidve%CC%81gi%2C+Khashayar+Etemadi%2C+Sofia+Bobadilla%2C+and+Martin+Monperrus%2E+2024%2E+Cigar%3A+Cost%2Defficient+program+repair+with+llms%2E+arXiv+preprint+arXiv%3A2402%2E06598+%282024%29%2E+%E2%80%94+%5BarXiv%3A2402%2E06598%5D%28https%3A%2F%2Far)
41. Jie Huang, Xinyun Chen, Swaroop Mishra, Huaixiu Steven Zheng, Adams Wei Yu, Xinying Song, and Denny Zhou. 2024. [Large Language Models Cannot Self-Correct Reasoning Yet](https://md.research.elamur.ai/free/en/papers/huang-2023-large-language-models-cannot-self/). In The Twelfth International Conference on Learning Representations. [https://openreview.net/forum?id=IkmD3fKBPQ](https://openreview.net/forum?id=IkmD3fKBPQ) [↗](https://openreview.net/forum?id=IkmD3fKBPQ)
42. Nan Jiang, Kevin Liu, Thibaud Lutellier, and Lin Tan. 2023. Impact of Code Language Models on Automated Program Repair. In 2023 IEEE/ACM 45th International Conference on Software Engineering (ICSE). 1430–1442. https://doi.org/10.1109/ICSE48619. 2023.00125 — [doi:10.1109/icse48619](https://doi.org/10.1109/icse48619) [↗](https://doi.org/10.1109/ICSE48619)
43. Nan Jiang, Thibaud Lutellier, and Lin Tan. 2021. CURE: Code-Aware Neural Machine Translation for Automatic Program Repair. 2021 IEEE/ACM 43rd International Conference on Software Engineering (ICSE) (May 2021). [find](https://scholar.google.com/scholar?q=Nan+Jiang%2C+Thibaud+Lutellier%2C+and+Lin+Tan%2E+2021%2E+CURE%3A+Code%2DAware+Neural+Machine+Translation+for+Automatic+Program+Repair%2E+2021+IEEE%2FACM+43rd+International+Conference+on+Software+Engineering+%28ICSE%29+%28M)
44. Carlos E Jimenez, John Yang, Alexander Wettig, Shunyu Yao, Kexin Pei, Ofir Press, and Karthik R Narasimhan. 2024. [SWE-bench: Can Language Models Resolve Real-world Github Issues?](https://md.research.elamur.ai/free/en/papers/jimenez-2024-swe-bench-can-language-models/). In The Twelfth International Conference on Learning Representations. [https://openreview.net/forum?id=VTF8yNQM66](https://openreview.net/forum?id=VTF8yNQM66) [↗](https://openreview.net/forum?id=VTF8yNQM66)
45. Carlos E Jimenez, John Yang, Alexander Wettig, Shunyu Yao, Kexin Pei, Ofir Press, and Karthik R Narasimhan. 2024. SWE-bench Leaderboard. [https://www.swebench.com/](https://www.swebench.com/). [↗](https://www.swebench.com/)
46. Wei Jin and Alessandro Orso. 2012. Bugredux: Reproducing field failures for in-house debugging. In 2012 34th international conference on software engineering (ICSE). IEEE, 474–484. [find](https://scholar.google.com/scholar?q=Wei+Jin+and+Alessandro+Orso%2E+2012%2E+Bugredux%3A+Reproducing+field+failures+for+in%2Dhouse+debugging%2E+In+2012+34th+international+conference+on+software+engineering+%28ICSE%29%2E+IEEE%2C+474%E2%80%93484%2E)
47. James A Jones and Mary Jean Harrold. 2005. Empirical evaluation of the tarantula automatic fault-localization technique. In Proceedings of the 20th IEEE/ACM international Conference on Automated software engineering. 273–282. [find](https://scholar.google.com/scholar?q=James+A+Jones+and+Mary+Jean+Harrold%2E+2005%2E+Empirical+evaluation+of+the+tarantula+automatic+fault%2Dlocalization+technique%2E+In+Proceedings+of+the+20th+IEEE%2FACM+international+Conference+on+Automated+softw)
48. Sungmin Kang, Gabin An, and Shin Yoo. 2024. A quantitative and qualitative evaluation of LLM-based explainable fault localization. Proceedings of the ACM on Software Engineering 1, FSE (2024), 1424–1446. [find](https://scholar.google.com/scholar?q=Sungmin+Kang%2C+Gabin+An%2C+and+Shin+Yoo%2E+2024%2E+A+quantitative+and+qualitative+evaluation+of+LLM%2Dbased+explainable+fault+localization%2E+Proceedings+of+the+ACM+on+Software+Engineering+1%2C+FSE+%282024%29%2C+1424%E2%80%9314)
49. Sophia D Kolak, Ruben Martins, Claire Le Goues, and Vincent Josua Hellendoorn. 2022. Patch Generation with Language Models: Feasibility and Scaling Behavior. In Deep Learning for Code Workshop. [find](https://scholar.google.com/scholar?q=Sophia+D+Kolak%2C+Ruben+Martins%2C+Claire+Le+Goues%2C+and+Vincent+Josua+Hellendoorn%2E+2022%2E+Patch+Generation+with+Language+Models%3A+Feasibility+and+Scaling+Behavior%2E+In+Deep+Learning+for+Code+Workshop%2E)
50. Xuan Bach D. Le, David Lo, and Claire Le Goues. 2016. History Driven Program Repair. In 2016 IEEE 23rd International Conference on Software Analysis, Evolution, and Reengineering (SANER), Vol. 1. 213–224. [find](https://scholar.google.com/scholar?q=Xuan+Bach+D%2E+Le%2C+David+Lo%2C+and+Claire+Le+Goues%2E+2016%2E+History+Driven+Program+Repair%2E+In+2016+IEEE+23rd+International+Conference+on+Software+Analysis%2C+Evolution%2C+and+Reengineering+%28SANER%29%2C+Vol%2E+1%2E+213%E2%80%93)
51. Claire Le Goues, ThanhVu Nguyen, Stephanie Forrest, and Westley Weimer. 2012. GenProg: A Generic Method for Automatic Software Repair. IEEE Transactions on Software Engineering 38, 1 (2012), 54–72. [find](https://scholar.google.com/scholar?q=Claire+Le+Goues%2C+ThanhVu+Nguyen%2C+Stephanie+Forrest%2C+and+Westley+Weimer%2E+2012%2E+GenProg%3A+A+Generic+Method+for+Automatic+Software+Repair%2E+IEEE+Transactions+on+Software+Engineering+38%2C+1+%282012%29%2C+54%E2%80%9372%2E)
52. Caroline Lemieux, Jeevana Priya Inala, Shuvendu K Lahiri, and Siddhartha Sen. 2023. CODAMOSA: Escaping Coverage Plateaus in Test Generation with Pre-trained Large Language Models. In 45th International Conference on Software Engineering (ICSE). [find](https://scholar.google.com/scholar?q=Caroline+Lemieux%2C+Jeevana+Priya+Inala%2C+Shuvendu+K+Lahiri%2C+and+Siddhartha+Sen%2E+2023%2E+CODAMOSA%3A+Escaping+Coverage+Plateaus+in+Test+Generation+with+Pre%2Dtrained+Large+Language+Models%2E+In+45th+Internationa)
53. Raymond Li, Loubna Ben Allal, Yangtian Zi, Niklas Muennighoff, Denis Kocetkov, Chenghao Mou, Marc Marone, Christopher Akiki, Jia Li, Jenny Chim, et al. 2023. Starcoder: may the source be with you! [find](https://scholar.google.com/scholar?q=Raymond+Li%2C+Loubna+Ben+Allal%2C+Yangtian+Zi%2C+Niklas+Muennighoff%2C+Denis+Kocetkov%2C+Chenghao+Mou%2C+Marc+Marone%2C+Christopher+Akiki%2C+Jia+Li%2C+Jenny+Chim%2C+et+al%2E+2023%2E+Starcoder%3A+may+the+source+be+with+you%21)
54. Xia Li, Wei Li, Yuqun Zhang, and Lingming Zhang. 2019. Deepfl: Integrating multiple fault diagnosis dimensions for deep fault localization. In Proceedings of the 28th ACM SIGSOFT international symposium on software testing and analysis. 169–180. [find](https://scholar.google.com/scholar?q=Xia+Li%2C+Wei+Li%2C+Yuqun+Zhang%2C+and+Lingming+Zhang%2E+2019%2E+Deepfl%3A+Integrating+multiple+fault+diagnosis+dimensions+for+deep+fault+localization%2E+In+Proceedings+of+the+28th+ACM+SIGSOFT+international+symposi)
55. Yi Li, Shaohua Wang, and Tien N. Nguyen. 2020. DLFix: Context-Based Code Transformation Learning for Automated Program Repair. In Proceedings of the ACM/IEEE 42nd International Conference on Software Engineering (Seoul, South Korea) (ICSE ’20). Association for Computing Machinery, New York, NY, USA,… [find](https://scholar.google.com/scholar?q=Yi+Li%2C+Shaohua+Wang%2C+and+Tien+N%2E+Nguyen%2E+2020%2E+DLFix%3A+Context%2DBased+Code+Transformation+Learning+for+Automated+Program+Repair%2E+In+Proceedings+of+the+ACM%2FIEEE+42nd+International+Conference+on+Software+)
56. Junwei Liu, Kaixin Wang, Yixuan Chen, Xin Peng, Zhenpeng Chen, Lingming Zhang, and Yiling Lou. 2024. Large Language Model-Based Agents for Software Engineering: A Survey. arXiv preprint arXiv:2409.02977 (2024). — [arXiv:2409.02977](https://arxiv.org/abs/2409.02977) [find](https://scholar.google.com/scholar?q=Junwei+Liu%2C+Kaixin+Wang%2C+Yixuan+Chen%2C+Xin+Peng%2C+Zhenpeng+Chen%2C+Lingming+Zhang%2C+and+Yiling+Lou%2E+2024%2E+Large+Language+Model%2DBased+Agents+for+Software+Engineering%3A+A+Survey%2E+arXiv+preprint+arXiv%3A2409%2E029)
57. Kui Liu, Anil Koyuncu, Dongsun Kim, and Tegawendé F. Bissyandé. 2019. TBar: Revisiting Template-Based Automated Program Repair. In Proceedings of the 28th ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA 2019). ACM, New York, NY, USA, 31–42. [find](https://scholar.google.com/scholar?q=Kui+Liu%2C+Anil+Koyuncu%2C+Dongsun+Kim%2C+and+Tegawende%CC%81+F%2E+Bissyande%CC%81%2E+2019%2E+TBar%3A+Revisiting+Template%2DBased+Automated+Program+Repair%2E+In+Proceedings+of+the+28th+ACM+SIGSOFT+International+Symposium+on+Soft)
58. Yizhou Liu, Pengfei Gao, Xinchen Wang, Chao Peng, and Zhao Zhang. 2024. MarsCode Agent: AI-native Automated Bug Fixing. arXiv preprint arXiv:2409.00899 (2024). — [arXiv:2409.00899](https://arxiv.org/abs/2409.00899) [find](https://scholar.google.com/scholar?q=Yizhou+Liu%2C+Pengfei+Gao%2C+Xinchen+Wang%2C+Chao+Peng%2C+and+Zhao+Zhang%2E+2024%2E+MarsCode+Agent%3A+AI%2Dnative+Automated+Bug+Fixing%2E+arXiv+preprint+arXiv%3A2409%2E00899+%282024%29%2E+%E2%80%94+%5BarXiv%3A2409%2E00899%5D%28https%3A%2F%2Farxiv%2Eorg%2Fa)
59. Zhe Liu, Chunyang Chen, Junjie Wang, Mengzhuo Chen, Boyu Wu, Xing Che, Dandan Wang, and Qing Wang. 2024. Make llm a testing expert: Bringing human-like interaction to mobile gui testing via functionality-aware decisions. In Proceedings of the IEEE/ACM 46th International Conference on Software Engine… [find](https://scholar.google.com/scholar?q=Zhe+Liu%2C+Chunyang+Chen%2C+Junjie+Wang%2C+Mengzhuo+Chen%2C+Boyu+Wu%2C+Xing+Che%2C+Dandan+Wang%2C+and+Qing+Wang%2E+2024%2E+Make+llm+a+testing+expert%3A+Bringing+human%2Dlike+interaction+to+mobile+gui+testing+via+functional)
60. LlamaIndex. 2024. LlamaIndex, Data Framework for LLM Applications. [https://www.llamaindex.ai/](https://www.llamaindex.ai/). [↗](https://www.llamaindex.ai/)
61. Fan Long and Martin Rinard. 2015. Staged Program Repair with Condition Synthesis. In Proceedings of the 2015 10th Joint Meeting on Foundations of Software Engineering (Bergamo, Italy) (ESEC/FSE 2015). New York, NY, USA, 166–178. [find](https://scholar.google.com/scholar?q=Fan+Long+and+Martin+Rinard%2E+2015%2E+Staged+Program+Repair+with+Condition+Synthesis%2E+In+Proceedings+of+the+2015+10th+Joint+Meeting+on+Foundations+of+Software+Engineering+%28Bergamo%2C+Italy%29+%28ESEC%2FFSE+2015%29%2E)
62. Fan Long and Martin Rinard. 2016. An analysis of the search spaces for generate and validate patch generation systems. In Proceedings of the 38th International Conference on Software Engineering. 702–713. [find](https://scholar.google.com/scholar?q=Fan+Long+and+Martin+Rinard%2E+2016%2E+An+analysis+of+the+search+spaces+for+generate+and+validate+patch+generation+systems%2E+In+Proceedings+of+the+38th+International+Conference+on+Software+Engineering%2E+702%E2%80%93)
63. Yiling Lou, Ali Ghanbari, Xia Li, Lingming Zhang, Haotian Zhang, Dan Hao, and Lu Zhang. 2020. Can automated program repair refine fault localization? a unified debugging approach. In Proceedings of the 29th ACM SIGSOFT International Symposium on Software Testing and Analysis. 75–87. [find](https://scholar.google.com/scholar?q=Yiling+Lou%2C+Ali+Ghanbari%2C+Xia+Li%2C+Lingming+Zhang%2C+Haotian+Zhang%2C+Dan+Hao%2C+and+Lu+Zhang%2E+2020%2E+Can+automated+program+repair+refine+fault+localization%3F+a+unified+debugging+approach%2E+In+Proceedings+of+th)
64. Yingwei Ma, Qingping Yang, Rongyu Cao, Binhua Li, Fei Huang, and Yongbin Li. 2024. How to Understand Whole Software Repository? arXiv preprint arXiv:2406.01422 (2024). — [arXiv:2406.01422](https://arxiv.org/abs/2406.01422) [find](https://scholar.google.com/scholar?q=Yingwei+Ma%2C+Qingping+Yang%2C+Rongyu+Cao%2C+Binhua+Li%2C+Fei+Huang%2C+and+Yongbin+Li%2E+2024%2E+How+to+Understand+Whole+Software+Repository%3F+arXiv+preprint+arXiv%3A2406%2E01422+%282024%29%2E+%E2%80%94+%5BarXiv%3A2406%2E01422%5D%28https%3A%2F%2Farx)
65. Sergey Mechtaev, Jooyong Yi, and Abhik Roychoudhury. 2016. Angelix: Scalable Multiline Program Patch Synthesis via Symbolic Analysis. In Proceedings of the 38th International Conference on Software Engineering (Austin, Texas) (ICSE ’16). 691–701. [find](https://scholar.google.com/scholar?q=Sergey+Mechtaev%2C+Jooyong+Yi%2C+and+Abhik+Roychoudhury%2E+2016%2E+Angelix%3A+Scalable+Multiline+Program+Patch+Synthesis+via+Symbolic+Analysis%2E+In+Proceedings+of+the+38th+International+Conference+on+Software+En)
66. Ruijie Meng, Martin Mirchev, Marcel Böhme, and Abhik Roychoudhury. 2024. Large language model guided protocol fuzzing. In Proceedings of the 31st Annual Network and Distributed System Security Symposium (NDSS). [find](https://scholar.google.com/scholar?q=Ruijie+Meng%2C+Martin+Mirchev%2C+Marcel+Bo%CC%88hme%2C+and+Abhik+Roychoudhury%2E+2024%2E+Large+language+model+guided+protocol+fuzzing%2E+In+Proceedings+of+the+31st+Annual+Network+and+Distributed+System+Security+Sympos)
67. Xiangxin Meng, Xu Wang, Hongyu Zhang, Hailong Sun, and Xudong Liu. 2022. Improving fault localization and program repair with deep semantic features and transferred knowledge. In Proceedings of the 44th International Conference on Software Engineering. 1169–1180. [find](https://scholar.google.com/scholar?q=Xiangxin+Meng%2C+Xu+Wang%2C+Hongyu+Zhang%2C+Hailong+Sun%2C+and+Xudong+Liu%2E+2022%2E+Improving+fault+localization+and+program+repair+with+deep+semantic+features+and+transferred+knowledge%2E+In+Proceedings+of+the+44)
68. Seokhyeon Moon, Yunho Kim, Moonzoo Kim, and Shin Yoo. 2014. Ask the mutants: Mutating faulty programs for fault localization. In 2014 IEEE Seventh International Conference on Software Testing, Verification and Validation. IEEE, 153–162. [find](https://scholar.google.com/scholar?q=Seokhyeon+Moon%2C+Yunho+Kim%2C+Moonzoo+Kim%2C+and+Shin+Yoo%2E+2014%2E+Ask+the+mutants%3A+Mutating+faulty+programs+for+fault+localization%2E+In+2014+IEEE+Seventh+International+Conference+on+Software+Testing%2C+Verific)
69. Theo X Olausson, Jeevana Priya Inala, Chenglong Wang, Jianfeng Gao, and Armando Solar-Lezama. 2023. [Is Self-Repair a Silver Bullet for Code Generation?](https://md.research.elamur.ai/free/en/papers/olausson-2024-self-repair-silver-bullet-code/). In The Twelfth International Conference on Learning Representations.
70. Yaroslav Oliinyk, Michael Scott, Ryan Tsang, Chongzhou Fang, Houman Homayoun, et al. 2024. Fuzzing BusyBox: Leveraging LLM and Crash Reuse for Embedded Bug Unearthing. arXiv preprint arXiv:2403.03897 (2024). — [arXiv:2403.03897](https://arxiv.org/abs/2403.03897) [find](https://scholar.google.com/scholar?q=Yaroslav+Oliinyk%2C+Michael+Scott%2C+Ryan+Tsang%2C+Chongzhou+Fang%2C+Houman+Homayoun%2C+et+al%2E+2024%2E+Fuzzing+BusyBox%3A+Leveraging+LLM+and+Crash+Reuse+for+Embedded+Bug+Unearthing%2E+arXiv+preprint+arXiv%3A2403%2E03897+)
71. OpenAI. 2023. [GPT-4 Technical Report](https://md.research.elamur.ai/free/en/papers/openai-2023-gpt-4-technical-report/). ArXiv abs/2303.08774 (2023).
72. OpenAI. 2024. Hello GPT-4o. [https://openai.com/index/hello-gpt-4o/](https://openai.com/index/hello-gpt-4o/). [↗](https://openai.com/index/hello-gpt-4o/)
73. OpenAI. 2024. New embedding models and API updates. [https://openai.com/](https://openai.com/) index/new-embedding-models-and-api-updates/. [↗](https://openai.com/)
74. OpenAI. 2024. [OpenAI o1 System Card](https://md.research.elamur.ai/free/en/papers/openai-2024-openai-o1-system-card/). openai-o1-system-card/. [https://openai.com/index/](https://openai.com/index/) [↗](https://openai.com/index/)
75. Xianfei Ou, Cong Li, Yanyan Jiang, and Chang Xu. 2024. The Mutators Reloaded: Fuzzing Compilers with Large Language Model Generated Mutation Operators. In ASPLOS. [find](https://scholar.google.com/scholar?q=Xianfei+Ou%2C+Cong+Li%2C+Yanyan+Jiang%2C+and+Chang+Xu%2E+2024%2E+The+Mutators+Reloaded%3A+Fuzzing+Compilers+with+Large+Language+Model+Generated+Mutation+Operators%2E+In+ASPLOS%2E)
76. Mike Papadakis, Marinos Kintis, Jie Zhang, Yue Jia, Yves Le Traon, and Mark Harman. 2019. Mutation testing advances: an analysis and survey. In Advances in computers. Vol. 112. Elsevier, 275–378. [find](https://scholar.google.com/scholar?q=Mike+Papadakis%2C+Marinos+Kintis%2C+Jie+Zhang%2C+Yue+Jia%2C+Yves+Le+Traon%2C+and+Mark+Harman%2E+2019%2E+Mutation+testing+advances%3A+an+analysis+and+survey%2E+In+Advances+in+computers%2E+Vol%2E+112%2E+Elsevier%2C+275%E2%80%93378%2E)
77. Mike Papadakis and Yves Le Traon. 2015. Metallaxis-FL: mutation-based fault localization. Software Testing, Verification and Reliability 25, 5-7 (2015), 605–628. [find](https://scholar.google.com/scholar?q=Mike+Papadakis+and+Yves+Le+Traon%2E+2015%2E+Metallaxis%2DFL%3A+mutation%2Dbased+fault+localization%2E+Software+Testing%2C+Verification+and+Reliability+25%2C+5%2D7+%282015%29%2C+605%E2%80%93628%2E)
78. Juan Altmayer Pizzorno and Emery D Berger. 2024. CoverUp: Coverage-Guided LLM-Based Test Generation. arXiv preprint arXiv:2403.16218 (2024). — [arXiv:2403.16218](https://arxiv.org/abs/2403.16218) [find](https://scholar.google.com/scholar?q=Juan+Altmayer+Pizzorno+and+Emery+D+Berger%2E+2024%2E+CoverUp%3A+Coverage%2DGuided+LLM%2DBased+Test+Generation%2E+arXiv+preprint+arXiv%3A2403%2E16218+%282024%29%2E+%E2%80%94+%5BarXiv%3A2403%2E16218%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2403%2E16218%29)
79. Yihao Qin, Shangwen Wang, Yiling Lou, Jinhao Dong, Kaixin Wang, Xiaoling Li, and Xiaoguang Mao. 2024. AgentFL: Scaling LLM-based Fault Localization to Project-Level Context. arXiv preprint arXiv:2403.16362 (2024). — [arXiv:2403.16362](https://arxiv.org/abs/2403.16362) [find](https://scholar.google.com/scholar?q=Yihao+Qin%2C+Shangwen+Wang%2C+Yiling+Lou%2C+Jinhao+Dong%2C+Kaixin+Wang%2C+Xiaoling+Li%2C+and+Xiaoguang+Mao%2E+2024%2E+AgentFL%3A+Scaling+LLM%2Dbased+Fault+Localization+to+Project%2DLevel+Context%2E+arXiv+preprint+arXiv%3A2403%2E)
80. Stephen Robertson, Hugo Zaragoza, et al. 2009. The probabilistic relevance framework: BM25 and beyond. Foundations and Trends® in Information Retrieval 3, 4 (2009), 333–389. [find](https://scholar.google.com/scholar?q=Stephen+Robertson%2C+Hugo+Zaragoza%2C+et+al%2E+2009%2E+The+probabilistic+relevance+framework%3A+BM25+and+beyond%2E+Foundations+and+Trends%C2%AE+in+Information+Retrieval+3%2C+4+%282009%29%2C+333%E2%80%93389%2E)
81. Haifeng Ruan, Yuntong Zhang, and Abhik Roychoudhury. 2024. SpecRover: Code Intent Extraction via LLMs. arXiv preprint arXiv:2408.02232 (2024). — [arXiv:2408.02232](https://arxiv.org/abs/2408.02232) [find](https://scholar.google.com/scholar?q=Haifeng+Ruan%2C+Yuntong+Zhang%2C+and+Abhik+Roychoudhury%2E+2024%2E+SpecRover%3A+Code+Intent+Extraction+via+LLMs%2E+arXiv+preprint+arXiv%3A2408%2E02232+%282024%29%2E+%E2%80%94+%5BarXiv%3A2408%2E02232%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2408%2E02232%29)
82. Ripon K Saha, Matthew Lease, Sarfraz Khurshid, and Dewayne E Perry. 2013. Improving bug localization using structured information retrieval. In 2013 28th IEEE/ACM International Conference on Automated Software Engineering (ASE). IEEE, 345–355. [find](https://scholar.google.com/scholar?q=Ripon+K+Saha%2C+Matthew+Lease%2C+Sarfraz+Khurshid%2C+and+Dewayne+E+Perry%2E+2013%2E+Improving+bug+localization+using+structured+information+retrieval%2E+In+2013+28th+IEEE%2FACM+International+Conference+on+Automated)
83. Max Schäfer, Sarah Nadi, Aryaz Eghbali, and Frank Tip. 2023. Adaptive Test Generation Using a Large Language Model. arXiv:2302.06527 [cs.SE] — [arXiv:2302.06527](https://arxiv.org/abs/2302.06527) [find](https://scholar.google.com/scholar?q=Max+Scha%CC%88fer%2C+Sarah+Nadi%2C+Aryaz+Eghbali%2C+and+Frank+Tip%2E+2023%2E+Adaptive+Test+Generation+Using+a+Large+Language+Model%2E+arXiv%3A2302%2E06527+%5Bcs%2ESE%5D+%E2%80%94+%5BarXiv%3A2302%2E06527%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2302%2E06527%29)
84. Freda Shi, Xinyun Chen, Kanishka Misra, Nathan Scales, David Dohan, Ed H Chi, Nathanael Schärli, and Denny Zhou. 2023. [Large language models can be easily distracted by irrelevant context](https://md.research.elamur.ai/free/en/papers/shi-2023-large-language-models-easily-distracted/). In International Conference on Machine Learning. PMLR, 31210–31227.
85. Amit Singhal et al. 2001. Modern information retrieval: A brief overview. IEEE Data Eng. Bull. 24, 4 (2001), 35–43. [find](https://scholar.google.com/scholar?q=Amit+Singhal+et+al%2E+2001%2E+Modern+information+retrieval%3A+A+brief+overview%2E+IEEE+Data+Eng%2E+Bull%2E+24%2C+4+%282001%29%2C+35%E2%80%9343%2E)
86. Jeongju Sohn and Shin Yoo. 2017. Fluccs: Using code and change metrics to improve fault localization. In Proceedings of the 26th ACM SIGSOFT International Symposium on Software Testing and Analysis. 273–283. [find](https://scholar.google.com/scholar?q=Jeongju+Sohn+and+Shin+Yoo%2E+2017%2E+Fluccs%3A+Using+code+and+change+metrics+to+improve+fault+localization%2E+In+Proceedings+of+the+26th+ACM+SIGSOFT+International+Symposium+on+Software+Testing+and+Analysis%2E+2)
87. Qianqian Wang, Chris Parnin, and Alessandro Orso. 2015. Evaluating the usefulness of ir-based fault localization techniques. In Proceedings of the 2015 international symposium on software testing and analysis. 1–11. [find](https://scholar.google.com/scholar?q=Qianqian+Wang%2C+Chris+Parnin%2C+and+Alessandro+Orso%2E+2015%2E+Evaluating+the+usefulness+of+ir%2Dbased+fault+localization+techniques%2E+In+Proceedings+of+the+2015+international+symposium+on+software+testing+and+)
88. Shaowei Wang and David Lo. 2014. Version history, similar report, and structure: Putting them together for improved bug localization. In Proceedings of the 22nd international conference on program comprehension. 53–63. [find](https://scholar.google.com/scholar?q=Shaowei+Wang+and+David+Lo%2E+2014%2E+Version+history%2C+similar+report%2C+and+structure%3A+Putting+them+together+for+improved+bug+localization%2E+In+Proceedings+of+the+22nd+international+conference+on+program+com)
89. Yuxiang Wei, Zhe Wang, Jiawei Liu, Yifeng Ding, and Lingming Zhang. 2023. Magicoder: Source Code Is All You Need. arXiv preprint arXiv:2312.02120 (2023). [find](https://scholar.google.com/scholar?q=Yuxiang+Wei%2C+Zhe+Wang%2C+Jiawei+Liu%2C+Yifeng+Ding%2C+and+Lingming+Zhang%2E+2023%2E+Magicoder%3A+Source+Code+Is+All+You+Need%2E+arXiv+preprint+arXiv%3A2312%2E02120+%282023%29%2E)
90. W Eric Wong, Ruizhi Gao, Yihao Li, Rui Abreu, and Franz Wotawa. 2016. A survey on software fault localization. IEEE Transactions on Software Engineering 42, 8 (2016), 707–740. [find](https://scholar.google.com/scholar?q=W+Eric+Wong%2C+Ruizhi+Gao%2C+Yihao+Li%2C+Rui+Abreu%2C+and+Franz+Wotawa%2E+2016%2E+A+survey+on+software+fault+localization%2E+IEEE+Transactions+on+Software+Engineering+42%2C+8+%282016%29%2C+707%E2%80%93740%2E)
91. Yonghao Wu, Zheng Li, Jie M Zhang, Mike Papadakis, Mark Harman, and Yong Liu. 2023. Large language models in fault localisation. arXiv preprint arXiv:2308.15276 (2023). — [arXiv:2308.15276](https://arxiv.org/abs/2308.15276) [find](https://scholar.google.com/scholar?q=Yonghao+Wu%2C+Zheng+Li%2C+Jie+M+Zhang%2C+Mike+Papadakis%2C+Mark+Harman%2C+and+Yong+Liu%2E+2023%2E+Large+language+models+in+fault+localisation%2E+arXiv+preprint+arXiv%3A2308%2E15276+%282023%29%2E+%E2%80%94+%5BarXiv%3A2308%2E15276%5D%28https%3A%2F%2Far)
92. Zhiheng Xi, Wenxiang Chen, Xin Guo, Wei He, Yiwen Ding, Boyang Hong, Ming Zhang, Junzhe Wang, Senjie Jin, Enyu Zhou, et al. 2023. [The rise and potential of large language model based agents: A survey](https://md.research.elamur.ai/free/en/papers/xi-2023-rise-potential-large-language/). arXiv preprint arXiv:2309.07864 (2023).
93. Chunqiu Steven Xia, Matteo Paltenghi, Jia Le Tian, Michael Pradel, and Lingming Zhang. 2024. Universal Fuzzing via Large Language Models. In 46th International Conference on Software Engineering (ICSE). [find](https://scholar.google.com/scholar?q=Chunqiu+Steven+Xia%2C+Matteo+Paltenghi%2C+Jia+Le+Tian%2C+Michael+Pradel%2C+and+Lingming+Zhang%2E+2024%2E+Universal+Fuzzing+via+Large+Language+Models%2E+In+46th+International+Conference+on+Software+Engineering+%28ICSE)
94. Chunqiu Steven Xia, Yuxiang Wei, and Lingming Zhang. 2023. Automated Program Repair in the Era of Large Pre-trained Language Models. In Proceedings of the ACM/IEEE 45th International Conference on Software Engineering (ICSE ’23). [find](https://scholar.google.com/scholar?q=Chunqiu+Steven+Xia%2C+Yuxiang+Wei%2C+and+Lingming+Zhang%2E+2023%2E+Automated+Program+Repair+in+the+Era+of+Large+Pre%2Dtrained+Language+Models%2E+In+Proceedings+of+the+ACM%2FIEEE+45th+International+Conference+on+Sof)
95. Chunqiu Steven Xia, Yuxiang Wei, and Lingming Zhang. 2023. Automated program repair in the era of large pre-trained language models. In Proceedings of the 45th International Conference on Software Engineering (ICSE 2023). Association for Computing Machinery. [find](https://scholar.google.com/scholar?q=Chunqiu+Steven+Xia%2C+Yuxiang+Wei%2C+and+Lingming+Zhang%2E+2023%2E+Automated+program+repair+in+the+era+of+large+pre%2Dtrained+language+models%2E+In+Proceedings+of+the+45th+International+Conference+on+Software+Eng)
96. Chunqiu Steven Xia and Lingming Zhang. 2022. Less training, more repairing please: revisiting automated program repair via zero-shot learning. In Proceedings of the 30th ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering. 959–971. [find](https://scholar.google.com/scholar?q=Chunqiu+Steven+Xia+and+Lingming+Zhang%2E+2022%2E+Less+training%2C+more+repairing+please%3A+revisiting+automated+program+repair+via+zero%2Dshot+learning%2E+In+Proceedings+of+the+30th+ACM+Joint+European+Software+En)
97. Chunqiu Steven Xia and Lingming Zhang. 2023. Keep the Conversation Going: Fixing 162 out of 337 bugs for $0.42 each using ChatGPT. arXiv preprint arXiv:2304.00385 (2023). — [arXiv:2304.00385](https://arxiv.org/abs/2304.00385) [find](https://scholar.google.com/scholar?q=Chunqiu+Steven+Xia+and+Lingming+Zhang%2E+2023%2E+Keep+the+Conversation+Going%3A+Fixing+162+out+of+337+bugs+for+%240%2E42+each+using+ChatGPT%2E+arXiv+preprint+arXiv%3A2304%2E00385+%282023%29%2E+%E2%80%94+%5BarXiv%3A2304%2E00385%5D%28https%3A%2F%2F)
98. Aidan ZH Yang, Claire Le Goues, Ruben Martins, and Vincent Hellendoorn. 2024. Large language models for test-free fault localization. In Proceedings of the 46th IEEE/ACM International Conference on Software Engineering. 1–12. [find](https://scholar.google.com/scholar?q=Aidan+ZH+Yang%2C+Claire+Le+Goues%2C+Ruben+Martins%2C+and+Vincent+Hellendoorn%2E+2024%2E+Large+language+models+for+test%2Dfree+fault+localization%2E+In+Proceedings+of+the+46th+IEEE%2FACM+International+Conference+on+So)
99. Chenyuan Yang, Yinlin Deng, Runyu Lu, Jiayi Yao, Jiawei Liu, Reyhaneh Jabbarvand, and Lingming Zhang. 2023. White-box Compiler Fuzzing Empowered by Large Language Models. arXiv:2310.15991 [cs.SE] — [arXiv:2310.15991](https://arxiv.org/abs/2310.15991) [find](https://scholar.google.com/scholar?q=Chenyuan+Yang%2C+Yinlin+Deng%2C+Runyu+Lu%2C+Jiayi+Yao%2C+Jiawei+Liu%2C+Reyhaneh+Jabbarvand%2C+and+Lingming+Zhang%2E+2023%2E+White%2Dbox+Compiler+Fuzzing+Empowered+by+Large+Language+Models%2E+arXiv%3A2310%2E15991+%5Bcs%2ESE%5D+%E2%80%94+%5Ba)
100. Chenyuan Yang, Zijie Zhao, and Lingming Zhang. 2023. Kernelgpt: Enhanced kernel fuzzing via large language models. arXiv preprint arXiv:2401.00563 (2023). — [arXiv:2401.00563](https://arxiv.org/abs/2401.00563) [find](https://scholar.google.com/scholar?q=Chenyuan+Yang%2C+Zijie+Zhao%2C+and+Lingming+Zhang%2E+2023%2E+Kernelgpt%3A+Enhanced+kernel+fuzzing+via+large+language+models%2E+arXiv+preprint+arXiv%3A2401%2E00563+%282023%29%2E+%E2%80%94+%5BarXiv%3A2401%2E00563%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F24)
101. John Yang, Carlos E Jimenez, Alexander Wettig, Kilian Lieret, Shunyu Yao, Karthik Narasimhan, and Ofir Press. 2024. [Swe-agent: Agent-computer interfaces enable automated software engineering](https://md.research.elamur.ai/free/en/papers/yang-2024-swe-agent-agent-computer-interfaces/). arXiv preprint arXiv:2405.15793 (2024).
102. Zhiqiang Yuan, Mingwei Liu, Shiji Ding, Kaixin Wang, Yixuan Chen, Xin Peng, and Yiling Lou. 2024. Evaluating and improving chatgpt for unit test generation. Proceedings of the ACM on Software Engineering 1, FSE (2024), 1703–1726. [find](https://scholar.google.com/scholar?q=Zhiqiang+Yuan%2C+Mingwei+Liu%2C+Shiji+Ding%2C+Kaixin+Wang%2C+Yixuan+Chen%2C+Xin+Peng%2C+and+Yiling+Lou%2E+2024%2E+Evaluating+and+improving+chatgpt+for+unit+test+generation%2E+Proceedings+of+the+ACM+on+Software+Engineer)
103. Andreas Zeller, Rahul Gopinath, Marcel Böhme, Gordon Fraser, and Christian Holler. 2019. The fuzzing book. [find](https://scholar.google.com/scholar?q=Andreas+Zeller%2C+Rahul+Gopinath%2C+Marcel+Bo%CC%88hme%2C+Gordon+Fraser%2C+and+Christian+Holler%2E+2019%2E+The+fuzzing+book%2E)
104. Kexun Zhang, Weiran Yao, Zuxin Liu, Yihao Feng, Zhiwei Liu, Rithesh Murthy, Tian Lan, Lei Li, Renze Lou, Jiacheng Xu, et al. 2024. Diversity Empowers Intelligence: Integrating Expertise of Software Engineering Agents. arXiv preprint arXiv:2408.07060 (2024). — [arXiv:2408.07060](https://arxiv.org/abs/2408.07060) [find](https://scholar.google.com/scholar?q=Kexun+Zhang%2C+Weiran+Yao%2C+Zuxin+Liu%2C+Yihao+Feng%2C+Zhiwei+Liu%2C+Rithesh+Murthy%2C+Tian+Lan%2C+Lei+Li%2C+Renze+Lou%2C+Jiacheng+Xu%2C+et+al%2E+2024%2E+Diversity+Empowers+Intelligence%3A+Integrating+Expertise+of+Software+En)
105. Lyuye Zhang, Kaixuan Li, Kairan Sun, Daoyuan Wu, Ye Liu, Haoye Tian, and Yang Liu. 2024. Acfix: Guiding llms with mined common rbac practices for context-aware repair of access control vulnerabilities in smart contracts. arXiv preprint arXiv:2403.06838 (2024). — [arXiv:2403.06838](https://arxiv.org/abs/2403.06838) [find](https://scholar.google.com/scholar?q=Lyuye+Zhang%2C+Kaixuan+Li%2C+Kairan+Sun%2C+Daoyuan+Wu%2C+Ye+Liu%2C+Haoye+Tian%2C+and+Yang+Liu%2E+2024%2E+Acfix%3A+Guiding+llms+with+mined+common+rbac+practices+for+context%2Daware+repair+of+access+control+vulnerabilities)
106. Quanjun Zhang, Chunrong Fang, Yang Xie, YuXiang Ma, Weisong Sun, and Yun Yang Zhenyu Chen. 2024. A Systematic Literature Review on Large Language Models for Automated Program Repair. arXiv preprint arXiv:2405.01466 (2024). — [arXiv:2405.01466](https://arxiv.org/abs/2405.01466) [find](https://scholar.google.com/scholar?q=Quanjun+Zhang%2C+Chunrong+Fang%2C+Yang+Xie%2C+YuXiang+Ma%2C+Weisong+Sun%2C+and+Yun+Yang+Zhenyu+Chen%2E+2024%2E+A+Systematic+Literature+Review+on+Large+Language+Models+for+Automated+Program+Repair%2E+arXiv+preprint+ar)
107. Yue Zhang, Yafu Li, Leyang Cui, Deng Cai, Lemao Liu, Tingchen Fu, Xinting Huang, Enbo Zhao, Yu Zhang, Yulong Chen, et al. 2023. [Siren’s song in the AI ocean: a survey on hallucination in large language models](https://md.research.elamur.ai/free/en/papers/zhang-2023-siren-s-song-ai-ocean/). arXiv preprint arXiv:2309.01219 (2023).
108. Yuntong Zhang, Haifeng Ruan, Zhiyu Fan, and Abhik Roychoudhury. 2024. AutoCodeRover: Autonomous Program Improvement. arXiv:2404.05427 [cs.SE] — [arXiv:2404.05427](https://arxiv.org/abs/2404.05427) [find](https://scholar.google.com/scholar?q=Yuntong+Zhang%2C+Haifeng+Ruan%2C+Zhiyu+Fan%2C+and+Abhik+Roychoudhury%2E+2024%2E+AutoCodeRover%3A+Autonomous+Program+Improvement%2E+arXiv%3A2404%2E05427+%5Bcs%2ESE%5D+%E2%80%94+%5BarXiv%3A2404%2E05427%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2404%2E05427%29)
109. Jian Zhou, Hongyu Zhang, and David Lo. 2012. Where should the bugs be fixed? more accurate information retrieval-based bug localization based on bug reports. In 2012 34th International conference on software engineering (ICSE). IEEE, 14–24. [find](https://scholar.google.com/scholar?q=Jian+Zhou%2C+Hongyu+Zhang%2C+and+David+Lo%2E+2012%2E+Where+should+the+bugs+be+fixed%3F+more+accurate+information+retrieval%2Dbased+bug+localization+based+on+bug+reports%2E+In+2012+34th+International+conference+on+s)
110. Qihao Zhu, Zeyu Sun, Yuan-an Xiao, Wenjie Zhang, Kang Yuan, Yingfei Xiong, and Lu Zhang. 2021. A Syntax-Guided Edit Decoder for Neural Program Repair. In Proceedings of the 29th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering. AC… [find](https://scholar.google.com/scholar?q=Qihao+Zhu%2C+Zeyu+Sun%2C+Yuan%2Dan+Xiao%2C+Wenjie+Zhang%2C+Kang+Yuan%2C+Yingfei+Xiong%2C+and+Lu+Zhang%2E+2021%2E+A+Syntax%2DGuided+Edit+Decoder+for+Neural+Program+Repair%2E+In+Proceedings+of+the+29th+ACM+Joint+Meeting+on+E)

*Updated 2 November 2026*