---
title: "When Can LLMs Actually Correct Their Own — Kamoi 2024"
description: "Studies disagree on whether large language models can fix their own errors at inference time."
authors: ["Ryo Kamoi", "Yusen Zhang", "Nan Zhang", "Jiawei Han", "Rui Zhang"]
year: 2024
venue: "Transactions of the Association for Computational Linguistics"
source: "https://aclanthology.org/2024.tacl-1.78/"
doi: "10.1162/tacl_a_00713"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/kamoi-2024-when-can-llms-actually-correct/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/kamoi-2024-when-can-llms-actually-correct/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2024](https://md.research.elamur.ai/free/en/papers/2024/page/5/) / When Can LLMs Actually Correct Their Own Mistakes? A Critical Survey of Self-Correction of LLMs

# When Can LLMs Actually Correct Their Own Mistakes? A Critical Survey of Self-Correction of LLMs

**Authors:** Ryo Kamoi, Yusen Zhang, Nan Zhang, [Jiawei Han](https://md.research.elamur.ai/free/en/authors/jiawei-han/), [Rui Zhang](https://md.research.elamur.ai/free/en/authors/rui-zhang/) · **Year:** 2024 · **Venue:** Transactions of the Association for Computational Linguistics · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2024.tacl-1.78/) · [PDF](https://aclanthology.org/2024.tacl-1.78.pdf) · [DOI 10.1162/tacl\_a\_00713](https://doi.org/10.1162/tacl_a_00713) · [OpenAlex](https://openalex.org/W4404131920)

**Citations:** 84 citations (openalex-snapshot · 2026-10-14) · [75](#cited-by) in our library, of the 140884 papers whose bibliographies we read

## Summary
Studies disagree on whether large language models can fix their own errors at inference time. Some report gains from self-correction, while others find that models fail to detect or repair their mistakes. Kamoi et al. review a wide range of self-correction work, covering intrinsic prompting, external tools or knowledge, and fine-tuned feedback models. Their aim is to pin down when the approach actually helps. They argue that many papers leave their research question vague and rely on unrealistic setups, for example oracle ground-truth signals or deliberately weak initial prompts, which inflate the apparent benefit. To fix this, the authors sort the field into three research questions, state which experimental requirements each one demands, and propose a checklist for experiment design and for reporting negative results. Re-examined under this framework, the evidence points to feedback generation as the main bottleneck. Self-correction succeeds when trustworthy external feedback exists or when large-scale fine-tuning is used. Prompting alone helps only in unusually favorable tasks. The survey matters for anyone building or evaluating LLM refinement pipelines.

## Key points
- The authors split self-correction into three stages: initial response, feedback, and refinement. They then classify frameworks along three axes: feedback source (intrinsic, external information, fine-tuning), post-hoc vs. generation-time correction, and same-model vs. cross-model correction.
- They define three research questions:
  - **RQ1:** self-correction using only the model's own abilities.
  - **RQ2:** correcting best-possible initial responses with external information.
  - **RQ3:** whether self-correction outperforms other methods.
- No reviewed study shows reliable intrinsic self-correction on general tasks. Huang et al. (2024a) report negative results with GPT-4-T and GPT-3.5-T on GSM8k, CSQA and HotpotQA. Positive intrinsic cases involve exceptionally suitable tasks, such as those with decomposable responses (e.g., CoVe).
- Frameworks with dependable external signals work well, including Self-Debug (code interpreter), CRITIC (interpreter, web search) and Baldur (proof assistant). Fine-tuned approaches such as SelFee, REFINER and RL4F also succeed, but they need large training data.
- Self-correction is rarely compared against strong baselines. Suggested future directions include better feedback via LLM-based evaluation metrics, fine-tuning on small data, and pre-training aimed at self-correction (TACL vol. 12, pp. 1417–1440).

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

Fields: [Natural language processing](https://md.research.elamur.ai/free/en/shelves/nlp/)

## Cited by

Cited by

75 papers of our library cite it (of the 140884 whose bibliographies we read); 3 of them have a page here, 72 not yet.

- [Do NOT Think That Much for 2+3=? On the Overthinking of o1-Like LLMs](https://md.research.elamur.ai/free/en/papers/chen-2024-think-that-much-2-3/)
- [Training Language Models to Self-Correct via Reinforcement Learning](https://md.research.elamur.ai/free/en/papers/kumar-2024-training-language-models-self-correct/)
- [ProcessBench: Identifying Process Errors in Mathematical Reasoning](https://md.research.elamur.ai/free/en/papers/zheng-2025-processbench-identifying-process-errors-mathematical/)

## Cite

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@article{kamoi2024when,
  title = {{When Can LLMs Actually Correct Their Own Mistakes? A Critical Survey of Self-Correction of LLMs}},
  author = {Kamoi, Ryo and Zhang, Yusen and Zhang, Nan and Han, Jiawei and Zhang, Rui},
  journal = {Transactions of the Association for Computational Linguistics},
  year = {2024},
  doi = {10.1162/tacl\_a\_00713},
  url = {https://doi.org/10.1162/tacl_a_00713}
}
```

- [← previous What to Teach: Bilingual Arabic Teachers' Beliefs and Stances about Pedagogical Translanguaging and Transdialecting](https://md.research.elamur.ai/free/en/papers/deiri-2024-teach-bilingual-arabic-teachers-beliefs/)
- [next → WildChat: 1M ChatGPT Interaction Logs in the Wild](https://md.research.elamur.ai/free/en/papers/zhao-2024-wildchat-1m-chatgpt-interaction-logs/)
- [↑ back to papers of 2024](https://md.research.elamur.ai/free/en/papers/2024/page/5/)

## References

122 references, 44 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Afra Feyza Akyurek, Ekin Akyurek, Ashwin Kalyan, Peter Clark, Derry Tanti Wijaya, and Niket Tandon. 2023. RL4F: Generating natural language feedback with reinforcement learning for repairing model outputs. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Vo… [find](https://scholar.google.com/scholar?q=Afra+Feyza+Akyurek%2C+Ekin+Akyurek%2C+Ashwin+Kalyan%2C+Peter+Clark%2C+Derry+Tanti+Wijaya%2C+and+Niket+Tandon%2E+2023%2E+RL4F%3A+Generating+natural+language+feedback+with+reinforcement+learning+for+repairing+model+out)
2. Akari Asai, Zeqiu Wu, Yizhong Wang, Avirup Sil, and Hannaneh Hajishirzi. 2024. Self-RAG: Learning to retrieve, generate, and critique through self-reflection. In The Twelfth International Conference on Learning Representations. [find](https://scholar.google.com/scholar?q=Akari+Asai%2C+Zeqiu+Wu%2C+Yizhong+Wang%2C+Avirup+Sil%2C+and+Hannaneh+Hajishirzi%2E+2024%2E+Self%2DRAG%3A+Learning+to+retrieve%2C+generate%2C+and+critique+through+self%2Dreflection%2E+In+The+Twelfth+International+Conference+o) [our card](https://md.research.elamur.ai/free/en/papers/asai-2023-self-rag-learning-retrieve-generate/)
3. Johannes Bader, Andrew Scott, Michael Pradel, and Satish Chandra. 2019. Getafix: Learning to fix bugs automatically. Proceedings of the ACM on Programming Languages, 3(OOPSLA). https://doi.org/10.1145/3360585 [↗](https://doi.org/10.1145/3360585)
4. Yuntao Bai, Saurav Kadavath, Sandipan Kundu, Amanda Askell, Jackson Kernion, Andy Jones, Anna Chen, Anna Goldie, Azalia Mirhoseini, Cameron McKinnon, Carol Chen, Catherine Olsson, Christopher Olah, Danny Hernandez, Dawn Drain, Deep Ganguli, Dustin Li, Eli Tran-Johnson, Ethan Perez, Jamie Kerr, Jared… [find](https://scholar.google.com/scholar?q=Yuntao+Bai%2C+Saurav+Kadavath%2C+Sandipan+Kundu%2C+Amanda+Askell%2C+Jackson+Kernion%2C+Andy+Jones%2C+Anna+Chen%2C+Anna+Goldie%2C+Azalia+Mirhoseini%2C+Cameron+McKinnon%2C+Carol+Chen%2C+Catherine+Olsson%2C+Christopher+Olah%2C+Da) [our card](https://md.research.elamur.ai/free/en/papers/blodgett-2020-language-technology-is-power-critical/)
5. Meng Cao, Yue Dong, Jiapeng Wu, and Jackie Chi Kit Cheung. 2020. Factual error correction for abstractive summarization models. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 6251–6258, Online. Association for Computational Linguistics. https… [find](https://scholar.google.com/scholar?q=Meng+Cao%2C+Yue+Dong%2C+Jiapeng+Wu%2C+and+Jackie+Chi+Kit+Cheung%2E+2020%2E+Factual+error+correction+for+abstractive+summarization+models%2E+In+Proceedings+of+the+2020+Conference+on+Empirical+Methods+in+Natural+La) [our card](https://md.research.elamur.ai/free/en/papers/cao-2020-factual-error-correction-abstractive-summarization/)
6. Chi-Min Chan, Weize Chen, Yusheng Su, Jianxuan Yu, Wei Xue, Shanghang Zhang, Jie Fu, and Zhiyuan Liu. 2024. Chateval: Towards better LLM-based evaluators through multi-agent debate. In The Twelfth International Conference on Learning Representations. [find](https://scholar.google.com/scholar?q=Chi%2DMin+Chan%2C+Weize+Chen%2C+Yusheng+Su%2C+Jianxuan+Yu%2C+Wei+Xue%2C+Shanghang+Zhang%2C+Jie+Fu%2C+and+Zhiyuan+Liu%2E+2024%2E+Chateval%3A+Towards+better+LLM%2Dbased+evaluators+through+multi%2Dagent+debate%2E+In+The+Twelfth+Int) [our card](https://md.research.elamur.ai/free/en/papers/chan-2023-chateval-better-llm-based/)
7. Yiannis Charalambous, Norbert Tihanyi, Ridhi Jain, Youcheng Sun, Mohamed Amine Ferrag, and Lucas C. Cordeiro. 2023. A new era in software security: Towards self-healing software via large language models and formal verification. arXiv preprint arXiv:2305.14752. — [arXiv:2305.14752](https://arxiv.org/abs/2305.14752) [find](https://scholar.google.com/scholar?q=Yiannis+Charalambous%2C+Norbert+Tihanyi%2C+Ridhi+Jain%2C+Youcheng+Sun%2C+Mohamed+Amine+Ferrag%2C+and+Lucas+C%2E+Cordeiro%2E+2023%2E+A+new+era+in+software+security%3A+Towards+self%2Dhealing+software+via+large+language+mod)
8. Angelica Chen, Jérémy Scheurer, Jon Ander Campos, Tomasz Korbak, Jun Shern Chan, Samuel R. Bowman, Kyunghyun Cho, and Ethan Perez. 2024a. Learning from natural language feedback. Transactions on Machine Learning Research. [find](https://scholar.google.com/scholar?q=Angelica+Chen%2C+Je%CC%81re%CC%81my+Scheurer%2C+Jon+Ander+Campos%2C+Tomasz+Korbak%2C+Jun+Shern+Chan%2C+Samuel+R%2E+Bowman%2C+Kyunghyun+Cho%2C+and+Ethan+Perez%2E+2024a%2E+Learning+from+natural+language+feedback%2E+Transactions+on+Mac)
9. Bei Chen, Fengji Zhang, Anh Nguyen, Daoguang Zan, Zeqi Lin, Jian-Guang Lou, and Weizhu Chen. 2023a. Codet: Code generation with generated tests. In The Eleventh International Conference on Learning Representations. [find](https://scholar.google.com/scholar?q=Bei+Chen%2C+Fengji+Zhang%2C+Anh+Nguyen%2C+Daoguang+Zan%2C+Zeqi+Lin%2C+Jian%2DGuang+Lou%2C+and+Weizhu+Chen%2E+2023a%2E+Codet%3A+Code+generation+with+generated+tests%2E+In+The+Eleventh+International+Conference+on+Learning+Re)
10. Canyu Chen and Kai Shu. 2024. Can LLM-generated misinformation be detected? In The Twelfth International Conference on Learning Representations. [find](https://scholar.google.com/scholar?q=Canyu+Chen+and+Kai+Shu%2E+2024%2E+Can+LLM%2Dgenerated+misinformation+be+detected%3F+In+The+Twelfth+International+Conference+on+Learning+Representations%2E)
11. Justin Chih-Yao Chen, Swarnadeep Saha, and Mohit Bansal. 2024b. Reconcile: Round-table conference improves reasoning via consensus among diverse LLMs. arXiv preprint arXiv: 2309.13007. [find](https://scholar.google.com/scholar?q=Justin+Chih%2DYao+Chen%2C+Swarnadeep+Saha%2C+and+Mohit+Bansal%2E+2024b%2E+Reconcile%3A+Round%2Dtable+conference+improves+reasoning+via+consensus+among+diverse+LLMs%2E+arXiv+preprint+arXiv%3A+2309%2E13007%2E) [our card](https://md.research.elamur.ai/free/en/papers/chen-2024-reconcile-round-table-conference-improves/)
12. Pinzhen Chen, Zhicheng Guo, Barry Haddow, and Kenneth Heafield. 2023b. Iterative translation refinement with large language models. arXiv preprint arXiv:2306.03856. [find](https://scholar.google.com/scholar?q=Pinzhen+Chen%2C+Zhicheng+Guo%2C+Barry+Haddow%2C+and+Kenneth+Heafield%2E+2023b%2E+Iterative+translation+refinement+with+large+language+models%2E+arXiv+preprint+arXiv%3A2306%2E03856%2E) [our card](https://md.research.elamur.ai/free/en/papers/chen-2024-iterative-translation-refinement-large-language/)
13. Xinyun Chen, Renat Aksitov, Uri Alon, Jie Ren, Kefan Xiao, Pengcheng Yin, Sushant Prakash, Charles Sutton, Xuezhi Wang, and Denny Zhou. 2024c. Universal self-consistency for large language models. In ICML 2024 Workshop on In-Context Learning. [find](https://scholar.google.com/scholar?q=Xinyun+Chen%2C+Renat+Aksitov%2C+Uri+Alon%2C+Jie+Ren%2C+Kefan+Xiao%2C+Pengcheng+Yin%2C+Sushant+Prakash%2C+Charles+Sutton%2C+Xuezhi+Wang%2C+and+Denny+Zhou%2E+2024c%2E+Universal+self%2Dconsistency+for+large+language+models%2E+In+)
14. Xinyun Chen, Maxwell Lin, Nathanael Schärli, and Denny Zhou. 2024d. Teaching large language models to self-debug. In The Twelfth International Conference on Learning Representations. [find](https://scholar.google.com/scholar?q=Xinyun+Chen%2C+Maxwell+Lin%2C+Nathanael+Scha%CC%88rli%2C+and+Denny+Zhou%2E+2024d%2E+Teaching+large+language+models+to+self%2Ddebug%2E+In+The+Twelfth+International+Conference+on+Learning+Representations%2E)
15. Zimin Chen, Steve Kommrusch, Michele Tufano, Louis-Noël Pouchet, Denys Poshyvanyk, and Martin Monperrus. 2021. Sequencer: Sequence-to-sequence learning for end-to-end program repair. IEEE Transactions on Software Engineering, 47(9):1943–1959. https://doi.org/10.1109/TSE.2019.2940179 [↗](https://doi.org/10.1109/TSE.2019.2940179)
16. I-Chun Chern, Steffi Chern, Shiqi Chen, Weizhe Yuan, Kehua Feng, Chunting Zhou, Junxian He, Graham Neubig, and Pengfei Liu. 2023. Factool: Factuality detection in generative AI – a tool augmented framework for multi-task and multi-domain scenarios. arXiv preprint arXiv:2307.13528. [find](https://scholar.google.com/scholar?q=I%2DChun+Chern%2C+Steffi+Chern%2C+Shiqi+Chen%2C+Weizhe+Yuan%2C+Kehua+Feng%2C+Chunting+Zhou%2C+Junxian+He%2C+Graham+Neubig%2C+and+Pengfei+Liu%2E+2023%2E+Factool%3A+Factuality+detection+in+generative+AI+%E2%80%93+a+tool+augmented+fram) [our card](https://md.research.elamur.ai/free/en/papers/chern-2023-factool-factuality-detection-generative-ai/)
17. Cheng-Han Chiang and Hung-yi Lee. 2023. Can large language models be an alternative to human evaluations? In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 15607–15631, Toronto, Canada. Association for Computational Linguistics.… [find](https://scholar.google.com/scholar?q=Cheng%2DHan+Chiang+and+Hung%2Dyi+Lee%2E+2023%2E+Can+large+language+models+be+an+alternative+to+human+evaluations%3F+In+Proceedings+of+the+61st+Annual+Meeting+of+the+Association+for+Computational+Linguistics+%28Vo)
18. Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, Christopher Hesse, and John Schulman. 2021. Training verifiers to solve math word problems. arXiv preprint arXiv:2110.14168. [find](https://scholar.google.com/scholar?q=Karl+Cobbe%2C+Vineet+Kosaraju%2C+Mohammad+Bavarian%2C+Mark+Chen%2C+Heewoo+Jun%2C+Lukasz+Kaiser%2C+Matthias+Plappert%2C+Jerry+Tworek%2C+Jacob+Hilton%2C+Reiichiro+Nakano%2C+Christopher+Hesse%2C+and+John+Schulman%2E+2021%2E+Train) [our card](https://md.research.elamur.ai/free/en/papers/cobbe-2021-training-verifiers-solve-math-word/)
19. Roi Cohen, May Hamri, Mor Geva, and Amir Globerson. 2023. LM vs LM: Detecting factual errors via cross examination. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 12621–12640, Singapore. Association for Computational Linguistics. https://doi .org/10.… [find](https://scholar.google.com/scholar?q=Roi+Cohen%2C+May+Hamri%2C+Mor+Geva%2C+and+Amir+Globerson%2E+2023%2E+LM+vs+LM%3A+Detecting+factual+errors+via+cross+examination%2E+In+Proceedings+of+the+2023+Conference+on+Empirical+Methods+in+Natural+Language+Proce)
20. Antonia Creswell and Murray Shanahan. 2022. Faithful reasoning using large language models. arXiv preprint arXiv:2208.14271. [find](https://scholar.google.com/scholar?q=Antonia+Creswell+and+Murray+Shanahan%2E+2022%2E+Faithful+reasoning+using+large+language+models%2E+arXiv+preprint+arXiv%3A2208%2E14271%2E)
21. Shehzaad Dhuliawala, Mojtaba Komeili, Jing Xu, Roberta Raileanu, Xian Li, Asli Celikyilmaz, and Jason Weston. 2023. Chain-of-verification reduces hallucination in large language models. arXiv preprint arXiv:2309.11495. [find](https://scholar.google.com/scholar?q=Shehzaad+Dhuliawala%2C+Mojtaba+Komeili%2C+Jing+Xu%2C+Roberta+Raileanu%2C+Xian+Li%2C+Asli+Celikyilmaz%2C+and+Jason+Weston%2E+2023%2E+Chain%2Dof%2Dverification+reduces+hallucination+in+large+language+models%2E+arXiv+preprint) [our card](https://md.research.elamur.ai/free/en/papers/dhuliawala-2023-chain-verification-reduces-hallucination-large/)
22. Yilun Du, Shuang Li, Antonio Torralba, Joshua B. Tenenbaum, and Igor Mordatch. 2023. Improving factuality and reasoning in language models through multiagent debate. arXiv preprint arXiv:2305.14325. [find](https://scholar.google.com/scholar?q=Yilun+Du%2C+Shuang+Li%2C+Antonio+Torralba%2C+Joshua+B%2E+Tenenbaum%2C+and+Igor+Mordatch%2E+2023%2E+Improving+factuality+and+reasoning+in+language+models+through+multiagent+debate%2E+arXiv+preprint+arXiv%3A2305%2E14325%2E) [our card](https://md.research.elamur.ai/free/en/papers/du-2023-improving-factuality-reasoning-language-models/)
23. Esin Durmus, He He, and Mona Diab. 2020. FEQA: A question answering evaluation framework for faithfulness assessment in abstractive summarization. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 5055–5070, Online. Association for Computational Lingui… [find](https://scholar.google.com/scholar?q=Esin+Durmus%2C+He+He%2C+and+Mona+Diab%2E+2020%2E+FEQA%3A+A+question+answering+evaluation+framework+for+faithfulness+assessment+in+abstractive+summarization%2E+In+Proceedings+of+the+58th+Annual+Meeting+of+the+Asso) [our card](https://md.research.elamur.ai/free/en/papers/durmus-2020-feqa-question-answering-evaluation-framework/)
24. Mohamed Elaraby, Mengyin Lu, Jacob Dunn, Xueying Zhang, Yu Wang, Shizhu Liu, Pingchuan Tian, Yuping Wang, and Yuxuan Wang. 2023. Halo: Estimation and reduction of hallucinations in open-source weak large language models. arXiv preprint arXiv:2308 .11764. [find](https://scholar.google.com/scholar?q=Mohamed+Elaraby%2C+Mengyin+Lu%2C+Jacob+Dunn%2C+Xueying+Zhang%2C+Yu+Wang%2C+Shizhu+Liu%2C+Pingchuan+Tian%2C+Yuping+Wang%2C+and+Yuxuan+Wang%2E+2023%2E+Halo%3A+Estimation+and+reduction+of+hallucinations+in+open%2Dsource+weak+la)
25. Shangbin Feng, Chan Young Park, Yuhan Liu, and Yulia Tsvetkov. 2023. From pretraining data to language models to downstream tasks: Tracking the trails of political biases leading to unfair NLP models. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume… [find](https://scholar.google.com/scholar?q=Shangbin+Feng%2C+Chan+Young+Park%2C+Yuhan+Liu%2C+and+Yulia+Tsvetkov%2E+2023%2E+From+pretraining+data+to+language+models+to+downstream+tasks%3A+Tracking+the+trails+of+political+biases+leading+to+unfair+NLP+models%2E) [our card](https://md.research.elamur.ai/free/en/papers/feng-2023-pretraining-data-language-models-downstream/)
26. Chen. 2024. CRITIC: Large language models can self-correct with tool-interactive critiquing. In The Twelfth International Conference on Learning Representations. https://doi.org/10.18653/v1/2023 .acl-long.656 [↗](https://doi.org/10.18653/v1/2023) [our card](https://md.research.elamur.ai/free/en/papers/gou-2023-critic-large-language-models-self/)
27. Emily First, Markus Rabe, Talia Ringer, and Yuriy Brun. 2023. Baldur: Whole-proof generation and repair with large language models. In Proceedings of the 31st ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering, ESEC/FSE 2023, pages 1229–1241,… [find](https://scholar.google.com/scholar?q=Emily+First%2C+Markus+Rabe%2C+Talia+Ringer%2C+and+Yuriy+Brun%2E+2023%2E+Baldur%3A+Whole%2Dproof+generation+and+repair+with+large+language+models%2E+In+Proceedings+of+the+31st+ACM+Joint+European+Software+Engineering+C)
28. Jinlan Fu, See-Kiong Ng, Zhengbao Jiang, and Pengfei Liu. 2024. GPTScore: Evaluate as you desire. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), pages 6556–6576, Mexico City, M… [find](https://scholar.google.com/scholar?q=Jinlan+Fu%2C+See%2DKiong+Ng%2C+Zhengbao+Jiang%2C+and+Pengfei+Liu%2E+2024%2E+GPTScore%3A+Evaluate+as+you+desire%2E+In+Proceedings+of+the+2024+Conference+of+the+North+American+Chapter+of+the+Association+for+Computation)
29. Luyu Gao, Zhuyun Dai, Panupong Pasupat, Anthony Chen, Arun Tejasvi Chaganty, Yicheng Fan, Vincent Zhao, Ni Lao, Hongrae Lee, Da-Cheng Juan, and Kelvin Guu. 2023. RARR: Researching and revising what language models say, using language models. In Proceedings of the 61st Annual Meeting of the Associati… [find](https://scholar.google.com/scholar?q=Luyu+Gao%2C+Zhuyun+Dai%2C+Panupong+Pasupat%2C+Anthony+Chen%2C+Arun+Tejasvi+Chaganty%2C+Yicheng+Fan%2C+Vincent+Zhao%2C+Ni+Lao%2C+Hongrae+Lee%2C+Da%2DCheng+Juan%2C+and+Kelvin+Guu%2E+2023%2E+RARR%3A+Researching+and+revising+what+la) [our card](https://md.research.elamur.ai/free/en/papers/gao-2023-rarr-researching-revising-what-language/)
30. Jiaxin Ge, Sanjay Subramanian, Trevor Darrell, and Boyi Li. 2023. From wrong to right: A recursive approach towards vision-language explanation. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 1173–1185, Singapore. Association for Computational Lingui… [find](https://scholar.google.com/scholar?q=Jiaxin+Ge%2C+Sanjay+Subramanian%2C+Trevor+Darrell%2C+and+Boyi+Li%2E+2023%2E+From+wrong+to+right%3A+A+recursive+approach+towards+vision%2Dlanguage+explanation%2E+In+Proceedings+of+the+2023+Conference+on+Empirical+Meth)
31. Zelalem Gero, Chandan Singh, Hao Cheng, Tristan Naumann, Michel Galley, Jianfeng Gao, and Hoifung Poon. 2023. Self-verification improves few-shot clinical information extraction. In ICML 3rd Workshop on Interpretable Machine Learning in Healthcare (IMLH). [find](https://scholar.google.com/scholar?q=Zelalem+Gero%2C+Chandan+Singh%2C+Hao+Cheng%2C+Tristan+Naumann%2C+Michel+Galley%2C+Jianfeng+Gao%2C+and+Hoifung+Poon%2E+2023%2E+Self%2Dverification+improves+few%2Dshot+clinical+information+extraction%2E+In+ICML+3rd+Workshop+)
32. Zhibin Gou, Zhihong Shao, Yeyun Gong, yelong shen, Yujiu Yang, Nan Duan, and Weizhu Caglar Gulcehre, Tom Le Paine, Srivatsan Srinivasan, Ksenia Konyushkova, Lotte Weerts, Abhishek Sharma, Aditya Siddhant, Alex Ahern, Miaosen Wang, Chenjie Gu, Wolfgang Macherey, Arnaud Doucet, Orhan Firat, and Nando… [find](https://scholar.google.com/scholar?q=Zhibin+Gou%2C+Zhihong+Shao%2C+Yeyun+Gong%2C+yelong+shen%2C+Yujiu+Yang%2C+Nan+Duan%2C+and+Weizhu+Caglar+Gulcehre%2C+Tom+Le+Paine%2C+Srivatsan+Srinivasan%2C+Ksenia+Konyushkova%2C+Lotte+Weerts%2C+Abhishek+Sharma%2C+Aditya+Siddh) [our card](https://md.research.elamur.ai/free/en/papers/gulcehre-2023-reinforced-self-training-rest-language/)
33. Rahul Gupta, Soham Pal, Aditya Kanade, and Shirish Shevade. 2017. Deepfix: Fixing common c language errors by deep learning. Proceedings of the AAAI Conference on Artificial Intelligence, 31(1). https://doi.org/10 .1609/aaai.v31i1.10742 [find](https://scholar.google.com/scholar?q=Rahul+Gupta%2C+Soham+Pal%2C+Aditya+Kanade%2C+and+Shirish+Shevade%2E+2017%2E+Deepfix%3A+Fixing+common+c+language+errors+by+deep+learning%2E+Proceedings+of+the+AAAI+Conference+on+Artificial+Intelligence%2C+31%281%29%2E+https)
34. Haixia Han, Jiaqing Liang, Jie Shi, Qianyu He, and Yanghua Xiao. 2024. Small language model can self-correct. Proceedings of the AAAI Conference on Artificial Intelligence, 38(16):18162–18170. https://doi.org/10 .1609/aaai.v38i16.29774 [find](https://scholar.google.com/scholar?q=Haixia+Han%2C+Jiaqing+Liang%2C+Jie+Shi%2C+Qianyu+He%2C+and+Yanghua+Xiao%2E+2024%2E+Small+language+model+can+self%2Dcorrect%2E+Proceedings+of+the+AAAI+Conference+on+Artificial+Intelligence%2C+38%2816%29%3A18162%E2%80%9318170%2E+https%3A%2F)
35. Shibo Hao, Yi Gu, Haodi Ma, Joshua Hong, Zhen Wang, Daisy Wang, and Zhiting Hu. 2023. Reasoning with language model is planning with world model. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 8154–8173, Singapore. Association for Computational Lingu… [find](https://scholar.google.com/scholar?q=Shibo+Hao%2C+Yi+Gu%2C+Haodi+Ma%2C+Joshua+Hong%2C+Zhen+Wang%2C+Daisy+Wang%2C+and+Zhiting+Hu%2E+2023%2E+Reasoning+with+language+model+is+planning+with+world+model%2E+In+Proceedings+of+the+2023+Conference+on+Empirical+Met)
36. Ruixin Hong, Hongming Zhang, Xinyu Pang, Dong Yu, and Changshui Zhang. 2024. A closer look at the self-verification abilities of large language models in logical reasoning. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Lan… [find](https://scholar.google.com/scholar?q=Ruixin+Hong%2C+Hongming+Zhang%2C+Xinyu+Pang%2C+Dong+Yu%2C+and+Changshui+Zhang%2E+2024%2E+A+closer+look+at+the+self%2Dverification+abilities+of+large+language+models+in+logical+reasoning%2E+In+Proceedings+of+the+2024+)
37. Jiaxin Huang, Shixiang Gu, Le Hou, Yuexin Wu, Xuezhi Wang, Hongkun Yu, and Jiawei Han. 2023. Large language models can self-improve. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 1051–1068, Singapore. Association for Computational Linguistics. https… [find](https://scholar.google.com/scholar?q=Jiaxin+Huang%2C+Shixiang+Gu%2C+Le+Hou%2C+Yuexin+Wu%2C+Xuezhi+Wang%2C+Hongkun+Yu%2C+and+Jiawei+Han%2E+2023%2E+Large+language+models+can+self%2Dimprove%2E+In+Proceedings+of+the+2023+Conference+on+Empirical+Methods+in+Natur) [our card](https://md.research.elamur.ai/free/en/papers/huang-2023-large-language-models-can-self/)
38. Jie Huang, Xinyun Chen, Swaroop Mishra, Huaixiu Steven Zheng, Adams Wei Yu, Xinying Song, and Denny Zhou. 2024a. Large language models cannot self-correct reasoning yet. In The Twelfth International Conference on Learning Representations. [find](https://scholar.google.com/scholar?q=Jie+Huang%2C+Xinyun+Chen%2C+Swaroop+Mishra%2C+Huaixiu+Steven+Zheng%2C+Adams+Wei+Yu%2C+Xinying+Song%2C+and+Denny+Zhou%2E+2024a%2E+Large+language+models+cannot+self%2Dcorrect+reasoning+yet%2E+In+The+Twelfth+International+C) [our card](https://md.research.elamur.ai/free/en/papers/huang-2023-large-language-models-cannot-self/)
39. Kung-Hsiang Huang, Mingyang Zhou, Hou Pong Chan, Yi Fung, Zhenhailong Wang, Lingyu Zhang, Shih-Fu Chang, and Heng Ji. 2024b. Do LVLMs understand charts? Analyzing and correcting factual errors in chart captioning. In Findings of the Association for Computational Linguistics ACL 2024, pages 730–749,… [find](https://scholar.google.com/scholar?q=Kung%2DHsiang+Huang%2C+Mingyang+Zhou%2C+Hou+Pong+Chan%2C+Yi+Fung%2C+Zhenhailong+Wang%2C+Lingyu+Zhang%2C+Shih%2DFu+Chang%2C+and+Heng+Ji%2E+2024b%2E+Do+LVLMs+understand+charts%3F+Analyzing+and+correcting+factual+errors+in+char)
40. Robert Iv, Alexandre Passos, Sameer Singh, and Ming-Wei Chang. 2022. FRUIT: Faithfully reflecting updated information in text. In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 3670–3686, Seattle,… [find](https://scholar.google.com/scholar?q=Robert+Iv%2C+Alexandre+Passos%2C+Sameer+Singh%2C+and+Ming%2DWei+Chang%2E+2022%2E+FRUIT%3A+Faithfully+reflecting+updated+information+in+text%2E+In+Proceedings+of+the+2022+Conference+of+the+North+American+Chapter+of+th)
41. Dongwei Jiang, Jingyu Zhang, Orion Weller, Nathaniel Weir, Benjamin Van Durme, and Daniel Khashabi. 2024. Self-[in]correct: Llms struggle with refining self-generated responses. arXiv preprint arXiv:2404.04298. — [arXiv:2404.04298](https://arxiv.org/abs/2404.04298) [find](https://scholar.google.com/scholar?q=Dongwei+Jiang%2C+Jingyu+Zhang%2C+Orion+Weller%2C+Nathaniel+Weir%2C+Benjamin+Van+Durme%2C+and+Daniel+Khashabi%2E+2024%2E+Self%2D%5Bin%5Dcorrect%3A+Llms+struggle+with+refining+self%2Dgenerated+responses%2E+arXiv+preprint+arXiv%3A2)
42. Shuyang Jiang, Yuhao Wang, and Yu Wang. 2023a. Selfevolve: A code evolution framework via large language models. arXiv preprint arXiv:2306.02907. — [arXiv:2306.02907](https://arxiv.org/abs/2306.02907) [find](https://scholar.google.com/scholar?q=Shuyang+Jiang%2C+Yuhao+Wang%2C+and+Yu+Wang%2E+2023a%2E+Selfevolve%3A+A+code+evolution+framework+via+large+language+models%2E+arXiv+preprint+arXiv%3A2306%2E02907%2E+%E2%80%94+%5BarXiv%3A2306%2E02907%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2306%2E02907%29)
43. Zhengbao Jiang, Frank Xu, Luyu Gao, Zhiqing Sun, Qian Liu, Jane Dwivedi-Yu, Yiming Yang, Jamie Callan, and Graham Neubig. 2023b. Active retrieval augmented generation. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 7969–7992, Singapore. Association f… [find](https://scholar.google.com/scholar?q=Zhengbao+Jiang%2C+Frank+Xu%2C+Luyu+Gao%2C+Zhiqing+Sun%2C+Qian+Liu%2C+Jane+Dwivedi%2DYu%2C+Yiming+Yang%2C+Jamie+Callan%2C+and+Graham+Neubig%2E+2023b%2E+Active+retrieval+augmented+generation%2E+In+Proceedings+of+the+2023+Confe)
44. Jaehun Jung, Lianhui Qin, Sean Welleck, Faeze Brahman, Chandra Bhagavatula, Ronan Le Bras, and Yejin Choi. 2022. Maieutic prompting: Logically consistent reasoning with recursive explanations. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 1266–1279,… [find](https://scholar.google.com/scholar?q=Jaehun+Jung%2C+Lianhui+Qin%2C+Sean+Welleck%2C+Faeze+Brahman%2C+Chandra+Bhagavatula%2C+Ronan+Le+Bras%2C+and+Yejin+Choi%2E+2022%2E+Maieutic+prompting%3A+Logically+consistent+reasoning+with+recursive+explanations%2E+In+Proc) [our card](https://md.research.elamur.ai/free/en/papers/jung-2022-maieutic-prompting-logically-consistent-reasoning/)
45. Ryo Kamoi, Sarkar Snigdha Sarathi Das, Renze Lou, Jihyun Janice Ahn, Yilun Zhao, Xiaoxin Lu, Nan Zhang, Yusen Zhang, Ranran Haoran Zhang, Sujeeth Reddy Vummanthala, Salika Dave, Shaobo Qin, Arman Cohan, Wenpeng Yin, and Rui Zhang. 2024. Evaluating LLMs at detecting errors in LLM responses. arXiv pre… — [arXiv:2404.03602](https://arxiv.org/abs/2404.03602) [find](https://scholar.google.com/scholar?q=Ryo+Kamoi%2C+Sarkar+Snigdha+Sarathi+Das%2C+Renze+Lou%2C+Jihyun+Janice+Ahn%2C+Yilun+Zhao%2C+Xiaoxin+Lu%2C+Nan+Zhang%2C+Yusen+Zhang%2C+Ranran+Haoran+Zhang%2C+Sujeeth+Reddy+Vummanthala%2C+Salika+Dave%2C+Shaobo+Qin%2C+Arman+Coha)
46. Dayeon Ki and Marine Carpuat. 2024. Guiding large language models to post-edit machine translation with error annotations. In Findings of the Association for Computational Linguistics: NAACL 2024, pages 4253–4273, Mexico City, Mexico. Association for Computational Linguistics. https://doi.org/10.186… [find](https://scholar.google.com/scholar?q=Dayeon+Ki+and+Marine+Carpuat%2E+2024%2E+Guiding+large+language+models+to+post%2Dedit+machine+translation+with+error+annotations%2E+In+Findings+of+the+Association+for+Computational+Linguistics%3A+NAACL+2024%2C+pag)
47. Geunwoo Kim, Pierre Baldi, and Stephen McAleer. 2023. Language models can solve computer tasks. In Advances in Neural Information Processing Systems, volume 36, pages 39648–39677. Curran Associates, Inc. [find](https://scholar.google.com/scholar?q=Geunwoo+Kim%2C+Pierre+Baldi%2C+and+Stephen+McAleer%2E+2023%2E+Language+models+can+solve+computer+tasks%2E+In+Advances+in+Neural+Information+Processing+Systems%2C+volume+36%2C+pages+39648%E2%80%9339677%2E+Curran+Associates%2C+I) [our card](https://md.research.elamur.ai/free/en/papers/kim-2023-language-models-solve-computer-tasks/)
48. Hung Le, Yue Wang, Akhilesh Deepak Gotmare, Silvio Savarese, and Steven Chu Hong Hoi. 2022. Coderl: Mastering code generation through pretrained models and deep reinforcement learning. In Advances in Neural Information Processing Systems, volume 35, pages 21314–21328. Curran Associates, Inc. [find](https://scholar.google.com/scholar?q=Hung+Le%2C+Yue+Wang%2C+Akhilesh+Deepak+Gotmare%2C+Silvio+Savarese%2C+and+Steven+Chu+Hong+Hoi%2E+2022%2E+Coderl%3A+Mastering+code+generation+through+pretrained+models+and+deep+reinforcement+learning%2E+In+Advances+in+) [our card](https://md.research.elamur.ai/free/en/papers/le-2022-coderl-mastering-code-generation-through/)
49. Seongyun Lee, Sue Park, Yongrae Jo, and Minjoon Seo. 2024. Volcano: Mitigating multimodal hallucination through self-feedback guided revision. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1:… [find](https://scholar.google.com/scholar?q=Seongyun+Lee%2C+Sue+Park%2C+Yongrae+Jo%2C+and+Minjoon+Seo%2E+2024%2E+Volcano%3A+Mitigating+multimodal+hallucination+through+self%2Dfeedback+guided+revision%2E+In+Proceedings+of+the+2024+Conference+of+the+North+Americ)
50. Loka Li, Zhenhao Chen, Guangyi Chen, Yixuan Zhang, Yusheng Su, Eric Xing, and Kun Zhang. 2024a. Confidence matters: Revisiting intrinsic self-correction capabilities of large language models. arXiv preprint arXiv:2402 .12563. [find](https://scholar.google.com/scholar?q=Loka+Li%2C+Zhenhao+Chen%2C+Guangyi+Chen%2C+Yixuan+Zhang%2C+Yusheng+Su%2C+Eric+Xing%2C+and+Kun+Zhang%2E+2024a%2E+Confidence+matters%3A+Revisiting+intrinsic+self%2Dcorrection+capabilities+of+large+language+models%2E+arXiv+pr)
51. Ruosen Li, Teerth Patel, and Xinya Du. 2023. Prd: Peer rank and discussion improve large language model based evaluations. arXiv preprint arXiv:2307.02762. https://doi.org/10 .48550/arXiv.2307.02762 [find](https://scholar.google.com/scholar?q=Ruosen+Li%2C+Teerth+Patel%2C+and+Xinya+Du%2E+2023%2E+Prd%3A+Peer+rank+and+discussion+improve+large+language+model+based+evaluations%2E+arXiv+preprint+arXiv%3A2307%2E02762%2E+https%3A%2F%2Fdoi%2Eorg%2F10+%2E48550%2FarXiv%2E2307%2E02762)
52. Yanhong Li, Chenghao Yang, and Allyson Ettinger. 2024b. When hindsight is not 20/20: Testing limits on reflective thinking in large language models. In Findings of the Association for Computational Linguistics: NAACL 2024, pages 3741–3753, Mexico City, Mexico. Association for Computational Linguisti… [find](https://scholar.google.com/scholar?q=Yanhong+Li%2C+Chenghao+Yang%2C+and+Allyson+Ettinger%2E+2024b%2E+When+hindsight+is+not+20%2F20%3A+Testing+limits+on+reflective+thinking+in+large+language+models%2E+In+Findings+of+the+Association+for+Computational+Li)
53. Tian Liang, Zhiwei He, Wenxiang Jiao, Xing Wang, Yan Wang, Rui Wang, Yujiu Yang, Zhaopeng Tu, and Shuming Shi. 2023. Encouraging divergent thinking in large language models through multi-agent debate. arXiv preprint arXiv:2305.19118. [find](https://scholar.google.com/scholar?q=Tian+Liang%2C+Zhiwei+He%2C+Wenxiang+Jiao%2C+Xing+Wang%2C+Yan+Wang%2C+Rui+Wang%2C+Yujiu+Yang%2C+Zhaopeng+Tu%2C+and+Shuming+Shi%2E+2023%2E+Encouraging+divergent+thinking+in+large+language+models+through+multi%2Dagent+debate%2E) [our card](https://md.research.elamur.ai/free/en/papers/liang-2023-encouraging-divergent-thinking-large-language/)
54. Jared Lichtarge, Chris Alberti, Shankar Kumar, Noam Shazeer, Niki Parmar, and Simon Tong. 2019. Corpora generation for grammatical error correction. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volu… [find](https://scholar.google.com/scholar?q=Jared+Lichtarge%2C+Chris+Alberti%2C+Shankar+Kumar%2C+Noam+Shazeer%2C+Niki+Parmar%2C+and+Simon+Tong%2E+2019%2E+Corpora+generation+for+grammatical+error+correction%2E+In+Proceedings+of+the+2019+Conference+of+the+North+)
55. Hunter Lightman, Vineet Kosaraju, Yuri Burda, Harrison Edwards, Bowen Baker, Teddy Lee, Jan Leike, John Schulman, Ilya Sutskever, and Karl Cobbe. 2024. Let’s verify step by step. In The Twelfth International Conference on Learning Representations. [find](https://scholar.google.com/scholar?q=Hunter+Lightman%2C+Vineet+Kosaraju%2C+Yuri+Burda%2C+Harrison+Edwards%2C+Bowen+Baker%2C+Teddy+Lee%2C+Jan+Leike%2C+John+Schulman%2C+Ilya+Sutskever%2C+and+Karl+Cobbe%2E+2024%2E+Let%E2%80%99s+verify+step+by+step%2E+In+The+Twelfth+Intern) [our card](https://md.research.elamur.ai/free/en/papers/lightman-2023-let-s-verify-step-step/)
56. Guangliang Liu, Haitao Mao, Bochuan Cao, Zhiyu Xue, Kristen Johnson, Jiliang Tang, and Rongrong Wang. 2024. On the intrinsic self-correction capability of LLMs: Uncertainty and latent concept. arXiv preprint arXiv:2406.02378. — [arXiv:2406.02378](https://arxiv.org/abs/2406.02378) [find](https://scholar.google.com/scholar?q=Guangliang+Liu%2C+Haitao+Mao%2C+Bochuan+Cao%2C+Zhiyu+Xue%2C+Kristen+Johnson%2C+Jiliang+Tang%2C+and+Rongrong+Wang%2E+2024%2E+On+the+intrinsic+self%2Dcorrection+capability+of+LLMs%3A+Uncertainty+and+latent+concept%2E+arXiv+p)
57. Yang Liu, Dan Iter, Yichong Xu, Shuohang Wang, Ruochen Xu, and Chenguang Zhu. 2023. G-eval: NLG evaluation using gpt-4 with better human alignment. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 2511–2522, Singapore. Association for Computational Lin… [find](https://scholar.google.com/scholar?q=Yang+Liu%2C+Dan+Iter%2C+Yichong+Xu%2C+Shuohang+Wang%2C+Ruochen+Xu%2C+and+Chenguang+Zhu%2E+2023%2E+G%2Deval%3A+NLG+evaluation+using+gpt%2D4+with+better+human+alignment%2E+In+Proceedings+of+the+2023+Conference+on+Empirical+M)
58. Aman Madaan, Niket Tandon, Prakhar Gupta, Skyler Hallinan, Luyu Gao, Sarah Wiegreffe, Uri Alon, Nouha Dziri, Shrimai Prabhumoye, Yiming Yang, Shashank Gupta, Bodhisattwa Prasad Majumder, Katherine Hermann, Sean Welleck, Amir Yazdanbakhsh, and Peter Clark. 2023. Self-refine: Iterative refinement with… [find](https://scholar.google.com/scholar?q=Aman+Madaan%2C+Niket+Tandon%2C+Prakhar+Gupta%2C+Skyler+Hallinan%2C+Luyu+Gao%2C+Sarah+Wiegreffe%2C+Uri+Alon%2C+Nouha+Dziri%2C+Shrimai+Prabhumoye%2C+Yiming+Yang%2C+Shashank+Gupta%2C+Bodhisattwa+Prasad+Majumder%2C+Katherine+Her)
59. Potsawee Manakul, Adian Liusie, and Mark Gales. 2023. SelfCheckGPT: Zero-resource black-box hallucination detection for generative large language models. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 9004–9017, Singapore. Association for Computation… [find](https://scholar.google.com/scholar?q=Potsawee+Manakul%2C+Adian+Liusie%2C+and+Mark+Gales%2E+2023%2E+SelfCheckGPT%3A+Zero%2Dresource+black%2Dbox+hallucination+detection+for+generative+large+language+models%2E+In+Proceedings+of+the+2023+Conference+on+Empir)
60. Ninareh Mehrabi, Palash Goyal, Christophe Dupuy, Qian Hu, Shalini Ghosh, Richard Zemel, Kai-Wei Chang, Aram Galstyan, and Rahul Gupta. 2023. Flirt: Feedback loop in-context red teaming. arXiv preprint arXiv: 2308.04265. [find](https://scholar.google.com/scholar?q=Ninareh+Mehrabi%2C+Palash+Goyal%2C+Christophe+Dupuy%2C+Qian+Hu%2C+Shalini+Ghosh%2C+Richard+Zemel%2C+Kai%2DWei+Chang%2C+Aram+Galstyan%2C+and+Rahul+Gupta%2E+2023%2E+Flirt%3A+Feedback+loop+in%2Dcontext+red+teaming%2E+arXiv+preprint)
61. Yu Meng, Jiaxin Huang, Yu Zhang, and Jiawei Han. 2022. Generating training data with language models: Towards zero-shot language understanding. In Advances in Neural Information Processing Systems, volume 35, pages 462–477. Curran Associates, Inc. [find](https://scholar.google.com/scholar?q=Yu+Meng%2C+Jiaxin+Huang%2C+Yu+Zhang%2C+and+Jiawei+Han%2E+2022%2E+Generating+training+data+with+language+models%3A+Towards+zero%2Dshot+language+understanding%2E+In+Advances+in+Neural+Information+Processing+Systems%2C+vo) [our card](https://md.research.elamur.ai/free/en/papers/meng-2022-generating-training-data-language-models/)
62. Ali Mesbah, Andrew Rice, Emily Johnston, Nick Glorioso, and Edward Aftandilian. 2019. Deepdelta: Learning to repair compilation errors. In Proceedings of the 2019 27th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering, ESEC/ FSE 20… — [doi:10.1145/3338906](https://doi.org/10.1145/3338906) [find](https://scholar.google.com/scholar?q=Ali+Mesbah%2C+Andrew+Rice%2C+Emily+Johnston%2C+Nick+Glorioso%2C+and+Edward+Aftandilian%2E+2019%2E+Deepdelta%3A+Learning+to+repair+compilation+errors%2E+In+Proceedings+of+the+2019+27th+ACM+Joint+Meeting+on+European+So)
63. Ning Miao, Yee Whye Teh, and Tom Rainforth. 2024. Selfcheck: Using LLMs to zero-shot check their own step-by-step reasoning. In The Twelfth International Conference on Learning Representations. [find](https://scholar.google.com/scholar?q=Ning+Miao%2C+Yee+Whye+Teh%2C+and+Tom+Rainforth%2E+2024%2E+Selfcheck%3A+Using+LLMs+to+zero%2Dshot+check+their+own+step%2Dby%2Dstep+reasoning%2E+In+The+Twelfth+International+Conference+on+Learning+Representations%2E)
64. Theo X. Olausson, Jeevana Priya Inala, Chenglong Wang, Jianfeng Gao, and Armando Solar-Lezama. 2024. Is self-repair a silver bullet for code generation? In The Twelfth International Conference on Learning Representations. [find](https://scholar.google.com/scholar?q=Theo+X%2E+Olausson%2C+Jeevana+Priya+Inala%2C+Chenglong+Wang%2C+Jianfeng+Gao%2C+and+Armando+Solar%2DLezama%2E+2024%2E+Is+self%2Drepair+a+silver+bullet+for+code+generation%3F+In+The+Twelfth+International+Conference+on+Lear)
65. Sewon Min, Kalpesh Krishna, Xinxi Lyu, Mike Lewis, Wen-tau Yih, Pang Koh, Mohit Iyyer, Luke Zettlemoyer, and Hannaneh Hajishirzi. 2023. FActScore: Fine-grained atomic evaluation of factual precision in long form text generation. In Proceedings of the 2023 Conference on Empirical Methods in Natural L… [find](https://scholar.google.com/scholar?q=Sewon+Min%2C+Kalpesh+Krishna%2C+Xinxi+Lyu%2C+Mike+Lewis%2C+Wen%2Dtau+Yih%2C+Pang+Koh%2C+Mohit+Iyyer%2C+Luke+Zettlemoyer%2C+and+Hannaneh+Hajishirzi%2E+2023%2E+FActScore%3A+Fine%2Dgrained+atomic+evaluation+of+factual+precision+i) [our card](https://md.research.elamur.ai/free/en/papers/min-2023-factscore-fine-grained-atomic-evaluation/)
66. Liangming Pan, Alon Albalak, Xinyi Wang, and William Wang. 2023. Logic-LM: Empowering large language models with symbolic solvers for faithful logical reasoning. In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 3806–3824, Singapore. Association for Computational Lingui… [find](https://scholar.google.com/scholar?q=Liangming+Pan%2C+Alon+Albalak%2C+Xinyi+Wang%2C+and+William+Wang%2E+2023%2E+Logic%2DLM%3A+Empowering+large+language+models+with+symbolic+solvers+for+faithful+logical+reasoning%2E+In+Findings+of+the+Association+for+Com) [our card](https://md.research.elamur.ai/free/en/papers/pan-2023-logic-lm-empowering-large-language/)
67. Abhika Mishra, Akari Asai, Vidhisha Balachandran, Yizhong Wang, Graham Neubig, Yulia Tsvetkov, and Hannaneh Hajishirzi. 2024. Fine-grained hallucination detection and editing for language models. arXiv preprint arXiv:2401.06855. [find](https://scholar.google.com/scholar?q=Abhika+Mishra%2C+Akari+Asai%2C+Vidhisha+Balachandran%2C+Yizhong+Wang%2C+Graham+Neubig%2C+Yulia+Tsvetkov%2C+and+Hannaneh+Hajishirzi%2E+2024%2E+Fine%2Dgrained+hallucination+detection+and+editing+for+language+models%2E+arXi)
68. Deepak Nathani, David Wang, Liangming Pan, and William Wang. 2023. MAF: Multi-aspect feedback for improving reasoning in large language models. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 6591–6616, Singapore. Association for Computational Linguis… [find](https://scholar.google.com/scholar?q=Deepak+Nathani%2C+David+Wang%2C+Liangming+Pan%2C+and+William+Wang%2E+2023%2E+MAF%3A+Multi%2Daspect+feedback+for+improving+reasoning+in+large+language+models%2E+In+Proceedings+of+the+2023+Conference+on+Empirical+Metho)
69. Hwee Tou Ng, Siew Mei Wu, Ted Briscoe, Christian Hadiwinoto, Raymond Hendy Susanto, and Christopher Bryant. 2014. The CoNLL-2014 shared task on grammatical error correction. In Proceedings of the Eighteenth Conference on Computational Natural Language Learning: Shared Task, pages 1–14, Baltimore, Ma… [find](https://scholar.google.com/scholar?q=Hwee+Tou+Ng%2C+Siew+Mei+Wu%2C+Ted+Briscoe%2C+Christian+Hadiwinoto%2C+Raymond+Hendy+Susanto%2C+and+Christopher+Bryant%2E+2014%2E+The+CoNLL%2D2014+shared+task+on+grammatical+error+correction%2E+In+Proceedings+of+the+Eigh)
70. Ansong Ni, Srini Iyer, Dragomir Radev, Veselin Stoyanov, Wen-Tau Yih, Sida Wang, and Xi Victoria Lin. 2023. LEVER: Learning to verify language-to-code generation with execution. In Proceedings of the 40th International Conference on Machine Learning, volume 202 [find](https://scholar.google.com/scholar?q=Ansong+Ni%2C+Srini+Iyer%2C+Dragomir+Radev%2C+Veselin+Stoyanov%2C+Wen%2DTau+Yih%2C+Sida+Wang%2C+and+Xi+Victoria+Lin%2E+2023%2E+LEVER%3A+Learning+to+verify+language%2Dto%2Dcode+generation+with+execution%2E+In+Proceedings+of+the+) [our card](https://md.research.elamur.ai/free/en/papers/ni-2023-lever-learning-verify-language-code/)
71. Liangming Pan, Michael Saxon, Wenda Xu, Deepak Nathani, Xinyi Wang, and William Yang Wang. 2024. Automatically correcting large language models: Surveying the landscape of diverse automated correction strategies. Transactions of the Association for Computational Linguistics, 12:484–506. [https://doi.…](https://doi.…) [↗](https://doi.…)
72. Jing-Cheng Pang, Pengyuan Wang, Kaiyuan Li, Xiong-Hui Chen, Jiacheng Xu, Zongzhang Zhang, and Yang Yu. 2024. Language model self-improvement by reinforcement learning contemplation. In The Twelfth International Conference on Learning Representations. [find](https://scholar.google.com/scholar?q=Jing%2DCheng+Pang%2C+Pengyuan+Wang%2C+Kaiyuan+Li%2C+Xiong%2DHui+Chen%2C+Jiacheng+Xu%2C+Zongzhang+Zhang%2C+and+Yang+Yu%2E+2024%2E+Language+model+self%2Dimprovement+by+reinforcement+learning+contemplation%2E+In+The+Twelfth+Int)
73. Debjit Paul, Mete Ismayilzada, Maxime Peyrard, Beatriz Borges, Antoine Bosselut, Robert West, and Boi Faltings. 2024. REFINER: Reasoning feedback on intermediate representations. In Proceedings of the 18th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1:… [find](https://scholar.google.com/scholar?q=Debjit+Paul%2C+Mete+Ismayilzada%2C+Maxime+Peyrard%2C+Beatriz+Borges%2C+Antoine+Bosselut%2C+Robert+West%2C+and+Boi+Faltings%2E+2024%2E+REFINER%3A+Reasoning+feedback+on+intermediate+representations%2E+In+Proceedings+of+the) [our card](https://md.research.elamur.ai/free/en/papers/paul-2023-refiner-reasoning-feedback-intermediate-representations/)
74. Baolin Peng, Michel Galley, Pengcheng He, Hao Cheng, Yujia Xie, Yu Hu, Qiuyuan Huang, Lars Liden, Zhou Yu, Weizhu Chen, and Jianfeng Gao. 2023. Check your facts and try again: Improving large language models with external knowledge and automated feedback. arXiv preprint arXiv:2302.12813. [find](https://scholar.google.com/scholar?q=Baolin+Peng%2C+Michel+Galley%2C+Pengcheng+He%2C+Hao+Cheng%2C+Yujia+Xie%2C+Yu+Hu%2C+Qiuyuan+Huang%2C+Lars+Liden%2C+Zhou+Yu%2C+Weizhu+Chen%2C+and+Jianfeng+Gao%2E+2023%2E+Check+your+facts+and+try+again%3A+Improving+large+language) [our card](https://md.research.elamur.ai/free/en/papers/peng-2023-check-facts-try-again-improving/)
75. Mansi Phute, Alec Helbling, Matthew Hull, ShengYun Peng, Sebastian Szyller, Cory Cornelius, and Duen Horng Chau. 2024. Llm self defense: By self examination, llms know they are being tricked. arXiv preprint arXiv:2308.07308. [find](https://scholar.google.com/scholar?q=Mansi+Phute%2C+Alec+Helbling%2C+Matthew+Hull%2C+ShengYun+Peng%2C+Sebastian+Szyller%2C+Cory+Cornelius%2C+and+Duen+Horng+Chau%2E+2024%2E+Llm+self+defense%3A+By+self+examination%2C+llms+know+they+are+being+tricked%2E+arXiv+pr)
76. Reid Pryzant, Dan Iter, Jerry Li, Yin Lee, Chenguang Zhu, and Michael Zeng. 2023. Automatic prompt optimization with ‘‘gradient descent’’ and beam search. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 7957–7968, Singapore. Association for Computatio… [find](https://scholar.google.com/scholar?q=Reid+Pryzant%2C+Dan+Iter%2C+Jerry+Li%2C+Yin+Lee%2C+Chenguang+Zhu%2C+and+Michael+Zeng%2E+2023%2E+Automatic+prompt+optimization+with+%E2%80%98%E2%80%98gradient+descent%E2%80%99%E2%80%99+and+beam+search%2E+In+Proceedings+of+the+2023+Conference+on+Empi) [our card](https://md.research.elamur.ai/free/en/papers/pryzant-2023-automatic-prompt-optimization-gradient-descent/)
77. Maribeth Rauh, John F. J. Mellor, Jonathan Uesato, Po-Sen Huang, Johannes Welbl, Laura Weidinger, Sumanth Dathathri, Amelia Glaese, Geoffrey Irving, Iason Gabriel, William Isaac, and Lisa Anne Hendricks. 2022. Characteristics of harmful text: Towards rigorous benchmarking of language models. In Thir… [find](https://scholar.google.com/scholar?q=Maribeth+Rauh%2C+John+F%2E+J%2E+Mellor%2C+Jonathan+Uesato%2C+Po%2DSen+Huang%2C+Johannes+Welbl%2C+Laura+Weidinger%2C+Sumanth+Dathathri%2C+Amelia+Glaese%2C+Geoffrey+Irving%2C+Iason+Gabriel%2C+William+Isaac%2C+and+Lisa+Anne+Hendric)
78. Vikas Raunak, Amr Sharaf, Yiren Wang, Hany Awadalla, and Arul Menezes. 2023. Leveraging GPT-4 for automatic translation post-editing. In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 12009–12024, Singapore. Association for Computational Linguistics. https://doi .org/10… [find](https://scholar.google.com/scholar?q=Vikas+Raunak%2C+Amr+Sharaf%2C+Yiren+Wang%2C+Hany+Awadalla%2C+and+Arul+Menezes%2E+2023%2E+Leveraging+GPT%2D4+for+automatic+translation+post%2Dediting%2E+In+Findings+of+the+Association+for+Computational+Linguistics%3A+EMNL)
79. Swarnadeep Saha, Omer Levy, Asli Celikyilmaz, Mohit Bansal, Jason Weston, and Xian Li. 2024. Branch-solve-merge improves large language model evaluation and generation. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Languag… [find](https://scholar.google.com/scholar?q=Swarnadeep+Saha%2C+Omer+Levy%2C+Asli+Celikyilmaz%2C+Mohit+Bansal%2C+Jason+Weston%2C+and+Xian+Li%2E+2024%2E+Branch%2Dsolve%2Dmerge+improves+large+language+model+evaluation+and+generation%2E+In+Proceedings+of+the+2024+Conf)
80. William Saunders, Catherine Yeh, Jeff Wu, Steven Bills, Long Ouyang, Jonathan Ward, and Jan Leike. 2022. Self-critiquing models for assisting human evaluators. arXiv preprint arXiv:2206.05802. [find](https://scholar.google.com/scholar?q=William+Saunders%2C+Catherine+Yeh%2C+Jeff+Wu%2C+Steven+Bills%2C+Long+Ouyang%2C+Jonathan+Ward%2C+and+Jan+Leike%2E+2022%2E+Self%2Dcritiquing+models+for+assisting+human+evaluators%2E+arXiv+preprint+arXiv%3A2206%2E05802%2E) [our card](https://md.research.elamur.ai/free/en/papers/saunders-2022-self-critiquing-models-assisting-human/)
81. Timo Schick, Sahana Udupa, and Hinrich Schütze. 2021. Self-diagnosis and self-debiasing: A proposal for reducing corpus-based bias in NLP. Transactions of the Association for Computational Linguistics, 9:1408–1424. https://doi.org/10.1162/tacl a 00434 [↗](https://doi.org/10.1162/tacl) [our card](https://md.research.elamur.ai/free/en/papers/schick-2021-self-diagnosis-self-debiasing-proposal/)
82. Timo Schick, Jane A. Yu, Zhengbao Jiang, Fabio Petroni, Patrick Lewis, Gautier Izacard, Qingfei You, Christoforos Nalmpantis, Edouard Grave, and Sebastian Riedel. 2023. PEER: A collaborative language model. In The Eleventh International Conference on Learning Representations. [find](https://scholar.google.com/scholar?q=Timo+Schick%2C+Jane+A%2E+Yu%2C+Zhengbao+Jiang%2C+Fabio+Petroni%2C+Patrick+Lewis%2C+Gautier+Izacard%2C+Qingfei+You%2C+Christoforos+Nalmpantis%2C+Edouard+Grave%2C+and+Sebastian+Riedel%2E+2023%2E+PEER%3A+A+collaborative+language+)
83. Thomas Scialom, Paul-Alexis Dray, Sylvain Lamprier, Benjamin Piwowarski, Jacopo Staiano, Alex Wang, and Patrick Gallinari. 2021. QuestEval: Summarization asks for fact-based evaluation. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 6594–6604, Online… [find](https://scholar.google.com/scholar?q=Thomas+Scialom%2C+Paul%2DAlexis+Dray%2C+Sylvain+Lamprier%2C+Benjamin+Piwowarski%2C+Jacopo+Staiano%2C+Alex+Wang%2C+and+Patrick+Gallinari%2E+2021%2E+QuestEval%3A+Summarization+asks+for+fact%2Dbased+evaluation%2E+In+Proceedings) [our card](https://md.research.elamur.ai/free/en/papers/scialom-2021-questeval-summarization-asks-fact-based/)
84. Darsh Shah, Tal Schuster, and Regina Barzilay. 2020. Automatic fact-guided sentence modification. Proceedings of the AAAI Conference on Artificial Intelligence, 34(05):8791–8798. https://doi.org/10.1609/aaai.v34i05 .6406 — [doi:10.1609/aaai.v34i05](https://doi.org/10.1609/aaai.v34i05) [↗](https://doi.org/10.1609/aaai.v34i05)
85. Jianhao Shen, Yichun Yin, Lin Li, Lifeng Shang, Xin Jiang, Ming Zhang, and Qun Liu. 2021. Generate & rank: A multi-task framework for math word problems. In Findings of the Association for Computational Linguistics: EMNLP 2021, pages 2269–2279, Punta Cana, Dominican Republic. Association for Computa… [find](https://scholar.google.com/scholar?q=Jianhao+Shen%2C+Yichun+Yin%2C+Lin+Li%2C+Lifeng+Shang%2C+Xin+Jiang%2C+Ming+Zhang%2C+and+Qun+Liu%2E+2021%2E+Generate+%26+rank%3A+A+multi%2Dtask+framework+for+math+word+problems%2E+In+Findings+of+the+Association+for+Computation)
86. Freda Shi, Daniel Fried, Marjan Ghazvininejad, Luke Zettlemoyer, and Sida I. Wang. 2022. Natural language to code translation with execution. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 3533–3546, Abu Dhabi, United Arab Emirates. Association for C… [find](https://scholar.google.com/scholar?q=Freda+Shi%2C+Daniel+Fried%2C+Marjan+Ghazvininejad%2C+Luke+Zettlemoyer%2C+and+Sida+I%2E+Wang%2E+2022%2E+Natural+language+to+code+translation+with+execution%2E+In+Proceedings+of+the+2022+Conference+on+Empirical+Methods)
87. Noah Shinn, Federico Cassano, Ashwin Gopinath, Karthik Narasimhan, and Shunyu Yao. 2023. Reflexion: Language agents with verbal reinforcement learning. In Advances in Neural Information Processing Systems, volume 36, pages 8634–8652. Curran Associates, Inc. solving planning tasks. In NeurIPS 2023 Fo… [find](https://scholar.google.com/scholar?q=Noah+Shinn%2C+Federico+Cassano%2C+Ashwin+Gopinath%2C+Karthik+Narasimhan%2C+and+Shunyu+Yao%2E+2023%2E+Reflexion%3A+Language+agents+with+verbal+reinforcement+learning%2E+In+Advances+in+Neural+Information+Processing+Sys) [our card](https://md.research.elamur.ai/free/en/papers/shinn-2023-reflexion-language-agents-verbal-reinforcement/)
88. Kaya Stechly, Matthew Marquez, and Subbarao Kambhampati. 2023. GPT-4 doesn’t know it’s wrong: An analysis of iterative prompting for reasoning problems. In NeurIPS 2023 Foundation Models for Decision Making Workshop. [find](https://scholar.google.com/scholar?q=Kaya+Stechly%2C+Matthew+Marquez%2C+and+Subbarao+Kambhampati%2E+2023%2E+GPT%2D4+doesn%E2%80%99t+know+it%E2%80%99s+wrong%3A+An+analysis+of+iterative+prompting+for+reasoning+problems%2E+In+NeurIPS+2023+Foundation+Models+for+Decision+)
89. Neeraj Varshney, Wenlin Yao, Hongming Zhang, Jianshu Chen, and Dong Yu. 2023. A stitch in time saves nine: Detecting and mitigating hallucinations of LLMs by validating low-confidence generation. arXiv preprint arXiv: 2307.03987. [find](https://scholar.google.com/scholar?q=Neeraj+Varshney%2C+Wenlin+Yao%2C+Hongming+Zhang%2C+Jianshu+Chen%2C+and+Dong+Yu%2E+2023%2E+A+stitch+in+time+saves+nine%3A+Detecting+and+mitigating+hallucinations+of+LLMs+by+validating+low%2Dconfidence+generation%2E+arXi) [our card](https://md.research.elamur.ai/free/en/papers/varshney-2023-stitch-time-saves-nine-detecting/)
90. Elias Stengel-Eskin, Archiki Prasad, and Mohit Bansal. 2024. Regal: Refactoring programs to discover generalizable abstractions. arXiv preprint arXiv:2401.16467. [find](https://scholar.google.com/scholar?q=Elias+Stengel%2DEskin%2C+Archiki+Prasad%2C+and+Mohit+Bansal%2E+2024%2E+Regal%3A+Refactoring+programs+to+discover+generalizable+abstractions%2E+arXiv+preprint+arXiv%3A2401%2E16467%2E)
91. Oyvind Tafjord, Bhavana Dalvi Mishra, and Peter Clark. 2022. Entailer: Answering questions with faithful and truthful chains of reasoning. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 2078–2093, Abu Dhabi, United Arab Emirates. Association for Comp… [find](https://scholar.google.com/scholar?q=Oyvind+Tafjord%2C+Bhavana+Dalvi+Mishra%2C+and+Peter+Clark%2E+2022%2E+Entailer%3A+Answering+questions+with+faithful+and+truthful+chains+of+reasoning%2E+In+Proceedings+of+the+2022+Conference+on+Empirical+Methods+in)
92. James Thorne and Andreas Vlachos. 2021. Evidence-based factual error correction. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 3298–3309, Online. Ass… [find](https://scholar.google.com/scholar?q=James+Thorne+and+Andreas+Vlachos%2E+2021%2E+Evidence%2Dbased+factual+error+correction%2E+In+Proceedings+of+the+59th+Annual+Meeting+of+the+Association+for+Computational+Linguistics+and+the+11th+International+J)
93. Gladys Tyen, Hassan Mansoor, Victor Carbune, Peter Chen, and Tony Mak. 2024. LLMs cannot find reasoning errors, but can correct them given the error location. In Findings of the Association for Computational Linguistics ACL 2024, pages 13894–13908, Bangkok, Thailand and virtual meeting. Association… [find](https://scholar.google.com/scholar?q=Gladys+Tyen%2C+Hassan+Mansoor%2C+Victor+Carbune%2C+Peter+Chen%2C+and+Tony+Mak%2E+2024%2E+LLMs+cannot+find+reasoning+errors%2C+but+can+correct+them+given+the+error+location%2E+In+Findings+of+the+Association+for+Comput)
94. Jonathan Uesato, Nate Kushman, Ramana Kumar, Francis Song, Noah Siegel, Lisa Wang, Antonia Creswell, Geoffrey Irving, and Irina Higgins. 2022. Solving math word problems with process- and outcome-based feedback. arXiv preprint arXiv:2211.14275. [find](https://scholar.google.com/scholar?q=Jonathan+Uesato%2C+Nate+Kushman%2C+Ramana+Kumar%2C+Francis+Song%2C+Noah+Siegel%2C+Lisa+Wang%2C+Antonia+Creswell%2C+Geoffrey+Irving%2C+and+Irina+Higgins%2E+2022%2E+Solving+math+word+problems+with+process%2D+and+outcome%2Dbase) [our card](https://md.research.elamur.ai/free/en/papers/uesato-2022-solving-math-word-problems-process/)
95. Karthik Valmeekam, Matthew Marquez, and Subbarao Kambhampati. 2023. Investigating the effectiveness of self-critiquing in LLMs Alex Wang, Kyunghyun Cho, and Mike Lewis. 2020. Asking and answering questions to evaluate the factual consistency of summaries. In Proceedings of the 58th Annual Meeting of… [find](https://scholar.google.com/scholar?q=Karthik+Valmeekam%2C+Matthew+Marquez%2C+and+Subbarao+Kambhampati%2E+2023%2E+Investigating+the+effectiveness+of+self%2Dcritiquing+in+LLMs+Alex+Wang%2C+Kyunghyun+Cho%2C+and+Mike+Lewis%2E+2020%2E+Asking+and+answering+ques)
96. Qineng Wang, Zihao Wang, Ying Su, Hanghang Tong, and Yangqiu Song. 2024a. Rethinking the bounds of LLM reasoning: Are multi-agent discussions the key? In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 6106–6131, Bangkok, Thailan… [find](https://scholar.google.com/scholar?q=Qineng+Wang%2C+Zihao+Wang%2C+Ying+Su%2C+Hanghang+Tong%2C+and+Yangqiu+Song%2E+2024a%2E+Rethinking+the+bounds+of+LLM+reasoning%3A+Are+multi%2Dagent+discussions+the+key%3F+In+Proceedings+of+the+62nd+Annual+Meeting+of+the+)
97. Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc V. Le, Ed H. Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou. 2023. Self-consistency improves chain of thought reasoning in language models. In The Eleventh International Conference on Learning Representations. [find](https://scholar.google.com/scholar?q=Xuezhi+Wang%2C+Jason+Wei%2C+Dale+Schuurmans%2C+Quoc+V%2E+Le%2C+Ed+H%2E+Chi%2C+Sharan+Narang%2C+Aakanksha+Chowdhery%2C+and+Denny+Zhou%2E+2023%2E+Self%2Dconsistency+improves+chain+of+thought+reasoning+in+language+models%2E+In+Th) [our card](https://md.research.elamur.ai/free/en/papers/wang-2022-self-consistency-improves-chain-thought/)
98. Yifei Wang, Yuyang Wu, Zeming Wei, Stefanie Jegelka, and Yisen Wang. 2024b. A theoretical understanding of self-correction through in-context alignment. In ICML 2024 Workshop on In-Context Learning. [find](https://scholar.google.com/scholar?q=Yifei+Wang%2C+Yuyang+Wu%2C+Zeming+Wei%2C+Stefanie+Jegelka%2C+and+Yisen+Wang%2E+2024b%2E+A+theoretical+understanding+of+self%2Dcorrection+through+in%2Dcontext+alignment%2E+In+ICML+2024+Workshop+on+In%2DContext+Learning%2E)
99. Sean Welleck, Ximing Lu, Peter West, Faeze Brahman, Tianxiao Shen, Daniel Khashabi, and Yejin Choi. 2023. Generating sequences by learning to self-correct. In The Eleventh International Conference on Learning Representations. [find](https://scholar.google.com/scholar?q=Sean+Welleck%2C+Ximing+Lu%2C+Peter+West%2C+Faeze+Brahman%2C+Tianxiao+Shen%2C+Daniel+Khashabi%2C+and+Yejin+Choi%2E+2023%2E+Generating+sequences+by+learning+to+self%2Dcorrect%2E+In+The+Eleventh+International+Conference+on+) [our card](https://md.research.elamur.ai/free/en/papers/welleck-2022-generating-sequences-learning-self-correct/)
100. Yixuan Weng, Minjun Zhu, Fei Xia, Bin Li, Shizhu He, Shengping Liu, Bin Sun, Kang Liu, and Jun Zhao. 2023. Large language models are better reasoners with self-verification. In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 2550–2575, Singapore. Association for Computat… [find](https://scholar.google.com/scholar?q=Yixuan+Weng%2C+Minjun+Zhu%2C+Fei+Xia%2C+Bin+Li%2C+Shizhu+He%2C+Shengping+Liu%2C+Bin+Sun%2C+Kang+Liu%2C+and+Jun+Zhao%2E+2023%2E+Large+language+models+are+better+reasoners+with+self%2Dverification%2E+In+Findings+of+the+Associa) [our card](https://md.research.elamur.ai/free/en/papers/weng-2023-large-language-models-are-better/)
101. Zhenyu Wu, Qingkai Zeng, Zhihan Zhang, Zhaoxuan Tan, Chao Shen, and Meng Jiang. 2024. Large language models can self-correct with minimal effort. arXiv preprint arXiv: 2405.14092. — [arXiv:2405.14092](https://arxiv.org/abs/2405.14092) [find](https://scholar.google.com/scholar?q=Zhenyu+Wu%2C+Qingkai+Zeng%2C+Zhihan+Zhang%2C+Zhaoxuan+Tan%2C+Chao+Shen%2C+and+Meng+Jiang%2E+2024%2E+Large+language+models+can+self%2Dcorrect+with+minimal+effort%2E+arXiv+preprint+arXiv%3A+2405%2E14092%2E+%E2%80%94+%5BarXiv%3A2405%2E14092%5D)
102. Yuxi Xie, Kenji Kawaguchi, Yiran Zhao, Xu Zhao, Min-Yen Kan, Junxian He, and Qizhe Xie. 2023. Self-evaluation guided beam search for reasoning. In Thirty-seventh Conference on Neural Information Processing Systems. [find](https://scholar.google.com/scholar?q=Yuxi+Xie%2C+Kenji+Kawaguchi%2C+Yiran+Zhao%2C+Xu+Zhao%2C+Min%2DYen+Kan%2C+Junxian+He%2C+and+Qizhe+Xie%2E+2023%2E+Self%2Devaluation+guided+beam+search+for+reasoning%2E+In+Thirty%2Dseventh+Conference+on+Neural+Information+Proce) [our card](https://md.research.elamur.ai/free/en/papers/xie-2023-self-evaluation-guided-beam-search/)
103. Wenda Xu, Danqing Wang, Liangming Pan, Zhenqiao Song, Markus Freitag, William Yang Wang, and Lei Li. 2023. INSTRUCTSCORE: Towards explainable text generation evaluation with automatic feedback. In The 2023 Conference on Empirical Methods in Natural Language Processing. https://doi.org/10 .18653/v1/2… [find](https://scholar.google.com/scholar?q=Wenda+Xu%2C+Danqing+Wang%2C+Liangming+Pan%2C+Zhenqiao+Song%2C+Markus+Freitag%2C+William+Yang+Wang%2C+and+Lei+Li%2E+2023%2E+INSTRUCTSCORE%3A+Towards+explainable+text+generation+evaluation+with+automatic+feedback%2E+In+The) [our card](https://md.research.elamur.ai/free/en/papers/xu-2023-instructscore-explainable-text-generation-evaluation/)
104. Chengrun Yang, Xuezhi Wang, Yifeng Lu, Hanxiao Liu, Quoc V. Le, Denny Zhou, and Xinyun Chen. 2024. Large language models as optimizers. In The Twelfth International Conference on Learning Representations. [find](https://scholar.google.com/scholar?q=Chengrun+Yang%2C+Xuezhi+Wang%2C+Yifeng+Lu%2C+Hanxiao+Liu%2C+Quoc+V%2E+Le%2C+Denny+Zhou%2C+and+Xinyun+Chen%2E+2024%2E+Large+language+models+as+optimizers%2E+In+The+Twelfth+International+Conference+on+Learning+Representati) [our card](https://md.research.elamur.ai/free/en/papers/yang-2023-large-language-models-optimizers/)
105. Kaiyu Yang, Jia Deng, and Danqi Chen. 2022a. Generating natural language proofs with verifier-guided search. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 89–105, Abu Dhabi, United Arab Emirates. Association for Computational Linguistics. https://do… [find](https://scholar.google.com/scholar?q=Kaiyu+Yang%2C+Jia+Deng%2C+and+Danqi+Chen%2E+2022a%2E+Generating+natural+language+proofs+with+verifier%2Dguided+search%2E+In+Proceedings+of+the+2022+Conference+on+Empirical+Methods+in+Natural+Language+Processing%2C+)
106. Kevin Yang, Yuandong Tian, Nanyun Peng, and Dan Klein. 2022b. Re3: Generating longer stories with recursive reprompting and revision. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 4393–4479, Abu Dhabi, United Arab Emirates. Association for Computati… [find](https://scholar.google.com/scholar?q=Kevin+Yang%2C+Yuandong+Tian%2C+Nanyun+Peng%2C+and+Dan+Klein%2E+2022b%2E+Re3%3A+Generating+longer+stories+with+recursive+reprompting+and+revision%2E+In+Proceedings+of+the+2022+Conference+on+Empirical+Methods+in+Natu)
107. Shunyu Yao, Dian Yu, Jeffrey Zhao, Izhak Shafran, Tom Griffiths, Yuan Cao, and Karthik Narasimhan. 2023. Tree of thoughts: Deliberate problem solving with large language models. [find](https://scholar.google.com/scholar?q=Shunyu+Yao%2C+Dian+Yu%2C+Jeffrey+Zhao%2C+Izhak+Shafran%2C+Tom+Griffiths%2C+Yuan+Cao%2C+and+Karthik+Narasimhan%2E+2023%2E+Tree+of+thoughts%3A+Deliberate+problem+solving+with+large+language+models%2E)
108. In Advances in Neural Information Processing Systems, volume 36, pages 11809–11822. Curran Associates, Inc. [find](https://scholar.google.com/scholar?q=In+Advances+in+Neural+Information+Processing+Systems%2C+volume+36%2C+pages+11809%E2%80%9311822%2E+Curran+Associates%2C+Inc%2E)
109. Michihiro Yasunaga and Percy Liang. 2020. Graph-based, self-supervised program repair from diagnostic feedback. In Proceedings of the 37th International Conference on Machine Learning, volume 119 of Proceedings of Machine Learning Research, pages 10799–10808. PMLR. [find](https://scholar.google.com/scholar?q=Michihiro+Yasunaga+and+Percy+Liang%2E+2020%2E+Graph%2Dbased%2C+self%2Dsupervised+program+repair+from+diagnostic+feedback%2E+In+Proceedings+of+the+37th+International+Conference+on+Machine+Learning%2C+volume+119+of+P)
110. Michihiro Yasunaga and Percy Liang. 2021. Break-it-fix-it: Unsupervised learning for program repair. In Proceedings of the 38th International Conference on Machine Learning, volume 139 of Proceedings of Machine Learning Research, pages 11941–11952. PMLR. [find](https://scholar.google.com/scholar?q=Michihiro+Yasunaga+and+Percy+Liang%2E+2021%2E+Break%2Dit%2Dfix%2Dit%3A+Unsupervised+learning+for+program+repair%2E+In+Proceedings+of+the+38th+International+Conference+on+Machine+Learning%2C+volume+139+of+Proceedings+)
111. Seonghyeon Ye, Yongrae Jo, Doyoung Kim, Sungdong Kim, Hyeonbin Hwang, and Minjoon Seo. 2023. Selfee: Iterative self-revising LLM empowered by self-feedback generation. Blog post. [find](https://scholar.google.com/scholar?q=Seonghyeon+Ye%2C+Yongrae+Jo%2C+Doyoung+Kim%2C+Sungdong+Kim%2C+Hyeonbin+Hwang%2C+and+Minjoon+Seo%2E+2023%2E+Selfee%3A+Iterative+self%2Drevising+LLM+empowered+by+self%2Dfeedback+generation%2E+Blog+post%2E)
112. Shukang Yin, Chaoyou Fu, Sirui Zhao, Tong Xu, Hao Wang, Dianbo Sui, Yunhang Shen, Ke Li, Xing Sun, and Enhong Chen. 2023. Woodpecker: Hallucination correction for multimodal large language models. arXiv preprint arXiv:2310.16045. [find](https://scholar.google.com/scholar?q=Shukang+Yin%2C+Chaoyou+Fu%2C+Sirui+Zhao%2C+Tong+Xu%2C+Hao+Wang%2C+Dianbo+Sui%2C+Yunhang+Shen%2C+Ke+Li%2C+Xing+Sun%2C+and+Enhong+Chen%2E+2023%2E+Woodpecker%3A+Hallucination+correction+for+multimodal+large+language+models%2E+arX)
113. Wenhao Yu, Zhihan Zhang, Zhenwen Liang, Meng Jiang, and Ashish Sabharwal. 2023. Improving language models via plug-and-play retrieval feedback. arXiv preprint arXiv: 2305.14002. [find](https://scholar.google.com/scholar?q=Wenhao+Yu%2C+Zhihan+Zhang%2C+Zhenwen+Liang%2C+Meng+Jiang%2C+and+Ashish+Sabharwal%2E+2023%2E+Improving+language+models+via+plug%2Dand%2Dplay+retrieval+feedback%2E+arXiv+preprint+arXiv%3A+2305%2E14002%2E)
114. Eric Zelikman, Yuhuai Wu, Jesse Mu, and Noah Goodman. 2022. Star: Bootstrapping reasoning with reasoning. In Advances in Neural Information Processing Systems, volume 35, pages 15476–15488. Curran Associates, Inc. [find](https://scholar.google.com/scholar?q=Eric+Zelikman%2C+Yuhuai+Wu%2C+Jesse+Mu%2C+and+Noah+Goodman%2E+2022%2E+Star%3A+Bootstrapping+reasoning+with+reasoning%2E+In+Advances+in+Neural+Information+Processing+Systems%2C+volume+35%2C+pages+15476%E2%80%9315488%2E+Curran+Ass) [our card](https://md.research.elamur.ai/free/en/papers/zelikman-2022-star-bootstrapping-reasoning-reasoning/)
115. Jintian Zhang, Xin Xu, and Shumin Deng. 2023a. Exploring collaboration mechanisms for llm agents: A social psychology view. arXiv preprint arXiv:2310.02124. [find](https://scholar.google.com/scholar?q=Jintian+Zhang%2C+Xin+Xu%2C+and+Shumin+Deng%2E+2023a%2E+Exploring+collaboration+mechanisms+for+llm+agents%3A+A+social+psychology+view%2E+arXiv+preprint+arXiv%3A2310%2E02124%2E)
116. Kechi Zhang, Zhuo Li, Jia Li, Ge Li, and Zhi Jin. 2023b. Self-edit: Fault-aware code editor for code generation. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 769–787, Toronto, Canada. Association for Computational Linguisti… [find](https://scholar.google.com/scholar?q=Kechi+Zhang%2C+Zhuo+Li%2C+Jia+Li%2C+Ge+Li%2C+and+Zhi+Jin%2E+2023b%2E+Self%2Dedit%3A+Fault%2Daware+code+editor+for+code+generation%2E+In+Proceedings+of+the+61st+Annual+Meeting+of+the+Association+for+Computational+Linguist)
117. Muru Zhang, Ofir Press, William Merrill, Alisa Liu, and Noah A. Smith. 2023c. How language model hallucinations can snowball. arXiv preprint arXiv:2305.13534. [find](https://scholar.google.com/scholar?q=Muru+Zhang%2C+Ofir+Press%2C+William+Merrill%2C+Alisa+Liu%2C+and+Noah+A%2E+Smith%2E+2023c%2E+How+language+model+hallucinations+can+snowball%2E+arXiv+preprint+arXiv%3A2305%2E13534%2E) [our card](https://md.research.elamur.ai/free/en/papers/zhang-2023-language-model-hallucinations-snowball/)
118. Tianyi Zhang, Tao Yu, Tatsunori Hashimoto, Mike Lewis, Wen-Tau Yih, Daniel Fried, and Sida Wang. 2023d. Coder reviewer reranking for code generation. In Proceedings of the 40th International Conference on Machine Learning, volume 202 of Proceedings of Machine Learning Research, pages 41832–41846. PM… [find](https://scholar.google.com/scholar?q=Tianyi+Zhang%2C+Tao+Yu%2C+Tatsunori+Hashimoto%2C+Mike+Lewis%2C+Wen%2DTau+Yih%2C+Daniel+Fried%2C+and+Sida+Wang%2E+2023d%2E+Coder+reviewer+reranking+for+code+generation%2E+In+Proceedings+of+the+40th+International+Conferenc)
119. Yunxiang Zhang, Muhammad Khalifa, Lajanugen Logeswaran, Jaekyeom Kim, Moontae Lee, Honglak Lee, and Lu Wang. 2024. Small language models need strong verifiers to self-correct reasoning. In Findings of the Association for Computational Linguistics ACL 2024, pages 15637–15653, Bangkok, Thailand and vi… [find](https://scholar.google.com/scholar?q=Yunxiang+Zhang%2C+Muhammad+Khalifa%2C+Lajanugen+Logeswaran%2C+Jaekyeom+Kim%2C+Moontae+Lee%2C+Honglak+Lee%2C+and+Lu+Wang%2E+2024%2E+Small+language+models+need+strong+verifiers+to+self%2Dcorrect+reasoning%2E+In+Findings+of)
120. Ruochen Zhao, Xingxuan Li, Shafiq Joty, Chengwei Qin, and Lidong Bing. 2023. Verify-and-edit: A knowledge-enhanced chain-of-thought framework. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 5823–5840, Toronto, Canada. Associa… [find](https://scholar.google.com/scholar?q=Ruochen+Zhao%2C+Xingxuan+Li%2C+Shafiq+Joty%2C+Chengwei+Qin%2C+and+Lidong+Bing%2E+2023%2E+Verify%2Dand%2Dedit%3A+A+knowledge%2Denhanced+chain%2Dof%2Dthought+framework%2E+In+Proceedings+of+the+61st+Annual+Meeting+of+the+Associat) [our card](https://md.research.elamur.ai/free/en/papers/zhao-2023-verify-edit-knowledge-enhanced-chain/)
121. Yiyang Zhou, Chenhang Cui, Jaehong Yoon, Linjun Zhang, Zhun Deng, Chelsea Finn, Mohit Bansal, and Huaxiu Yao. 2024. Analyzing and mitigating object hallucination in large vision-language models. In The Twelfth International Conference on Learning Representations. [find](https://scholar.google.com/scholar?q=Yiyang+Zhou%2C+Chenhang+Cui%2C+Jaehong+Yoon%2C+Linjun+Zhang%2C+Zhun+Deng%2C+Chelsea+Finn%2C+Mohit+Bansal%2C+and+Huaxiu+Yao%2E+2024%2E+Analyzing+and+mitigating+object+hallucination+in+large+vision%2Dlanguage+models%2E+In+Th)
122. Initial Response Prompt Feedback Prompt We want to iteratively improve the provided responses. To help improve, scores for each response on desired traits are provided: 1) Relevant, 2) Informative, 3) Interesting, 4) Consistent, 5) Helpful, 6) Engaging, 7) Specific, 8) Safe, 9) User understanding, a… [find](https://scholar.google.com/scholar?q=Initial+Response+Prompt+Feedback+Prompt+We+want+to+iteratively+improve+the+provided+responses%2E+To+help+improve%2C+scores+for+each+response+on+desired+traits+are+provided%3A+1%29+Relevant%2C+2%29+Informative%2C+3%29)