---
title: "Automatically Correcting Large Language Models: Surveying the Landscape — paper by Pan et al. (2024)"
description: "Large language models still make characteristic mistakes. 153 references, cited by 56 papers here."
authors: ["Liangming Pan", "Michael Saxon", "Wenda Xu", "Deepak Nathani", "Xinyi Wang", "William Yang Wang"]
year: 2024
venue: "Transactions of the Association for Computational Linguistics"
source: "https://aclanthology.org/2024.tacl-1.27/"
doi: "10.1162/tacl_a_00660"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/pan-2024-automatically-correcting-large-language-models/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/pan-2024-automatically-correcting-large-language-models/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2024](https://md.research.elamur.ai/free/en/papers/2024/page/2/) / Automatically Correcting Large Language Models: Surveying the Landscape of Diverse Automated Correction Strategies

# Automatically Correcting Large Language Models: Surveying the Landscape of Diverse Automated Correction Strategies

**Authors:** [Liangming Pan](https://md.research.elamur.ai/free/en/authors/liangming-pan/), Michael Saxon, [Wenda Xu](https://md.research.elamur.ai/free/en/authors/wenda-xu/), [Deepak Nathani](https://md.research.elamur.ai/free/en/authors/deepak-nathani/), [Xinyi Wang](https://md.research.elamur.ai/free/en/authors/xinyi-wang/) and [William Yang Wang](https://md.research.elamur.ai/free/en/authors/william-yang-wang/) · **Year:** 2024 · **Venue:** [Transactions of the Association for Computational Linguistics](https://md.research.elamur.ai/free/en/venues/tacl/) ([TACL 2024](https://md.research.elamur.ai/free/en/venues/tacl/2024/)) · **Publisher:** [The MIT Press](https://md.research.elamur.ai/free/en/publishers/the-mit-press/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2024.tacl-1.27/) · [PDF](https://aclanthology.org/2024.tacl-1.27.pdf) · [DOI 10.1162/tacl\_a\_00660](https://doi.org/10.1162/tacl_a_00660) · [OpenAlex](https://openalex.org/W4396665924)

**Citations:** 84 citations (OpenAlex, 2026-10-14) · [56](https://md.research.elamur.ai/free/en/papers/pan-2024-automatically-correcting-large-language-models/cited-by/) in our library, of the 141,840 papers whose bibliographies we read

## Summary
Large language models still make characteristic mistakes. They invent facts, reach conclusions that do not follow from their own reasoning, and sometimes produce toxic or biased text. Fixing these flaws with human feedback (e.g., RLHF) is expensive and cannot happen in real time. This TACL survey therefore focuses on corrections driven by automated signals, which come either from the model itself (self-correction) or from external models, tools, knowledge sources, and metrics.

The authors frame the process as a medical analogy. A language model plays the patient, a critic model diagnoses problems and issues feedback, and a refine model treats either the output or the model itself. Existing work is organized along five dimensions: the error targeted, the feedback source, the feedback format, the refinement strategy, and the learning method. Methods are also grouped by when correction happens: during training, during generation, or after generation.

The paper reports no new experiments. Its contribution is a structured map of a fast-moving field, connections to earlier research, and an agenda of open problems.

## Key points
- Three error classes are targeted: hallucination, unfaithful reasoning, and toxic/biased/harmful content.
- Feedback is either a scalar score, which is easy to plug into training or decoding but not very descriptive, or natural language, which is richer. Self-Debug, for example, combines code explanations with execution results.
- The timing taxonomy has three branches:
  - **Training-time:** often impractical for closed models like GPT-4.
  - **Generation-time:** e.g., step-level guidance in reasoning or proof search.
  - **Post-hoc:** iterative revision with no parameter updates; the most flexible and explainable option.
- Proposed future directions include:
  - benchmarks and diagnostic datasets for automated correction,
  - continual lifelong self-improvement,
  - combining self-correction with model editing,
  - multi-modal self-correction.
- Published in TACL vol. 12 (2024), pp. 484–506. A continually updated reading list is hosted at github.com/teacherpeterpan/self-correction-llm-papers.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [University of California, Santa Barbara](https://md.research.elamur.ai/free/en/universities/university-of-california-santa-barbara/) — University, Santa Barbara, United States · 35 works · 9,287 citations

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 9,086 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 8,889 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 9,296 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 9,206 works

🧭 Topics (1): [🧠 Language models & pretraining](https://md.research.elamur.ai/free/en/topics/language-models/) · 1,343 works

## Cite

Cite as: Liangming Pan, Michael Saxon, Wenda Xu, Deepak Nathani, Xinyi Wang and William Yang Wang. 2024. Automatically Correcting Large Language Models: Surveying the Landscape of Diverse Automated Correction Strategies. Transactions of the Association for Computational Linguistics.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@article{pan2024automatically,
  title = {{Automatically Correcting Large Language Models: Surveying the Landscape of Diverse Automated Correction Strategies}},
  author = {Liangming Pan, Michael Saxon, Wenda Xu, Deepak Nathani, Xinyi Wang and William Yang Wang},
  journal = {Transactions of the Association for Computational Linguistics},
  year = {2024},
  doi = {10.1162/tacl\_a\_00660},
  url = {https://doi.org/10.1162/tacl_a_00660}
}
```

- [← previous Justice or Prejudice? Quantifying Biases in LLM-as-a-Judge](https://md.research.elamur.ai/free/en/papers/ye-2024-justice-prejudice-quantifying-biases-llm/)
- [next → LongRoPE: Extending LLM Context Window Beyond 2 Million Tokens](https://md.research.elamur.ai/free/en/papers/ding-2024-longrope-extending-llm-context/)
- [↑ back to papers of 2024](https://md.research.elamur.ai/free/en/papers/2024/page/2/)

## References

153 references, 72 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Afra Feyza Akyürek, Ekin Akyürek, Ashwin Kalyan, Peter Clark, Derry Tanti Wijaya, and Niket Tandon. 2023. RL4F: Generating natural language feedback with reinforcement learning for repairing model outputs. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (… [our card](https://md.research.elamur.ai/free/en/papers/akyurek-2023-rl4f-generating-natural-language-feedback/)
2. Gasper Begus, Maksymilian Dabkowski, and Ryan Rhodes. 2023. Large linguistic models: Analyzing theoretical linguistic abilities of LLMs. CoRR, abs/2305.00948. — [arXiv:2305.00948](https://arxiv.org/abs/2305.00948) [find](https://scholar.google.com/scholar?q=Gasper+Begus%2C+Maksymilian+Dabkowski%2C+and+Ryan+Rhodes%2E+2023%2E+Large+linguistic+models%3A+Analyzing+theoretical+linguistic+abilities+of+LLMs%2E+CoRR%2C+abs%2F2305%2E00948%2E+%E2%80%94+%5BarXiv%3A2305%2E00948%5D%28https%3A%2F%2Farxiv%2Eorg%2Fab)
3. Vicent Alabau, Christian Buck, Michael Carl, Francisco Casacuberta, Mercedes Garcı́a-Martı́nez, Ulrich Germann, Jesús González-Rubio, Robin L. Hill, Philipp Koehn, Luis A. Leiva, Bartolomé Mesa-Lao, Daniel Ortiz-Martı́nez, Herve Saint-Amand, Germán Sanchis-Trilles, and Chara Tsoukala. 2014. [find](https://scholar.google.com/scholar?q=Vicent+Alabau%2C+Christian+Buck%2C+Michael+Carl%2C+Francisco+Casacuberta%2C+Mercedes+Garc%C4%B1%CC%81a%2DMart%C4%B1%CC%81nez%2C+Ulrich+Germann%2C+Jesu%CC%81s+Gonza%CC%81lez%2DRubio%2C+Robin+L%2E+Hill%2C+Philipp+Koehn%2C+Luis+A%2E+Leiva%2C+Bartolome%CC%81+Mesa%2DLao)
4. Henrik Bellhäuser, Charlotte Dignath, and Maria Theobald. 2023. Daily automated feedback enhances self-regulated learning: A longitudinal randomized field experiment. Frontiers in Psychology, 14:1125873. https://doi.org/10.3389/fpsyg.2023.1125873, PubMed: 37275690 — [doi:10.3389/fpsyg.2023.1125873](https://doi.org/10.3389/fpsyg.2023.1125873) [↗](https://doi.org/10.3389/fpsyg.2023.1125873)
5. Justin Chih-Yao Chen, Swarnadeep Saha, and Mohit Bansal. 2023c. Reconcile: Round-table conference improves reasoning via consensus among diverse LLMs. CoRR, abs/2309.13007. [our card](https://md.research.elamur.ai/free/en/papers/chen-2024-reconcile-round-table-conference-improves/)
6. Ben Bogin, Matt Gardner, and Jonathan Berant. 2019. Global reasoning over database structures for text-to-SQL parsing. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), p… [find](https://scholar.google.com/scholar?q=Ben+Bogin%2C+Matt+Gardner%2C+and+Jonathan+Berant%2E+2019%2E+Global+reasoning+over+database+structures+for+text%2Dto%2DSQL+parsing%2E+In+Proceedings+of+the+2019+Conference+on+Empirical+Methods+in+Natural+Language+Pr)
7. Pinzhen Chen, Zhicheng Guo, Barry Haddow, and Kenneth Heafield. 2023d. Iterative translation refinement with large language models. CoRR, abs/2306.03856. [our card](https://md.research.elamur.ai/free/en/papers/chen-2024-iterative-translation-refinement-large-language/)
8. Xinyun Chen, Maxwell Lin, Nathanael Schärli, and Denny Zhou. 2023e. Teaching large language models to self-debug. CoRR, abs/2304 .05128. [find](https://scholar.google.com/scholar?q=Xinyun+Chen%2C+Maxwell+Lin%2C+Nathanael+Scha%CC%88rli%2C+and+Denny+Zhou%2E+2023e%2E+Teaching+large+language+models+to+self%2Ddebug%2E+CoRR%2C+abs%2F2304+%2E05128%2E)
9. Evelyn M. Boyd and Ann W. Fales. 1983. Reflective learning: Key to learning from experience. Journal of Humanistic Psychology, 23(2):99–117. https://doi.org/10 .1177/0022167883232011 [find](https://scholar.google.com/scholar?q=Evelyn+M%2E+Boyd+and+Ann+W%2E+Fales%2E+1983%2E+Reflective+learning%3A+Key+to+learning+from+experience%2E+Journal+of+Humanistic+Psychology%2C+23%282%29%3A99%E2%80%93117%2E+https%3A%2F%2Fdoi%2Eorg%2F10+%2E1177%2F0022167883232011)
10. I-Chun Chern, Steffi Chern, Shiqi Chen, Weizhe Yuan, Kehua Feng, Chunting Zhou, Junxian He, Graham Neubig, and Pengfei Liu. 2023. Factool: Factuality detection in generative AI – a tool augmented framework for multi-task and multi-domain scenarios. CoRR, abs/2307 .13528. [our card](https://md.research.elamur.ai/free/en/papers/chern-2023-factool-factuality-detection-generative-ai/)
11. Meng Cao, Yue Dong, Jiapeng Wu, and Jackie Chi Kit Cheung. 2020. Factual error correction for abstractive summarization models. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 6251–6258. https://doi.org/10 .18653/v1/2020.emnlp-main.506 [our card](https://md.research.elamur.ai/free/en/papers/cao-2020-factual-error-correction-abstractive-summarization/)
12. Nicola De Cao, Wilker Aziz, and Ivan Titov. 2021. Editing factual knowledge in language models. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 6491–6506. https://doi.org/10.18653/v1/2021 .emnlp-main.522 [↗](https://doi.org/10.18653/v1/2021)
13. Yiannis Charalambous, Norbert Tihanyi, Ridhi Jain, Youcheng Sun, Mohamed Amine Ferrag, and Lucas C. Cordeiro. 2023. A new era in software security: Towards self-healing software via large language models and formal verification. CoRR, abs/2305.14752. — [arXiv:2305.14752](https://arxiv.org/abs/2305.14752) [find](https://scholar.google.com/scholar?q=Yiannis+Charalambous%2C+Norbert+Tihanyi%2C+Ridhi+Jain%2C+Youcheng+Sun%2C+Mohamed+Amine+Ferrag%2C+and+Lucas+C%2E+Cordeiro%2E+2023%2E+A+new+era+in+software+security%3A+Towards+self%2Dhealing+software+via+large+language+mod)
14. Elizabeth Clark, Tal August, Sofia Serrano, Nikita Haduong, Suchin Gururangan, and Noah A. Smith. 2021. All that’s ‘human’ is not gold: Evaluating human evaluation of generated text. In Processings of the 59th Annual Meeting of the Association for Computational Linguistics (ACL), pages 7282–7296. ht… [find](https://scholar.google.com/scholar?q=Elizabeth+Clark%2C+Tal+August%2C+Sofia+Serrano%2C+Nikita+Haduong%2C+Suchin+Gururangan%2C+and+Noah+A%2E+Smith%2E+2021%2E+All+that%E2%80%99s+%E2%80%98human%E2%80%99+is+not+gold%3A+Evaluating+human+evaluation+of+generated+text%2E+In+Processings+of)
15. Roi Cohen, May Hamri, Mor Geva, and Amir Globerson. 2023. LM vs LM: Detecting factual errors via cross examination. CoRR, abs/2305 .13281. https://doi.org/10.18653/v1/2023.emnlp-main.778 [↗](https://doi.org/10.18653/v1/2023.emnlp-main.778) [our card](https://md.research.elamur.ai/free/en/papers/cohen-2023-lm-vs-lm-detecting-factual/)
16. Antonia Creswell and Murray Shanahan. 2022. Faithful reasoning using large language models. CoRR, abs/2208.14271. [find](https://scholar.google.com/scholar?q=Antonia+Creswell+and+Murray+Shanahan%2E+2022%2E+Faithful+reasoning+using+large+language+models%2E+CoRR%2C+abs%2F2208%2E14271%2E)
17. Ishita Dasgupta, Andrew K. Lampinen, Stephanie C. Y. Chan, Antonia Creswell, Dharshan Kumaran, James L. McClelland, and Felix Hill. 2022. Language models show human-like content effects on reasoning. CoRR, abs/2207 .07051. [find](https://scholar.google.com/scholar?q=Ishita+Dasgupta%2C+Andrew+K%2E+Lampinen%2C+Stephanie+C%2E+Y%2E+Chan%2C+Antonia+Creswell%2C+Dharshan+Kumaran%2C+James+L%2E+McClelland%2C+and+Felix+Hill%2E+2022%2E+Language+models+show+human%2Dlike+content+effects+on+reasoning%2E+)
18. Angelica Chen, Jérémy Scheurer, Tomasz Korbak, Jon Ander Campos, Jun Shern Chan, Samuel R. Bowman, Kyunghyun Cho, and Ethan Perez. 2023a. Improving code generation by training with natural language feedback. CoRR, abs/ 2303.16749. [find](https://scholar.google.com/scholar?q=Angelica+Chen%2C+Je%CC%81re%CC%81my+Scheurer%2C+Tomasz+Korbak%2C+Jon+Ander+Campos%2C+Jun+Shern+Chan%2C+Samuel+R%2E+Bowman%2C+Kyunghyun+Cho%2C+and+Ethan+Perez%2E+2023a%2E+Improving+code+generation+by+training+with+natural+language+)
19. Sumanth Dathathri, Andrea Madotto, Janice Lan, Jane Hung, Eric Frank, Piero Molino, Jason Yosinski, and Rosanne Liu. 2020. Plug and play language models: A simple approach to controlled text generation. In Proceedings of the 8th International Conference on Learning Representations (ICLR). [our card](https://md.research.elamur.ai/free/en/papers/dathathri-2019-plug-play-language-models-simple/)
20. Bei Chen, Fengji Zhang, Anh Nguyen, Daoguang Zan, Zeqi Lin, Jian-Guang Lou, and Weizhu Chen. 2023b. Codet: Code generation with generated tests. In Proceedings of the 11th International Conference on Learning Representations (ICLR). [find](https://scholar.google.com/scholar?q=Bei+Chen%2C+Fengji+Zhang%2C+Anh+Nguyen%2C+Daoguang+Zan%2C+Zeqi+Lin%2C+Jian%2DGuang+Lou%2C+and+Weizhu+Chen%2E+2023b%2E+Codet%3A+Code+generation+with+generated+tests%2E+In+Proceedings+of+the+11th+International+Conference+on+)
21. Félix do Carmo, Dimitar Shterionov, Joss Moorkens, Joachim Wagner, Murhaf Hossari, Eric Paquin, Dag Schmidtke, Declan Groves, and Andy Way. 2021. A review of the state-of-the-art in automatic post-editing. Machine Translation, 35(2):101–143. https://doi.org/10.1007/s10590-020-09252-y, PubMed: 3472… — [doi:10.1007/s10590-020-09252-y](https://doi.org/10.1007/s10590-020-09252-y) [↗](https://doi.org/10.1007/s10590-020-09252-y)
22. Markus Freitag, David Grangier, Qijun Tan, and Bowen Liang. 2022. High quality rather than high model probability: Minimum bayes risk decoding with neural metrics. Transactions of the Association for Computational Linguistics (TACL), pages 811–825. https://doi.org/10.1162/tacl_a_00491 [↗](https://doi.org/10.1162/tacl_a_00491) [our card](https://md.research.elamur.ai/free/en/papers/freitag-2022-high-quality-rather-than-high/)
23. Yao Fu, Hao Peng, Tushar Khot, and Mirella Lapata. 2023. Improving language model negotiation with self-play and in-context learning from AI feedback. CoRR, abs/2305.10142. [our card](https://md.research.elamur.ai/free/en/papers/fu-2023-improving-language-model-negotiation-self/)
24. Yilun Du, Shuang Li, Antonio Torralba, Joshua B. Tenenbaum, and Igor Mordatch. 2023. Improving factuality and reasoning in language models through multiagent debate. CoRR, abs/2305.14325. [our card](https://md.research.elamur.ai/free/en/papers/du-2023-improving-factuality-reasoning-language-models/)
25. Deep Ganguli, Amanda Askell, Nicholas Schiefer, Thomas I. Liao, Kamile Lukosiute, Anna Chen, Anna Goldie, Azalia Mirhoseini, Catherine Olsson, Danny Hernandez, Dawn Drain, Dustin Li, Eli Tran-Johnson, Ethan Perez, Jackson Kernion, Jamie Kerr, Jared Mueller, Joshua Landau, Kamal Ndousse, Karina Nguye… [our card](https://md.research.elamur.ai/free/en/papers/ganguli-2023-capacity-moral-self-correction-large/)
26. Yann Dubois, Xuechen Li, Rohan Taori, Tianyi Zhang, Ishaan Gulrajani, Jimmy Ba, Carlos Guestrin, Percy Liang, and Tatsunori B. Hashimoto. 2023. Alpacafarm: A simulation framework for methods that learn from human feedback. CoRR, abs/2305.14387. [our card](https://md.research.elamur.ai/free/en/papers/dubois-2023-alpacafarm-simulation-framework-methods-that/)
27. Tobias Falke, Leonardo F. R. Ribeiro, Prasetya Ajie Utama, Ido Dagan, and Iryna Gurevych. 2019. Ranking generated summaries by correctness: An interesting but challenging application for natural language inference. In Proceedings of the 57st Annual Meeting of the Association for Computational Lingui… [our card](https://md.research.elamur.ai/free/en/papers/falke-2019-ranking-generated-summaries-correctness-interesting/)
28. Patrick Fernandes, Aman Madaan, Emmy Liu, António Farinhas, Pedro Henrique Martins, Amanda Bertsch, José G. C. de Souza, Shuyan Zhou, Tongshuang Wu, Graham Neubig, and André F. T. Martins. 2023. Bridging the gap: A survey on integrating (human) feedback for natural language generation. CoRR, abs/… [our card](https://md.research.elamur.ai/free/en/papers/fernandes-2023-bridging-gap-survey-integrating-human/)
29. Ge Gao, Hung-Ting Chen, Yoav Artzi, and Eunsol Choi. 2023a. Continually improving extractive QA via human feedback. CoRR, abs/ 2305.12473. https://doi.org/10.18653/v1/2023.emnlp-main.27 [↗](https://doi.org/10.18653/v1/2023.emnlp-main.27)
30. Luyu Gao, Zhuyun Dai, Panupong Pasupat, Anthony Chen, Arun Tejasvi Chaganty, Yicheng Fan, Vincent Y. Zhao, Ni Lao, Hongrae Lee, Da-Cheng Juan, and Kelvin Guu. 2023b. Rarr: Researching and revising what language models say, using language models. In Proceedings of the 61th Annual Meeting of the Assoc… [our card](https://md.research.elamur.ai/free/en/papers/gao-2023-rarr-researching-revising-what-language/)
31. Emanuela Ferretti, Kristina Rohde, Gregory P. Moore, and Thierry Daboval. 2019. Catch the moment: The power of turning mistakes into ‘precious’ learning opportunities. Paediatrics & Child Health, 24(3):156–159. https://doi.org/10.1093/pch/pxy102, PubMed: 31111832 — [doi:10.1093/pch/pxy102](https://doi.org/10.1093/pch/pxy102) [↗](https://doi.org/10.1093/pch/pxy102)
32. Emily First, Markus N. Rabe, Talia Ringer, and Yuriy Brun. 2023. Baldur: Whole-proof generation and repair with large language models. CoRR, abs/2303.04910. https://doi.org/10.1145/3611643.3616243 [↗](https://doi.org/10.1145/3611643.3616243)
33. Samuel Gehman, Suchin Gururangan, Maarten Sap, Yejin Choi, and Noah A. Smith. 2020. RealToxicityPrompts: Evaluating neural toxic degeneration in language models. In Findings of the Association for Computational Linguistics: EMNLP 2020, pages 3356–3369. https://doi.org/10.18653/v1/2020 .findings-emnl… [↗](https://doi.org/10.18653/v1/2020) [our card](https://md.research.elamur.ai/free/en/papers/gehman-2020-realtoxicityprompts-evaluating-neural-toxic-degeneration/)
34. Zelalem Gero, Chandan Singh, Hao Cheng, Tristan Naumann, Michel Galley, Jianfeng Gao, and Hoifung Poon. 2023. Self-verification improves few-shot clinical information extraction. CoRR, abs/2306.00024. training (rest) for language modeling. CoRR, abs/2308.08998. [find](https://scholar.google.com/scholar?q=Zelalem+Gero%2C+Chandan+Singh%2C+Hao+Cheng%2C+Tristan+Naumann%2C+Michel+Galley%2C+Jianfeng+Gao%2C+and+Hoifung+Poon%2E+2023%2E+Self%2Dverification+improves+few%2Dshot+clinical+information+extraction%2E+CoRR%2C+abs%2F2306%2E00024%2E)
35. Biyang Guo, Xin Zhang, Ziyuan Wang, Minqi Jiang, Jinran Nie, Yuxuan Ding, Jianwei Yue, and Yupeng Wu. 2023. How close is chatgpt to human experts? Comparison corpus, evaluation, and detection. CoRR, abs/2301.07597. [our card](https://md.research.elamur.ai/free/en/papers/guo-2023-close-chatgpt-human-experts-comparison/)
36. Shibo Hao, Yi Gu, Haodi Ma, Joshua Jiahua Hong, Zhen Wang, Daisy Zhe Wang, and Zhiting Hu. 2023. Reasoning with language model is planning with world model. CoRR, abs/2305.14992. https://doi.org/10 .18653/v1/2023.emnlp-main.507 [find](https://scholar.google.com/scholar?q=Shibo+Hao%2C+Yi+Gu%2C+Haodi+Ma%2C+Joshua+Jiahua+Hong%2C+Zhen+Wang%2C+Daisy+Zhe+Wang%2C+and+Zhiting+Hu%2E+2023%2E+Reasoning+with+language+model+is+planning+with+world+model%2E+CoRR%2C+abs%2F2305%2E14992%2E+https%3A%2F%2Fdoi%2Eorg%2F10+%2E1)
37. Amelia Glaese, Nat McAleese, Maja Trebacz, John Aslanides, Vlad Firoiu, Timo Ewalds, Maribeth Rauh, Laura Weidinger, Martin J. Chadwick, Phoebe Thacker, Lucy Campbell-Gillingham, Jonathan Uesato, Po-Sen Huang, Ramona Comanescu, Fan Yang, Abigail See, Sumanth Dathathri, Rory Greig, Charlie Chen, Doug… [our card](https://md.research.elamur.ai/free/en/papers/glaese-2022-improving-alignment-dialogue-agents-targeted/)
38. Hangfeng He, Hongming Zhang, and Dan Roth. 2023. Rethinking with retrieval: Faithful large language model inference. CoRR, abs/2301 .00303. [find](https://scholar.google.com/scholar?q=Hangfeng+He%2C+Hongming+Zhang%2C+and+Dan+Roth%2E+2023%2E+Rethinking+with+retrieval%3A+Faithful+large+language+model+inference%2E+CoRR%2C+abs%2F2301+%2E00303%2E)
39. Pengcheng He, Xiaodong Liu, Jianfeng Gao, and Weizhu Chen. 2021. Deberta: Decoding-enhanced bert with disentangled attention. In Proceedings of The 9th International Conference on Learning Representations (ICLR). [our card](https://md.research.elamur.ai/free/en/papers/he-2020-deberta-decoding-enhanced-bert-disentangled/)
40. Alec Helbling, Mansi Phute, Matthew Hull, and Duen Horng Chau. 2023. LLM self defense: By self examination, LLMs know they are being tricked. CoRR, abs/2308.07308. [find](https://scholar.google.com/scholar?q=Alec+Helbling%2C+Mansi+Phute%2C+Matthew+Hull%2C+and+Duen+Horng+Chau%2E+2023%2E+LLM+self+defense%3A+By+self+examination%2C+LLMs+know+they+are+being+tricked%2E+CoRR%2C+abs%2F2308%2E07308%2E)
41. Dongyoung Go, Tomasz Korbak, Germán Kruszewski, Jos Rozen, Nahyeon Ryu, and Marc Dymetman. 2023. Aligning language models with preferences through f-divergence minimization. CoRR, abs/2302.08215. [find](https://scholar.google.com/scholar?q=Dongyoung+Go%2C+Tomasz+Korbak%2C+Germa%CC%81n+Kruszewski%2C+Jos+Rozen%2C+Nahyeon+Ryu%2C+and+Marc+Dymetman%2E+2023%2E+Aligning+language+models+with+preferences+through+f%2Ddivergence+minimization%2E+CoRR%2C+abs%2F2302%2E08215%2E)
42. Jason Hoelscher-Obermaier, Julia Persson, Esben Kran, Ioannis Konstas, and Fazl Barez. 2023. Detecting edit failures in large language models: An improved specificity benchmark. In Findings of the Association for Computational Linguistics: ACL 2023, pages 11548–11559. https://doi.org/10.18653/v1/202… [↗](https://doi.org/10.18653/v1/202…)
43. Olga Golovneva, Moya Chen, Spencer Poff, Martin Corredor, Luke Zettlemoyer, Maryam Fazel-Zarandi, and Asli Celikyilmaz. 2023. ROSCOE: A suite of metrics for scoring step-by-step reasoning. In Proceedings of the 11th International Conference on Learning Representations (ICLR). [our card](https://md.research.elamur.ai/free/en/papers/golovneva-2022-roscoe-suite-metrics-scoring-step/)
44. Ari Holtzman, Jan Buys, Maxwell Forbes, Antoine Bosselut, David Golub, and Yejin Choi. 2018. Learning to write with cooperative discriminators. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (ACL), pages 1638–1649. https://doi.org/10.18653/v1/P18 -1152 [↗](https://doi.org/10.18653/v1/P18) [our card](https://md.research.elamur.ai/free/en/papers/holtzman-2018-learning-write-cooperative-discriminators/)
45. Zhibin Gou, Zhihong Shao, Yeyun Gong, Yelong Shen, Yujiu Yang, Nan Duan, and Weizhu Chen. 2023. CRITIC: Large language models can self-correct with tool-interactive critiquing. CoRR, abs/2305.11738. [our card](https://md.research.elamur.ai/free/en/papers/gou-2023-critic-large-language-models-self/)
46. Caglar Gulcehre, Tom Le Paine, Srivatsan Srinivasan, Ksenia Konyushkova, Lotte Weerts, Abhishek Sharma, Aditya Siddhant, Alex Ahern, Miaosen Wang, Chenjie Gu, Wolfgang Macherey, Arnaud Doucet, Orhan Firat, and Nando de Freitas. 2023. Reinforced self-Ruixin Hong, Hongming Zhang, Xinyu Pang, Dong Yu,… [find](https://scholar.google.com/scholar?q=Caglar+Gulcehre%2C+Tom+Le+Paine%2C+Srivatsan+Srinivasan%2C+Ksenia+Konyushkova%2C+Lotte+Weerts%2C+Abhishek+Sharma%2C+Aditya+Siddhant%2C+Alex+Ahern%2C+Miaosen+Wang%2C+Chenjie+Gu%2C+Wolfgang+Macherey%2C+Arnaud+Doucet%2C+Orhan+F)
47. Jie Huang, Xinyun Chen, Swaroop Mishra, Huaixiu Steven Zheng, Adams Wei Yu, Xinying Song, and Denny Zhou. 2023. Large language models cannot self-correct reasoning yet. CoRR, abs/2310.01798. [our card](https://md.research.elamur.ai/free/en/papers/huang-2023-large-language-models-cannot-self/)
48. Muhammad Khalifa, Lajanugen Logeswaran, Moontae Lee, Honglak Lee, and Lu Wang. 2023. Discriminator-guided multi-step reasoning with language models. CoRR, abs/2305 .14934. [find](https://scholar.google.com/scholar?q=Muhammad+Khalifa%2C+Lajanugen+Logeswaran%2C+Moontae+Lee%2C+Honglak+Lee%2C+and+Lu+Wang%2E+2023%2E+Discriminator%2Dguided+multi%2Dstep+reasoning+with+language+models%2E+CoRR%2C+abs%2F2305+%2E14934%2E)
49. Jiaxin Huang, Shixiang Shane Gu, Le Hou, Yuexin Wu, Xuezhi Wang, Hongkun Yu, and Jiawei Han. 2022. Large language models can self-improve. CoRR, abs/2210.11610. https://doi.org/10.18653/v1/2023 .emnlp-main.67 [↗](https://doi.org/10.18653/v1/2023) [our card](https://md.research.elamur.ai/free/en/papers/huang-2023-large-language-models-can-self/)
50. Geunwoo Kim, Pierre Baldi, and Stephen McAleer. 2023. Language models can solve computer tasks. CoRR, abs/2303.17491. [our card](https://md.research.elamur.ai/free/en/papers/kim-2023-language-models-solve-computer-tasks/)
51. Shuyang Jiang, Yuhao Wang, and Yu Wang. 2023. Selfevolve: A code evolution framework via large language models. CoRR, abs/2306.02907. — [arXiv:2306.02907](https://arxiv.org/abs/2306.02907) [find](https://scholar.google.com/scholar?q=Shuyang+Jiang%2C+Yuhao+Wang%2C+and+Yu+Wang%2E+2023%2E+Selfevolve%3A+A+code+evolution+framework+via+large+language+models%2E+CoRR%2C+abs%2F2306%2E02907%2E+%E2%80%94+%5BarXiv%3A2306%2E02907%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2306%2E02907%29)
52. Jaehun Jung, Lianhui Qin, Sean Welleck, Faeze Brahman, Chandra Bhagavatula, Ronan Le Bras, and Yejin Choi. 2022. Maieutic prompting: Logically consistent reasoning with recursive explanations. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 12… [our card](https://md.research.elamur.ai/free/en/papers/jung-2022-maieutic-prompting-logically-consistent-reasoning/)
53. James Kirkpatrick, Razvan Pascanu, Neil C. Rabinowitz, Joel Veness, Guillaume Desjardins, Andrei A. Rusu, Kieran Milan, John Quan, Tiago Ramalho, Agnieszka Grabska-Barwinska, Demis Hassabis, Claudia Clopath, Dharshan Kumaran, and Raia Hadsell. 2016. Overcoming catastrophic forgetting in neural netwo… [find](https://scholar.google.com/scholar?q=James+Kirkpatrick%2C+Razvan+Pascanu%2C+Neil+C%2E+Rabinowitz%2C+Joel+Veness%2C+Guillaume+Desjardins%2C+Andrei+A%2E+Rusu%2C+Kieran+Milan%2C+John+Quan%2C+Tiago+Ramalho%2C+Agnieszka+Grabska%2DBarwinska%2C+Demis+Hassabis%2C+Claudia+C)
54. Takeshi Kojima, Shixiang Shane Gu, Machel Reid, Yutaka Matsuo, and Yusuke Iwasawa. 2022. Large language models are zero-shot reasoners. In Proceedings of the 2022 Annual Conference on Neural Information Processing Systems (NeurIPS). [our card](https://md.research.elamur.ai/free/en/papers/kojima-2022-large-language-models-zero-shot/)
55. Julia Kreutzer, Shahram Khadivi, Evgeny Matusov, and Stefan Riezler. 2018. Can neural machine translation be improved with user feedback? In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HIT). htt… [find](https://scholar.google.com/scholar?q=Julia+Kreutzer%2C+Shahram+Khadivi%2C+Evgeny+Matusov%2C+and+Stefan+Riezler%2E+2018%2E+Can+neural+machine+translation+be+improved+with+user+feedback%3F+In+Proceedings+of+the+2018+Conference+of+the+North+American+Ch)
56. Saurav Kadavath, Tom Conerly, Amanda Askell, Tom Henighan, Dawn Drain, Ethan Perez, Nicholas Schiefer, Zac Hatfield-Dodds, Nova DasSarma, Eli Tran-Johnson, Scott Johnston, Sheer El Showk, Andy Jones, Nelson Elhage, Tristan Hume, Anna Chen, Yuntao Bai, Sam Bowman, Stanislav Fort, Deep Ganguli, Danny… [find](https://scholar.google.com/scholar?q=Saurav+Kadavath%2C+Tom+Conerly%2C+Amanda+Askell%2C+Tom+Henighan%2C+Dawn+Drain%2C+Ethan+Perez%2C+Nicholas+Schiefer%2C+Zac+Hatfield%2DDodds%2C+Nova+DasSarma%2C+Eli+Tran%2DJohnson%2C+Scott+Johnston%2C+Sheer+El+Showk%2C+Andy+Jones%2C+)
57. Hung Le, Yue Wang, Akhilesh Deepak Gotmare, Silvio Savarese, and Steven Chu-Hong Hoi. 2022. Coderl: Mastering code generation through pretrained models and deep reinforcement learning. In Proceedings of the Annual Conference on Neural Information Processing Systems (NeurIPS). [our card](https://md.research.elamur.ai/free/en/papers/le-2022-coderl-mastering-code-generation-through/)
58. Dongjun Lee. 2019. Clause-wise and recursive decoding for complex and cross-domain text-to-SQL generation. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 6045–60… [find](https://scholar.google.com/scholar?q=Dongjun+Lee%2E+2019%2E+Clause%2Dwise+and+recursive+decoding+for+complex+and+cross%2Ddomain+text%2Dto%2DSQL+generation%2E+In+Proceedings+of+the+2019+Conference+on+Empirical+Methods+in+Natural+Language+Processing+and)
59. Pei Ke, Bosi Wen, Zhuoer Feng, Xiao Liu, Xuanyu Lei, Jiale Cheng, Shengyuan Wang, Aohan Zeng, Yuxiao Dong, Hongning Wang, Jie Tang, and Minlie Huang. 2023. CritiqueLLM: Scaling LLM-as-critic for effective and explainable evaluation of large language model generation. CoRR, abs/2311.18702. [our card](https://md.research.elamur.ai/free/en/papers/ke-2023-critiquellm-informative-critique-generation/)
60. Amol Kelkar, Rohan Relan, Vaishali Bhardwaj, Saurabh Vaichal, and Peter Relan. 2020. Bertrand-dr: Improving text-to-sql using a discriminative re-ranker. CoRR, abs/2002.00557. — [arXiv:2002.00557](https://arxiv.org/abs/2002.00557) [find](https://scholar.google.com/scholar?q=Amol+Kelkar%2C+Rohan+Relan%2C+Vaishali+Bhardwaj%2C+Saurabh+Vaichal%2C+and+Peter+Relan%2E+2020%2E+Bertrand%2Ddr%3A+Improving+text%2Dto%2Dsql+using+a+discriminative+re%2Dranker%2E+CoRR%2C+abs%2F2002%2E00557%2E+%E2%80%94+%5BarXiv%3A2002%2E00557%5D%28htt)
61. Kyungjae Lee, Wookje Han, Seung-won Hwang, Hwaran Lee, Joonsuk Park, and Sang-Woo Lee. 2022. Plug-and-play adaptation for continuously-updated QA. In Findings of the Association for Computational Linguistics: ACL 2022, pages 438–447. https://doi.org/10.18653/v1/2022.findings-acl.37 [↗](https://doi.org/10.18653/v1/2022.findings-acl.37)
62. Yifei Li, Zeqi Lin, Shizhuo Zhang, Qiang Fu, Bei Chen, Jian-Guang Lou, and Weizhu Chen. 2023d. Making language models better reasoners with step-aware verifier. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (ACL), pages 5315–5333. https://doi.org/10 .1865… [our card](https://md.research.elamur.ai/free/en/papers/li-2023-making-language-models-better-reasoners/)
63. WonKee Lee, Baikjin Jung, Jaehun Shin, and Jong-Hyeok Lee. 2021. Adaptation of back-translation to automatic post-editing for synthetic data generation. In Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics (EACL), pages 3685–3691. https://doi… [find](https://scholar.google.com/scholar?q=WonKee+Lee%2C+Baikjin+Jung%2C+Jaehun+Shin%2C+and+Jong%2DHyeok+Lee%2E+2021%2E+Adaptation+of+back%2Dtranslation+to+automatic+post%2Dediting+for+synthetic+data+generation%2E+In+Proceedings+of+the+16th+Conference+of+the+Eu)
64. Hunter Lightman, Vineet Kosaraju, Yura Burda, Harrison Edwards, Bowen Baker, Teddy Lee, Jan Leike, John Schulman, Ilya Sutskever, and Karl Cobbe. 2023. Let’s verify step by step. CoRR, abs/2305.20050. [our card](https://md.research.elamur.ai/free/en/papers/lightman-2023-let-s-verify-step-step/)
65. Sharon Levy, Emily Allaway, Melanie Subbiah, Lydia Chilton, Desmond Patton, Kathleen McKeown, and William Yang Wang. 2022. SafeText: A benchmark for exploring physical safety in language models. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages… [find](https://scholar.google.com/scholar?q=Sharon+Levy%2C+Emily+Allaway%2C+Melanie+Subbiah%2C+Lydia+Chilton%2C+Desmond+Patton%2C+Kathleen+McKeown%2C+and+William+Yang+Wang%2E+2022%2E+SafeText%3A+A+benchmark+for+exploring+physical+safety+in+language+models%2E+In+Pr)
66. Stephanie Lin, Jacob Hilton, and Owain Evans. 2022. TruthfulQA: Measuring how models mimic human falsehoods. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (ACL), pages 3214–3252. https://doi.org/10 .18653/v1/2022.acl-long.229 [our card](https://md.research.elamur.ai/free/en/papers/lin-2022-truthfulqa-measuring-how-models-mimic/)
67. Yen-Ting Lin and Yun-Nung Chen. 2023. LLM-eval: Unified multi-dimensional automatic evaluation for open-domain conversations with large language models. CoRR, abs/2305.13711. [our card](https://md.research.elamur.ai/free/en/papers/lin-2023-llm-eval-unified-multi-dimensional/)
68. Sharon Levy, Michael Saxon, and William Yang Wang. 2021. Investigating memorization of conspiracy theories in text generation. In Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021, pages 4718–4729, Online. Association for Computational Linguistics. https://doi.org/10.18653/v… [find](https://scholar.google.com/scholar?q=Sharon+Levy%2C+Michael+Saxon%2C+and+William+Yang+Wang%2E+2021%2E+Investigating+memorization+of+conspiracy+theories+in+text+generation%2E+In+Findings+of+the+Association+for+Computational+Linguistics%3A+ACL%2DIJCNLP+)
69. Zhen Lin, Shubhendu Trivedi, and Jimeng Sun. 2023. Generating with confidence: Uncertainty quantification for black-box large language models. CoRR, abs/2305.19187. [find](https://scholar.google.com/scholar?q=Zhen+Lin%2C+Shubhendu+Trivedi%2C+and+Jimeng+Sun%2E+2023%2E+Generating+with+confidence%3A+Uncertainty+quantification+for+black%2Dbox+large+language+models%2E+CoRR%2C+abs%2F2305%2E19187%2E)
70. Hao Liu, Carmelo Sferrazza, and Pieter Abbeel. 2023a. Chain of hindsight aligns language models with feedback. CoRR, abs/2302.02676. [find](https://scholar.google.com/scholar?q=Hao+Liu%2C+Carmelo+Sferrazza%2C+and+Pieter+Abbeel%2E+2023a%2E+Chain+of+hindsight+aligns+language+models+with+feedback%2E+CoRR%2C+abs%2F2302%2E02676%2E)
71. Junyi Li, Xiaoxue Cheng, Wayne Xin Zhao, Jian-Yun Nie, and Ji-Rong Wen. 2023a. Halueval: A large-scale hallucination evaluation benchmark for large language models. CoRR, abs/2305.11747. [our card](https://md.research.elamur.ai/free/en/papers/li-2023-halueval-large-scale-hallucination-evaluation/)
72. Jiacheng Liu, Ramakanth Pasunuru, Hannaneh Hajishirzi, Yejin Choi, and Asli Celikyilmaz. 2023b. Crystal: Introspective reasoners reinforced with self-feedback. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 11557–11572. https://doi.org/10 .18… [find](https://scholar.google.com/scholar?q=Jiacheng+Liu%2C+Ramakanth+Pasunuru%2C+Hannaneh+Hajishirzi%2C+Yejin+Choi%2C+and+Asli+Celikyilmaz%2E+2023b%2E+Crystal%3A+Introspective+reasoners+reinforced+with+self%2Dfeedback%2E+In+Proceedings+of+the+2023+Conference+on)
73. Miaoran Li, Baolin Peng, and Zhu Zhang. 2023b. Self-checker: Plug-and-play modules for fact-checking with large language models. CoRR, abs/2305.14623. [our card](https://md.research.elamur.ai/free/en/papers/li-2024-self-checker-plug-play-modules/)
74. Ruosen Li, Teerth Patel, and Xinya Du. 2023c. PRD: Peer rank and discussion improve large language model based evaluations. CoRR, abs/ 2307.02762. [find](https://scholar.google.com/scholar?q=Ruosen+Li%2C+Teerth+Patel%2C+and+Xinya+Du%2E+2023c%2E+PRD%3A+Peer+rank+and+discussion+improve+large+language+model+based+evaluations%2E+CoRR%2C+abs%2F+2307%2E02762%2E)
75. Yixin Liu and Pengfei Liu. 2021. Simcls: A simple framework for contrastive learning of abstractive summarization. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (ACL/IJCNLP), pages… [our card](https://md.research.elamur.ai/free/en/papers/liu-2021-simcls-simple-framework-contrastive-learning/)
76. Xiang Li, John Thickstun, Ishaan Gulrajani, Percy Liang, and Tatsunori B. Hashimoto. 2022. Diffusion-lm improves controllable text generation. In Proceedings of the Annual Conference on Neural Information Processing Systems (NeurIPS). [our card](https://md.research.elamur.ai/free/en/papers/li-2022-diffusion-lm-improves-controllable-text/)
77. Manuel London, Valerie I. Sessa, and Loren A. Shelley. 2023. Developing self-awareness: Learning processes for self-and interpersonal growth. Annual Review of Organizational Psychology and Organizational Behavior, 10:261–288. https://doi.org/10.1146/annurev-orgpsych-120920-044531 [↗](https://doi.org/10.1146/annurev-orgpsych-120920-044531)
78. Rahul Gupta. 2023. Flirt: Feedback loop in-context red teaming. CoRR, abs/2308.04265. [find](https://scholar.google.com/scholar?q=Rahul+Gupta%2E+2023%2E+Flirt%3A+Feedback+loop+in%2Dcontext+red+teaming%2E+CoRR%2C+abs%2F2308%2E04265%2E)
79. Janet Metcalfe. 2017. Learning from errors. Annual Review of Psychology, 68:465–489. https://doi.org/10.1146/annurev -psych-010416-044022, PubMed: 27648988 [↗](https://doi.org/10.1146/annurev)
80. Ning Miao, Yee Whye Teh, and Tom Rainforth. 2023. Selfcheck: Using LLMs to zero-shot check their own step-by-step reasoning. CoRR, abs/2308.00436. [find](https://scholar.google.com/scholar?q=Ning+Miao%2C+Yee+Whye+Teh%2C+and+Tom+Rainforth%2E+2023%2E+Selfcheck%3A+Using+LLMs+to+zero%2Dshot+check+their+own+step%2Dby%2Dstep+reasoning%2E+CoRR%2C+abs%2F2308%2E00436%2E)
81. Ximing Lu, Sean Welleck, Jack Hessel, Liwei Jiang, Lianhui Qin, Peter West, Prithviraj Ammanabrolu, and Yejin Choi. 2022. QUARK: Controllable text generation with reinforced unlearning. In Proceedings of the Annual Conference on Neural Information Processing Systems (NeurIPS). [our card](https://md.research.elamur.ai/free/en/papers/lu-2022-quark-controllable-text-generation-reinforced/)
82. Chenyang Lyu, Jitao Xu, and Longyue Wang. 2023a. New trends in machine translation using large language models: Case examples with chatgpt. CoRR, abs/2305.01181. https://doi.org/10.18653/v1/2023.emnlp-main.1036 [↗](https://doi.org/10.18653/v1/2023.emnlp-main.1036) [our card](https://md.research.elamur.ai/free/en/papers/wang-2023-document-level-machine-translation-large/)
83. Qing Lyu, Shreya Havaldar, Adam Stein, Li Zhang, Delip Rao, Eric Wong, Marianna Apidianaki, and Chris Callison-Burch. 2023b. Faithful chain-of-thought reasoning. CoRR, abs/2301.13379. https://doi.org/10.18653/v1/2023.ijcnlp-main.20 [↗](https://doi.org/10.18653/v1/2023.ijcnlp-main.20) [our card](https://md.research.elamur.ai/free/en/papers/lyu-2023-faithful-chain-thought-reasoning/)
84. Aman Madaan, Niket Tandon, Peter Clark, and Yiming Yang. 2022. Memory-assisted prompt editing to improve GPT-3 after deployment. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 2833–2861. https://doi.org/10.18653/v1/2022.emnlp-main.183 [↗](https://doi.org/10.18653/v1/2022.emnlp-main.183)
85. Sewon Min, Kalpesh Krishna, Xinxi Lyu, Mike Lewis, Wen-tau Yih, Pang Wei Koh, Mohit Iyyer, Luke Zettlemoyer, and Hannaneh Hajishirzi. 2023. Factscore: Fine-grained atomic evaluation of factual precision in long form text generation. CoRR, abs/2305.14251. https://doi.org/10.18653/v1/2023 .emnlp-main.… [↗](https://doi.org/10.18653/v1/2023) [our card](https://md.research.elamur.ai/free/en/papers/min-2023-factscore-fine-grained-atomic-evaluation/)
86. Shikhar Murty, Christopher D. Manning, Scott M. Lundberg, and Marco Túlio Ribeiro. 2022. Fixing model bugs with natural language patches. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 11600–11613. https://doi.org/10.18653/v1/2022 .emnlp-mai… [↗](https://doi.org/10.18653/v1/2022)
87. Deepak Nathani, David Wang, Liangming Pan, and William Wang. 2023. MAF: Multi-aspect feedback for improving reasoning in large language models. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 6591–6616. https://doi.org/10 .18653/v1/2023.emnlp-… [find](https://scholar.google.com/scholar?q=Deepak+Nathani%2C+David+Wang%2C+Liangming+Pan%2C+and+William+Wang%2E+2023%2E+MAF%3A+Multi%2Daspect+feedback+for+improving+reasoning+in+large+language+models%2E+In+Proceedings+of+the+2023+Conference+on+Empirical+Metho)
88. Aman Madaan, Niket Tandon, Prakhar Gupta, Skyler Hallinan, Luyu Gao, Sarah Wiegreffe, Uri Alon, Nouha Dziri, Shrimai Prabhumoye, Yiming Yang, Sean Welleck, Bodhisattwa Prasad Majumder, Shashank Gupta, Amir Yazdanbakhsh, and Peter Clark. 2023. Self-refine: Iterative refinement with self-feedback. CoR… [find](https://scholar.google.com/scholar?q=Aman+Madaan%2C+Niket+Tandon%2C+Prakhar+Gupta%2C+Skyler+Hallinan%2C+Luyu+Gao%2C+Sarah+Wiegreffe%2C+Uri+Alon%2C+Nouha+Dziri%2C+Shrimai+Prabhumoye%2C+Yiming+Yang%2C+Sean+Welleck%2C+Bodhisattwa+Prasad+Majumder%2C+Shashank+Gupta%2C)
89. Ansong Ni, Srini Iyer, Dragomir Radev, Ves Stoyanov, Wen-tau Yih, Sida I. Wang, and Xi Victoria Lin. 2023. LEVER: Learning to verify language-to-code generation with execution. In Proceedings of the 40th International Conference on Machine Learning (ICML). [our card](https://md.research.elamur.ai/free/en/papers/ni-2023-lever-learning-verify-language-code/)
90. Potsawee Manakul, Adian Liusie, and Mark J. F. Gales. 2023. Selfcheckgpt: Zero-resource black-box hallucination detection for generative large language models. CoRR, abs/2303.08896. https://doi.org/10.18653/v1/2023 .emnlp-main.557 [↗](https://doi.org/10.18653/v1/2023)
91. Ninareh Mehrabi, Palash Goyal, Christophe Dupuy, Qian Hu, Shalini Ghosh, Richard Zemel, Kai-Wei Chang, Aram Galstyan, and Theo X. Olausson, Jeevana Priya Inala, Chenglong Wang, Jianfeng Gao, and Armando Solar-Lezama. 2023. Demystifying GPT self-repair for code generation. CoRR, abs/2306 .09896. [find](https://scholar.google.com/scholar?q=Ninareh+Mehrabi%2C+Palash+Goyal%2C+Christophe+Dupuy%2C+Qian+Hu%2C+Shalini+Ghosh%2C+Richard+Zemel%2C+Kai%2DWei+Chang%2C+Aram+Galstyan%2C+and+Theo+X%2E+Olausson%2C+Jeevana+Priya+Inala%2C+Chenglong+Wang%2C+Jianfeng+Gao%2C+and+Arman)
92. Yasumasa Onoe, Michael J. Q. Zhang, Shankar Padmanabhan, Greg Durrett, and Eunsol Choi. 2023. Can lms learn new entities from descriptions? Challenges in propagating injected knowledge. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (ACL), pages 5469–5485.… [find](https://scholar.google.com/scholar?q=Yasumasa+Onoe%2C+Michael+J%2E+Q%2E+Zhang%2C+Shankar+Padmanabhan%2C+Greg+Durrett%2C+and+Eunsol+Choi%2E+2023%2E+Can+lms+learn+new+entities+from+descriptions%3F+Challenges+in+propagating+injected+knowledge%2E+In+Proceedings)
93. Chengwei Qin, Aston Zhang, Zhuosheng Zhang, Jiaao Chen, Michihiro Yasunaga, and Diyi Yang. 2023. Is chatgpt a general-purpose natural language processing task solver? CoRR, abs/2302.06476. [our card](https://md.research.elamur.ai/free/en/papers/qin-2023-chatgpt-general-purpose-natural-language/)
94. OpenAI. 2023. GPT-4 technical report. CoRR, abs/2303.08774. [our card](https://md.research.elamur.ai/free/en/papers/openai-2023-gpt-4-technical-report/)
95. Vikas Raunak, Amr Sharaf, Hany Hassan Awadallah, and Arul Menezes. 2023. Leveraging GPT-4 for automatic translation post-editing. CoRR, abs/2305.14878. https://doi.org/10.18653/v1/2023.findings-emnlp .804 [↗](https://doi.org/10.18653/v1/2023.findings-emnlp) [our card](https://md.research.elamur.ai/free/en/papers/raunak-2023-leveraging-gpt-4-automatic-translation/)
96. Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll L. Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, John Schulman, Jacob Hilton, Fraser Kelton, Luke Miller, Maddie Simens, Amanda Askell, Peter Welinder, Paul F. Christiano, Jan Leike, and Ryan Lowe. 2022.… [our card](https://md.research.elamur.ai/free/en/papers/ouyang-2022-training-language-models-follow-instructions/)
97. Danilo Neves Ribeiro, Shen Wang, Xiaofei Ma, Henry Zhu, Rui Dong, Deguang Kong, Juliette Burger, Anjelica Ramos, William Yang Wang, Zhiheng Huang, George Karypis, Bing Xiang, and Dan Roth. 2023. STREET: A multi-task structured reasoning and explanation benchmark. In Proceedings of the 11th Internati… [find](https://scholar.google.com/scholar?q=Danilo+Neves+Ribeiro%2C+Shen+Wang%2C+Xiaofei+Ma%2C+Henry+Zhu%2C+Rui+Dong%2C+Deguang+Kong%2C+Juliette+Burger%2C+Anjelica+Ramos%2C+William+Yang+Wang%2C+Zhiheng+Huang%2C+George+Karypis%2C+Bing+Xiang%2C+and+Dan+Roth%2E+2023%2E+STREE)
98. Liangming Pan, Alon Albalak, Xinyi Wang, and William Yang Wang. 2023. Logic-LM: Empowering large language models with symbolic solvers for faithful logical reasoning. CoRR, abs/2305.12295. https://doi.org/10.18653/v1/2023.findings-emnlp.248 [↗](https://doi.org/10.18653/v1/2023.findings-emnlp.248) [our card](https://md.research.elamur.ai/free/en/papers/pan-2023-logic-lm-empowering-large-language/)
99. Manasvi Sagarkar, John Wieting, Lifu Tu, and Kevin Gimpel. 2018. Quality signals in generated stories. In Proceedings of the Seventh Joint Conference on Lexical and Computational Semantics (SEM@NAACL-HLT 2018), pages 192–202. https://doi.org/10.18653/v1/S18-2024 [↗](https://doi.org/10.18653/v1/S18-2024)
100. Jing-Cheng Pang, Pengyuan Wang, Kaiyuan Li, Xiong-Hui Chen, Jiacheng Xu, Zongzhang Zhang, and Yang Yu. 2023. Language model self-improvement by reinforcement learning contemplation. CoRR, abs/2305.14483. [find](https://scholar.google.com/scholar?q=Jing%2DCheng+Pang%2C+Pengyuan+Wang%2C+Kaiyuan+Li%2C+Xiong%2DHui+Chen%2C+Jiacheng+Xu%2C+Zongzhang+Zhang%2C+and+Yang+Yu%2E+2023%2E+Language+model+self%2Dimprovement+by+reinforcement+learning+contemplation%2E+CoRR%2C+abs%2F2305%2E144)
101. Swarnadeep Saha, Sayan Ghosh, Shashank Srivastava, and Mohit Bansal. 2020. PRover: Proof generation for interpretable reasoning over rules. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 122–136. https://doi.org/10.18653/v1/2020 .emnlp-main.9 [↗](https://doi.org/10.18653/v1/2020) [our card](https://md.research.elamur.ai/free/en/papers/saha-2020-prover-proof-generation-interpretable-reasoning/)
102. Debjit Paul, Mete Ismayilzada, Maxime Peyrard, Beatriz Borges, Antoine Bosselut, Robert West, and Boi Faltings. 2023. REFINER: Reasoning feedback on intermediate representations. CoRR, abs/2304.01904. [our card](https://md.research.elamur.ai/free/en/papers/paul-2023-refiner-reasoning-feedback-intermediate-representations/)
103. William Saunders, Catherine Yeh, Jeff Wu, Steven Bills, Long Ouyang, Jonathan Ward, and Jan Leike. 2022. Self-critiquing models for assisting human evaluators. CoRR, abs/2206 .05802. [our card](https://md.research.elamur.ai/free/en/papers/saunders-2022-self-critiquing-models-assisting-human/)
104. Baolin Peng, Michel Galley, Pengcheng He, Hao Cheng, Yujia Xie, Yu Hu, Qiuyuan Huang, Lars Liden, Zhou Yu, Weizhu Chen, and Jianfeng Gao. 2023. Check your facts and try again: Improving large language models with external knowledge and automated feedback. CoRR, abs/2302.12813. [our card](https://md.research.elamur.ai/free/en/papers/peng-2023-check-facts-try-again-improving/)
105. Michael Saxon, Xinyi Wang, Wenda Xu, and William Yang Wang. 2023. PECO: Examining single sentence label leakage in natural language inference datasets through progressive evaluation of cluster outliers. In Proceedings of the 17th Conference of the European Chapter of the Association for Computationa… [find](https://scholar.google.com/scholar?q=Michael+Saxon%2C+Xinyi+Wang%2C+Wenda+Xu%2C+and+William+Yang+Wang%2E+2023%2E+PECO%3A+Examining+single+sentence+label+leakage+in+natural+language+inference+datasets+through+progressive+evaluation+of+cluster+outlier)
106. Dongqi Pu and Vera Demberg. 2023. Chatgpt vs human-authored text: Insights into controllable text summarization and sentence style transfer. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics: Student Research Workshop (ACL), pages 1–18. [our card](https://md.research.elamur.ai/free/en/papers/liu-2023-chatgpt-vs-human-authored-text/)
107. Jérémy Scheurer, Jon Ander Campos, Tomasz Korbak, Jun Shern Chan, Angelica Chen, Kyunghyun Cho, and Ethan Perez. 2023. Training language models with language feedback at scale. CoRR, abs/2303.16755. [find](https://scholar.google.com/scholar?q=Je%CC%81re%CC%81my+Scheurer%2C+Jon+Ander+Campos%2C+Tomasz+Korbak%2C+Jun+Shern+Chan%2C+Angelica+Chen%2C+Kyunghyun+Cho%2C+and+Ethan+Perez%2E+2023%2E+Training+language+models+with+language+feedback+at+scale%2E+CoRR%2C+abs%2F2303%2E16755%2E)
108. Chung, Aakanksha Chowdhery, Quoc Le, Ed Chi, Denny Zhou, and Jason Wei. 2023. Challenging big-bench tasks and whether chain-of-thought can solve them. In Findings of the Association for Computational Linguistics: ACL 2023, pages 13003–13051. https://doi.org/10.18653/v1/2023.findings-acl.824 [↗](https://doi.org/10.18653/v1/2023.findings-acl.824) [our card](https://md.research.elamur.ai/free/en/papers/suzgun-2023-challenging-big-bench-tasks-whether/)
109. Timo Schick, Jane A. Yu, Zhengbao Jiang, Fabio Petroni, Patrick S. H. Lewis, Gautier Izacard, Qingfei You, Christoforos Nalmpantis, Edouard Grave, and Sebastian Riedel. 2023. PEER: A collaborative language model. In Proceedings of the 11th International Conference on Learning Representations (ICLR). [find](https://scholar.google.com/scholar?q=Timo+Schick%2C+Jane+A%2E+Yu%2C+Zhengbao+Jiang%2C+Fabio+Petroni%2C+Patrick+S%2E+H%2E+Lewis%2C+Gautier+Izacard%2C+Qingfei+You%2C+Christoforos+Nalmpantis%2C+Edouard+Grave%2C+and+Sebastian+Riedel%2E+2023%2E+PEER%3A+A+collaborative+lan)
110. Oyvind Tafjord, Bhavana Dalvi, and Peter Clark. 2021. ProofWriter: Generating implications, proofs, and abductive statements over natural language. In Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021, pages 3621–3634. https://doi.org/10 .18653/v1/2021.findings-acl.317 [our card](https://md.research.elamur.ai/free/en/papers/tafjord-2021-proofwriter-generating-implications-proofs-abductive/)
111. John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. 2017. Proximal policy optimization algorithms. CoRR, abs/1707.06347. [find](https://scholar.google.com/scholar?q=John+Schulman%2C+Filip+Wolski%2C+Prafulla+Dhariwal%2C+Alec+Radford%2C+and+Oleg+Klimov%2E+2017%2E+Proximal+policy+optimization+algorithms%2E+CoRR%2C+abs%2F1707%2E06347%2E)
112. Oyvind Tafjord, Bhavana Dalvi Mishra, and Peter Clark. 2022. Entailer: Answering questions with faithful and truthful chains of reasoning. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 2078–2093. https://doi.org/10.18653/v1/2022.emnlp-main… [↗](https://doi.org/10.18653/v1/2022.emnlp-main…)
113. Omar Shaikh, Hongxin Zhang, William Held, Michael Bernstein, and Diyi Yang. 2023. On second thought, let’s not think step by step! Bias and toxicity in zero-shot reasoning. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (ACL), pages 4454–4470. [https://doi.…](https://doi.…) [↗](https://doi.…) [our card](https://md.research.elamur.ai/free/en/papers/shaikh-2023-second-thought-let-s-not/)
114. Ryutaro Tanno, Melanie F. Pradier, Aditya V. Nori, and Yingzhen Li. 2022. Repairing neural networks by leaving the right past behind. In Proceedings of the 2022 Annual Conference on Neural Information Processing Systems (NeurIPS). [find](https://scholar.google.com/scholar?q=Ryutaro+Tanno%2C+Melanie+F%2E+Pradier%2C+Aditya+V%2E+Nori%2C+and+Yingzhen+Li%2E+2022%2E+Repairing+neural+networks+by+leaving+the+right+past+behind%2E+In+Proceedings+of+the+2022+Annual+Conference+on+Neural+Information)
115. Noah Shinn, Federico Cassano, Beck Labash, Ashwin Gopinath, Karthik Narasimhan, and Shunyu Yao. 2023. Reflexion: Language agents with verbal reinforcement learning. CoRR, abs/2303.11366. [our card](https://md.research.elamur.ai/free/en/papers/shinn-2023-reflexion-language-agents-verbal-reinforcement/)
116. Anton Sinitsin, Vsevolod Plokhotnyuk, Dmitry V. Pyrkin, Sergei Popov, and Artem Babenko. 2020. Editable neural networks. In Proceedings of the 8th International Conference on Learning Representations (ICLR). [find](https://scholar.google.com/scholar?q=Anton+Sinitsin%2C+Vsevolod+Plokhotnyuk%2C+Dmitry+V%2E+Pyrkin%2C+Sergei+Popov%2C+and+Artem+Babenko%2E+2020%2E+Editable+neural+networks%2E+In+Proceedings+of+the+8th+International+Conference+on+Learning+Representations+)
117. Gladys Tyen, Hassan Mansoor, Peter Chen, Tony Mak, and Victor Carbune. 2023. LLMs cannot find reasoning errors, but can correct them! CoRR, abs/2311.08516. — [arXiv:2311.08516](https://arxiv.org/abs/2311.08516) [find](https://scholar.google.com/scholar?q=Gladys+Tyen%2C+Hassan+Mansoor%2C+Peter+Chen%2C+Tony+Mak%2C+and+Victor+Carbune%2E+2023%2E+LLMs+cannot+find+reasoning+errors%2C+but+can+correct+them%21+CoRR%2C+abs%2F2311%2E08516%2E+%E2%80%94+%5BarXiv%3A2311%2E08516%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2)
118. Jonathan Uesato, Nate Kushman, Ramana Kumar, H. Francis Song, Noah Y. Siegel, Lisa Wang, Antonia Creswell, Geoffrey Irving, and Irina Higgins. 2022. Solving math word problems with process- and outcome-based feedback. CoRR, abs/2211.14275. [our card](https://md.research.elamur.ai/free/en/papers/uesato-2022-solving-math-word-problems-process/)
119. Neha Srikanth and Rachel Rudinger. 2022. Partial-input baselines show that NLI models can ignore context, but they don’t. In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT), pages 4753–4763. ht… [our card](https://md.research.elamur.ai/free/en/papers/srikanth-2022-partial-input-baselines-show-that/)
120. Inigo Jauregi Unanue, Jacob Parnell, and Massimo Piccardi. 2021. Berttune: Fine-tuning neural machine translation with bertscore. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (ACL… [find](https://scholar.google.com/scholar?q=Inigo+Jauregi+Unanue%2C+Jacob+Parnell%2C+and+Massimo+Piccardi%2E+2021%2E+Berttune%3A+Fine%2Dtuning+neural+machine+translation+with+bertscore%2E+In+Proceedings+of+the+59th+Annual+Meeting+of+the+Association+for+Compu)
121. Kaya Stechly, Matthew Marquez, and Subbarao Kambhampati. 2023. GPT-4 doesn’t know it’s wrong: An analysis of iterative prompting for reasoning problems. CoRR, abs/2310.12397. — [arXiv:2310.12397](https://arxiv.org/abs/2310.12397) [find](https://scholar.google.com/scholar?q=Kaya+Stechly%2C+Matthew+Marquez%2C+and+Subbarao+Kambhampati%2E+2023%2E+GPT%2D4+doesn%E2%80%99t+know+it%E2%80%99s+wrong%3A+An+analysis+of+iterative+prompting+for+reasoning+problems%2E+CoRR%2C+abs%2F2310%2E12397%2E+%E2%80%94+%5BarXiv%3A2310%2E12397%5D%28http)
122. Mirac Suzgun, Nathan Scales, Nathanael Schärli, Sebastian Gehrmann, Yi Tay, Hyung Won Karthik Valmeekam, Matthew Marquez, and Subbarao Kambhampati. 2023. Can large language models really improve by self-critiquing their own plans? CoRR, abs/2310.08118. abilities of large language models. CoRR, abs/… — [arXiv:2310.08118](https://arxiv.org/abs/2310.08118) [find](https://scholar.google.com/scholar?q=Mirac+Suzgun%2C+Nathan+Scales%2C+Nathanael+Scha%CC%88rli%2C+Sebastian+Gehrmann%2C+Yi+Tay%2C+Hyung+Won+Karthik+Valmeekam%2C+Matthew+Marquez%2C+and+Subbarao+Kambhampati%2E+2023%2E+Can+large+language+models+really+improve+by+s)
123. Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed H. Chi, Quoc V. Le, and Denny Zhou. 2022b. Chain-of-thought prompting elicits reasoning in large language models. In Proceedings of the Annual Conference on Neural Information Processing Systems (NeurIPS). [our card](https://md.research.elamur.ai/free/en/papers/wei-2022-chain-thought-prompting-elicits-reasoning/)
124. Neeraj Varshney, Wenlin Yao, Hongming Zhang, Jianshu Chen, and Dong Yu. 2023. A stitch in time saves nine: Detecting and mitigating hallucinations of LLMs by validating low-confidence generation. CoRR, abs/2307.03987. [our card](https://md.research.elamur.ai/free/en/papers/varshney-2023-stitch-time-saves-nine-detecting/)
125. David Wan and Mohit Bansal. 2022. Factpegasus: Factuality-aware pre-training and fine-tuning for abstractive summarization. In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT), pages 1010–1028.… [our card](https://md.research.elamur.ai/free/en/papers/wan-2022-factpegasus-factuality-aware-pre-training/)
126. Sean Welleck, Ximing Lu, Peter West, Faeze Brahman, Tianxiao Shen, Daniel Khashabi, and Yejin Choi. 2023. Generating sequences by learning to self-correct. In Proceedings of The 11th International Conference on Learning Representations (ICLR). [our card](https://md.research.elamur.ai/free/en/papers/welleck-2022-generating-sequences-learning-self-correct/)
127. Yixuan Weng, Minjun Zhu, Fei Xia, Bin Li, Shizhu He, Kang Liu, and Jun Zhao. 2023. Large language models are better reasoners with self-verification. CoRR, abs/2212.09561. https://doi.org/10.18653/v1/2023 .findings-emnlp.167 [↗](https://doi.org/10.18653/v1/2023) [our card](https://md.research.elamur.ai/free/en/papers/weng-2023-large-language-models-are-better/)
128. Boxin Wang, Weixin Chen, Hengzhi Pei, Chulin Xie, Mintong Kang, Chenhui Zhang, Chejian Xu, Zidi Xiong, Ritik Dutta, Rylan Schaeffer, Sang T. Truong, Simran Arora, Mantas Mazeika, Dan Hendrycks, Zinan Lin, Yu Cheng, Sanmi Koyejo, Dawn Song, and Bo Li. 2023a. Decodingtrust: A comprehensive assessment… [our card](https://md.research.elamur.ai/free/en/papers/wang-2023-decodingtrust-comprehensive-assessment-trustworthiness-gpt/)
129. Zeqiu Wu, Yushi Hu, Weijia Shi, Nouha Dziri, Alane Suhr, Prithviraj Ammanabrolu, Noah A. Smith, Mari Ostendorf, and Hannaneh Hajishirzi. 2023a. Fine-grained human feedback gives better rewards for language model training. CoRR, abs/2306.01693. [our card](https://md.research.elamur.ai/free/en/papers/wu-2023-fine-grained-human-feedback-gives/)
130. Haotian Wang, Xiyuan Du, Weijiang Yu, Qianglong Chen, Kun Zhu, Zheng Chu, Lian Yan, and Yi Guan. 2023b. Apollo’s oracle: Retrieval-augmented reasoning in multi-agent debates. CoRR, abs/2312.04854. — [arXiv:2312.04854](https://arxiv.org/abs/2312.04854) [find](https://scholar.google.com/scholar?q=Haotian+Wang%2C+Xiyuan+Du%2C+Weijiang+Yu%2C+Qianglong+Chen%2C+Kun+Zhu%2C+Zheng+Chu%2C+Lian+Yan%2C+and+Yi+Guan%2E+2023b%2E+Apollo%E2%80%99s+oracle%3A+Retrieval%2Daugmented+reasoning+in+multi%2Dagent+debates%2E+CoRR%2C+abs%2F2312%2E04854%2E+%E2%80%94+%5B)
131. Zhaofeng Wu, Linlu Qiu, Alexis Ross, Ekin Akyürek, Boyuan Chen, Bailin Wang, Najoung Kim, Jacob Andreas, and Yoon Kim. 2023b. Reasoning or reciting? Exploring the capabilities and limitations of language models through counterfactual tasks. CoRR, abs/2307.02477. [find](https://scholar.google.com/scholar?q=Zhaofeng+Wu%2C+Linlu+Qiu%2C+Alexis+Ross%2C+Ekin+Akyu%CC%88rek%2C+Boyuan+Chen%2C+Bailin+Wang%2C+Najoung+Kim%2C+Jacob+Andreas%2C+and+Yoon+Kim%2E+2023b%2E+Reasoning+or+reciting%3F+Exploring+the+capabilities+and+limitations+of+lang)
132. Liyuan Wang, Xingxing Zhang, Hang Su, and Jun Zhu. 2023c. A comprehensive survey of continual learning: Theory, method and application. CoRR, abs/2302.00487. [find](https://scholar.google.com/scholar?q=Liyuan+Wang%2C+Xingxing+Zhang%2C+Hang+Su%2C+and+Jun+Zhu%2E+2023c%2E+A+comprehensive+survey+of+continual+learning%3A+Theory%2C+method+and+application%2E+CoRR%2C+abs%2F2302%2E00487%2E)
133. Yuxi Xie, Kenji Kawaguchi, Yiran Zhao, Xu Zhao, Min-Yen Kan, Junxian He, and Qizhe Xie. 2023. Decomposition enhances reasoning via self-evaluation guided decoding. CoRR, abs/2305.00633. [our card](https://md.research.elamur.ai/free/en/papers/xie-2023-self-evaluation-guided-beam-search/)
134. Tong Wang, Ping Chen, and Boyang Li. 2017. Predicting the quality of short narratives from social media. In Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence (IJCAI), pages 3859–3865. https://doi.org/10.24963/ijcai.2017/539 — [doi:10.24963/ijcai.2017](https://doi.org/10.24963/ijcai.2017) [↗](https://doi.org/10.24963/ijcai.2017/539)
135. Wenda Xu, Danqing Wang, Liangming Pan, Zhenqiao Song, Markus Freitag, William Yang Wang, and Lei Li. 2023. INSTRUCTSCORE: Towards explainable text generation evaluation with automatic feedback. CoRR, abs/2305 .14282. https://doi.org/10.18653/v1/2023.emnlp-main.365 [↗](https://doi.org/10.18653/v1/2023.emnlp-main.365) [our card](https://md.research.elamur.ai/free/en/papers/xu-2023-instructscore-explainable-text-generation-evaluation/)
136. Jason Wei, Yi Tay, Rishi Bommasani, Colin Raffel, Barret Zoph, Sebastian Borgeaud, Dani Yogatama, Maarten Bosma, Denny Zhou, Donald Metzler, Ed H. Chi, Tatsunori Hashimoto, Oriol Vinyals, Percy Liang, Jeff Dean, and William Fedus. 2022a. [find](https://scholar.google.com/scholar?q=Jason+Wei%2C+Yi+Tay%2C+Rishi+Bommasani%2C+Colin+Raffel%2C+Barret+Zoph%2C+Sebastian+Borgeaud%2C+Dani+Yogatama%2C+Maarten+Bosma%2C+Denny+Zhou%2C+Donald+Metzler%2C+Ed+H%2E+Chi%2C+Tatsunori+Hashimoto%2C+Oriol+Vinyals%2C+Percy+Liang%2C)
137. Emergent Xiaojun Xu, Chang Liu, and Dawn Song. 2017. Sqlnet: Generating structured queries from natural language without reinforcement learning. CoRR, abs/1711.04436. [find](https://scholar.google.com/scholar?q=Emergent+Xiaojun+Xu%2C+Chang+Liu%2C+and+Dawn+Song%2E+2017%2E+Sqlnet%3A+Generating+structured+queries+from+natural+language+without+reinforcement+learning%2E+CoRR%2C+abs%2F1711%2E04436%2E)
138. Tao Yu, Michihiro Yasunaga, Kai Yang, Rui Zhang, Dongxu Wang, Zifan Li, and Dragomir Radev. 2018. SyntaxSQLNet: Syntax tree networks for complex and cross-domain text-to-SQL task. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 1653–1663. http… [find](https://scholar.google.com/scholar?q=Tao+Yu%2C+Michihiro+Yasunaga%2C+Kai+Yang%2C+Rui+Zhang%2C+Dongxu+Wang%2C+Zifan+Li%2C+and+Dragomir+Radev%2E+2018%2E+SyntaxSQLNet%3A+Syntax+tree+networks+for+complex+and+cross%2Ddomain+text%2Dto%2DSQL+task%2E+In+Proceedings+of+th)
139. Hao Yan, Saurabh Srivastava, Yintao Tai, Sida I. Wang, Wen-tau Yih, and Ziyu Yao. 2023. Learning to simulate natural language feedback for interactive semantic parsing. In Proceedings of the 61th Annual Meeting of the Association for Computational Linguistics (ACL), pages 3149–3170. https://doi.org/… [find](https://scholar.google.com/scholar?q=Hao+Yan%2C+Saurabh+Srivastava%2C+Yintao+Tai%2C+Sida+I%2E+Wang%2C+Wen%2Dtau+Yih%2C+and+Ziyu+Yao%2E+2023%2E+Learning+to+simulate+natural+language+feedback+for+interactive+semantic+parsing%2E+In+Proceedings+of+the+61th+Annu)
140. Wenhao Yu, Zhihan Zhang, Zhenwen Liang, Meng Jiang, and Ashish Sabharwal. 2023. Improving language models via plug-and-play retrieval feedback. CoRR, abs/2305.14002. [find](https://scholar.google.com/scholar?q=Wenhao+Yu%2C+Zhihan+Zhang%2C+Zhenwen+Liang%2C+Meng+Jiang%2C+and+Ashish+Sabharwal%2E+2023%2E+Improving+language+models+via+plug%2Dand%2Dplay+retrieval+feedback%2E+CoRR%2C+abs%2F2305%2E14002%2E)
141. Kaiyu Yang, Jia Deng, and Danqi Chen. 2022a. Generating natural language proofs with verifier-guided search. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 89–105. https://doi.org/10.18653/v1/2022.emnlp-main.7 [↗](https://doi.org/10.18653/v1/2022.emnlp-main.7)
142. Weizhe Yuan, Kyunghyun Cho, and Jason Weston. 2023. System-level natural language feedback. CoRR, abs/2306.13588. [find](https://scholar.google.com/scholar?q=Weizhe+Yuan%2C+Kyunghyun+Cho%2C+and+Jason+Weston%2E+2023%2E+System%2Dlevel+natural+language+feedback%2E+CoRR%2C+abs%2F2306%2E13588%2E)
143. Eric Zelikman, Yuhuai Wu, Jesse Mu, and Noah D. Goodman. 2022. Star: Bootstrapping reasoning with reasoning. In Proceedings of the Annual Conference on Neural Information Processing Systems (NeurIPS). [our card](https://md.research.elamur.ai/free/en/papers/zelikman-2022-star-bootstrapping-reasoning-reasoning/)
144. Kevin Yang and Dan Klein. 2021. FUDGE: Controlled text generation with future discriminators. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT), pages 3511–3535. https://doi.org/10 .18653/v1/2… [our card](https://md.research.elamur.ai/free/en/papers/yang-2021-fudge-controlled-text-generation-future/)
145. Kevin Yang, Yuandong Tian, Nanyun Peng, and Dan Klein. 2022b. Re3: Generating longer stories with recursive reprompting and revision. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 4393–4479. https://doi.org/10.18653/v1/2022 .emnlp-main.296 [↗](https://doi.org/10.18653/v1/2022)
146. Shunyu Yao, Dian Yu, Jeffrey Zhao, Izhak Shafran, Thomas L. Griffiths, Yuan Cao, and Karthik Narasimhan. 2023a. Tree of thoughts: Deliberate problem solving with large language models. CoRR, abs/2305.10601. [find](https://scholar.google.com/scholar?q=Shunyu+Yao%2C+Dian+Yu%2C+Jeffrey+Zhao%2C+Izhak+Shafran%2C+Thomas+L%2E+Griffiths%2C+Yuan+Cao%2C+and+Karthik+Narasimhan%2E+2023a%2E+Tree+of+thoughts%3A+Deliberate+problem+solving+with+large+language+models%2E+CoRR%2C+abs%2F2305%2E)
147. Kechi Zhang, Zhuo Li, Jia Li, Ge Li, and Zhi Jin. 2023a. Self-edit: Fault-aware code editor for code generation. CoRR, abs/2305.04087. https://doi.org/10.18653/v1/2023 .acl-long.45 [↗](https://doi.org/10.18653/v1/2023)
148. Kexun Zhang, Danqing Wang, Jingtao Xia, William Yang Wang, and Lei Li. 2023b. Algo: Synthesizing algorithmic programs with generated oracle verifiers. CoRR, abs/2305.14591. [find](https://scholar.google.com/scholar?q=Kexun+Zhang%2C+Danqing+Wang%2C+Jingtao+Xia%2C+William+Yang+Wang%2C+and+Lei+Li%2E+2023b%2E+Algo%3A+Synthesizing+algorithmic+programs+with+generated+oracle+verifiers%2E+CoRR%2C+abs%2F2305%2E14591%2E)
149. Muru Zhang, Ofir Press, William Merrill, Alisa Liu, and Noah A. Smith. 2023c. How language model hallucinations can snowball. CoRR, abs/2305.13534. [our card](https://md.research.elamur.ai/free/en/papers/zhang-2023-language-model-hallucinations-snowball/)
150. Xinyu Zhu, Junjie Wang, Lin Zhang, Yuxiang Zhang, Yongfeng Huang, Ruyi Gan, Jiaxing Zhang, and Yujiu Yang. 2023. Solving math word problems via cooperative reasoning induced language models. In Processings of the 61th Annual Meeting of the Association for Computational Linguistics (ACL), pages 4471–… [our card](https://md.research.elamur.ai/free/en/papers/zhu-2023-solving-math-word-problems-cooperative/)
151. Yunzhi Yao, Peng Wang, Bozhong Tian, Siyuan Cheng, Zhoubo Li, Shumin Deng, Huajun Chen, and Ningyu Zhang. 2023b. Editing large language models: Problems, methods, and opportunities. CoRR, abs/2305.13172. [find](https://scholar.google.com/scholar?q=Yunzhi+Yao%2C+Peng+Wang%2C+Bozhong+Tian%2C+Siyuan+Cheng%2C+Zhoubo+Li%2C+Shumin+Deng%2C+Huajun+Chen%2C+and+Ningyu+Zhang%2E+2023b%2E+Editing+large+language+models%3A+Problems%2C+methods%2C+and+opportunities%2E+CoRR%2C+abs%2F2305%2E131)
152. Seonghyeon Ye, Yongrae Jo, Doyoung Kim, Sungdong Kim, Hyeonbin Hwang, and Minjoon Seo. 2023. Selfee: Iterative self-revising LLM empowered by self-feedback generation. Blog post. [find](https://scholar.google.com/scholar?q=Seonghyeon+Ye%2C+Yongrae+Jo%2C+Doyoung+Kim%2C+Sungdong+Kim%2C+Hyeonbin+Hwang%2C+and+Minjoon+Seo%2E+2023%2E+Selfee%3A+Iterative+self%2Drevising+LLM+empowered+by+self%2Dfeedback+generation%2E+Blog+post%2E)
153. Terry Yue Zhuo, Yujin Huang, Chunyang Chen, and Zhenchang Xing. 2023. Red teaming chatgpt via jailbreaking: Bias, robustness, reliability and toxicity. CoRR, abs/2301.12867. [our card](https://md.research.elamur.ai/free/en/papers/zhuo-2023-red-teaming-chatgpt-jailbreaking-bias/)

*Updated 24 October 2026*