---
title: "Generative Verifiers — Zhang 2024"
description: "Best-of-N selection lets an LLM sample many candidate solutions and relies on a learned verifier to choose the best one."
authors: ["Lunjun Zhang", "Arian Hosseini", "Hritik Bansal", "Mehran Kazemi", "Aviral Kumar", "Rishabh Agarwal"]
year: 2024
venue: "arXiv:2408.15240"
source: "https://arxiv.org/abs/2408.15240"
doi: "10.48550/arxiv.2408.15240"
arxiv: "2408.15240"
licence: "arXiv — non-exclusive licence to distribute"
licence_url: "https://arxiv.org/licenses/nonexclusive-distrib/1.0/license.html"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/zhang-2024-generative-verifiers-reward-modeling-as/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/zhang-2024-generative-verifiers-reward-modeling-as/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2024](https://md.research.elamur.ai/free/en/papers/2024/page/2/) / Generative Verifiers: Reward Modeling as Next-Token Prediction

# Generative Verifiers: Reward Modeling as Next-Token Prediction

**Authors:** Lunjun Zhang, [Arian Hosseini](https://md.research.elamur.ai/free/en/authors/arian-hosseini/), [Hritik Bansal](https://md.research.elamur.ai/free/en/authors/hritik-bansal/), Mehran Kazemi, [Aviral Kumar](https://md.research.elamur.ai/free/en/authors/aviral-kumar/), [Rishabh Agarwal](https://md.research.elamur.ai/free/en/authors/rishabh-agarwal/) · **Year:** 2024 · **Venue:** arXiv:2408.15240 · **Licence:** [arXiv — non-exclusive licence to distribute](https://arxiv.org/licenses/nonexclusive-distrib/1.0/license.html)

**Links:** [Publisher's page](https://arxiv.org/abs/2408.15240) · [PDF](https://arxiv.org/pdf/2408.15240v3) · [DOI 10.48550/arxiv.2408.15240](https://doi.org/10.48550/arxiv.2408.15240) · [arXiv 2408.15240](https://arxiv.org/abs/2408.15240) · [OpenAlex](https://openalex.org/W4402705340)

**Citations:** 2 citations (openalex-snapshot · 2026-10-14) · [141](#cited-by) in our library, of the 140673 papers whose bibliographies we read

## Summary
Best-of-N selection lets an LLM sample many candidate solutions and relies on a learned verifier to choose the best one. These verifiers are usually trained as discriminative classifiers that output a single score, which leaves the model's text-generation abilities unused. The authors, from Google DeepMind and partner universities, propose GenRM, which trains the verifier with ordinary next-token prediction. The verifier is asked whether a solution is correct, and its score is the probability it gives to the "Yes" token. The variant GenRM-CoT first writes a step-by-step verification rationale and then gives its verdict. At test time, several rationales are sampled and their "Yes" probabilities are averaged by majority voting. The same model can also be trained on verification and solution generation together. Training rationales are synthetic, generated by Gemini 1.0 Pro and filtered for correctness. On algorithmic string tasks and math reasoning, GenRM outperforms discriminative reward models, DPO-based verifiers, LLM-as-a-Judge and self-consistency. This shows that recasting reward modeling as generation unlocks chain-of-thought reasoning and extra inference compute for verification.

## Key points
- **Algorithmic tasks:** Gemma-2B verifiers were tested on Last Letter Concat and BBH Word Sorting. Best-of-N success rose from 5% to 45.3%, close to what an oracle verifier achieves.
- **GSM8K:** a Gemma2-9B GenRM-CoT verifier ranked solutions produced by Gemini 1.0 Pro. Problems solved rose from 73% to 93.4%, which beats GPT-4 and Gemini 1.5 Pro.
- **Easy-to-hard transfer:** verifiers trained only on grade-school math improved Best-of-32 on MATH500 from 28% to 44.6%, and on MMLU abstract algebra from 37.9% to 53.5%.
- **Synthetic rationales are enough:** model-generated verification rationales, though imperfect, were sufficient to teach the verifier to catch subtle reasoning errors that discriminative reward models miss.
- **Scaling:** generative verifiers improve more than discriminative ones as model size grows. They also gain from more test-time compute through majority voting. The work appeared at ICLR 2025 (arXiv:2408.15240).

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

Fields: [Mathematics & statistics](https://md.research.elamur.ai/free/en/shelves/mathematics/)

## Cited by

Cited by

141 papers of our library cite it (of the 140673 whose bibliographies we read); 4 of them have a page here, 137 not yet.

- [Towards Reasoning Era: A Survey of Long Chain-of-Thought for Reasoning Large Language Models](https://md.research.elamur.ai/free/en/papers/chen-2025-towards-reasoning-era-survey-long/)
- [Kimi k1.5: Scaling Reinforcement Learning with LLMs](https://md.research.elamur.ai/free/en/papers/kimi-2025-team-kimi-k1-5-scaling-reinforcement/)
- [Siren's Song in the AI Ocean: A Survey on Hallucination in Large Language Models](https://md.research.elamur.ai/free/en/papers/zhang-2023-siren-s-song-ai-ocean/)
- [The Lessons of Developing Process Reward Models in Mathematical Reasoning](https://md.research.elamur.ai/free/en/papers/zhang-2025-lessons-developing-process-reward-models/)

## Cite

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@misc{zhang2024generative,
  title = {{Generative Verifiers: Reward Modeling as Next-Token Prediction}},
  author = {Zhang, Lunjun and Hosseini, Arian and Bansal, Hritik and Kazemi, Mehran and Kumar, Aviral and Agarwal, Rishabh},
  year = {2024},
  doi = {10.48550/arxiv.2408.15240},
  eprint = {2408.15240},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2408.15240}
}
```

- [← previous Generative Representational Instruction Tuning](https://md.research.elamur.ai/free/en/papers/muennighoff-2024-generative-representational-instruction-tuning/)
- [next → GPT-4o System Card](https://md.research.elamur.ai/free/en/papers/openai-2024-gpt-4o-system-card/)
- [↑ back to papers of 2024](https://md.research.elamur.ai/free/en/papers/2024/page/2/)

## References

58 references, 27 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. J. Achiam, S. Adler, S. Agarwal, L. Ahmad, I. Akkaya, F. L. Aleman, D. Almeida, J. Altenschmidt, S. Altman, S. Anadkat, et al. Gpt-4 technical report. arXiv preprint arXiv:2303.08774, 2023. [find](https://scholar.google.com/scholar?q=J%2E+Achiam%2C+S%2E+Adler%2C+S%2E+Agarwal%2C+L%2E+Ahmad%2C+I%2E+Akkaya%2C+F%2E+L%2E+Aleman%2C+D%2E+Almeida%2C+J%2E+Altenschmidt%2C+S%2E+Altman%2C+S%2E+Anadkat%2C+et+al%2E+Gpt%2D4+technical+report%2E+arXiv+preprint+arXiv%3A2303%2E08774%2C+2023%2E) [our card](https://md.research.elamur.ai/free/en/papers/openai-2023-gpt-4-technical-report/)
2. R. Agarwal, A. Singh, L. M. Zhang, B. Bohnet, S. Chan, A. Anand, Z. Abbas, A. Nova, J. D. Co-Reyes, E. Chu, et al. Many-shot in-context learning. arXiv preprint arXiv:2404.11018, 2024. [find](https://scholar.google.com/scholar?q=R%2E+Agarwal%2C+A%2E+Singh%2C+L%2E+M%2E+Zhang%2C+B%2E+Bohnet%2C+S%2E+Chan%2C+A%2E+Anand%2C+Z%2E+Abbas%2C+A%2E+Nova%2C+J%2E+D%2E+Co%2DReyes%2C+E%2E+Chu%2C+et+al%2E+Many%2Dshot+in%2Dcontext+learning%2E+arXiv+preprint+arXiv%3A2404%2E11018%2C+2024%2E) [our card](https://md.research.elamur.ai/free/en/papers/agarwal-2024-many-shot-context-learning/)
3. Z. Ankner, M. Paul, B. Cui, J. D. Chang, and P. Ammanabrolu. Critique-out-loud reward models. arXiv preprint arXiv:2408.11791, 2024. [find](https://scholar.google.com/scholar?q=Z%2E+Ankner%2C+M%2E+Paul%2C+B%2E+Cui%2C+J%2E+D%2E+Chang%2C+and+P%2E+Ammanabrolu%2E+Critique%2Dout%2Dloud+reward+models%2E+arXiv+preprint+arXiv%3A2408%2E11791%2C+2024%2E)
4. Y. Bai, S. Kadavath, S. Kundu, A. Askell, J. Kernion, A. Jones, A. Chen, A. Goldie, A. Mirhoseini, C. McKinnon, et al. Constitutional ai: Harmlessness from ai feedback. arXiv preprint arXiv:2212.08073, 2022. [find](https://scholar.google.com/scholar?q=Y%2E+Bai%2C+S%2E+Kadavath%2C+S%2E+Kundu%2C+A%2E+Askell%2C+J%2E+Kernion%2C+A%2E+Jones%2C+A%2E+Chen%2C+A%2E+Goldie%2C+A%2E+Mirhoseini%2C+C%2E+McKinnon%2C+et+al%2E+Constitutional+ai%3A+Harmlessness+from+ai+feedback%2E+arXiv+preprint+arXiv%3A2212%2E08073) [our card](https://md.research.elamur.ai/free/en/papers/bai-2022-constitutional-ai-harmlessness-ai-feedback/)
5. M. Besta, L. Paleari, A. Kubicek, P. Nyczyk, R. Gerstenberger, P. Iff, T. Lehmann, H. Niewiadomski, and T. Hoefler. Checkembed: Effective verification of llm solutions to open-ended tasks. arXiv preprint arXiv:2406.02524, 2024. — [arXiv:2406.02524](https://arxiv.org/abs/2406.02524) [find](https://scholar.google.com/scholar?q=M%2E+Besta%2C+L%2E+Paleari%2C+A%2E+Kubicek%2C+P%2E+Nyczyk%2C+R%2E+Gerstenberger%2C+P%2E+Iff%2C+T%2E+Lehmann%2C+H%2E+Niewiadomski%2C+and+T%2E+Hoefler%2E+Checkembed%3A+Effective+verification+of+llm+solutions+to+open%2Dended+tasks%2E+arXiv+prepr)
6. S. Borgeaud, A. Mensch, J. Hoffmann, T. Cai, E. Rutherford, K. Millican, G. B. Van Den Driessche, J.-B. Lespiau, B. Damoc, A. Clark, et al. Improving language models by retrieving from trillions of tokens. In International conference on machine learning, pages 2206–2240. PMLR, 2022. [find](https://scholar.google.com/scholar?q=S%2E+Borgeaud%2C+A%2E+Mensch%2C+J%2E+Hoffmann%2C+T%2E+Cai%2C+E%2E+Rutherford%2C+K%2E+Millican%2C+G%2E+B%2E+Van+Den+Driessche%2C+J%2E%2DB%2E+Lespiau%2C+B%2E+Damoc%2C+A%2E+Clark%2C+et+al%2E+Improving+language+models+by+retrieving+from+trillions+of+to)
7. B. Brown, J. Juravsky, R. Ehrlich, R. Clark, Q. V. Le, C. Ré, and A. Mirhoseini. Large language monkeys: Scaling inference compute with repeated sampling. arXiv preprint arXiv:2407.21787, 2024. [find](https://scholar.google.com/scholar?q=B%2E+Brown%2C+J%2E+Juravsky%2C+R%2E+Ehrlich%2C+R%2E+Clark%2C+Q%2E+V%2E+Le%2C+C%2E+R%C3%A9%2C+and+A%2E+Mirhoseini%2E+Large+language+monkeys%3A+Scaling+inference+compute+with+repeated+sampling%2E+arXiv+preprint+arXiv%3A2407%2E21787%2C+2024%2E) [our card](https://md.research.elamur.ai/free/en/papers/brown-2024-large-language-monkeys-scaling-inference/)
8. E. Charniak and M. Johnson. Coarse-to-fine n-best parsing and maxent discriminative reranking. In Proceedings of the 43rd Annual Meeting of the Association for Computational Linguistics (ACL’05), pages 173–180, 2005. [find](https://scholar.google.com/scholar?q=E%2E+Charniak+and+M%2E+Johnson%2E+Coarse%2Dto%2Dfine+n%2Dbest+parsing+and+maxent+discriminative+reranking%2E+In+Proceedings+of+the+43rd+Annual+Meeting+of+the+Association+for+Computational+Linguistics+%28ACL%E2%80%9905%29%2C+page) [our card](https://md.research.elamur.ai/free/en/papers/narrative-2005-charniak-coarse-fine-n-best-parsing/)
9. A. Chowdhery, S. Narang, J. Devlin, M. Bosma, G. Mishra, A. Roberts, P. Barham, H. W. Chung, C. Sutton, S. Gehrmann, et al. Palm: Scaling language modeling with pathways. Journal of Machine Learning Research, 24(240):1–113, 2023. [find](https://scholar.google.com/scholar?q=A%2E+Chowdhery%2C+S%2E+Narang%2C+J%2E+Devlin%2C+M%2E+Bosma%2C+G%2E+Mishra%2C+A%2E+Roberts%2C+P%2E+Barham%2C+H%2E+W%2E+Chung%2C+C%2E+Sutton%2C+S%2E+Gehrmann%2C+et+al%2E+Palm%3A+Scaling+language+modeling+with+pathways%2E+Journal+of+Machine+Learning+R)
10. H. W. Chung, L. Hou, S. Longpre, B. Zoph, Y. Tay, W. Fedus, Y. Li, X. Wang, M. Dehghani, S. Brahma, et al. Scaling instruction-finetuned language models. arXiv preprint arXiv:2210.11416, 2022. [find](https://scholar.google.com/scholar?q=H%2E+W%2E+Chung%2C+L%2E+Hou%2C+S%2E+Longpre%2C+B%2E+Zoph%2C+Y%2E+Tay%2C+W%2E+Fedus%2C+Y%2E+Li%2C+X%2E+Wang%2C+M%2E+Dehghani%2C+S%2E+Brahma%2C+et+al%2E+Scaling+instruction%2Dfinetuned+language+models%2E+arXiv+preprint+arXiv%3A2210%2E11416%2C+2022%2E)
11. K. Cobbe, V. Kosaraju, M. Bavarian, M. Chen, H. Jun, L. Kaiser, M. Plappert, J. Tworek, J. Hilton, R. Nakano, et al. Training verifiers to solve math word problems. arXiv preprint arXiv:2110.14168, 2021. [find](https://scholar.google.com/scholar?q=K%2E+Cobbe%2C+V%2E+Kosaraju%2C+M%2E+Bavarian%2C+M%2E+Chen%2C+H%2E+Jun%2C+L%2E+Kaiser%2C+M%2E+Plappert%2C+J%2E+Tworek%2C+J%2E+Hilton%2C+R%2E+Nakano%2C+et+al%2E+Training+verifiers+to+solve+math+word+problems%2E+arXiv+preprint+arXiv%3A2110%2E14168%2C+20) [our card](https://md.research.elamur.ai/free/en/papers/cobbe-2021-training-verifiers-solve-math-word/)
12. Gemma Team, T. Mesnard, C. Hardin, R. Dadashi, S. Bhupatiraju, S. Pathak, L. Sifre, M. Rivière, M. S. Kale, J. Love, et al. Gemma: Open models based on gemini research and technology. arXiv preprint arXiv:2403.08295, 2024a. [find](https://scholar.google.com/scholar?q=Gemma+Team%2C+T%2E+Mesnard%2C+C%2E+Hardin%2C+R%2E+Dadashi%2C+S%2E+Bhupatiraju%2C+S%2E+Pathak%2C+L%2E+Sifre%2C+M%2E+Rivi%C3%A8re%2C+M%2E+S%2E+Kale%2C+J%2E+Love%2C+et+al%2E+Gemma%3A+Open+models+based+on+gemini+research+and+technology%2E+arXiv+preprint+a)
13. Gemma Team, M. Riviere, S. Pathak, P. G. Sessa, C. Hardin, S. Bhupatiraju, L. Hussenot, T. Mesnard, B. Shahriari, A. Ramé, et al. Gemma 2: Improving open language models at a practical size. arXiv preprint arXiv:2408.00118, 2024b. [find](https://scholar.google.com/scholar?q=Gemma+Team%2C+M%2E+Riviere%2C+S%2E+Pathak%2C+P%2E+G%2E+Sessa%2C+C%2E+Hardin%2C+S%2E+Bhupatiraju%2C+L%2E+Hussenot%2C+T%2E+Mesnard%2C+B%2E+Shahriari%2C+A%2E+Ram%C3%A9%2C+et+al%2E+Gemma+2%3A+Improving+open+language+models+at+a+practical+size%2E+arXiv+pre)
14. G. T. Google, R. Anil, S. Borgeaud, Y. Wu, J.-B. Alayrac, J. Yu, R. Soricut, J. Schalkwyk, A. M. Dai, A. Hauth, et al. Gemini: a family of highly capable multimodal models. arXiv preprint arXiv:2312.11805, 2023. [find](https://scholar.google.com/scholar?q=G%2E+T%2E+Google%2C+R%2E+Anil%2C+S%2E+Borgeaud%2C+Y%2E+Wu%2C+J%2E%2DB%2E+Alayrac%2C+J%2E+Yu%2C+R%2E+Soricut%2C+J%2E+Schalkwyk%2C+A%2E+M%2E+Dai%2C+A%2E+Hauth%2C+et+al%2E+Gemini%3A+a+family+of+highly+capable+multimodal+models%2E+arXiv+preprint+arXiv%3A2312%2E1) [our card](https://md.research.elamur.ai/free/en/papers/gemini-2023-team-gemini-family-highly-capable-multimodal/)
15. D. Hendrycks, C. Burns, S. Basart, A. Zou, M. Mazeika, D. Song, and J. Steinhardt. Measuring massive multitask language understanding. arXiv preprint arXiv:2009.03300, 2020. [find](https://scholar.google.com/scholar?q=D%2E+Hendrycks%2C+C%2E+Burns%2C+S%2E+Basart%2C+A%2E+Zou%2C+M%2E+Mazeika%2C+D%2E+Song%2C+and+J%2E+Steinhardt%2E+Measuring+massive+multitask+language+understanding%2E+arXiv+preprint+arXiv%3A2009%2E03300%2C+2020%2E) [our card](https://md.research.elamur.ai/free/en/papers/hendrycks-2020-measuring-massive-multitask-language-understanding/)
16. D. Hendrycks, C. Burns, S. Kadavath, A. Arora, S. Basart, E. Tang, D. Song, and J. Steinhardt. Measuring mathematical problem solving with the math dataset. arXiv preprint arXiv:2103.03874, 2021. [find](https://scholar.google.com/scholar?q=D%2E+Hendrycks%2C+C%2E+Burns%2C+S%2E+Kadavath%2C+A%2E+Arora%2C+S%2E+Basart%2C+E%2E+Tang%2C+D%2E+Song%2C+and+J%2E+Steinhardt%2E+Measuring+mathematical+problem+solving+with+the+math+dataset%2E+arXiv+preprint+arXiv%3A2103%2E03874%2C+2021%2E) [our card](https://md.research.elamur.ai/free/en/papers/hendrycks-2021-measuring-mathematical-problem-solving-math/)
17. A. Hosseini, X. Yuan, N. Malkin, A. Courville, A. Sordoni, and R. Agarwal. V-star: Training verifiers for self-taught reasoners. arXiv preprint arXiv:2402.06457, 2024. [find](https://scholar.google.com/scholar?q=A%2E+Hosseini%2C+X%2E+Yuan%2C+N%2E+Malkin%2C+A%2E+Courville%2C+A%2E+Sordoni%2C+and+R%2E+Agarwal%2E+V%2Dstar%3A+Training+verifiers+for+self%2Dtaught+reasoners%2E+arXiv+preprint+arXiv%3A2402%2E06457%2C+2024%2E) [our card](https://md.research.elamur.ai/free/en/papers/hosseini-2024-v-star-training-verifiers-self/)
18. S. Kapoor, N. Gruver, M. Roberts, K. Collins, A. Pal, U. Bhatt, A. Weller, S. Dooley, M. Goldblum, and A. G. Wilson. Large language models must be taught to know what they don’t know. arXiv preprint arXiv:2406.08391, 2024. [find](https://scholar.google.com/scholar?q=S%2E+Kapoor%2C+N%2E+Gruver%2C+M%2E+Roberts%2C+K%2E+Collins%2C+A%2E+Pal%2C+U%2E+Bhatt%2C+A%2E+Weller%2C+S%2E+Dooley%2C+M%2E+Goldblum%2C+and+A%2E+G%2E+Wilson%2E+Large+language+models+must+be+taught+to+know+what+they+don%E2%80%99t+know%2E+arXiv+preprint+a)
19. S. Kim, J. Shin, Y. Cho, J. Jang, S. Longpre, H. Lee, S. Yun, S. Shin, S. Kim, J. Thorne, et al. Prometheus: Inducing fine-grained evaluation capability in language models. In The Twelfth International Conference on Learning Representations, 2023. [find](https://scholar.google.com/scholar?q=S%2E+Kim%2C+J%2E+Shin%2C+Y%2E+Cho%2C+J%2E+Jang%2C+S%2E+Longpre%2C+H%2E+Lee%2C+S%2E+Yun%2C+S%2E+Shin%2C+S%2E+Kim%2C+J%2E+Thorne%2C+et+al%2E+Prometheus%3A+Inducing+fine%2Dgrained+evaluation+capability+in+language+models%2E+In+The+Twelfth+Internationa) [our card](https://md.research.elamur.ai/free/en/papers/kim-2023-prometheus-inducing-fine-grained-evaluation/)
20. D. P. Kingma. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980, 2014. [find](https://scholar.google.com/scholar?q=D%2E+P%2E+Kingma%2E+Adam%3A+A+method+for+stochastic+optimization%2E+arXiv+preprint+arXiv%3A1412%2E6980%2C+2014%2E)
21. N. Lambert, V. Pyatkin, J. Morrison, L. Miranda, B. Y. Lin, K. Chandu, N. Dziri, S. Kumar, T. Zick, Y. Choi, et al. Rewardbench: Evaluating reward models for language modeling. arXiv preprint arXiv:2403.13787, 2024. [find](https://scholar.google.com/scholar?q=N%2E+Lambert%2C+V%2E+Pyatkin%2C+J%2E+Morrison%2C+L%2E+Miranda%2C+B%2E+Y%2E+Lin%2C+K%2E+Chandu%2C+N%2E+Dziri%2C+S%2E+Kumar%2C+T%2E+Zick%2C+Y%2E+Choi%2C+et+al%2E+Rewardbench%3A+Evaluating+reward+models+for+language+modeling%2E+arXiv+preprint+arXiv%3A24)
22. H. Lightman, V. Kosaraju, Y. Burda, H. Edwards, B. Baker, T. Lee, J. Leike, J. Schulman, I. Sutskever, and K. Cobbe. Let’s verify step by step. arXiv preprint arXiv:2305.20050, 2023. [find](https://scholar.google.com/scholar?q=H%2E+Lightman%2C+V%2E+Kosaraju%2C+Y%2E+Burda%2C+H%2E+Edwards%2C+B%2E+Baker%2C+T%2E+Lee%2C+J%2E+Leike%2C+J%2E+Schulman%2C+I%2E+Sutskever%2C+and+K%2E+Cobbe%2E+Let%E2%80%99s+verify+step+by+step%2E+arXiv+preprint+arXiv%3A2305%2E20050%2C+2023%2E) [our card](https://md.research.elamur.ai/free/en/papers/lightman-2023-let-s-verify-step-step/)
23. Z. Lin, D. Pathak, B. Li, J. Li, X. Xia, G. Neubig, P. Zhang, and D. Ramanan. Evaluating text-to-visual generation with image-to-text generation. arXiv preprint arXiv:2404.01291, 2024. — [arXiv:2404.01291](https://arxiv.org/abs/2404.01291) [find](https://scholar.google.com/scholar?q=Z%2E+Lin%2C+D%2E+Pathak%2C+B%2E+Li%2C+J%2E+Li%2C+X%2E+Xia%2C+G%2E+Neubig%2C+P%2E+Zhang%2C+and+D%2E+Ramanan%2E+Evaluating+text%2Dto%2Dvisual+generation+with+image%2Dto%2Dtext+generation%2E+arXiv+preprint+arXiv%3A2404%2E01291%2C+2024%2E+%E2%80%94+%5BarXiv%3A2404%2E0)
24. Z. Ling, Y. Fang, X. Li, Z. Huang, M. Lee, R. Memisevic, and H. Su. Deductive verification of chain-of-thought reasoning. Advances in Neural Information Processing Systems, 36, 2024. [find](https://scholar.google.com/scholar?q=Z%2E+Ling%2C+Y%2E+Fang%2C+X%2E+Li%2C+Z%2E+Huang%2C+M%2E+Lee%2C+R%2E+Memisevic%2C+and+H%2E+Su%2E+Deductive+verification+of+chain%2Dof%2Dthought+reasoning%2E+Advances+in+Neural+Information+Processing+Systems%2C+36%2C+2024%2E) [our card](https://md.research.elamur.ai/free/en/papers/ling-2023-deductive-verification-chain-thought-reasoning/)
25. Y. Liu, A. Singh, C. D. Freeman, J. D. Co-Reyes, and P. J. Liu. Improving large language model fine-tuning for solving math problems. arXiv preprint arXiv:2310.10047, 2023. — [arXiv:2310.10047](https://arxiv.org/abs/2310.10047) [find](https://scholar.google.com/scholar?q=Y%2E+Liu%2C+A%2E+Singh%2C+C%2E+D%2E+Freeman%2C+J%2E+D%2E+Co%2DReyes%2C+and+P%2E+J%2E+Liu%2E+Improving+large+language+model+fine%2Dtuning+for+solving+math+problems%2E+arXiv+preprint+arXiv%3A2310%2E10047%2C+2023%2E+%E2%80%94+%5BarXiv%3A2310%2E10047%5D%28https%3A)
26. I. Loshchilov and F. Hutter. Decoupled weight decay regularization. arXiv preprint arXiv:1711.05101, 2017. [find](https://scholar.google.com/scholar?q=I%2E+Loshchilov+and+F%2E+Hutter%2E+Decoupled+weight+decay+regularization%2E+arXiv+preprint+arXiv%3A1711%2E05101%2C+2017%2E)
27. L. Luo, Y. Liu, R. Liu, S. Phatale, H. Lara, Y. Li, L. Shu, Y. Zhu, L. Meng, J. Sun, et al. Improve mathematical reasoning in language models by automated process supervision. arXiv preprint arXiv:2406.06592, 2024. [find](https://scholar.google.com/scholar?q=L%2E+Luo%2C+Y%2E+Liu%2C+R%2E+Liu%2C+S%2E+Phatale%2C+H%2E+Lara%2C+Y%2E+Li%2C+L%2E+Shu%2C+Y%2E+Zhu%2C+L%2E+Meng%2C+J%2E+Sun%2C+et+al%2E+Improve+mathematical+reasoning+in+language+models+by+automated+process+supervision%2E+arXiv+preprint+arXiv%3A240) [our card](https://md.research.elamur.ai/free/en/papers/luo-2024-improve-mathematical-reasoning-language-models/)
28. N. McAleese, R. M. Pokorny, J. F. C. Uribe, E. Nitishinskaya, M. Trebacz, and J. Leike. Llm critics help catch llm bugs. arXiv preprint arXiv:2407.00215, 2024. — [arXiv:2407.00215](https://arxiv.org/abs/2407.00215) [find](https://scholar.google.com/scholar?q=N%2E+McAleese%2C+R%2E+M%2E+Pokorny%2C+J%2E+F%2E+C%2E+Uribe%2C+E%2E+Nitishinskaya%2C+M%2E+Trebacz%2C+and+J%2E+Leike%2E+Llm+critics+help+catch+llm+bugs%2E+arXiv+preprint+arXiv%3A2407%2E00215%2C+2024%2E+%E2%80%94+%5BarXiv%3A2407%2E00215%5D%28https%3A%2F%2Farxiv%2Eorg%2Fa)
29. A. Meurer, C. P. Smith, M. Paprocki, O. Čertík, S. B. Kirpichev, M. Rocklin, A. Kumar, S. Ivanov, J. K. Moore, S. Singh, et al. Sympy: symbolic computing in python. PeerJ Computer Science, 3:e103, 2017. [find](https://scholar.google.com/scholar?q=A%2E+Meurer%2C+C%2E+P%2E+Smith%2C+M%2E+Paprocki%2C+O%2E+%C4%8Cert%C3%ADk%2C+S%2E+B%2E+Kirpichev%2C+M%2E+Rocklin%2C+A%2E+Kumar%2C+S%2E+Ivanov%2C+J%2E+K%2E+Moore%2C+S%2E+Singh%2C+et+al%2E+Sympy%3A+symbolic+computing+in+python%2E+PeerJ+Computer+Science%2C+3%3Ae103%2C+201)
30. R. Nakano, J. Hilton, S. Balaji, J. Wu, L. Ouyang, C. Kim, C. Hesse, S. Jain, V. Kosaraju, W. Saunders, et al. Webgpt: Browser-assisted question-answering with human feedback. arXiv preprint arXiv:2112.09332, 2021. [find](https://scholar.google.com/scholar?q=R%2E+Nakano%2C+J%2E+Hilton%2C+S%2E+Balaji%2C+J%2E+Wu%2C+L%2E+Ouyang%2C+C%2E+Kim%2C+C%2E+Hesse%2C+S%2E+Jain%2C+V%2E+Kosaraju%2C+W%2E+Saunders%2C+et+al%2E+Webgpt%3A+Browser%2Dassisted+question%2Danswering+with+human+feedback%2E+arXiv+preprint+arXiv%3A211) [our card](https://md.research.elamur.ai/free/en/papers/nakano-2021-webgpt-browser-assisted-question-answering/)
31. A. Pal, D. Karkhanis, S. Dooley, M. Roberts, S. Naidu, and C. White. Smaug: Fixing failure modes of preference optimisation with dpo-positive. arXiv preprint arXiv:2402.13228, 2024. [find](https://scholar.google.com/scholar?q=A%2E+Pal%2C+D%2E+Karkhanis%2C+S%2E+Dooley%2C+M%2E+Roberts%2C+S%2E+Naidu%2C+and+C%2E+White%2E+Smaug%3A+Fixing+failure+modes+of+preference+optimisation+with+dpo%2Dpositive%2E+arXiv+preprint+arXiv%3A2402%2E13228%2C+2024%2E)
32. R. Y. Pang, W. Yuan, K. Cho, H. He, S. Sukhbaatar, and J. Weston. Iterative reasoning preference optimization. arXiv preprint arXiv:2404.19733, 2024. [find](https://scholar.google.com/scholar?q=R%2E+Y%2E+Pang%2C+W%2E+Yuan%2C+K%2E+Cho%2C+H%2E+He%2C+S%2E+Sukhbaatar%2C+and+J%2E+Weston%2E+Iterative+reasoning+preference+optimization%2E+arXiv+preprint+arXiv%3A2404%2E19733%2C+2024%2E) [our card](https://md.research.elamur.ai/free/en/papers/pang-2024-iterative-reasoning-preference-optimization/)
33. R. Rafailov, A. Sharma, E. Mitchell, C. D. Manning, S. Ermon, and C. Finn. Direct preference optimization: Your language model is secretly a reward model. Advances in Neural Information Processing Systems, 36, 2024. [find](https://scholar.google.com/scholar?q=R%2E+Rafailov%2C+A%2E+Sharma%2C+E%2E+Mitchell%2C+C%2E+D%2E+Manning%2C+S%2E+Ermon%2C+and+C%2E+Finn%2E+Direct+preference+optimization%3A+Your+language+model+is+secretly+a+reward+model%2E+Advances+in+Neural+Information+Processing+Sys)
34. A. Roberts, H. W. Chung, A. Levskaya, G. Mishra, J. Bradbury, D. Andor, S. Narang, B. Lester, C. Gaffney, A. Mohiuddin, C. Hawthorne, A. Lewkowycz, A. Salcianu, M. van Zee, J. Austin, S. Goodman, L. B. Soares, H. Hu, S. Tsvyashchenko, A. Chowdhery, J. Bastings, J. Bulian, X. Garcia, J. Ni, A. Chen,… [find](https://scholar.google.com/scholar?q=A%2E+Roberts%2C+H%2E+W%2E+Chung%2C+A%2E+Levskaya%2C+G%2E+Mishra%2C+J%2E+Bradbury%2C+D%2E+Andor%2C+S%2E+Narang%2C+B%2E+Lester%2C+C%2E+Gaffney%2C+A%2E+Mohiuddin%2C+C%2E+Hawthorne%2C+A%2E+Lewkowycz%2C+A%2E+Salcianu%2C+M%2E+van+Zee%2C+J%2E+Austin%2C+S%2E+Goodman%2C+L%2E+B)
35. W. Saunders, C. Yeh, J. Wu, S. Bills, L. Ouyang, J. Ward, and J. Leike. Self-critiquing models for assisting human evaluators. arXiv preprint arXiv:2206.05802, 2022. [find](https://scholar.google.com/scholar?q=W%2E+Saunders%2C+C%2E+Yeh%2C+J%2E+Wu%2C+S%2E+Bills%2C+L%2E+Ouyang%2C+J%2E+Ward%2C+and+J%2E+Leike%2E+Self%2Dcritiquing+models+for+assisting+human+evaluators%2E+arXiv+preprint+arXiv%3A2206%2E05802%2C+2022%2E) [our card](https://md.research.elamur.ai/free/en/papers/saunders-2022-self-critiquing-models-assisting-human/)
36. T. Schick, J. Dwivedi-Yu, R. Dessì, R. Raileanu, M. Lomeli, E. Hambro, L. Zettlemoyer, N. Cancedda, and T. Scialom. Toolformer: Language models can teach themselves to use tools. Advances in Neural Information Processing Systems, 36, 2024. [find](https://scholar.google.com/scholar?q=T%2E+Schick%2C+J%2E+Dwivedi%2DYu%2C+R%2E+Dess%C3%AC%2C+R%2E+Raileanu%2C+M%2E+Lomeli%2C+E%2E+Hambro%2C+L%2E+Zettlemoyer%2C+N%2E+Cancedda%2C+and+T%2E+Scialom%2E+Toolformer%3A+Language+models+can+teach+themselves+to+use+tools%2E+Advances+in+Neural+In) [our card](https://md.research.elamur.ai/free/en/papers/schick-2023-toolformer-language-models-can-teach/)
37. A. Setlur, S. Garg, X. Geng, N. Garg, V. Smith, and A. Kumar. Rl on incorrect synthetic data scales the efficiency of llm math reasoning by eight-fold. arXiv preprint arXiv:2406.14532, 2024. [find](https://scholar.google.com/scholar?q=A%2E+Setlur%2C+S%2E+Garg%2C+X%2E+Geng%2C+N%2E+Garg%2C+V%2E+Smith%2C+and+A%2E+Kumar%2E+Rl+on+incorrect+synthetic+data+scales+the+efficiency+of+llm+math+reasoning+by+eight%2Dfold%2E+arXiv+preprint+arXiv%3A2406%2E14532%2C+2024%2E)
38. A. Singh, J. D. Co-Reyes, R. Agarwal, A. Anand, P. Patil, P. J. Liu, J. Harrison, J. Lee, K. Xu, A. Parisi, et al. Beyond human data: Scaling self-training for problem-solving with language models. arXiv preprint arXiv:2312.06585, 2023. [find](https://scholar.google.com/scholar?q=A%2E+Singh%2C+J%2E+D%2E+Co%2DReyes%2C+R%2E+Agarwal%2C+A%2E+Anand%2C+P%2E+Patil%2C+P%2E+J%2E+Liu%2C+J%2E+Harrison%2C+J%2E+Lee%2C+K%2E+Xu%2C+A%2E+Parisi%2C+et+al%2E+Beyond+human+data%3A+Scaling+self%2Dtraining+for+problem%2Dsolving+with+language+models%2E+ar) [our card](https://md.research.elamur.ai/free/en/papers/singh-2023-human-data-scaling-self-training/)
39. N. Stiennon, L. Ouyang, J. Wu, D. Ziegler, R. Lowe, C. Voss, A. Radford, D. Amodei, and P. F. Christiano. Learning to summarize with human feedback. Advances in Neural Information Processing Systems, 33: 3008–3021, 2020. [find](https://scholar.google.com/scholar?q=N%2E+Stiennon%2C+L%2E+Ouyang%2C+J%2E+Wu%2C+D%2E+Ziegler%2C+R%2E+Lowe%2C+C%2E+Voss%2C+A%2E+Radford%2C+D%2E+Amodei%2C+and+P%2E+F%2E+Christiano%2E+Learning+to+summarize+with+human+feedback%2E+Advances+in+Neural+Information+Processing+Systems%2C+)
40. Z. Sun, L. Yu, Y. Shen, W. Liu, Y. Yang, S. Welleck, and C. Gan. Easy-to-hard generalization: Scalable alignment beyond human supervision. arXiv preprint arXiv:2403.09472, 2024. [find](https://scholar.google.com/scholar?q=Z%2E+Sun%2C+L%2E+Yu%2C+Y%2E+Shen%2C+W%2E+Liu%2C+Y%2E+Yang%2C+S%2E+Welleck%2C+and+C%2E+Gan%2E+Easy%2Dto%2Dhard+generalization%3A+Scalable+alignment+beyond+human+supervision%2E+arXiv+preprint+arXiv%3A2403%2E09472%2C+2024%2E)
41. M. Suzgun, N. Scales, N. Schärli, S. Gehrmann, Y. Tay, H. W. Chung, A. Chowdhery, Q. V. Le, E. H. Chi, D. Zhou, et al. Challenging big-bench tasks and whether chain-of-thought can solve them. arXiv preprint arXiv:2210.09261, 2022. [find](https://scholar.google.com/scholar?q=M%2E+Suzgun%2C+N%2E+Scales%2C+N%2E+Sch%C3%A4rli%2C+S%2E+Gehrmann%2C+Y%2E+Tay%2C+H%2E+W%2E+Chung%2C+A%2E+Chowdhery%2C+Q%2E+V%2E+Le%2C+E%2E+H%2E+Chi%2C+D%2E+Zhou%2C+et+al%2E+Challenging+big%2Dbench+tasks+and+whether+chain%2Dof%2Dthought+can+solve+them%2E+arXiv+pr) [our card](https://md.research.elamur.ai/free/en/papers/suzgun-2023-challenging-big-bench-tasks-whether/)
42. G. Team, M. Reid, N. Savinov, D. Teplyashin, T. Lillicrap, J.-b. Alayrac, R. Soricut, A. Lazaridou, O. Firat, J. Schrittwieser, et al. Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context. arXiv e-prints, pages arXiv–2403, 2024. [find](https://scholar.google.com/scholar?q=G%2E+Team%2C+M%2E+Reid%2C+N%2E+Savinov%2C+D%2E+Teplyashin%2C+T%2E+Lillicrap%2C+J%2E%2Db%2E+Alayrac%2C+R%2E+Soricut%2C+A%2E+Lazaridou%2C+O%2E+Firat%2C+J%2E+Schrittwieser%2C+et+al%2E+Gemini+1%2E5%3A+Unlocking+multimodal+understanding+across+millions+of) [our card](https://md.research.elamur.ai/free/en/papers/gemini-2024-team-gemini-1-5-unlocking-multimodal/)
43. J. Uesato, N. Kushman, R. Kumar, F. Song, N. Siegel, L. Wang, A. Creswell, G. Irving, and I. Higgins. Solving math word problems with process-and outcome-based feedback. arXiv preprint arXiv:2211.14275, 2022. [find](https://scholar.google.com/scholar?q=J%2E+Uesato%2C+N%2E+Kushman%2C+R%2E+Kumar%2C+F%2E+Song%2C+N%2E+Siegel%2C+L%2E+Wang%2C+A%2E+Creswell%2C+G%2E+Irving%2C+and+I%2E+Higgins%2E+Solving+math+word+problems+with+process%2Dand+outcome%2Dbased+feedback%2E+arXiv+preprint+arXiv%3A2211%2E1427) [our card](https://md.research.elamur.ai/free/en/papers/uesato-2022-solving-math-word-problems-process/)
44. P. Wang, L. Li, Z. Shao, R. Xu, D. Dai, Y. Li, D. Chen, Y. Wu, and Z. Sui. Math-shepherd: A label-free step-by-step verifier for llms in mathematical reasoning. arXiv preprint arXiv:2312.08935, 2023. [find](https://scholar.google.com/scholar?q=P%2E+Wang%2C+L%2E+Li%2C+Z%2E+Shao%2C+R%2E+Xu%2C+D%2E+Dai%2C+Y%2E+Li%2C+D%2E+Chen%2C+Y%2E+Wu%2C+and+Z%2E+Sui%2E+Math%2Dshepherd%3A+A+label%2Dfree+step%2Dby%2Dstep+verifier+for+llms+in+mathematical+reasoning%2E+arXiv+preprint+arXiv%3A2312%2E08935%2C+2023%2E) [our card](https://md.research.elamur.ai/free/en/papers/wang-2024-math-shepherd-verify-reinforce-llms/)
45. T. Wang, I. Kulikov, O. Golovneva, P. Yu, W. Yuan, J. Dwivedi-Yu, R. Y. Pang, M. Fazel-Zarandi, J. Weston, and X. Li. Self-taught evaluators. arXiv preprint arXiv:2408.02666, 2024. [find](https://scholar.google.com/scholar?q=T%2E+Wang%2C+I%2E+Kulikov%2C+O%2E+Golovneva%2C+P%2E+Yu%2C+W%2E+Yuan%2C+J%2E+Dwivedi%2DYu%2C+R%2E+Y%2E+Pang%2C+M%2E+Fazel%2DZarandi%2C+J%2E+Weston%2C+and+X%2E+Li%2E+Self%2Dtaught+evaluators%2E+arXiv+preprint+arXiv%3A2408%2E02666%2C+2024%2E)
46. X. Wang, J. Wei, D. Schuurmans, Q. Le, E. Chi, S. Narang, A. Chowdhery, and D. Zhou. Self-consistency improves chain of thought reasoning in language models. arXiv preprint arXiv:2203.11171, 2022. [find](https://scholar.google.com/scholar?q=X%2E+Wang%2C+J%2E+Wei%2C+D%2E+Schuurmans%2C+Q%2E+Le%2C+E%2E+Chi%2C+S%2E+Narang%2C+A%2E+Chowdhery%2C+and+D%2E+Zhou%2E+Self%2Dconsistency+improves+chain+of+thought+reasoning+in+language+models%2E+arXiv+preprint+arXiv%3A2203%2E11171%2C+2022%2E) [our card](https://md.research.elamur.ai/free/en/papers/wang-2022-self-consistency-improves-chain-thought/)
47. J. Wei, X. Wang, D. Schuurmans, M. Bosma, F. Xia, E. Chi, Q. V. Le, D. Zhou, et al. Chain-of-thought prompting elicits reasoning in large language models. Advances in neural information processing systems, 35:24824–24837, 2022. [find](https://scholar.google.com/scholar?q=J%2E+Wei%2C+X%2E+Wang%2C+D%2E+Schuurmans%2C+M%2E+Bosma%2C+F%2E+Xia%2C+E%2E+Chi%2C+Q%2E+V%2E+Le%2C+D%2E+Zhou%2C+et+al%2E+Chain%2Dof%2Dthought+prompting+elicits+reasoning+in+large+language+models%2E+Advances+in+neural+information+processing+sys) [our card](https://md.research.elamur.ai/free/en/papers/wei-2022-chain-thought-prompting-elicits-reasoning/)
48. M. Wortsman, P. J. Liu, L. Xiao, K. Everett, A. Alemi, B. Adlam, J. D. Co-Reyes, I. Gur, A. Kumar, R. Novak, et al. Small-scale proxies for large-scale transformer training instabilities. arXiv preprint arXiv:2309.14322, 2023. [find](https://scholar.google.com/scholar?q=M%2E+Wortsman%2C+P%2E+J%2E+Liu%2C+L%2E+Xiao%2C+K%2E+Everett%2C+A%2E+Alemi%2C+B%2E+Adlam%2C+J%2E+D%2E+Co%2DReyes%2C+I%2E+Gur%2C+A%2E+Kumar%2C+R%2E+Novak%2C+et+al%2E+Small%2Dscale+proxies+for+large%2Dscale+transformer+training+instabilities%2E+arXiv+prepri)
49. T. Wu, W. Yuan, O. Golovneva, J. Xu, Y. Tian, J. Jiao, J. Weston, and S. Sukhbaatar. Meta-rewarding language models: Self-improving alignment with llm-as-a-meta-judge. arXiv preprint arXiv:2407.19594, 2024. [find](https://scholar.google.com/scholar?q=T%2E+Wu%2C+W%2E+Yuan%2C+O%2E+Golovneva%2C+J%2E+Xu%2C+Y%2E+Tian%2C+J%2E+Jiao%2C+J%2E+Weston%2C+and+S%2E+Sukhbaatar%2E+Meta%2Drewarding+language+models%3A+Self%2Dimproving+alignment+with+llm%2Das%2Da%2Dmeta%2Djudge%2E+arXiv+preprint+arXiv%3A2407%2E19594%2C)
50. R. Yang, R. Ding, Y. Lin, H. Zhang, and T. Zhang. Regularizing hidden states enables learning generalizable reward model for llms. arXiv preprint arXiv:2406.10216, 2024. — [arXiv:2406.10216](https://arxiv.org/abs/2406.10216) [find](https://scholar.google.com/scholar?q=R%2E+Yang%2C+R%2E+Ding%2C+Y%2E+Lin%2C+H%2E+Zhang%2C+and+T%2E+Zhang%2E+Regularizing+hidden+states+enables+learning+generalizable+reward+model+for+llms%2E+arXiv+preprint+arXiv%3A2406%2E10216%2C+2024%2E+%E2%80%94+%5BarXiv%3A2406%2E10216%5D%28https%3A%2F%2Fa)
51. S. Yao, D. Yu, J. Zhao, I. Shafran, T. Griffiths, Y. Cao, and K. Narasimhan. Tree of thoughts: Deliberate problem solving with large language models. Advances in Neural Information Processing Systems, 36, 2024. [find](https://scholar.google.com/scholar?q=S%2E+Yao%2C+D%2E+Yu%2C+J%2E+Zhao%2C+I%2E+Shafran%2C+T%2E+Griffiths%2C+Y%2E+Cao%2C+and+K%2E+Narasimhan%2E+Tree+of+thoughts%3A+Deliberate+problem+solving+with+large+language+models%2E+Advances+in+Neural+Information+Processing+Systems%2C)
52. Z. Ye, F. Greenlee-Scott, M. Bartolo, P. Blunsom, J. A. Campos, and M. Gallé. Improving reward models with synthetic critiques. arXiv preprint arXiv:2405.20850, 2024. [find](https://scholar.google.com/scholar?q=Z%2E+Ye%2C+F%2E+Greenlee%2DScott%2C+M%2E+Bartolo%2C+P%2E+Blunsom%2C+J%2E+A%2E+Campos%2C+and+M%2E+Gall%C3%A9%2E+Improving+reward+models+with+synthetic+critiques%2E+arXiv+preprint+arXiv%3A2405%2E20850%2C+2024%2E)
53. F. Yu, A. Gao, and B. Wang. Ovm, outcome-supervised value models for planning in mathematical reasoning. In Findings of the Association for Computational Linguistics: NAACL 2024, pages 858–875, 2024. [find](https://scholar.google.com/scholar?q=F%2E+Yu%2C+A%2E+Gao%2C+and+B%2E+Wang%2E+Ovm%2C+outcome%2Dsupervised+value+models+for+planning+in+mathematical+reasoning%2E+In+Findings+of+the+Association+for+Computational+Linguistics%3A+NAACL+2024%2C+pages+858%E2%80%93875%2C+2024%2E)
54. W. Yuan, R. Y. Pang, K. Cho, S. Sukhbaatar, J. Xu, and J. Weston. Self-rewarding language models. arXiv preprint arXiv:2401.10020, 2024. [find](https://scholar.google.com/scholar?q=W%2E+Yuan%2C+R%2E+Y%2E+Pang%2C+K%2E+Cho%2C+S%2E+Sukhbaatar%2C+J%2E+Xu%2C+and+J%2E+Weston%2E+Self%2Drewarding+language+models%2E+arXiv+preprint+arXiv%3A2401%2E10020%2C+2024%2E)
55. E. Zelikman, Y. Wu, J. Mu, and N. Goodman. Star: Bootstrapping reasoning with reasoning. Advances in Neural Information Processing Systems, 35:15476–15488, 2022. [find](https://scholar.google.com/scholar?q=E%2E+Zelikman%2C+Y%2E+Wu%2C+J%2E+Mu%2C+and+N%2E+Goodman%2E+Star%3A+Bootstrapping+reasoning+with+reasoning%2E+Advances+in+Neural+Information+Processing+Systems%2C+35%3A15476%E2%80%9315488%2C+2022%2E) [our card](https://md.research.elamur.ai/free/en/papers/zelikman-2022-star-bootstrapping-reasoning-reasoning/)
56. E. Zhang, V. Zhu, N. Saphra, A. Kleiman, B. L. Edelman, M. Tambe, S. M. Kakade, and E. Malach. Transcendence: Generative models can outperform the experts that train them. arXiv preprint arXiv:2406.11741, 2024. [find](https://scholar.google.com/scholar?q=E%2E+Zhang%2C+V%2E+Zhu%2C+N%2E+Saphra%2C+A%2E+Kleiman%2C+B%2E+L%2E+Edelman%2C+M%2E+Tambe%2C+S%2E+M%2E+Kakade%2C+and+E%2E+Malach%2E+Transcendence%3A+Generative+models+can+outperform+the+experts+that+train+them%2E+arXiv+preprint+arXiv%3A2406%2E11)
57. Y. Zhang, Y. Li, L. Cui, D. Cai, L. Liu, T. Fu, X. Huang, E. Zhao, Y. Zhang, Y. Chen, et al. Siren’s song in the ai ocean: a survey on hallucination in large language models. arXiv preprint arXiv:2309.01219, 2023. [find](https://scholar.google.com/scholar?q=Y%2E+Zhang%2C+Y%2E+Li%2C+L%2E+Cui%2C+D%2E+Cai%2C+L%2E+Liu%2C+T%2E+Fu%2C+X%2E+Huang%2C+E%2E+Zhao%2C+Y%2E+Zhang%2C+Y%2E+Chen%2C+et+al%2E+Siren%E2%80%99s+song+in+the+ai+ocean%3A+a+survey+on+hallucination+in+large+language+models%2E+arXiv+preprint+arXiv%3A2309) [our card](https://md.research.elamur.ai/free/en/papers/zhang-2023-siren-s-song-ai-ocean/)
58. L. Zheng, W.-L. Chiang, Y. Sheng, S. Zhuang, Z. Wu, Y. Zhuang, Z. Lin, Z. Li, D. Li, E. Xing, et al. Judging llm-as-a-judge with mt-bench and chatbot arena. Advances in Neural Information Processing Systems, 36, 2024. [find](https://scholar.google.com/scholar?q=L%2E+Zheng%2C+W%2E%2DL%2E+Chiang%2C+Y%2E+Sheng%2C+S%2E+Zhuang%2C+Z%2E+Wu%2C+Y%2E+Zhuang%2C+Z%2E+Lin%2C+Z%2E+Li%2C+D%2E+Li%2C+E%2E+Xing%2C+et+al%2E+Judging+llm%2Das%2Da%2Djudge+with+mt%2Dbench+and+chatbot+arena%2E+Advances+in+Neural+Information+Processing+S)