---
title: "A Survey of Reinforcement Learning from Human Feedback (2023)"
description: "The resulting map of design choices helps researchers working across application domains… 427 references, cited by 57 papers here."
authors: ["Timo Kaufmann", "Paul Weng", "Viktor Bengs", "Eyke Hullermeier"]
year: 2023
venue: "arXiv:2312.14925 · Transactions on Machine Learning Research, 2025"
source: "https://arxiv.org/abs/2312.14925"
doi: "10.48550/arxiv.2312.14925"
arxiv: "2312.14925"
licence: "CC BY 4.0 (arXiv)"
licence_url: "http://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/kaufmann-2023-survey-reinforcement-learning-human-feedback/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/kaufmann-2023-survey-reinforcement-learning-human-feedback/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2023](https://md.research.elamur.ai/free/en/papers/2023/page/8/) / A Survey of Reinforcement Learning from Human Feedback

# A Survey of Reinforcement Learning from Human Feedback

**Authors:** [Timo Kaufmann](https://md.research.elamur.ai/free/en/authors/timo-kaufmann/), Paul Weng, Viktor Bengs and [Eyke Hullermeier](https://md.research.elamur.ai/free/en/authors/eyke-hullermeier/) · **Year:** 2023 · **Venue:** [arXiv:2312.14925 · Transactions on Machine Learning Research, 2025](https://md.research.elamur.ai/free/en/venues/transactions-on-machine-learning-research/) ([Transactions on Machine Learning Research 2023](https://md.research.elamur.ai/free/en/venues/transactions-on-machine-learning-research/2023/)) · **Licence:** [CC BY 4.0 (arXiv)](http://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://arxiv.org/abs/2312.14925) · [PDF](https://arxiv.org/pdf/2312.14925) · [DOI 10.48550/arxiv.2312.14925](https://doi.org/10.48550/arxiv.2312.14925) · [arXiv 2312.14925](https://arxiv.org/abs/2312.14925) · [OpenAlex](https://openalex.org/W4390215524)

**Citations:** 35 citations (OpenAlex, 2026-10-14) · [57](https://md.research.elamur.ai/free/en/papers/kaufmann-2023-survey-reinforcement-learning-human-feedback/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
Hand-crafting reward functions for reinforcement learning is hard. Success is often vague, sparse signals slow learning, and shaped rewards invite reward hacking. This survey by Kaufmann, Weng, Bengs and Hüllermeier, published in TMLR, reviews reinforcement learning from human feedback (RLHF) as an alternative. In RLHF, a reward model is learned from human judgments and then used to train a policy. The authors place RLHF within its predecessor, preference-based RL (PbRL), and bring up to date the most recent review of similar scope (Wirth et al., 2017).

The survey's main contribution is a structured overview. It classifies feedback types by attributes such as arity, involvement and granularity, then covers label collection and active query selection, reward model training, policy learning (including bandit framings and direct policy optimization), theory, and benchmarks and datasets. Coverage is broad across domains, with the most depth in control and robotics. A separate appendix addresses LLM fine-tuning.

The resulting map of design choices helps researchers working across application domains, including LLM alignment.

## Key points
- **Why not inverse RL alone:** The authors argue that interactive feedback is easier for humans to give than demonstrations. It also does not require experts to perform near-optimally and can capture behavior people actually want from a machine. Demonstrations, by contrast, rarely identify rewards robustly.
- **Feedback taxonomy:** Feedback types are described along six attributes: arity, involvement, intent, explicitness, granularity and abstraction. One example is the similarity query (Bobu et al., 2023), which asks a human to compare an anchor trajectory against two alternatives.
- **Combining feedback types:** The survey identifies three strategies:
  - an initialize-then-refine pipeline, e.g., demonstrations followed by pairwise comparisons (Ibarz et al., 2018);
  - a primary feedback type paired with a cheap supplementary signal;
  - a blend of several primary types.
- **Application domains covered:** LLM fine-tuning, image generation, music generation, continuous control, games and robotics.
- **Methodological trends surveyed:**
  - active learning and query synthesis;
  - psychology-informed labeling;
  - meta-learning for transferring preferences;
  - data augmentation and semi-supervised learning;
  - Nash learning from human feedback.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Ludwig-Maximilians-Universität München](https://md.research.elamur.ai/free/en/universities/ludwig-maximilians-universitat-munchen/) — University, Munich, Germany · 113 works · 56,817 citations
- [LMU Klinikum](https://md.research.elamur.ai/free/en/hospitals/lmu-klinikum/) — Hospital, Munich, Germany · 2 works · 102 citations

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 18,614 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 17,965 works · [🎯 Decision Sciences](https://md.research.elamur.ai/free/en/shelves/decision-sciences/) · 249 works · [🎯 Management Science and Operations Research](https://md.research.elamur.ai/free/en/shelves/decision-sciences/management-science-and-operations-research/) · 134 works

## Cite

Cite as: Timo Kaufmann, Paul Weng, Viktor Bengs and Eyke Hullermeier. 2023. A Survey of Reinforcement Learning from Human Feedback. arXiv:2312.14925 · Transactions on Machine Learning Research, 2025.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@misc{kaufmann2023survey,
  title = {{A Survey of Reinforcement Learning from Human Feedback}},
  author = {Timo Kaufmann, Paul Weng, Viktor Bengs and Eyke Hullermeier},
  year = {2023},
  doi = {10.48550/arxiv.2312.14925},
  eprint = {2312.14925},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2312.14925}
}
```

- [← previous How well do Large Language Models perform in Arithmetic tasks?](https://md.research.elamur.ai/free/en/papers/yuan-2023-well-large-language-models-perform/)
- [next → ChatHaruhi: Reviving Anime Character in Reality via Large Language Model](https://md.research.elamur.ai/free/en/papers/li-2023-chatharuhi-reviving-anime-character-reality/)
- [↑ back to papers of 2023](https://md.research.elamur.ai/free/en/papers/2023/page/8/)

## References

427 references, 45 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Youssef Abdelkareem, Shady Shehata, and Fakhri Karray. Advances in Preference-based Reinforcement Learning: A Review. In Proceedings of IEEE International Conference on Systems, Man, and Cybernetics (SMC), 2022. doi: 10.1109/SMC53654.2022.9945333. — [doi:10.1109/smc53654.2022.9945333](https://doi.org/10.1109/smc53654.2022.9945333) [find](https://scholar.google.com/scholar?q=Youssef+Abdelkareem%2C+Shady+Shehata%2C+and+Fakhri+Karray%2E+Advances+in+Preference%2Dbased+Reinforcement+Learning%3A+A+Review%2E+In+Proceedings+of+IEEE+International+Conference+on+Systems%2C+Man%2C+and+Cybernetics+%28)
2. Josh Abramson, Arun Ahuja, Federico Carnevale, Petko Georgiev, Alex Goldin, Alden Hung, Jessica Landon, Jirka Lhotka, Timothy Lillicrap, Alistair Muldal, George Powell, Adam Santoro, Guy Scully, Sanjana Srivastava, Tamara von Glehn, Greg Wayne, Nathaniel Wong, Chen Yan, and Rui Zhu. Improving Multim… — [arXiv:2211.11602](https://arxiv.org/abs/2211.11602) [find](https://scholar.google.com/scholar?q=Josh+Abramson%2C+Arun+Ahuja%2C+Federico+Carnevale%2C+Petko+Georgiev%2C+Alex+Goldin%2C+Alden+Hung%2C+Jessica+Landon%2C+Jirka+Lhotka%2C+Timothy+Lillicrap%2C+Alistair+Muldal%2C+George+Powell%2C+Adam+Santoro%2C+Guy+Scully%2C+Sanja)
3. Riad Akrour, Marc Schoenauer, and Michele Sebag. Preference-Based Policy Learning. In Proceedings of Machine Learning and Knowledge Discovery in Databases (ECML PKDD). Springer, 2011. doi: 10.1007/978-3-642-23780-5_11. — [doi:10.1007/978-3-642-23780-5_11](https://doi.org/10.1007/978-3-642-23780-5_11) [find](https://scholar.google.com/scholar?q=Riad+Akrour%2C+Marc+Schoenauer%2C+and+Michele+Sebag%2E+Preference%2DBased+Policy+Learning%2E+In+Proceedings+of+Machine+Learning+and+Knowledge+Discovery+in+Databases+%28ECML+PKDD%29%2E+Springer%2C+2011%2E+doi%3A+10%2E1007%2F978)
4. Mayer Alvo and Philip L.H. Yu. Statistical Methods for Ranking Data. Springer, 2014. doi: 10.1007/978-1-4939-1471-5. — [doi:10.1007/978-1-4939-1471-5](https://doi.org/10.1007/978-1-4939-1471-5) [find](https://scholar.google.com/scholar?q=Mayer+Alvo+and+Philip+L%2EH%2E+Yu%2E+Statistical+Methods+for+Ranking+Data%2E+Springer%2C+2014%2E+doi%3A+10%2E1007%2F978%2D1%2D4939%2D1471%2D5%2E+%E2%80%94+%5Bdoi%3A10%2E1007%2F978%2D1%2D4939%2D1471%2D5%5D%28https%3A%2F%2Fdoi%2Eorg%2F10%2E1007%2F978%2D1%2D4939%2D1471%2D5%29)
5. Dario Amodei, Chris Olah, Jacob Steinhardt, Paul Christiano, John Schulman, and Dan Mané. [Concrete Problems in AI Safety](https://md.research.elamur.ai/free/en/papers/amodei-2016-concrete-problems-ai-safety/), 2016. arXiv: 1606.06565. preprint.
6. Dario Amodei, Paul Christiano, and Alex Ray. Learning from human preferences, 2017. URL [https://openai.com/research/learning-from-human-preferences](https://openai.com/research/learning-from-human-preferences). (accessed 2023-05-25). [↗](https://openai.com/research/learning-from-human-preferences)
7. Gaon An, Junhyeok Lee, Xingdong Zuo, Norio Kosaka, Kyung-Min Kim, and Hyun Oh Song. Direct Preference-based Policy Optimization without Reward Modeling. In Advances in Neural Information Processing Systems (NeurIPS), volume 36, 2023. URL [https://papers.nips.cc/paper_files/paper/](https://papers.nips.cc/paper_files/paper/) 2023/hash/de8bd6b2b0… [↗](https://papers.nips.cc/paper_files/paper/)
8. Stuart Armstrong, Jan Leike, Laurent Orseau, and Shane Legg. Pitfalls of Learning a Reward Function Online. In Proceedings of the International Joint Conference on Artificial Intelligence (IJCAI), volume 2, 2020. doi: 10.24963/ijcai.2020/221. — [doi:10.24963/ijcai.2020/221](https://doi.org/10.24963/ijcai.2020/221) [find](https://scholar.google.com/scholar?q=Stuart+Armstrong%2C+Jan+Leike%2C+Laurent+Orseau%2C+and+Shane+Legg%2E+Pitfalls+of+Learning+a+Reward+Function+Online%2E+In+Proceedings+of+the+International+Joint+Conference+on+Artificial+Intelligence+%28IJCAI%29%2C+vol)
9. Saurabh Arora and Prashant Doshi. A Survey of Inverse Reinforcement Learning: Challenges, Methods and Progress. Artificial Intelligence, 297:103500, 2021. doi: 10.1016/j.artint.2021.103500. — [doi:10.1016/j.artint.2021.103500](https://doi.org/10.1016/j.artint.2021.103500) [find](https://scholar.google.com/scholar?q=Saurabh+Arora+and+Prashant+Doshi%2E+A+Survey+of+Inverse+Reinforcement+Learning%3A+Challenges%2C+Methods+and+Progress%2E+Artificial+Intelligence%2C+297%3A103500%2C+2021%2E+doi%3A+10%2E1016%2Fj%2Eartint%2E2021%2E103500%2E+%E2%80%94+%5Bdoi%3A10%2E)
10. Dilip Arumugam, Jun Ki Lee, Sophie Saskin, and Michael L. Littman. Deep Reinforcement Learning from Policy-Dependent Human Feedback, 2019. arXiv: 1902.04257. preprint. — [arXiv:1902.04257](https://arxiv.org/abs/1902.04257) [find](https://scholar.google.com/scholar?q=Dilip+Arumugam%2C+Jun+Ki+Lee%2C+Sophie+Saskin%2C+and+Michael+L%2E+Littman%2E+Deep+Reinforcement+Learning+from+Policy%2DDependent+Human+Feedback%2C+2019%2E+arXiv%3A+1902%2E04257%2E+preprint%2E+%E2%80%94+%5BarXiv%3A1902%2E04257%5D%28https%3A%2F%2Farx)
11. Amanda Askell, Yuntao Bai, Anna Chen, Dawn Drain, Deep Ganguli, Tom Henighan, Andy Jones, Nicholas Joseph, Ben Mann, Nova DasSarma, Nelson Elhage, Zac Hatfield-Dodds, Danny Hernandez, Jackson Kernion, Kamal Ndousse, Catherine Olsson, Dario Amodei, Tom Brown, Jack Clark, Sam McCandlish, Chris Olah, a… [find](https://scholar.google.com/scholar?q=Amanda+Askell%2C+Yuntao+Bai%2C+Anna+Chen%2C+Dawn+Drain%2C+Deep+Ganguli%2C+Tom+Henighan%2C+Andy+Jones%2C+Nicholas+Joseph%2C+Ben+Mann%2C+Nova+DasSarma%2C+Nelson+Elhage%2C+Zac+Hatfield%2DDodds%2C+Danny+Hernandez%2C+Jackson+Kernion%2C)
12. Müslüm Atas, Alexander Felfernig, Seda Polat-Erdeniz, Andrei Popescu, Thi Ngoc Trang Tran, and Mathias Uta. Towards psychology-aware preference construction in recommender systems: Overview and research issues. Journal of Intelligent Information Systems, 57(3):467–489, 2021. doi: 10.1007/s10844-021-… — [doi:10.1007/s10844-021-00674-5](https://doi.org/10.1007/s10844-021-00674-5) [find](https://scholar.google.com/scholar?q=M%C3%BCsl%C3%BCm+Atas%2C+Alexander+Felfernig%2C+Seda+Polat%2DErdeniz%2C+Andrei+Popescu%2C+Thi+Ngoc+Trang+Tran%2C+and+Mathias+Uta%2E+Towards+psychology%2Daware+preference+construction+in+recommender+systems%3A+Overview+and+resear)
13. Mohammad Gheshlaghi Azar, Zhaohan Daniel Guo, Bilal Piot, Remi Munos, Mark Rowland, Michal Valko, and Daniele Calandriello. [A General Theoretical Paradigm to Understand Learning from Human Preferences](https://md.research.elamur.ai/free/en/papers/azar-2023-general-theoretical-paradigm-understand-learning/). In Proceedings of the International Conference on Artificial Intelligence and Statistics (AISTATS)…
14. Yuntao Bai, Andy Jones, Kamal Ndousse, Amanda Askell, Anna Chen, Nova DasSarma, Dawn Drain, Stanislav Fort, Deep Ganguli, Tom Henighan, Nicholas Joseph, Saurav Kadavath, Jackson Kernion, Tom Conerly, Sheer El-Showk, Nelson Elhage, Zac Hatfield-Dodds, Danny Hernandez, Tristan Hume, Scott Johnston, Sh… [find](https://scholar.google.com/scholar?q=Yuntao+Bai%2C+Andy+Jones%2C+Kamal+Ndousse%2C+Amanda+Askell%2C+Anna+Chen%2C+Nova+DasSarma%2C+Dawn+Drain%2C+Stanislav+Fort%2C+Deep+Ganguli%2C+Tom+Henighan%2C+Nicholas+Joseph%2C+Saurav+Kadavath%2C+Jackson+Kernion%2C+Tom+Conerly%2C+)
15. Yuntao Bai, Saurav Kadavath, Sandipan Kundu, Amanda Askell, Jackson Kernion, Andy Jones, Anna Chen, Anna Goldie, Azalia Mirhoseini, Cameron McKinnon, Carol Chen, Catherine Olsson, Christopher Olah, Danny Hernandez, Dawn Drain, Deep Ganguli, Dustin Li, Eli Tran-Johnson, Ethan Perez, Jamie Kerr, Jared… — [Constitutional AI: Harmlessness from AI Feedback](https://md.research.elamur.ai/free/en/papers/bai-2022-constitutional-ai-harmlessness-ai-feedback/)
16. Bowen Baker, Ingmar Kanitscheider, Todor Markov, Yi Wu, Glenn Powell, Bob McGrew, and Igor Mordatch. Emergent Tool Use From Multi-Agent Autocurricula. In Proceedings of the International Conference on Learning Representations (ICLR), 2020. URL [https://openreview.net/forum?id=SkxpxJBKwS](https://openreview.net/forum?id=SkxpxJBKwS). [↗](https://openreview.net/forum?id=SkxpxJBKwS)
17. Michiel Bakker, Martin Chadwick, Hannah Sheahan, Michael Tessler, Lucy Campbell-Gillingham, Jan Balaguer, Nat McAleese, Amelia Glaese, John Aslanides, Matt Botvinick, and Christopher Summerfield. Fine-tuning language models to find agreement among humans with diverse preferences. In Advances in Neur… [find](https://scholar.google.com/scholar?q=Michiel+Bakker%2C+Martin+Chadwick%2C+Hannah+Sheahan%2C+Michael+Tessler%2C+Lucy+Campbell%2DGillingham%2C+Jan+Balaguer%2C+Nat+McAleese%2C+Amelia+Glaese%2C+John+Aslanides%2C+Matt+Botvinick%2C+and+Christopher+Summerfield%2E+Fine)
18. Hritik Bansal, John Dang, and Aditya Grover. Peering Through Preferences: Unraveling Feedback Acquisition for Aligning Large Language Models. In Proceedings of the International Conference on Learning Representations (ICLR), 2024. URL [https://openreview.net/forum?id=dKl6lMwbCy](https://openreview.net/forum?id=dKl6lMwbCy). [↗](https://openreview.net/forum?id=dKl6lMwbCy)
19. Peter Barnett, Rachel Freedman, Justin Svegliato, and Stuart Russell. Active Reward Learning from Multiple Teachers. In AAAI 2023 Workshop on Artificial Intelligence Safety, 2023. [find](https://scholar.google.com/scholar?q=Peter+Barnett%2C+Rachel+Freedman%2C+Justin+Svegliato%2C+and+Stuart+Russell%2E+Active+Reward+Learning+from+Multiple+Teachers%2E+In+AAAI+2023+Workshop+on+Artificial+Intelligence+Safety%2C+2023%2E)
20. Alejandro Barredo Arrieta, Natalia Díaz-Rodríguez, Javier Del Ser, Adrien Bennetot, Siham Tabik, Alberto Barbado, Salvador Garcia, Sergio Gil-Lopez, Daniel Molina, Richard Benjamins, Raja Chatila, and Francisco Herrera. Explainable Artificial Intelligence (XAI): Concepts, taxonomies, opportunities a… [find](https://scholar.google.com/scholar?q=Alejandro+Barredo+Arrieta%2C+Natalia+D%C3%ADaz%2DRodr%C3%ADguez%2C+Javier+Del+Ser%2C+Adrien+Bennetot%2C+Siham+Tabik%2C+Alberto+Barbado%2C+Salvador+Garcia%2C+Sergio+Gil%2DLopez%2C+Daniel+Molina%2C+Richard+Benjamins%2C+Raja+Chatila%2C+and)
21. Chandrayee Basu, Qian Yang, David Hungerman, Mukesh Singhal, and Anca D. Dragan. Do You Want Your Autonomous Car To Drive Like You? In Proceedings of the ACM/IEEE International Conference on Human-Robot Interaction (HRI). Association for Computing Machinery, 2017. doi: 10.1145/2909824.3020250. — [doi:10.1145/2909824.3020250](https://doi.org/10.1145/2909824.3020250) [find](https://scholar.google.com/scholar?q=Chandrayee+Basu%2C+Qian+Yang%2C+David+Hungerman%2C+Mukesh+Singhal%2C+and+Anca+D%2E+Dragan%2E+Do+You+Want+Your+Autonomous+Car+To+Drive+Like+You%3F+In+Proceedings+of+the+ACM%2FIEEE+International+Conference+on+Human%2DRob)
22. Chandrayee Basu, Mukesh Singhal, and Anca D. Dragan. Learning from Richer Human Guidance: Augmenting Comparison-Based Learning with Feature Queries. In Proceedings of the ACM/IEEE International Conference on Human-Robot Interaction (HRI). Association for Computing Machinery, 2018. doi: 10.1145/31712… — [doi:10.1145/3171221.3171284](https://doi.org/10.1145/3171221.3171284) [find](https://scholar.google.com/scholar?q=Chandrayee+Basu%2C+Mukesh+Singhal%2C+and+Anca+D%2E+Dragan%2E+Learning+from+Richer+Human+Guidance%3A+Augmenting+Comparison%2DBased+Learning+with+Feature+Queries%2E+In+Proceedings+of+the+ACM%2FIEEE+International+Confer)
23. Chandrayee Basu, Erdem Bıyık, Zhixun He, Mukesh Singhal, and Dorsa Sadigh. Active Learning of Reward Dynamics from Hierarchical Queries. In Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2019. doi: 10.1109/IROS40897.2019.8968522. — [doi:10.1109/iros40897.2019.8968522](https://doi.org/10.1109/iros40897.2019.8968522) [find](https://scholar.google.com/scholar?q=Chandrayee+Basu%2C+Erdem+B%C4%B1y%C4%B1k%2C+Zhixun+He%2C+Mukesh+Singhal%2C+and+Dorsa+Sadigh%2E+Active+Learning+of+Reward+Dynamics+from+Hierarchical+Queries%2E+In+Proceedings+of+the+IEEE%2FRSJ+International+Conference+on+Inte)
24. Emmanuel Bengio, Moksh Jain, Maksym Korablyov, Doina Precup, and Yoshua Bengio. Flow Network based Generative Models for Non-Iterative Diverse Candidate Generation. In Advances in Neural Information Processing Systems (NeurIPS), volume 34. Curran Associates, Inc., 2021. URL https://proceedings. neur… [find](https://scholar.google.com/scholar?q=Emmanuel+Bengio%2C+Moksh+Jain%2C+Maksym+Korablyov%2C+Doina+Precup%2C+and+Yoshua+Bengio%2E+Flow+Network+based+Generative+Models+for+Non%2DIterative+Diverse+Candidate+Generation%2E+In+Advances+in+Neural+Information+P)
25. Viktor Bengs and Eyke Hüllermeier. Preselection Bandits. In Proceedings of the International Conference on Machine Learning (ICML). PMLR, 2020. URL [https://proceedings.mlr.press/v119/bengs20a.html](https://proceedings.mlr.press/v119/bengs20a.html). [↗](https://proceedings.mlr.press/v119/bengs20a.html)
26. Viktor Bengs, Róbert Busa-Fekete, Adil El Mesaoudi-Paul, and Eyke Hüllermeier. Preference-based Online Learning with Dueling Bandits: A Survey. Journal of Machine Learning Research, 22(7):1–108, 2021. ISSN 1533-7928. URL [http://jmlr.org/papers/v22/18-546.html](http://jmlr.org/papers/v22/18-546.html). [↗](http://jmlr.org/papers/v22/18-546.html)
27. Viktor Bengs, Aadirupa Saha, and Eyke Hüllermeier. Stochastic Contextual Dueling Bandits under Linear Stochastic Transitivity Models. In Proceedings of the International Conference on Machine Learning (ICML). PMLR, 2022. URL [https://proceedings.mlr.press/v162/bengs22a.html](https://proceedings.mlr.press/v162/bengs22a.html). [↗](https://proceedings.mlr.press/v162/bengs22a.html)
28. Tom Bewley and Freddy Lécué. Interpretable Preference-based Reinforcement Learning with Tree-Structured Reward Functions. In Proceedings of the International Conference on Autonomous Agents and Multiagent Systems (AAMAS). IFAAMAS, 2022. URL [https://www.ifaamas.org/Proceedings/aamas2022/pdfs/](https://www.ifaamas.org/Proceedings/aamas2022/pdfs/) p118.pd… [↗](https://www.ifaamas.org/Proceedings/aamas2022/pdfs/)
29. Tom Bewley, Jonathan Lawry, Arthur Richards, Rachel Craddock, and Ian Henderson. Reward Learning with Trees: Methods and Evaluation, 2022. arXiv: 2210.01007. preprint. — [arXiv:2210.01007](https://arxiv.org/abs/2210.01007) [find](https://scholar.google.com/scholar?q=Tom+Bewley%2C+Jonathan+Lawry%2C+Arthur+Richards%2C+Rachel+Craddock%2C+and+Ian+Henderson%2E+Reward+Learning+with+Trees%3A+Methods+and+Evaluation%2C+2022%2E+arXiv%3A+2210%2E01007%2E+preprint%2E+%E2%80%94+%5BarXiv%3A2210%2E01007%5D%28https%3A%2F%2Farx)
30. Tom Bewley, Jonathan Lawry, and Arthur Richards. Learning Interpretable Models of Aircraft Handling Behaviour by Reinforcement Learning from Human Feedback. In AIAA SCITECH 2024 Forum. American Institute of Aeronautics and Astronautics, 2024. doi: 10.2514/6.2024-1380. — [doi:10.2514/6.2024-1380](https://doi.org/10.2514/6.2024-1380) [find](https://scholar.google.com/scholar?q=Tom+Bewley%2C+Jonathan+Lawry%2C+and+Arthur+Richards%2E+Learning+Interpretable+Models+of+Aircraft+Handling+Behaviour+by+Reinforcement+Learning+from+Human+Feedback%2E+In+AIAA+SCITECH+2024+Forum%2E+American+Instit)
31. Adam Bignold, Francisco Cruz, Matthew E. Taylor, Tim Brys, Richard Dazeley, Peter Vamplew, and Cameron Foale. A conceptual framework for externally-influenced agents: An assisted reinforcement learning review. Journal of Ambient Intelligence and Humanized Computing, pp. 3621–3644, 2021. doi: 10.1007… — [doi:10.1007/s12652-021-03489-y](https://doi.org/10.1007/s12652-021-03489-y) [find](https://scholar.google.com/scholar?q=Adam+Bignold%2C+Francisco+Cruz%2C+Matthew+E%2E+Taylor%2C+Tim+Brys%2C+Richard+Dazeley%2C+Peter+Vamplew%2C+and+Cameron+Foale%2E+A+conceptual+framework+for+externally%2Dinfluenced+agents%3A+An+assisted+reinforcement+learnin)
32. Erdem Bıyık and Dorsa Sadigh. Batch Active Preference-Based Learning of Reward Functions. In Proceedings of the Conference on Robot Learning (CoRL). PMLR, 2018. URL [https://proceedings.mlr.press/v87/](https://proceedings.mlr.press/v87/) biyik18a.html. [↗](https://proceedings.mlr.press/v87/)
33. Erdem Bıyık, Malayandi Palan, Nicholas C. Landolfi, Dylan P. Losey, and Dorsa Sadigh. Asking Easy Questions: A User-Friendly Approach to Active Reward Learning. In Proceedings of the Conference on Robot Learning (CoRL). PMLR, 2019. URL [https://proceedings.mlr.press/v100/b-iy-ik20a.html](https://proceedings.mlr.press/v100/b-iy-ik20a.html). [↗](https://proceedings.mlr.press/v100/b-iy-ik20a.html)
34. Erdem Bıyık, Nicolas Huynh, Mykel Kochenderfer, and Dorsa Sadigh. Active Preference-Based Gaussian Process Regression for Reward Learning. In Proceedings of Robotics: Science and Systems (RSS), volume 16, 2020. URL [http://www.roboticsproceedings.org/rss16/p041.html](http://www.roboticsproceedings.org/rss16/p041.html). [↗](http://www.roboticsproceedings.org/rss16/p041.html)
35. Erdem Bıyık, Dylan P. Losey, Malayandi Palan, Nicholas C. Landolfi, Gleb Shevchuk, and Dorsa Sadigh. Learning reward functions from diverse sources of human feedback: Optimally integrating demonstrations and preferences. The International Journal of Robotics Research, 41(1):45–67, 2022a. doi: 10.117… — [doi:10.1177/02783649211041652](https://doi.org/10.1177/02783649211041652) [find](https://scholar.google.com/scholar?q=Erdem+B%C4%B1y%C4%B1k%2C+Dylan+P%2E+Losey%2C+Malayandi+Palan%2C+Nicholas+C%2E+Landolfi%2C+Gleb+Shevchuk%2C+and+Dorsa+Sadigh%2E+Learning+reward+functions+from+diverse+sources+of+human+feedback%3A+Optimally+integrating+demonstrati)
36. Erdem Bıyık, Aditi Talati, and Dorsa Sadigh. APReL: A Library for Active Preference-based Reward Learning Algorithms. In Proceedings of the ACM/IEEE International Conference on Human-Robot Interaction (HRI), 2022b. doi: 10.1109/HRI53351.2022.9889650. — [doi:10.1109/hri53351.2022.9889650](https://doi.org/10.1109/hri53351.2022.9889650) [find](https://scholar.google.com/scholar?q=Erdem+B%C4%B1y%C4%B1k%2C+Aditi+Talati%2C+and+Dorsa+Sadigh%2E+APReL%3A+A+Library+for+Active+Preference%2Dbased+Reward+Learning+Algorithms%2E+In+Proceedings+of+the+ACM%2FIEEE+International+Conference+on+Human%2DRobot+Interaction)
37. Erdem Bıyık, Nima Anari, and Dorsa Sadigh. Batch Active Learning of Reward Functions from Human Preferences. ACM Transactions on Human-Robot Interaction, 2024. doi: 10.1145/3649885. — [doi:10.1145/3649885](https://doi.org/10.1145/3649885) [find](https://scholar.google.com/scholar?q=Erdem+B%C4%B1y%C4%B1k%2C+Nima+Anari%2C+and+Dorsa+Sadigh%2E+Batch+Active+Learning+of+Reward+Functions+from+Human+Preferences%2E+ACM+Transactions+on+Human%2DRobot+Interaction%2C+2024%2E+doi%3A+10%2E1145%2F3649885%2E+%E2%80%94+%5Bdoi%3A10%2E1145%2F364)
38. Andreea Bobu, Andrea Bajcsy, Jaime F. Fisac, Sampada Deglurkar, and Anca D. Dragan. Quantifying Hypothesis Space Misspecification in Learning From Human–Robot Demonstrations and Physical Corrections. IEEE Transactions on Robotics, 36(3):835–854, 2020a. doi: 10.1109/TRO.2020.2971415. — [doi:10.1109/tro.2020.2971415](https://doi.org/10.1109/tro.2020.2971415) [find](https://scholar.google.com/scholar?q=Andreea+Bobu%2C+Andrea+Bajcsy%2C+Jaime+F%2E+Fisac%2C+Sampada+Deglurkar%2C+and+Anca+D%2E+Dragan%2E+Quantifying+Hypothesis+Space+Misspecification+in+Learning+From+Human%E2%80%93Robot+Demonstrations+and+Physical+Corrections%2E+)
39. Andreea Bobu, Dexter R. R. Scobee, Jaime F. Fisac, S. Shankar Sastry, and Anca D. Dragan. LESS is More: Rethinking Probabilistic Models of Human Behavior. In Proceedings of the ACM/IEEE International Conference on Human-Robot Interaction (HRI). Association for Computing Machinery, 2020b. doi: 10.114… — [doi:10.1145/3319502.3374811](https://doi.org/10.1145/3319502.3374811) [find](https://scholar.google.com/scholar?q=Andreea+Bobu%2C+Dexter+R%2E+R%2E+Scobee%2C+Jaime+F%2E+Fisac%2C+S%2E+Shankar+Sastry%2C+and+Anca+D%2E+Dragan%2E+LESS+is+More%3A+Rethinking+Probabilistic+Models+of+Human+Behavior%2E+In+Proceedings+of+the+ACM%2FIEEE+International+)
40. Andreea Bobu, Marius Wiggert, Claire Tomlin, and Anca D Dragan. Inducing structure in reward learning by learning features. The International Journal of Robotics Research, 41(5):497–518, 2022. doi: 10.1177/02783649221078031. — [doi:10.1177/02783649221078031](https://doi.org/10.1177/02783649221078031) [find](https://scholar.google.com/scholar?q=Andreea+Bobu%2C+Marius+Wiggert%2C+Claire+Tomlin%2C+and+Anca+D+Dragan%2E+Inducing+structure+in+reward+learning+by+learning+features%2E+The+International+Journal+of+Robotics+Research%2C+41%285%29%3A497%E2%80%93518%2C+2022%2E+doi%3A+10)
41. Andreea Bobu, Yi Liu, Rohin Shah, Daniel S. Brown, and Anca D. Dragan. SIRL: Similarity-based Implicit Representation Learning. In Proceedings of the ACM/IEEE International Conference on Human-Robot Interaction (HRI). Association for Computing Machinery, 2023. doi: 10.1145/3568162.3576989. — [doi:10.1145/3568162.3576989](https://doi.org/10.1145/3568162.3576989) [find](https://scholar.google.com/scholar?q=Andreea+Bobu%2C+Yi+Liu%2C+Rohin+Shah%2C+Daniel+S%2E+Brown%2C+and+Anca+D%2E+Dragan%2E+SIRL%3A+Similarity%2Dbased+Implicit+Representation+Learning%2E+In+Proceedings+of+the+ACM%2FIEEE+International+Conference+on+Human%2DRobot+I)
42. Florian Böhm, Yang Gao, Christian M. Meyer, Ori Shapira, Ido Dagan, and Iryna Gurevych. [Better Rewards Yield Better Summaries: Learning to Summarise Without References](https://md.research.elamur.ai/free/en/papers/bohm-2019-better-rewards-yield-better-summaries/). In Proceedings of the Conference on Empirical Methods in Natural Language Processing and the International Joint Conference on Natu…
43. Michael Bowling, John D. Martin, David Abel, and Will Dabney. Settling the Reward Hypothesis. In Proceedings of the International Conference on Machine Learning (ICML). PMLR, 2023. URL [https://proceedings.mlr.press/v202/bowling23a.html](https://proceedings.mlr.press/v202/bowling23a.html). [↗](https://proceedings.mlr.press/v202/bowling23a.html)
44. Ralph Allan Bradley and Milton E. Terry. Rank Analysis of Incomplete Block Designs: I. The Method of Paired Comparisons. Biometrika, 39(3/4):324–345, 1952. doi: 10.2307/2334029. — [doi:10.2307/2334029](https://doi.org/10.2307/2334029) [find](https://scholar.google.com/scholar?q=Ralph+Allan+Bradley+and+Milton+E%2E+Terry%2E+Rank+Analysis+of+Incomplete+Block+Designs%3A+I%2E+The+Method+of+Paired+Comparisons%2E+Biometrika%2C+39%283%2F4%29%3A324%E2%80%93345%2C+1952%2E+doi%3A+10%2E2307%2F2334029%2E+%E2%80%94+%5Bdoi%3A10%2E2307%2F2334029)
45. Felix Brandt, Vincent Conitzer, Ulle Endriss, Jérôme Lang, and Ariel D. Procaccia (eds.). Handbook of Computational Social Choice. Cambridge University Press, 2016. doi: 10.1017/CBO9781107446984. — [doi:10.1017/cbo9781107446984](https://doi.org/10.1017/cbo9781107446984) [find](https://scholar.google.com/scholar?q=Felix+Brandt%2C+Vincent+Conitzer%2C+Ulle+Endriss%2C+J%C3%A9r%C3%B4me+Lang%2C+and+Ariel+D%2E+Procaccia+%28eds%2E%29%2E+Handbook+of+Computational+Social+Choice%2E+Cambridge+University+Press%2C+2016%2E+doi%3A+10%2E1017%2FCBO9781107446984%2E+%E2%80%94+%5Bd)
46. Anthony Brohan, Noah Brown, Justice Carbajal, Yevgen Chebotar, Joseph Dabis, Chelsea Finn, Keerthana Gopalakrishnan, Karol Hausman, Alexander Herzog, Jasmine Hsu, Julian Ibarz, Brian Ichter, Alex Irpan, Tomas Jackson, Sally Jesmonth, Nikhil Joshi, Ryan Julian, Dmitry Kalashnikov, Yuheng Kuang, Isabe… [find](https://scholar.google.com/scholar?q=Anthony+Brohan%2C+Noah+Brown%2C+Justice+Carbajal%2C+Yevgen+Chebotar%2C+Joseph+Dabis%2C+Chelsea+Finn%2C+Keerthana+Gopalakrishnan%2C+Karol+Hausman%2C+Alexander+Herzog%2C+Jasmine+Hsu%2C+Julian+Ibarz%2C+Brian+Ichter%2C+Alex+Irpa)
47. Daniel S. Brown, Wonjoon Goo, Prabhat Nagarajan, and Scott Niekum. Extrapolating Beyond Suboptimal Demonstrations via Inverse Reinforcement Learning from Observations. In Proceedings of the International Conference on Machine Learning (ICML). PMLR, 2019. URL [https://proceedings.mlr.press/v97/](https://proceedings.mlr.press/v97/) brown1… [↗](https://proceedings.mlr.press/v97/)
48. Daniel S. Brown, Russell Coleman, Ravi Srinivasan, and Scott Niekum. Safe Imitation Learning via Fast Bayesian Reward Inference from Preferences. In Proceedings of the International Conference on Machine Learning (ICML). PMLR, 2020. URL [https://proceedings.mlr.press/v119/brown20a.html](https://proceedings.mlr.press/v119/brown20a.html). [↗](https://proceedings.mlr.press/v119/brown20a.html)
49. Róbert Busa-Fekete, Balázs Szörényi, Paul Weng, Weiwei Cheng, and Eyke Hüllermeier. Preference-based reinforcement learning: Evolutionary direct policy search using a preference-based racing algorithm. Machine Learning, 97(3):327–351, 2014. doi: 10.1007/s10994-014-5458-8. — [doi:10.1007/s10994-014-5458-8](https://doi.org/10.1007/s10994-014-5458-8) [find](https://scholar.google.com/scholar?q=R%C3%B3bert+Busa%2DFekete%2C+Bal%C3%A1zs+Sz%C3%B6r%C3%A9nyi%2C+Paul+Weng%2C+Weiwei+Cheng%2C+and+Eyke+H%C3%BCllermeier%2E+Preference%2Dbased+reinforcement+learning%3A+Evolutionary+direct+policy+search+using+a+preference%2Dbased+racing+algorithm)
50. Serkan Cabi, Sergio Gómez Colmenarejo, Alexander Novikov, Ksenia Konyushova, Scott Reed, Rae Jeong, Konrad Zolna, Yusuf Aytar, David Budden, Mel Vecerik, Oleg Sushkov, David Barker, Jonathan Scholz, Misha Denil, Nando de Freitas, and Ziyu Wang. Scaling data-driven robotics with reward sketching and… — [doi:10.15607/rss.2020.xvi.076](https://doi.org/10.15607/rss.2020.xvi.076) [find](https://scholar.google.com/scholar?q=Serkan+Cabi%2C+Sergio+G%C3%B3mez+Colmenarejo%2C+Alexander+Novikov%2C+Ksenia+Konyushova%2C+Scott+Reed%2C+Rae+Jeong%2C+Konrad+Zolna%2C+Yusuf+Aytar%2C+David+Budden%2C+Mel+Vecerik%2C+Oleg+Sushkov%2C+David+Barker%2C+Jonathan+Scholz%2C+M)
51. Haoyang Cao, Samuel Cohen, and Lukasz Szpruch. Identifiability in inverse reinforcement learning. In Advances in Neural Information Processing Systems (NeurIPS), volume 34. Curran Associates, Inc., 2021a. URL [https://proceedings.neurips.cc/paper/2021/hash/](https://proceedings.neurips.cc/paper/2021/hash/) 671f0311e2754fcdd37f70a8550379bc-Abstract.h… [↗](https://proceedings.neurips.cc/paper/2021/hash/)
52. Zehong Cao, KaiChiu Wong, and Chin-Teng Lin. Weak Human Preference Supervision for Deep Reinforcement Learning. IEEE Transactions on Neural Networks and Learning Systems, 32(12):5369–5378, 2021b. doi: 10.1109/TNNLS.2021.3084198. — [doi:10.1109/tnnls.2021.3084198](https://doi.org/10.1109/tnnls.2021.3084198) [find](https://scholar.google.com/scholar?q=Zehong+Cao%2C+KaiChiu+Wong%2C+and+Chin%2DTeng+Lin%2E+Weak+Human+Preference+Supervision+for+Deep+Reinforcement+Learning%2E+IEEE+Transactions+on+Neural+Networks+and+Learning+Systems%2C+32%2812%29%3A5369%E2%80%935378%2C+2021b%2E+doi%3A)
53. Micah Carroll, Alan Chan, Henry Ashton, and David Krueger. Characterizing Manipulation from AI Systems. In Proceedings of the ACM Conference on Equity and Access in Algorithms, Mechanisms, and Optimization (EAAMO). Association for Computing Machinery, 2023. doi: 10.1145/3617694.3623226. — [doi:10.1145/3617694.3623226](https://doi.org/10.1145/3617694.3623226) [find](https://scholar.google.com/scholar?q=Micah+Carroll%2C+Alan+Chan%2C+Henry+Ashton%2C+and+David+Krueger%2E+Characterizing+Manipulation+from+AI+Systems%2E+In+Proceedings+of+the+ACM+Conference+on+Equity+and+Access+in+Algorithms%2C+Mechanisms%2C+and+Optimiz)
54. Stephen Casper, Xander Davies, Claudia Shi, Thomas Krendl Gilbert, Jérémy Scheurer, Javier Rando, Rachel Freedman, Tomasz Korbak, David Lindner, Pedro Freire, Tony Tong Wang, Samuel Marks, Charbel-Raphael Segerie, Micah Carroll, Andi Peng, Phillip Christoffersen, Mehul Damani, Stewart Slocum, Usman… [find](https://scholar.google.com/scholar?q=Stephen+Casper%2C+Xander+Davies%2C+Claudia+Shi%2C+Thomas+Krendl+Gilbert%2C+J%C3%A9r%C3%A9my+Scheurer%2C+Javier+Rando%2C+Rachel+Freedman%2C+Tomasz+Korbak%2C+David+Lindner%2C+Pedro+Freire%2C+Tony+Tong+Wang%2C+Samuel+Marks%2C+Charbel%2DRap)
55. Louis Castricato, Alex Havrilla, Shahbuland Matiana, Duy V. Phung, Aman Tiwari, Jonathan Tow, and Maksym Zhuravinsky. trlX: A scalable framework for RLHF. Zenodo, 2023. URL [https://github.com/](https://github.com/) CarperAI/trlx. [↗](https://github.com/)
56. Louis Castricato, Nathan Lile, Suraj Anand, Hailey Schoelkopf, Siddharth Verma, and Stella Biderman. Suppressing Pink Elephants with Direct Principle Feedback, 2024. arXiv: 2402.07896. preprint. — [arXiv:2402.07896](https://arxiv.org/abs/2402.07896) [find](https://scholar.google.com/scholar?q=Louis+Castricato%2C+Nathan+Lile%2C+Suraj+Anand%2C+Hailey+Schoelkopf%2C+Siddharth+Verma%2C+and+Stella+Biderman%2E+Suppressing+Pink+Elephants+with+Direct+Principle+Feedback%2C+2024%2E+arXiv%3A+2402%2E07896%2E+preprint%2E+%E2%80%94+%5Bar)
57. Manuela Cattelan. Models for Paired Comparison Data: A Review with Emphasis on Dependent Data. Statistical Science, 27(3):412–433, 2012. doi: 10.1214/12-STS396. — [doi:10.1214/12-sts396](https://doi.org/10.1214/12-sts396) [find](https://scholar.google.com/scholar?q=Manuela+Cattelan%2E+Models+for+Paired+Comparison+Data%3A+A+Review+with+Emphasis+on+Dependent+Data%2E+Statistical+Science%2C+27%283%29%3A412%E2%80%93433%2C+2012%2E+doi%3A+10%2E1214%2F12%2DSTS396%2E+%E2%80%94+%5Bdoi%3A10%2E1214%2F12%2Dsts396%5D%28https%3A%2F%2Fdoi%2Eo)
58. Souradip Chakraborty, Amrit Bedi, Alec Koppel, Huazheng Wang, Dinesh Manocha, Mengdi Wang, and Furong Huang. PARL: A Unified Framework for Policy Alignment in Reinforcement Learning from Human Feedback. In Proceedings of the International Conference on Learning Representations (ICLR), 2024a. URL htt… [find](https://scholar.google.com/scholar?q=Souradip+Chakraborty%2C+Amrit+Bedi%2C+Alec+Koppel%2C+Huazheng+Wang%2C+Dinesh+Manocha%2C+Mengdi+Wang%2C+and+Furong+Huang%2E+PARL%3A+A+Unified+Framework+for+Policy+Alignment+in+Reinforcement+Learning+from+Human+Feedbac)
59. Souradip Chakraborty, Jiahao Qiu, Hui Yuan, Alec Koppel, Furong Huang, Dinesh Manocha, Amrit Bedi, and Mengdi Wang. MaxMin-RLHF: Towards Equitable Alignment of Large Language Models with Diverse Human Preferences. In ICML 2024 Workshop on Models of Human Feedback for AI Alignment (MHFAIA), 2024b. UR… [find](https://scholar.google.com/scholar?q=Souradip+Chakraborty%2C+Jiahao+Qiu%2C+Hui+Yuan%2C+Alec+Koppel%2C+Furong+Huang%2C+Dinesh+Manocha%2C+Amrit+Bedi%2C+and+Mengdi+Wang%2E+MaxMin%2DRLHF%3A+Towards+Equitable+Alignment+of+Large+Language+Models+with+Diverse+Human)
60. Alex James Chan, Hao Sun, Samuel Holt, and Mihaela van der Schaar. Dense Reward for Free in Reinforcement Learning from Human Feedback. In Proceedings of the International Conference on Machine Learning (ICML), 2024. URL [https://openreview.net/forum?id=eyxVRMrZ4m](https://openreview.net/forum?id=eyxVRMrZ4m). [↗](https://openreview.net/forum?id=eyxVRMrZ4m)
61. Lawrence Chan, Andrew Critch, and Anca Dragan. Human irrationality: Both bad and good for reward inference, 2021. arXiv: 2111.06956. preprint. — [arXiv:2111.06956](https://arxiv.org/abs/2111.06956) [find](https://scholar.google.com/scholar?q=Lawrence+Chan%2C+Andrew+Critch%2C+and+Anca+Dragan%2E+Human+irrationality%3A+Both+bad+and+good+for+reward+inference%2C+2021%2E+arXiv%3A+2111%2E06956%2E+preprint%2E+%E2%80%94+%5BarXiv%3A2111%2E06956%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2111%2E06956%29)
62. Niladri Chatterji, Aldo Pacchiano, Peter Bartlett, and Michael Jordan. On the Theory of Reinforcement Learning with Once-per-Episode Feedback. In Advances in Neural Information Processing Systems (NeurIPS), volume 34. Curran Associates, Inc., 2021. URL [https://proceedings.neurips.cc/paper/](https://proceedings.neurips.cc/paper/) 2021/hash… [↗](https://proceedings.neurips.cc/paper/)
63. Jiaao Chen, Mohan Dodda, and Diyi Yang. Human-in-the-loop Abstractive Dialogue Summarization. In Findings of the Association for Computational Linguistics (ACL). Association for Computational Linguistics, 2023. doi: 10.18653/v1/2023.findings-acl.584. [find](https://scholar.google.com/scholar?q=Jiaao+Chen%2C+Mohan+Dodda%2C+and+Diyi+Yang%2E+Human%2Din%2Dthe%2Dloop+Abstractive+Dialogue+Summarization%2E+In+Findings+of+the+Association+for+Computational+Linguistics+%28ACL%29%2E+Association+for+Computational+Linguist)
64. Li Chen, Marco de Gemmis, Alexander Felfernig, Pasquale Lops, Francesco Ricci, and Giovanni Semeraro. Human Decision Making and Recommender Systems. ACM Transactions on Interactive Intelligent Systems, 3(3):17:1–17:7, 2013. doi: 10.1145/2533670.2533675. — [doi:10.1145/2533670.2533675](https://doi.org/10.1145/2533670.2533675) [find](https://scholar.google.com/scholar?q=Li+Chen%2C+Marco+de+Gemmis%2C+Alexander+Felfernig%2C+Pasquale+Lops%2C+Francesco+Ricci%2C+and+Giovanni+Semeraro%2E+Human+Decision+Making+and+Recommender+Systems%2E+ACM+Transactions+on+Interactive+Intelligent+Systems)
65. Xiaoyu Chen, Han Zhong, Zhuoran Yang, Zhaoran Wang, and Liwei Wang. Human-in-the-loop: Provably Efficient Preference-based Reinforcement Learning with General Function Approximation. In Proceedings of the International Conference on Machine Learning (ICML). PMLR, 2022. URL https://proceedings. mlr.p… [find](https://scholar.google.com/scholar?q=Xiaoyu+Chen%2C+Han+Zhong%2C+Zhuoran+Yang%2C+Zhaoran+Wang%2C+and+Liwei+Wang%2E+Human%2Din%2Dthe%2Dloop%3A+Provably+Efficient+Preference%2Dbased+Reinforcement+Learning+with+General+Function+Approximation%2E+In+Proceedings+of)
66. Yu Chen, Yihan Du, Pihe Hu, Siwei Wang, Desheng Wu, and Longbo Huang. Provably Efficient Iterated CVaR Reinforcement Learning with Function Approximation and Human Feedback. In Proceedings of the International Conference on Learning Representations (ICLR), 2024. URL [https://openreview.net/](https://openreview.net/) forum?id=… [↗](https://openreview.net/)
67. Weiwei Cheng, Johannes Fürnkranz, Eyke Hüllermeier, and Sang-Hyeun Park. Preference-Based Policy Iteration: Leveraging Preference Learning for Reinforcement Learning. In Proceedings of Machine Learning and Knowledge Discovery in Databases (ECML PKDD). Springer, 2011. doi: 10.1007/978-3-642-23780-5_3… — [doi:10.1007/978-3-642-23780-5_30](https://doi.org/10.1007/978-3-642-23780-5_30) [find](https://scholar.google.com/scholar?q=Weiwei+Cheng%2C+Johannes+F%C3%BCrnkranz%2C+Eyke+H%C3%BCllermeier%2C+and+Sang%2DHyeun+Park%2E+Preference%2DBased+Policy+Iteration%3A+Leveraging+Preference+Learning+for+Reinforcement+Learning%2E+In+Proceedings+of+Machine+Learnin)
68. David Chhan, Ellen Novoseller, and Vernon J. Lawhern. Crowd-PrefRL: Preference-Based Reward Learning from Crowds, 2024. arXiv: 2401.10941. preprint. — [arXiv:2401.10941](https://arxiv.org/abs/2401.10941) [find](https://scholar.google.com/scholar?q=David+Chhan%2C+Ellen+Novoseller%2C+and+Vernon+J%2E+Lawhern%2E+Crowd%2DPrefRL%3A+Preference%2DBased+Reward+Learning+from+Crowds%2C+2024%2E+arXiv%3A+2401%2E10941%2E+preprint%2E+%E2%80%94+%5BarXiv%3A2401%2E10941%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2401%2E109)
69. Eugene Choi, Arash Ahmadian, Matthieu Geist, Olivier Pietquin, and Mohammad Gheshlaghi Azar. Self-Improving Robust Preference Optimization. In Proceedings of the International Conference on Learning Representations (ICLR), 2025. URL [https://openreview.net/forum?id=ZSdubdbOoi](https://openreview.net/forum?id=ZSdubdbOoi). [↗](https://openreview.net/forum?id=ZSdubdbOoi)
70. Jaedeug Choi and Kee-eung Kim. Nonparametric Bayesian Inverse Reinforcement Learning for Multiple Reward Functions. In Advances in Neural Information Processing Systems (NIPS), volume 25. Curran Associates, Inc., 2012. URL [https://proceedings.neurips.cc/paper/2012/hash/](https://proceedings.neurips.cc/paper/2012/hash/) 140f6969d5213fd0ece03148e62e4… [↗](https://proceedings.neurips.cc/paper/2012/hash/)
71. Sayak Ray Chowdhury and Xingyu Zhou. Differentially Private Reward Estimation from Preference Based Feedback. In ICML 2023 Workshop on The Many Facets of Preference-Based Learning (MFPL), 2023. URL [https://openreview.net/forum?id=TqzYmBPSGC](https://openreview.net/forum?id=TqzYmBPSGC). [↗](https://openreview.net/forum?id=TqzYmBPSGC)
72. Paul Christiano. Semi-supervised reinforcement learning, 2016. URL [https://ai-alignment.com/](https://ai-alignment.com/) semi-supervised-reinforcement-learning-cf7d5375197f. (accessed 2023-12-14). [↗](https://ai-alignment.com/)
73. Paul Christiano. Thoughts on the impact of RLHF research, 2023. URL [https://www.alignmentforum.org/posts/vwu4kegAEZTBtpT6p/thoughts-on-the-impact-of-rlhf-research](https://www.alignmentforum.org/posts/vwu4kegAEZTBtpT6p/thoughts-on-the-impact-of-rlhf-research). [↗](https://www.alignmentforum.org/posts/vwu4kegAEZTBtpT6p/thoughts-on-the-impact-of-rlhf-research)
74. Paul Christiano, Jan Leike, Tom Brown, Miljan Martic, Shane Legg, and Dario Amodei. Deep Reinforcement Learning from Human Preferences. In Advances in Neural Information Processing Systems (NIPS), volume 30. Curran Associates, Inc., 2017. URL [https://proceedings.neurips.cc/paper/2017/hash/](https://proceedings.neurips.cc/paper/2017/hash/) d5e2c0ada… [↗](https://proceedings.neurips.cc/paper/2017/hash/)
75. Paul Christiano, Buck Shlegeris, and Dario Amodei. Supervising strong learners by amplifying weak experts, 2018. arXiv: 1810.08575. preprint. [find](https://scholar.google.com/scholar?q=Paul+Christiano%2C+Buck+Shlegeris%2C+and+Dario+Amodei%2E+Supervising+strong+learners+by+amplifying+weak+experts%2C+2018%2E+arXiv%3A+1810%2E08575%2E+preprint%2E)
76. Geoffrey Cideron, Sertan Girgin, Mauro Verzetti, Damien Vincent, Matej Kastelic, Zalán Borsos, Brian Mcwilliams, Victor Ungureanu, Olivier Bachem, Olivier Pietquin, Matthieu Geist, Leonard Hussenot, Neil Zeghidour, and Andrea Agostinelli. MusicRL: Aligning Music Generation to Human Preferences. In P… [find](https://scholar.google.com/scholar?q=Geoffrey+Cideron%2C+Sertan+Girgin%2C+Mauro+Verzetti%2C+Damien+Vincent%2C+Matej+Kastelic%2C+Zal%C3%A1n+Borsos%2C+Brian+Mcwilliams%2C+Victor+Ungureanu%2C+Olivier+Bachem%2C+Olivier+Pietquin%2C+Matthieu+Geist%2C+Leonard+Hussenot%2C+N)
77. Jack Clark and Dario Amodei. Faulty Reward Functions in the Wild, 2016. URL [https://openai.com/](https://openai.com/) blog/faulty-reward-functions/. [↗](https://openai.com/)
78. Vincent Conitzer, Rachel Freedman, Jobst Heitzig, Wesley H. Holliday, Bob M. Jacobs, Nathan Lambert, Milan Mosse, Eric Pacuit, Stuart Russell, Hailey Schoelkopf, Emanuel Tewolde, and William S. Zwicker. Position: Social Choice Should Guide AI Alignment in Dealing with Diverse Human Feedback. In Proc… [find](https://scholar.google.com/scholar?q=Vincent+Conitzer%2C+Rachel+Freedman%2C+Jobst+Heitzig%2C+Wesley+H%2E+Holliday%2C+Bob+M%2E+Jacobs%2C+Nathan+Lambert%2C+Milan+Mosse%2C+Eric+Pacuit%2C+Stuart+Russell%2C+Hailey+Schoelkopf%2C+Emanuel+Tewolde%2C+and+William+S%2E+Zwicke)
79. Ryan Cosner, Maegan Tucker, Andrew Taylor, Kejun Li, Tamas Molnar, Wyatt Ubelacker, Anil Alan, Gabor Orosz, Yisong Yue, and Aaron D. Ames. Safety-Aware Preference-Based Learning for Safety-Critical Control. In Proceedings of the Annual Learning for Dynamics and Control Conference (L4DC). PMLR, 2022.… [find](https://scholar.google.com/scholar?q=Ryan+Cosner%2C+Maegan+Tucker%2C+Andrew+Taylor%2C+Kejun+Li%2C+Tamas+Molnar%2C+Wyatt+Ubelacker%2C+Anil+Alan%2C+Gabor+Orosz%2C+Yisong+Yue%2C+and+Aaron+D%2E+Ames%2E+Safety%2DAware+Preference%2DBased+Learning+for+Safety%2DCritical+Co)
80. Thomas Coste, Usman Anwar, Robert Kirk, and David Krueger. Reward Model Ensembles Help Mitigate Overoptimization. In Proceedings of the International Conference on Learning Representations (ICLR), 2024. URL [https://openreview.net/forum?id=dcjtMYkpXx](https://openreview.net/forum?id=dcjtMYkpXx). [↗](https://openreview.net/forum?id=dcjtMYkpXx)
81. Christian Arzate Cruz and Takeo Igarashi. A Survey on Interactive Reinforcement Learning: Design Principles and Open Challenges. In Proceedings of the ACM Designing Interactive Systems Conference (DIS). Association for Computing Machinery, 2020. doi: 10.1145/3357236.3395525. — [doi:10.1145/3357236.3395525](https://doi.org/10.1145/3357236.3395525) [find](https://scholar.google.com/scholar?q=Christian+Arzate+Cruz+and+Takeo+Igarashi%2E+A+Survey+on+Interactive+Reinforcement+Learning%3A+Design+Principles+and+Open+Challenges%2E+In+Proceedings+of+the+ACM+Designing+Interactive+Systems+Conference+%28DIS)
82. Yuchen Cui and Scott Niekum. Active Reward Learning from Critiques. In Proceedings of the IEEE International Conference on Robotics and Automation (ICRA), 2018. doi: 10.1109/ICRA.2018.8460854. — [doi:10.1109/icra.2018.8460854](https://doi.org/10.1109/icra.2018.8460854) [find](https://scholar.google.com/scholar?q=Yuchen+Cui+and+Scott+Niekum%2E+Active+Reward+Learning+from+Critiques%2E+In+Proceedings+of+the+IEEE+International+Conference+on+Robotics+and+Automation+%28ICRA%29%2C+2018%2E+doi%3A+10%2E1109%2FICRA%2E2018%2E8460854%2E+%E2%80%94+%5Bdoi%3A)
83. Yuchen Cui, Qiping Zhang, W. Bradley Knox, Alessandro Allievi, Peter Stone, and Scott Niekum. The EM-PATHIC Framework for Task Learning from Implicit Human Feedback. In Proceedings of the Conference on Robot Learning (CoRL). PMLR, 2021. URL [https://proceedings.mlr.press/v155/cui21a.html](https://proceedings.mlr.press/v155/cui21a.html). [↗](https://proceedings.mlr.press/v155/cui21a.html)
84. Jessica Dai and Eve Fleisig. Mapping Social Choice Theory to RLHF. In ICLR 2024 Workshop on Reliable and Responsible Foundation Models (R2-FM), 2024. URL [https://openreview.net/forum?id=](https://openreview.net/forum?id=) iYthn3WATc. [↗](https://openreview.net/forum?id=)
85. Josef Dai, Xuehai Pan, Ruiyang Sun, Jiaming Ji, Xinbo Xu, Mickel Liu, Yizhou Wang, and Yaodong Yang. [Safe RLHF: Safe Reinforcement Learning from Human Feedback](https://md.research.elamur.ai/free/en/papers/dai-2023-safe-rlhf-safe-reinforcement-learning/). In Proceedings of the International Conference on Learning Representations (ICLR), 2024. URL [https://openreview.net/forum?](https://openreview.net/forum?) id=TyFrPOKYXw. [↗](https://openreview.net/forum?)
86. Christian Daniel, Malte Viering, Jan Metz, Oliver Kroemer, and Jan Peters. Active Reward Learning. In Proceedings of Robotics: Science and Systems (RSS), volume 10, 2014. URL [http://www.roboticsproceedings.org/rss10/p31.html](http://www.roboticsproceedings.org/rss10/p31.html). [↗](http://www.roboticsproceedings.org/rss10/p31.html)
87. Oliver Daniels-Koch and Rachel Freedman. The Expertise Problem: Learning from Specialized Feedback. In NeurIPS 2022 Workshop on ML Safety, 2022. URL [https://openreview.net/forum?id=I7K975-H1Mg](https://openreview.net/forum?id=I7K975-H1Mg). [↗](https://openreview.net/forum?id=I7K975-H1Mg)
88. Brett Day, Ian J. Bateman, Richard T. Carson, Diane Dupont, Jordan J. Louviere, Sanae Morimoto, Riccardo Scarpa, and Paul Wang. Ordering effects and choice set awareness in repeat-response stated preference studies. Journal of Environmental Economics and Management, 63(1):73–91, 2012. doi: 10.1016/j… — [doi:10.1016/j.jeem.2011.09.001](https://doi.org/10.1016/j.jeem.2011.09.001) [find](https://scholar.google.com/scholar?q=Brett+Day%2C+Ian+J%2E+Bateman%2C+Richard+T%2E+Carson%2C+Diane+Dupont%2C+Jordan+J%2E+Louviere%2C+Sanae+Morimoto%2C+Riccardo+Scarpa%2C+and+Paul+Wang%2E+Ordering+effects+and+choice+set+awareness+in+repeat%2Dresponse+stated+pref)
89. Rémy Degenne, Thomas Nedelec, Clement Calauzenes, and Vianney Perchet. Bridging the gap between regret minimization and best arm identification, with application to A/B tests. In Proceedings of the International Conference on Artificial Intelligence and Statistics (AISTATS). PMLR, 2019. URL https: /… [find](https://scholar.google.com/scholar?q=R%C3%A9my+Degenne%2C+Thomas+Nedelec%2C+Clement+Calauzenes%2C+and+Vianney+Perchet%2E+Bridging+the+gap+between+regret+minimization+and+best+arm+identification%2C+with+application+to+A%2FB+tests%2E+In+Proceedings+of+the+In)
90. Zihan Ding, Yuanpei Chen, Allen Z. Ren, Shixiang Shane Gu, Hao Dong, and Chi Jin. Learning a Universal Human Prior for Dexterous Manipulation from Human Preference, 2023. arXiv: 2304.04602. preprint. — [arXiv:2304.04602](https://arxiv.org/abs/2304.04602) [find](https://scholar.google.com/scholar?q=Zihan+Ding%2C+Yuanpei+Chen%2C+Allen+Z%2E+Ren%2C+Shixiang+Shane+Gu%2C+Hao+Dong%2C+and+Chi+Jin%2E+Learning+a+Universal+Human+Prior+for+Dexterous+Manipulation+from+Human+Preference%2C+2023%2E+arXiv%3A+2304%2E04602%2E+preprint%2E+)
91. Hanze Dong, Wei Xiong, Deepanshu Goyal, Yihan Zhang, Winnie Chow, Rui Pan, Shizhe Diao, Jipeng Zhang, KaShun Shum, and Tong Zhang. [RAFT: Reward rAnked FineTuning for Generative Foundation Model Alignment](https://md.research.elamur.ai/free/en/papers/dong-2023-raft-reward-ranked-finetuning-generative/). Transactions on Machine Learning Research, 2023. ISSN 2835-8856. URL [https://openreview.net/fo…](https://openreview.net/fo…) [↗](https://openreview.net/fo…)
92. Hanze Dong, Wei Xiong, Bo Pang, Haoxiang Wang, Han Zhao, Yingbo Zhou, Nan Jiang, Doyen Sahoo, Caiming Xiong, and Tong Zhang. [RLHF Workflow: From Reward Modeling to Online RLHF](https://md.research.elamur.ai/free/en/papers/dong-2024-rlhf-workflow-reward-modeling-online/). In Transactions on Machine Learning Research, 2024a. URL [https://openreview.net/forum?id=a13aYUU9eU](https://openreview.net/forum?id=a13aYUU9eU). [↗](https://openreview.net/forum?id=a13aYUU9eU)
93. Zibin Dong, Yifu Yuan, Jianye Hao, Fei Ni, Yao Mu, Yan Zheng, Yujing Hu, Tangjie Lv, Changjie Fan, and Zhipeng Hu. AlignDiff: Aligning Diverse Human Preferences via Behavior-Customisable Diffusion Model. In Proceedings of the International Conference on Learning Representations (ICLR), 2024b. URL ht… [find](https://scholar.google.com/scholar?q=Zibin+Dong%2C+Yifu+Yuan%2C+Jianye+Hao%2C+Fei+Ni%2C+Yao+Mu%2C+Yan+Zheng%2C+Yujing+Hu%2C+Tangjie+Lv%2C+Changjie+Fan%2C+and+Zhipeng+Hu%2E+AlignDiff%3A+Aligning+Diverse+Human+Preferences+via+Behavior%2DCustomisable+Diffusion+Mod)
94. Yuqing Du, Ksenia Konyushkova, Misha Denil, Akhil Raju, Jessica Landon, Felix Hill, Nando de Freitas, and Serkan Cabi. Vision-Language Models as Success Detectors. In Proceedings of the Conference on Lifelong Learning Agents (CoLLAs). PMLR, 2023. URL [https://proceedings.mlr.press/v232/du23b.html](https://proceedings.mlr.press/v232/du23b.html). [↗](https://proceedings.mlr.press/v232/du23b.html)
95. Miroslav Dudík, Katja Hofmann, Robert E. Schapire, Aleksandrs Slivkins, and Masrour Zoghi. Contextual Dueling Bandits. In Proceedings of the Conference on Learning Theory (COLT). PMLR, 2015. URL [https://proceedings.mlr.press/v40/Dudik15.html](https://proceedings.mlr.press/v40/Dudik15.html). [↗](https://proceedings.mlr.press/v40/Dudik15.html)
96. Vincent Dumoulin, Daniel D. Johnson, Pablo Samuel Castro, Hugo Larochelle, and Yann Dauphin. A density estimation perspective on learning from pairwise human preferences. Transactions on Machine Learning Research, 2024. ISSN 2835-8856. URL [https://openreview.net/forum?id=YH3oERVYjF](https://openreview.net/forum?id=YH3oERVYjF). [↗](https://openreview.net/forum?id=YH3oERVYjF)
97. Krishnamurthy Dvijotham and Emanuel Todorov. Inverse optimal control with linearly-solvable MDPs. In Proceedings of the International Conference on Machine Learning (ICML). Omnipress, 2010. URL [https://icml.cc/Conferences/2010/papers/571.pdf](https://icml.cc/Conferences/2010/papers/571.pdf). [↗](https://icml.cc/Conferences/2010/papers/571.pdf)
98. Vikranth Dwaracherla, Seyed Mohammad Asghari, Botao Hao, and Benjamin Van Roy. Efficient Exploration for LLMs. In Proceedings of the International Conference on Machine Learning (ICML), 2024. URL [https://openreview.net/forum?id=PpPZ6W7rxy](https://openreview.net/forum?id=PpPZ6W7rxy). [↗](https://openreview.net/forum?id=PpPZ6W7rxy)
99. Cynthia Dwork. Differential Privacy: A Survey of Results. In Proceedings of Theory and Applications of Models of Computation (TAMC). Springer, 2008. doi: 10.1007/978-3-540-79228-4_1. — [doi:10.1007/978-3-540-79228-4_1](https://doi.org/10.1007/978-3-540-79228-4_1) [find](https://scholar.google.com/scholar?q=Cynthia+Dwork%2E+Differential+Privacy%3A+A+Survey+of+Results%2E+In+Proceedings+of+Theory+and+Applications+of+Models+of+Computation+%28TAMC%29%2E+Springer%2C+2008%2E+doi%3A+10%2E1007%2F978%2D3%2D540%2D79228%2D4%5F1%2E+%E2%80%94+%5Bdoi%3A10%2E1007%2F97)
100. Joseph Early, Tom Bewley, Christine Evers, and Sarvapali Ramchurn. Non-Markovian Reward Modelling from Trajectory Labels via Interpretable Multiple Instance Learning. In Advances in Neural Information Processing Systems (NeurIPS), volume 35, 2022. URL [https://proceedings.neurips.cc/paper/2022/](https://proceedings.neurips.cc/paper/2022/) hash/… [↗](https://proceedings.neurips.cc/paper/2022/)
101. André Eberhard, Houssam Metni, Georg Fahland, Alexander Stroh, and Pascal Friederich. Actively Learning Costly Reward Functions for Reinforcement Learning. In NeurIPS 2022 Workshop on AI for Accelerated Materials Design, 2022. URL [https://openreview.net/forum?id=eFHNEv6G9fF](https://openreview.net/forum?id=eFHNEv6G9fF). [↗](https://openreview.net/forum?id=eFHNEv6G9fF)
102. Jacob Eisenstein, Chirag Nagpal, Alekh Agarwal, Ahmad Beirami, Alexander Nicholas D’Amour, Krishnamurthy Dj Dvijotham, Adam Fisch, Katherine A. Heller, Stephen Robert Pfohl, Deepak Ramachandran, Peter Shaw, and Jonathan Berant. Helping or Herding? Reward Model Ensembles Mitigate but do not Eliminate… [find](https://scholar.google.com/scholar?q=Jacob+Eisenstein%2C+Chirag+Nagpal%2C+Alekh+Agarwal%2C+Ahmad+Beirami%2C+Alexander+Nicholas+D%E2%80%99Amour%2C+Krishnamurthy+Dj+Dvijotham%2C+Adam+Fisch%2C+Katherine+A%2E+Heller%2C+Stephen+Robert+Pfohl%2C+Deepak+Ramachandran%2C+Peter)
103. Evan Ellis, Gaurav R. Ghosal, Stuart J. Russell, Anca Dragan, and Erdem Bıyık. A Generalized Acquisition Function for Preference-based Reward Learning. In Proceedings of the IEEE International Conference on Robotics and Automation (ICRA), 2024. doi: 10.1109/ICRA57147.2024.10611472. — [doi:10.1109/icra57147.2024.10611472](https://doi.org/10.1109/icra57147.2024.10611472) [find](https://scholar.google.com/scholar?q=Evan+Ellis%2C+Gaurav+R%2E+Ghosal%2C+Stuart+J%2E+Russell%2C+Anca+Dragan%2C+and+Erdem+B%C4%B1y%C4%B1k%2E+A+Generalized+Acquisition+Function+for+Preference%2Dbased+Reward+Learning%2E+In+Proceedings+of+the+IEEE+International+Confere)
104. Kawin Ethayarajh, Yejin Choi, and Swabha Swayamdipta. Understanding Dataset Difficulty with V-Usable Information. In Proceedings of the International Conference on Machine Learning (ICML). PMLR, 2022. URL [https://proceedings.mlr.press/v162/ethayarajh22a.html](https://proceedings.mlr.press/v162/ethayarajh22a.html). [↗](https://proceedings.mlr.press/v162/ethayarajh22a.html)
105. Patrick Fernandes, Aman Madaan, Emmy Liu, António Farinhas, Pedro Henrique Martins, Amanda Bertsch, José G. C. de Souza, Shuyan Zhou, Tongshuang Wu, Graham Neubig, and André F. T. Martins. [Bridging the Gap: A Survey on Integrating (Human) Feedback for Natural Language Generation](https://md.research.elamur.ai/free/en/papers/fernandes-2023-bridging-gap-survey-integrating-human/). Transactions of the…
106. Tesca Fitzgerald, Pallavi Koppol, Patrick Callaghan, Russell Quinlan Jun Hei Wong, Reid Simmons, Oliver Kroemer, and Henny Admoni. INQUIRE: INteractive Querying for User-aware Informative REasoning. In Proceedings of the Conference on Robot Learning (CoRL). PMLR, 2022. URL https://proceedings. mlr.p… [find](https://scholar.google.com/scholar?q=Tesca+Fitzgerald%2C+Pallavi+Koppol%2C+Patrick+Callaghan%2C+Russell+Quinlan+Jun+Hei+Wong%2C+Reid+Simmons%2C+Oliver+Kroemer%2C+and+Henny+Admoni%2E+INQUIRE%3A+INteractive+Querying+for+User%2Daware+Informative+REasoning%2E+I)
107. Floyd J. Fowler, Jr. Survey Research Methods. SAGE Publications, 2013. ISBN 978-1-4833-1240-8. [find](https://scholar.google.com/scholar?q=Floyd+J%2E+Fowler%2C+Jr%2E+Survey+Research+Methods%2E+SAGE+Publications%2C+2013%2E+ISBN+978%2D1%2D4833%2D1240%2D8%2E)
108. Rachel Freedman, Rohin Shah, and Anca Dragan. Choice Set Misspecification in Reward Inference. In Proceedings of the Workshop on Artificial Intelligence Safety 2020 Co-Located with the 29th International Joint Conference on Artificial Intelligence and the 17th Pacific Rim International Conference on… [find](https://scholar.google.com/scholar?q=Rachel+Freedman%2C+Rohin+Shah%2C+and+Anca+Dragan%2E+Choice+Set+Misspecification+in+Reward+Inference%2E+In+Proceedings+of+the+Workshop+on+Artificial+Intelligence+Safety+2020+Co%2DLocated+with+the+29th+Internatio)
109. CEUR, 2021. URL [https://ceur-ws.org/Vol-2640/\#paper_14](https://ceur-ws.org/Vol-2640/\#paper_14). [↗](https://ceur-ws.org/Vol-2640/\#paper_14)
110. Rachel Freedman, Justin Svegliato, Kyle Wray, and Stuart Russell. Active teacher selection for reinforcement learning from human feedback, 2023. arXiv: 2310.15288. preprint. — [arXiv:2310.15288](https://arxiv.org/abs/2310.15288) [find](https://scholar.google.com/scholar?q=Rachel+Freedman%2C+Justin+Svegliato%2C+Kyle+Wray%2C+and+Stuart+Russell%2E+Active+teacher+selection+for+reinforcement+learning+from+human+feedback%2C+2023%2E+arXiv%3A+2310%2E15288%2E+preprint%2E+%E2%80%94+%5BarXiv%3A2310%2E15288%5D%28https)
111. Pedro Freire, Adam Gleave, Sam Toyer, and Stuart Russell. DERAIL: Diagnostic Environments for Reward And Imitation Learning. In NeurIPS 2020 Workshop on Deep Reinforcement Learning, 2020. [find](https://scholar.google.com/scholar?q=Pedro+Freire%2C+Adam+Gleave%2C+Sam+Toyer%2C+and+Stuart+Russell%2E+DERAIL%3A+Diagnostic+Environments+for+Reward+And+Imitation+Learning%2E+In+NeurIPS+2020+Workshop+on+Deep+Reinforcement+Learning%2C+2020%2E)
112. Justin Fu, Katie Luo, and Sergey Levine. Learning Robust Rewards with Adverserial Inverse Reinforcement Learning. In Proceedings of the International Conference on Learning Representations (ICLR), 2018a. URL [https://openreview.net/forum?id=rkHywl-A-](https://openreview.net/forum?id=rkHywl-A-). [↗](https://openreview.net/forum?id=rkHywl-A-)
113. Justin Fu, Avi Singh, Dibya Ghosh, Larry Yang, and Sergey Levine. Variational Inverse Control with Events: A General Framework for Data-Driven Reward Definition. In Advances in Neural Information Processing Systems (NIPS), volume 31. Curran Associates, Inc., 2018b. URL [https://proceedings.neurips.cc…](https://proceedings.neurips.cc…) [↗](https://proceedings.neurips.cc…)
114. Scott Fujimoto, Herke Hoof, and David Meger. Addressing Function Approximation Error in Actor-Critic Methods. In Proceedings of the International Conference on Machine Learning (ICML). PMLR, 2018. URL [https://proceedings.mlr.press/v80/fujimoto18a.html](https://proceedings.mlr.press/v80/fujimoto18a.html). [↗](https://proceedings.mlr.press/v80/fujimoto18a.html)
115. Johannes Fürnkranz, Eyke Hüllermeier, Weiwei Cheng, and Sang-Hyeun Park. Preference-based reinforcement learning: A formal framework and a policy iteration algorithm. Machine Learning, 89(1):123–156, 2012. doi: 10.1007/s10994-012-5313-8. — [doi:10.1007/s10994-012-5313-8](https://doi.org/10.1007/s10994-012-5313-8) [find](https://scholar.google.com/scholar?q=Johannes+F%C3%BCrnkranz%2C+Eyke+H%C3%BCllermeier%2C+Weiwei+Cheng%2C+and+Sang%2DHyeun+Park%2E+Preference%2Dbased+reinforcement+learning%3A+A+formal+framework+and+a+policy+iteration+algorithm%2E+Machine+Learning%2C+89%281%29%3A123%E2%80%93156%2C+)
116. R. Michael Furr. Psychometrics: An Introduction. SAGE Publications, 2021. ISBN 978-1-0718-2409-2. [find](https://scholar.google.com/scholar?q=R%2E+Michael+Furr%2E+Psychometrics%3A+An+Introduction%2E+SAGE+Publications%2C+2021%2E+ISBN+978%2D1%2D0718%2D2409%2D2%2E)
117. Iason Gabriel. Artificial Intelligence, Values, and Alignment. Minds and Machines, 30(3):411–437, 2020. doi: 10.1007/s11023-020-09539-2. — [doi:10.1007/s11023-020-09539-2](https://doi.org/10.1007/s11023-020-09539-2) [find](https://scholar.google.com/scholar?q=Iason+Gabriel%2E+Artificial+Intelligence%2C+Values%2C+and+Alignment%2E+Minds+and+Machines%2C+30%283%29%3A411%E2%80%93437%2C+2020%2E+doi%3A+10%2E1007%2Fs11023%2D020%2D09539%2D2%2E+%E2%80%94+%5Bdoi%3A10%2E1007%2Fs11023%2D020%2D09539%2D2%5D%28https%3A%2F%2Fdoi%2Eorg%2F10%2E1007%2Fs110)
118. Deep Ganguli, Amanda Askell, Nicholas Schiefer, Thomas I. Liao, Kamilė Lukošiūtė, Anna Chen, Anna Goldie, Azalia Mirhoseini, Catherine Olsson, Danny Hernandez, Dawn Drain, Dustin Li, Eli Tran-Johnson, Ethan Perez, Jackson Kernion, Jamie Kerr, Jared Mueller, Joshua Landau, Kamal Ndousse, Karina Ng… — [The Capacity for Moral Self-Correction in Large Language Models](https://md.research.elamur.ai/free/en/papers/ganguli-2023-capacity-moral-self-correction-large/)
119. Leo Gao, John Schulman, and Jacob Hilton. [Scaling Laws for Reward Model Overoptimization](https://md.research.elamur.ai/free/en/papers/gao-2022-scaling-laws-reward-model-overoptimization/). In Proceedings of the International Conference on Machine Learning (ICML). PMLR, 2023. URL [https://proceedings.mlr.press/v202/gao23h.html](https://proceedings.mlr.press/v202/gao23h.html). [↗](https://proceedings.mlr.press/v202/gao23h.html)
120. Yang Gao, Christian M. Meyer, and Iryna Gurevych. APRIL: Interactively Learning to Summarise by Combining Active Preference Learning and Reinforcement Learning. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing (EMNLP). Association for Computational Linguistic… [find](https://scholar.google.com/scholar?q=Yang+Gao%2C+Christian+M%2E+Meyer%2C+and+Iryna+Gurevych%2E+APRIL%3A+Interactively+Learning+to+Summarise+by+Combining+Active+Preference+Learning+and+Reinforcement+Learning%2E+In+Proceedings+of+the+2018+Conference+o)
121. Yang Gao, Christian M. Meyer, and Iryna Gurevych. Preference-based interactive multi-document summarisation. Information Retrieval Journal, 23(6):555–585, 2020. doi: 10.1007/s10791-019-09367-8. — [doi:10.1007/s10791-019-09367-8](https://doi.org/10.1007/s10791-019-09367-8) [find](https://scholar.google.com/scholar?q=Yang+Gao%2C+Christian+M%2E+Meyer%2C+and+Iryna+Gurevych%2E+Preference%2Dbased+interactive+multi%2Ddocument+summarisation%2E+Information+Retrieval+Journal%2C+23%286%29%3A555%E2%80%93585%2C+2020%2E+doi%3A+10%2E1007%2Fs10791%2D019%2D09367%2D8%2E+%E2%80%94+%5Bdoi)
122. Carles Gelada, Saurabh Kumar, Jacob Buckman, Ofir Nachum, and Marc G. Bellemare. DeepMDP: Learning Continuous Latent Space Models for Representation Learning. In Proceedings of the International Conference on Machine Learning (ICML). PMLR, 2019. URL [https://proceedings.mlr.press/v97/](https://proceedings.mlr.press/v97/) gelada19a.html. [↗](https://proceedings.mlr.press/v97/)
123. Hans-Otto Georgii. Gibbs Measures and Phase Transitions. Walter de Gruyter, 2011. ISBN 978-3-11-025029-9. [find](https://scholar.google.com/scholar?q=Hans%2DOtto+Georgii%2E+Gibbs+Measures+and+Phase+Transitions%2E+Walter+de+Gruyter%2C+2011%2E+ISBN+978%2D3%2D11%2D025029%2D9%2E)
124. Gaurav R. Ghosal, Matthew Zurek, Daniel S. Brown, and Anca D. Dragan. The Effect of Modeling Human Rationality Level on Learning Rewards from Multiple Feedback Types. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 37, 2023. doi: 10.1609/aaai.v37i5.25740. — [doi:10.1609/aaai.v37i5.25740](https://doi.org/10.1609/aaai.v37i5.25740) [find](https://scholar.google.com/scholar?q=Gaurav+R%2E+Ghosal%2C+Matthew+Zurek%2C+Daniel+S%2E+Brown%2C+and+Anca+D%2E+Dragan%2E+The+Effect+of+Modeling+Human+Rationality+Level+on+Learning+Rewards+from+Multiple+Feedback+Types%2E+In+Proceedings+of+the+AAAI+Confer)
125. Hugo Gilbert and Paul Weng. Quantile Reinforcement Learning. In Asian Workshop on Reinforcement Learning, 2016. [find](https://scholar.google.com/scholar?q=Hugo+Gilbert+and+Paul+Weng%2E+Quantile+Reinforcement+Learning%2E+In+Asian+Workshop+on+Reinforcement+Learning%2C+2016%2E)
126. Hugo Gilbert, Olivier Spanjaard, Paolo Viappiani, and Paul Weng. Reducing the Number of Queries in Interactive Value Iteration. In Proceedings of Algorithmic Decision Theory (ADT). Springer International Publishing, 2015. doi: 10.1007/978-3-319-23114-3_9. — [doi:10.1007/978-3-319-23114-3_9](https://doi.org/10.1007/978-3-319-23114-3_9) [find](https://scholar.google.com/scholar?q=Hugo+Gilbert%2C+Olivier+Spanjaard%2C+Paolo+Viappiani%2C+and+Paul+Weng%2E+Reducing+the+Number+of+Queries+in+Interactive+Value+Iteration%2E+In+Proceedings+of+Algorithmic+Decision+Theory+%28ADT%29%2E+Springer+Internatio)
127. Hugo Gilbert, Bruno Zanuttini, Paolo Viappiani, Paul Weng, and Esther Nicart. Model-Free Reinforcement Learning with Skew-Symmetric Bilinear Utilities. In Proceedings of the Conference on Uncertainty in Artificial Intelligence (UAI), 2016. URL [http://auai.org/uai2016/proceedings/papers/91.pdf](http://auai.org/uai2016/proceedings/papers/91.pdf). [↗](http://auai.org/uai2016/proceedings/papers/91.pdf)
128. Hugo Gilbert, Paul Weng, and Yan Xu. Optimizing Quantiles in Preference-Based Markov Decision Processes. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 31, 2017. doi: 10.1609/aaai.v31i1.11026. — [doi:10.1609/aaai.v31i1.11026](https://doi.org/10.1609/aaai.v31i1.11026) [find](https://scholar.google.com/scholar?q=Hugo+Gilbert%2C+Paul+Weng%2C+and+Yan+Xu%2E+Optimizing+Quantiles+in+Preference%2DBased+Markov+Decision+Processes%2E+In+Proceedings+of+the+AAAI+Conference+on+Artificial+Intelligence%2C+volume+31%2C+2017%2E+doi%3A+10%2E1609)
129. Amelia Glaese, Nat McAleese, Maja Trębacz, John Aslanides, Vlad Firoiu, Timo Ewalds, Maribeth Rauh, Laura Weidinger, Martin Chadwick, Phoebe Thacker, Lucy Campbell-Gillingham, Jonathan Uesato, Po-Sen Huang, Ramona Comanescu, Fan Yang, Abigail See, Sumanth Dathathri, Rory Greig, Charlie Chen, Doug Fr… — [Improving alignment of dialogue agents via targeted human judgements](https://md.research.elamur.ai/free/en/papers/glaese-2022-improving-alignment-dialogue-agents-targeted/)
130. Claire Glanois, Paul Weng, Matthieu Zimmer, Dong Li, Tianpei Yang, Jianye Hao, and Wulong Liu. A survey on interpretable reinforcement learning. Machine Learning, 113(8):5847–5890, 2024. doi: 10.1007/s10994-024-06543-w. — [doi:10.1007/s10994-024-06543-w](https://doi.org/10.1007/s10994-024-06543-w) [find](https://scholar.google.com/scholar?q=Claire+Glanois%2C+Paul+Weng%2C+Matthieu+Zimmer%2C+Dong+Li%2C+Tianpei+Yang%2C+Jianye+Hao%2C+and+Wulong+Liu%2E+A+survey+on+interpretable+reinforcement+learning%2E+Machine+Learning%2C+113%288%29%3A5847%E2%80%935890%2C+2024%2E+doi%3A+10%2E1007%2F)
131. Adam Gleave and Geoffrey Irving. Uncertainty Estimation for Language Reward Models, 2022. arXiv: 2203.07472. preprint. — [arXiv:2203.07472](https://arxiv.org/abs/2203.07472) [find](https://scholar.google.com/scholar?q=Adam+Gleave+and+Geoffrey+Irving%2E+Uncertainty+Estimation+for+Language+Reward+Models%2C+2022%2E+arXiv%3A+2203%2E07472%2E+preprint%2E+%E2%80%94+%5BarXiv%3A2203%2E07472%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2203%2E07472%29)
132. Adam Gleave, Michael D. Dennis, Shane Legg, Stuart Russell, and Jan Leike. Quantifying Differences in Reward Functions. In Proceedings of the International Conference on Learning Representations (ICLR), 2022a. URL [https://iclr.cc/virtual/2021/poster/3348](https://iclr.cc/virtual/2021/poster/3348). [↗](https://iclr.cc/virtual/2021/poster/3348)
133. Adam Gleave, Mohammad Taufeeque, Juan Rocamonde, Erik Jenner, Steven H. Wang, Sam Toyer, Maximilian Ernestus, Nora Belrose, Scott Emmons, and Stuart Russell. Imitation: Clean Imitation Learning Implementations, 2022b. arXiv: 2211.11972. preprint. — [arXiv:2211.11972](https://arxiv.org/abs/2211.11972) [find](https://scholar.google.com/scholar?q=Adam+Gleave%2C+Mohammad+Taufeeque%2C+Juan+Rocamonde%2C+Erik+Jenner%2C+Steven+H%2E+Wang%2C+Sam+Toyer%2C+Maximilian+Ernestus%2C+Nora+Belrose%2C+Scott+Emmons%2C+and+Stuart+Russell%2E+Imitation%3A+Clean+Imitation+Learning+Implem)
134. Lin Guan, Mudit Verma, Sihang Guo, Ruohan Zhang, and Subbarao Kambhampati. Widening the Pipeline in Human-Guided Reinforcement Learning with Explanation and Context-Aware Data Augmentation. In Advances in Neural Information Processing Systems (NeurIPS), 2021. URL https://proceedings. neurips.cc/pape… [find](https://scholar.google.com/scholar?q=Lin+Guan%2C+Mudit+Verma%2C+Sihang+Guo%2C+Ruohan+Zhang%2C+and+Subbarao+Kambhampati%2E+Widening+the+Pipeline+in+Human%2DGuided+Reinforcement+Learning+with+Explanation+and+Context%2DAware+Data+Augmentation%2E+In+Advance)
135. Lin Guan, Karthik Valmeekam, and Subbarao Kambhampati. Relative Behavioral Attributes: Filling the Gap between Symbolic Goal Specification and Reward Learning from Human Preferences. In Proceedings of the International Conference on Learning Representations (ICLR), 2023. URL https://openreview. net/… [find](https://scholar.google.com/scholar?q=Lin+Guan%2C+Karthik+Valmeekam%2C+and+Subbarao+Kambhampati%2E+Relative+Behavioral+Attributes%3A+Filling+the+Gap+between+Symbolic+Goal+Specification+and+Reward+Learning+from+Human+Preferences%2E+In+Proceedings+of)
136. Arnav Gudibande, Eric Wallace, Charlie Victor Snell, Xinyang Geng, Hao Liu, Pieter Abbeel, Sergey Levine, and Dawn Song. The False Promise of Imitating Proprietary Language Models. In Proceedings of the International Conference on Learning Representations (ICLR), 2024. URL [https://openreview.net/](https://openreview.net/) fo… [↗](https://openreview.net/)
137. Faruk Gul, Paulo Natenzon, and Wolfgang Pesendorfer. Random Choice as Behavioral Optimization. Econometrica, 82(5):1873–1912, 2014. doi: 10.3982/ECTA10621. — [doi:10.3982/ecta10621](https://doi.org/10.3982/ecta10621) [find](https://scholar.google.com/scholar?q=Faruk+Gul%2C+Paulo+Natenzon%2C+and+Wolfgang+Pesendorfer%2E+Random+Choice+as+Behavioral+Optimization%2E+Econometrica%2C+82%285%29%3A1873%E2%80%931912%2C+2014%2E+doi%3A+10%2E3982%2FECTA10621%2E+%E2%80%94+%5Bdoi%3A10%2E3982%2Fecta10621%5D%28https%3A%2F%2Fdoi%2Eorg%2F10)
138. Caglar Gulcehre, Tom Le Paine, Srivatsan Srinivasan, Ksenia Konyushkova, Lotte Weerts, Abhishek Sharma, Aditya Siddhant, Alex Ahern, Miaosen Wang, Chenjie Gu, Wolfgang Macherey, Arnaud Doucet, Orhan Firat, and Nando de Freitas. [Reinforced Self-Training (ReST) for Language Modeling](https://md.research.elamur.ai/free/en/papers/gulcehre-2023-reinforced-self-training-rest-language/), 2023. arXiv: 2308…
139. Tuomas Haarnoja, Haoran Tang, Pieter Abbeel, and Sergey Levine. Reinforcement Learning with Deep Energy-Based Policies. In Proceedings of the International Conference on Machine Learning (ICML). PMLR, 2017. URL [https://proceedings.mlr.press/v70/haarnoja17a.html](https://proceedings.mlr.press/v70/haarnoja17a.html). [↗](https://proceedings.mlr.press/v70/haarnoja17a.html)
140. Tuomas Haarnoja, Aurick Zhou, Pieter Abbeel, and Sergey Levine. Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor. In Proceedings of the International Conference on Machine Learning (ICML). PMLR, 2018. URL [https://proceedings.mlr.press/v80/](https://proceedings.mlr.press/v80/) haarnoja18b… [↗](https://proceedings.mlr.press/v80/)
141. Soheil Habibian, Ananth Jonnavittula, and Dylan P. Losey. Here’s What I’ve Learned: Asking Questions that Reveal Reward Learning. ACM Transactions on Human-Robot Interaction, 11(4):40:1–40:28, 2022. doi: 10.1145/3526107. — [doi:10.1145/3526107](https://doi.org/10.1145/3526107) [find](https://scholar.google.com/scholar?q=Soheil+Habibian%2C+Ananth+Jonnavittula%2C+and+Dylan+P%2E+Losey%2E+Here%E2%80%99s+What+I%E2%80%99ve+Learned%3A+Asking+Questions+that+Reveal+Reward+Learning%2E+ACM+Transactions+on+Human%2DRobot+Interaction%2C+11%284%29%3A40%3A1%E2%80%9340%3A28%2C+2022%2E+d)
142. Björn Haddenhorst, Eyke Hüllermeier, and Martin Kolb. Generalized transitivity: A systematic comparison of concepts with an application to preferences in the Babington Smith model. International Journal of Approximate Reasoning, 119:373–407, 2020. doi: 10.1016/j.ijar.2020.01.007. — [doi:10.1016/j.ijar.2020.01.007](https://doi.org/10.1016/j.ijar.2020.01.007) [find](https://scholar.google.com/scholar?q=Bj%C3%B6rn+Haddenhorst%2C+Eyke+H%C3%BCllermeier%2C+and+Martin+Kolb%2E+Generalized+transitivity%3A+A+systematic+comparison+of+concepts+with+an+application+to+preferences+in+the+Babington+Smith+model%2E+International+Journ)
143. Björn Haddenhorst, Viktor Bengs, Jasmin Brandt, and Eyke Hüllermeier. Testification of Condorcet Winners in dueling bandits. In Proceedings of the Conference on Uncertainty in Artificial Intelligence (UAI). PMLR, 2021. URL [https://proceedings.mlr.press/v161/haddenhorst21a.html](https://proceedings.mlr.press/v161/haddenhorst21a.html). [↗](https://proceedings.mlr.press/v161/haddenhorst21a.html)
144. Dylan Hadfield-Menell, Stuart J Russell, Pieter Abbeel, and Anca Dragan. Cooperative Inverse Reinforcement Learning. In Advances in Neural Information Processing Systems (NIPS), volume 29. Curran Associates, Inc., 2016. URL [https://proceedings.neurips.cc/paper/2016/hash/](https://proceedings.neurips.cc/paper/2016/hash/) c3395dd46c34fa7fd8d729d8cf88… [↗](https://proceedings.neurips.cc/paper/2016/hash/)
145. Dylan Hadfield-Menell, Anca Dragan, Pieter Abbeel, and Stuart Russell. The Off-Switch Game. In Proceedings of the International Joint Conference on Artificial Intelligence (IJCAI). International Joint Conferences on Artificial Intelligence Organization, 2017a. doi: 10.24963/ijcai.2017/32. — [doi:10.24963/ijcai.2017/32](https://doi.org/10.24963/ijcai.2017/32) [find](https://scholar.google.com/scholar?q=Dylan+Hadfield%2DMenell%2C+Anca+Dragan%2C+Pieter+Abbeel%2C+and+Stuart+Russell%2E+The+Off%2DSwitch+Game%2E+In+Proceedings+of+the+International+Joint+Conference+on+Artificial+Intelligence+%28IJCAI%29%2E+International+Joint)
146. Dylan Hadfield-Menell, Smitha Milli, Pieter Abbeel, Stuart J Russell, and Anca Dragan. Inverse Reward Design. In Advances in Neural Information Processing Systems (NIPS), volume 30. Curran Associates, Inc., 2017b. URL [https://proceedings.neurips.cc/paper/2017/hash/](https://proceedings.neurips.cc/paper/2017/hash/) 32fdab6559cdfa4f167f8c31b9199643-A… [↗](https://proceedings.neurips.cc/paper/2017/hash/)
147. Danijar Hafner, Timothy Lillicrap, Jimmy Ba, and Mohammad Norouzi. Dream to Control: Learning Behaviors by Latent Imagination. In Proceedings of the International Conference on Learning Representations (ICLR), 2020. URL [https://openreview.net/forum?id=S1lOTC4tDS](https://openreview.net/forum?id=S1lOTC4tDS). [↗](https://openreview.net/forum?id=S1lOTC4tDS)
148. Danijar Hafner, Jurgis Pasukonis, Jimmy Ba, and Timothy Lillicrap. Mastering Diverse Domains through World Models, 2023. arXiv: 2301.04104. preprint. [find](https://scholar.google.com/scholar?q=Danijar+Hafner%2C+Jurgis+Pasukonis%2C+Jimmy+Ba%2C+and+Timothy+Lillicrap%2E+Mastering+Diverse+Domains+through+World+Models%2C+2023%2E+arXiv%3A+2301%2E04104%2E+preprint%2E)
149. Thilo Hagendorff and Sarah Fabi. Methodological reflections for AI alignment research using human feedback, 2022. arXiv: 2301.06859. preprint. — [arXiv:2301.06859](https://arxiv.org/abs/2301.06859) [find](https://scholar.google.com/scholar?q=Thilo+Hagendorff+and+Sarah+Fabi%2E+Methodological+reflections+for+AI+alignment+research+using+human+feedback%2C+2022%2E+arXiv%3A+2301%2E06859%2E+preprint%2E+%E2%80%94+%5BarXiv%3A2301%2E06859%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2301%2E06859%29)
150. Jerry Zhi-Yang He and Anca D. Dragan. Assisted Robust Reward Design. In Proceedings of the Conference on Robot Learning (CoRL). PMLR, 2021. URL [https://proceedings.mlr.press/v164/he22a.html](https://proceedings.mlr.press/v164/he22a.html). [↗](https://proceedings.mlr.press/v164/he22a.html)
151. Donald Joseph Hejna and Dorsa Sadigh. Few-Shot Preference Learning for Human-in-the-Loop RL. In Proceedings of the Conference on Robot Learning (CoRL). PMLR, 2022. URL [https://proceedings.mlr.press/v205/iii23a.html](https://proceedings.mlr.press/v205/iii23a.html). [↗](https://proceedings.mlr.press/v205/iii23a.html)
152. Joey Hejna, Rafael Rafailov, Harshit Sikchi, Chelsea Finn, Scott Niekum, W. Bradley Knox, and Dorsa Sadigh. Contrastive Preference Learning: Learning from Human Feedback without Reinforcement Learning. In Proceedings of the International Conference on Learning Representations (ICLR), 2024. URL https… [find](https://scholar.google.com/scholar?q=Joey+Hejna%2C+Rafael+Rafailov%2C+Harshit+Sikchi%2C+Chelsea+Finn%2C+Scott+Niekum%2C+W%2E+Bradley+Knox%2C+and+Dorsa+Sadigh%2E+Contrastive+Preference+Learning%3A+Learning+from+Human+Feedback+without+Reinforcement+Learning)
153. Karl Moritz Hermann, Felix Hill, Simon Green, Fumin Wang, Ryan Faulkner, Hubert Soyer, David Szepesvari, Wojciech Marian Czarnecki, Max Jaderberg, Denis Teplyashin, Marcus Wainwright, Chris Apps, Demis Hassabis, and Phil Blunsom. [Grounded Language Learning in a Simulated 3D World](https://md.research.elamur.ai/free/en/papers/hermann-2017-grounded-language-learning-simulated-3d/), 2017. arXiv: 1706.…
154. Matteo Hessel, Joseph Modayil, Hado van Hasselt, Tom Schaul, Georg Ostrovski, Will Dabney, Dan Horgan, Bilal Piot, Mohammad Azar, and David Silver. Rainbow: Combining Improvements in Deep Reinforcement Learning. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 32, 2018. doi:… [find](https://scholar.google.com/scholar?q=Matteo+Hessel%2C+Joseph+Modayil%2C+Hado+van+Hasselt%2C+Tom+Schaul%2C+Georg+Ostrovski%2C+Will+Dabney%2C+Dan+Horgan%2C+Bilal+Piot%2C+Mohammad+Azar%2C+and+David+Silver%2E+Rainbow%3A+Combining+Improvements+in+Deep+Reinforcemen)
155. Matthew Hoffman, Eric Brochu, and Nando de Freitas. Portfolio Allocation for Bayesian Optimization. In Proceedings of the Conference on Uncertainty in Artificial Intelligence (UAI). AUAI Press, 2011. doi: 10.5555/3020548.3020587. — [doi:10.5555/3020548.3020587](https://doi.org/10.5555/3020548.3020587) [find](https://scholar.google.com/scholar?q=Matthew+Hoffman%2C+Eric+Brochu%2C+and+Nando+de+Freitas%2E+Portfolio+Allocation+for+Bayesian+Optimization%2E+In+Proceedings+of+the+Conference+on+Uncertainty+in+Artificial+Intelligence+%28UAI%29%2E+AUAI+Press%2C+2011%2E+)
156. Rachel Holladay, Shervin Javdani, Anca Dragan, and Siddhartha Srinivasa. Active comparison based learning incorporating user uncertainty and noise. In RSS 2016 Workshop on Model Learning for Human-Robot Communication, 2016. [find](https://scholar.google.com/scholar?q=Rachel+Holladay%2C+Shervin+Javdani%2C+Anca+Dragan%2C+and+Siddhartha+Srinivasa%2E+Active+comparison+based+learning+incorporating+user+uncertainty+and+noise%2E+In+RSS+2016+Workshop+on+Model+Learning+for+Human%2DRob)
157. Tom Hosking, Phil Blunsom, and Max Bartolo. Human Feedback is not Gold Standard. In Proceedings of the International Conference on Learning Representations (ICLR), 2024. URL [https://openreview.net/forum?id=7W3GLNImfS](https://openreview.net/forum?id=7W3GLNImfS). [↗](https://openreview.net/forum?id=7W3GLNImfS)
158. Edward J. Hu, Moksh Jain, Eric Elmoznino, Younesse Kaddar, Guillaume Lajoie, Yoshua Bengio, and Nikolay Malkin. Amortizing intractable inference in large language models. In Proceedings of the International Conference on Learning Representations (ICLR), 2024a. URL [https://openreview.net/forum?](https://openreview.net/forum?) id=Ou… [↗](https://openreview.net/forum?)
159. Xiao Hu, Jianxiong Li, Xianyuan Zhan, Qing-Shan Jia, and Ya-Qin Zhang. Query-Policy Misalignment in Preference-Based Reinforcement Learning. In Proceedings of the International Conference on Learning Representations (ICLR), 2024b. URL [https://openreview.net/forum?id=UoBymIwPJR](https://openreview.net/forum?id=UoBymIwPJR). [↗](https://openreview.net/forum?id=UoBymIwPJR)
160. Jie Huang, Jiangshan Hao, Rongshun Juan, Randy Gomez, Keisuke Nakamura, and Guangliang Li. GAN-Based Interactive Reinforcement Learning from Demonstration and Human Evaluative Feedback. In Proceedings of the IEEE International Conference on Robotics and Automation (ICRA), 2023. doi: 10.1109/ICRA4889… — [doi:10.1109/icra48891.2023.10160939](https://doi.org/10.1109/icra48891.2023.10160939) [find](https://scholar.google.com/scholar?q=Jie+Huang%2C+Jiangshan+Hao%2C+Rongshun+Juan%2C+Randy+Gomez%2C+Keisuke+Nakamura%2C+and+Guangliang+Li%2E+GAN%2DBased+Interactive+Reinforcement+Learning+from+Demonstration+and+Human+Evaluative+Feedback%2E+In+Proceedings)
161. Shengyi Huang, Michael Noukhovitch, Arian Hosseini, Kashif Rasul, Weixun Wang, and Lewis Tunstall. The N+ Implementation Details of RLHF with PPO: A Case Study on TL;DR Summarization. In Proceedings of the Conference on Language Modeling (COLM), 2024. URL [https://openreview.net/forum?id=](https://openreview.net/forum?id=) kHO2ZTa8e3\#… [↗](https://openreview.net/forum?id=)
162. Eyke Hüllermeier and Willem Waegeman. Aleatoric and epistemic uncertainty in machine learning: An introduction to concepts and methods. Machine Learning, 110(3):457–506, 2021. doi: 10.1007/s10994-021-05946-3. — [doi:10.1007/s10994-021-05946-3](https://doi.org/10.1007/s10994-021-05946-3) [find](https://scholar.google.com/scholar?q=Eyke+H%C3%BCllermeier+and+Willem+Waegeman%2E+Aleatoric+and+epistemic+uncertainty+in+machine+learning%3A+An+introduction+to+concepts+and+methods%2E+Machine+Learning%2C+110%283%29%3A457%E2%80%93506%2C+2021%2E+doi%3A+10%2E1007%2Fs10994%2D021%2D)
163. Minyoung Hwang, Gunmin Lee, Hogun Kee, Chan Woo Kim, Kyungjae Lee, and Songhwai Oh. Sequential Preference Ranking for Efficient Reinforcement Learning from Human Feedback. In Advances in Neural Information Processing Systems (NeurIPS), volume 36, 2023. URL [https://proceedings.neurips.cc/](https://proceedings.neurips.cc/) paper_files… [↗](https://proceedings.neurips.cc/)
164. Borja Ibarz, Jan Leike, Tobias Pohlen, Geoffrey Irving, Shane Legg, and Dario Amodei. Reward learning from human preferences and demonstrations in Atari. In Advances in Neural Information Processing Systems (NIPS), volume 31. Curran Associates, Inc., 2018. URL [https://proceedings.neurips.cc/](https://proceedings.neurips.cc/) paper/2… [↗](https://proceedings.neurips.cc/)
165. Alexander Irpan, Kanishka Rao, Konstantinos Bousmalis, Chris Harris, Julian Ibarz, and Sergey Levine. Off-Policy Evaluation via Off-Policy Classification. In Advances in Neural Information Processing Systems (NeurIPS), volume 32. Curran Associates, Inc., 2019. URL [https://proceedings.neurips.cc/pape…](https://proceedings.neurips.cc/pape…) [↗](https://proceedings.neurips.cc/pape…)
166. Charles Isbell, Christian R. Shelton, Michael Kearns, Satinder Singh, and Peter Stone. A social reinforcement learning agent. In Proceedings of the International Conference on Autonomous Agents (AGENTS). Association for Computing Machinery, 2001. doi: 10.1145/375735.376334. — [doi:10.1145/375735.376334](https://doi.org/10.1145/375735.376334) [find](https://scholar.google.com/scholar?q=Charles+Isbell%2C+Christian+R%2E+Shelton%2C+Michael+Kearns%2C+Satinder+Singh%2C+and+Peter+Stone%2E+A+social+reinforcement+learning+agent%2E+In+Proceedings+of+the+International+Conference+on+Autonomous+Agents+%28AGENT)
167. Ashesh Jain, Shikhar Sharma, Thorsten Joachims, and Ashutosh Saxena. Learning preferences for manipulation tasks from online coactive feedback. The International Journal of Robotics Research, 34(10): 1296–1313, 2015. doi: 10.1177/0278364915581193. — [doi:10.1177/0278364915581193](https://doi.org/10.1177/0278364915581193) [find](https://scholar.google.com/scholar?q=Ashesh+Jain%2C+Shikhar+Sharma%2C+Thorsten+Joachims%2C+and+Ashutosh+Saxena%2E+Learning+preferences+for+manipulation+tasks+from+online+coactive+feedback%2E+The+International+Journal+of+Robotics+Research%2C+34%2810%29%3A+)
168. Natasha Jaques, Shixiang Gu, Dzmitry Bahdanau, José Miguel Hernández-Lobato, Richard E. Turner, and Douglas Eck. Sequence Tutor: Conservative Fine-Tuning of Sequence Generation Models with KL-control. In Proceedings of the International Conference on Machine Learning (ICML). PMLR, 2017. URL https: /… [find](https://scholar.google.com/scholar?q=Natasha+Jaques%2C+Shixiang+Gu%2C+Dzmitry+Bahdanau%2C+Jos%C3%A9+Miguel+Hern%C3%A1ndez%2DLobato%2C+Richard+E%2E+Turner%2C+and+Douglas+Eck%2E+Sequence+Tutor%3A+Conservative+Fine%2DTuning+of+Sequence+Generation+Models+with+KL%2Dcontrol%2E)
169. Erik Jenner and Adam Gleave. Preprocessing Reward Functions for Interpretability. In NeurIPS 2021 Workshop on Cooperative AI, 2021. [find](https://scholar.google.com/scholar?q=Erik+Jenner+and+Adam+Gleave%2E+Preprocessing+Reward+Functions+for+Interpretability%2E+In+NeurIPS+2021+Workshop+on+Cooperative+AI%2C+2021%2E)
170. Erik Jenner, Joar Max Viktor Skalse, and Adam Gleave. A general framework for reward function distances. In NeurIPS 2022 Workshop on ML Safety, 2022. URL [https://openreview.net/forum?id=Hn21kZHiCK](https://openreview.net/forum?id=Hn21kZHiCK). [↗](https://openreview.net/forum?id=Hn21kZHiCK)
171. Hong Jun Jeon, Smitha Milli, and Anca Dragan. Reward-rational (implicit) choice: A unifying formalism for reward learning. In Advances in Neural Information Processing Systems (NeurIPS), volume 33. Curran Associates, Inc., 2020. URL [https://proceedings.neurips.cc/paper/2020/hash/](https://proceedings.neurips.cc/paper/2020/hash/) 2f10c1578a0706e06b6… [↗](https://proceedings.neurips.cc/paper/2020/hash/)
172. Jiaming Ji, Mickel Liu, Josef Dai, Xuehai Pan, Chi Zhang, Ce Bian, Boyuan Chen, Ruiyang Sun, Yizhou Wang, and Yaodong Yang. [BeaverTails: Towards Improved Safety Alignment of LLM via a Human-Preference Dataset](https://md.research.elamur.ai/free/en/papers/ji-2023-beavertails-towards-improved-safety-alignment/). In Advances in Neural Information Processing Systems (NeurIPS), volume 36, 2023a. URL http…
173. Jiaming Ji, Tianyi Qiu, Boyuan Chen, Borong Zhang, Hantao Lou, Kaile Wang, Yawen Duan, Zhonghao He, Jiayi Zhou, Zhaowei Zhang, Fanzhi Zeng, Kwan Yee Ng, Juntao Dai, Xuehai Pan, Aidan O’Gara, Yingshan Lei, Hua Xu, Brian Tse, Jie Fu, Stephen McAleer, Yaodong Yang, Yizhou Wang, Song-Chun Zhu, Yike Guo,… — [AI Alignment: A Comprehensive Survey](https://md.research.elamur.ai/free/en/papers/ji-2023-ai-alignment-comprehensive-survey/)
174. Xiang Ji, Huazheng Wang, Minshuo Chen, Tuo Zhao, and Mengdi Wang. Provable Benefits of Policy Learning from Human Preferences in Contextual Bandit Problems, 2023c. arXiv: 2307.12975. preprint. — [arXiv:2307.12975](https://arxiv.org/abs/2307.12975) [find](https://scholar.google.com/scholar?q=Xiang+Ji%2C+Huazheng+Wang%2C+Minshuo+Chen%2C+Tuo+Zhao%2C+and+Mengdi+Wang%2E+Provable+Benefits+of+Policy+Learning+from+Human+Preferences+in+Contextual+Bandit+Problems%2C+2023c%2E+arXiv%3A+2307%2E12975%2E+preprint%2E+%E2%80%94+%5BarXi)
175. Nan Jiang and Lihong Li. Doubly Robust Off-policy Value Evaluation for Reinforcement Learning. In Proceedings of the International Conference on Machine Learning (ICML). PMLR, 2016. URL [https://proceedings.mlr.press/v48/jiang16.html](https://proceedings.mlr.press/v48/jiang16.html). [↗](https://proceedings.mlr.press/v48/jiang16.html)
176. Chi Jin, Zhuoran Yang, Zhaoran Wang, and Michael I. Jordan. Provably efficient reinforcement learning with linear function approximation. In Proceedings of the Conference on Learning Theory (COLT). PMLR, 2020. URL [https://proceedings.mlr.press/v125/jin20a.html](https://proceedings.mlr.press/v125/jin20a.html). [↗](https://proceedings.mlr.press/v125/jin20a.html)
177. Ying Jin, Zhuoran Yang, and Zhaoran Wang. Is Pessimism Provably Efficient for Offline RL? In Proceedings of the International Conference on Machine Learning (ICML). PMLR, 2021. URL [https://proceedings.mlr.press/v139/jin21e.html](https://proceedings.mlr.press/v139/jin21e.html). [↗](https://proceedings.mlr.press/v139/jin21e.html)
178. Kshitij Judah, Saikat Roy, Alan Fern, and Thomas Dietterich. Reinforcement Learning Via Practice and Critique Advice. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 24, 2010. doi: 10.1609/aaai.v24i1.7690. — [doi:10.1609/aaai.v24i1.7690](https://doi.org/10.1609/aaai.v24i1.7690) [find](https://scholar.google.com/scholar?q=Kshitij+Judah%2C+Saikat+Roy%2C+Alan+Fern%2C+and+Thomas+Dietterich%2E+Reinforcement+Learning+Via+Practice+and+Critique+Advice%2E+In+Proceedings+of+the+AAAI+Conference+on+Artificial+Intelligence%2C+volume+24%2C+2010%2E)
179. Saurav Kadavath, Tom Conerly, Amanda Askell, Tom Henighan, Dawn Drain, Ethan Perez, Nicholas Schiefer, Zac Hatfield-Dodds, Nova DasSarma, Eli Tran-Johnson, Scott Johnston, Sheer El-Showk, Andy Jones, Nelson Elhage, Tristan Hume, Anna Chen, Yuntao Bai, Sam Bowman, Stanislav Fort, Deep Ganguli, Danny… [find](https://scholar.google.com/scholar?q=Saurav+Kadavath%2C+Tom+Conerly%2C+Amanda+Askell%2C+Tom+Henighan%2C+Dawn+Drain%2C+Ethan+Perez%2C+Nicholas+Schiefer%2C+Zac+Hatfield%2DDodds%2C+Nova+DasSarma%2C+Eli+Tran%2DJohnson%2C+Scott+Johnston%2C+Sheer+El%2DShowk%2C+Andy+Jones%2C+)
180. Gregory Kahn, Pieter Abbeel, and Sergey Levine. LaND: Learning to Navigate From Disengagements. IEEE Robotics and Automation Letters, 6(2):1872–1879, 2021. doi: 10.1109/LRA.2021.3060404. — [doi:10.1109/lra.2021.3060404](https://doi.org/10.1109/lra.2021.3060404) [find](https://scholar.google.com/scholar?q=Gregory+Kahn%2C+Pieter+Abbeel%2C+and+Sergey+Levine%2E+LaND%3A+Learning+to+Navigate+From+Disengagements%2E+IEEE+Robotics+and+Automation+Letters%2C+6%282%29%3A1872%E2%80%931879%2C+2021%2E+doi%3A+10%2E1109%2FLRA%2E2021%2E3060404%2E+%E2%80%94+%5Bdoi%3A10%2E110)
181. Akansha Kalra and Daniel S. Brown. Interpretable Reward Learning via Differentiable Decision Trees. In NeurIPS 2022 Workshop on ML Safety, 2022. URL [https://openreview.net/forum?id=3bk40MsYjet](https://openreview.net/forum?id=3bk40MsYjet). [↗](https://openreview.net/forum?id=3bk40MsYjet)
182. Akansha Kalra and Daniel S Brown. Can Differentiable Decision Trees Enable Interpretable Reward Learning from Human Feedback? Reinforcement Learning Journal, 4:1887–1910, 2024. [find](https://scholar.google.com/scholar?q=Akansha+Kalra+and+Daniel+S+Brown%2E+Can+Differentiable+Decision+Trees+Enable+Interpretable+Reward+Learning+from+Human+Feedback%3F+Reinforcement+Learning+Journal%2C+4%3A1887%E2%80%931910%2C+2024%2E)
183. Yachen Kang, Diyuan Shi, Jinxin Liu, Li He, and Donglin Wang. Beyond Reward: Offline Preference-guided Policy Optimization. In Proceedings of the International Conference on Machine Learning (ICML). PMLR, 2023. URL [https://proceedings.mlr.press/v202/kang23b.html](https://proceedings.mlr.press/v202/kang23b.html). [↗](https://proceedings.mlr.press/v202/kang23b.html)
184. Sydney M. Katz, Amir Maleki, Erdem Bıyık, and Mykel J. Kochenderfer. Preference-based Learning of Reward Function Features, 2021. arXiv: 2103.02727. preprint. — [arXiv:2103.02727](https://arxiv.org/abs/2103.02727) [find](https://scholar.google.com/scholar?q=Sydney+M%2E+Katz%2C+Amir+Maleki%2C+Erdem+B%C4%B1y%C4%B1k%2C+and+Mykel+J%2E+Kochenderfer%2E+Preference%2Dbased+Learning+of+Reward+Function+Features%2C+2021%2E+arXiv%3A+2103%2E02727%2E+preprint%2E+%E2%80%94+%5BarXiv%3A2103%2E02727%5D%28https%3A%2F%2Farxiv%2Eorg%2Fab)
185. Timo Kaufmann, Sarah Ball, Jacob Beck, Frauke Kreuter, and Eyke Hüllermeier. On the Challenges and Practices of Reinforcement Learning from Real Human Feedback. In ECML PKDD 2023 Workshop Towards Hybrid Human-Machine Learning and Decision Making, 2023. [find](https://scholar.google.com/scholar?q=Timo+Kaufmann%2C+Sarah+Ball%2C+Jacob+Beck%2C+Frauke+Kreuter%2C+and+Eyke+H%C3%BCllermeier%2E+On+the+Challenges+and+Practices+of+Reinforcement+Learning+from+Real+Human+Feedback%2E+In+ECML+PKDD+2023+Workshop+Towards+Hybr)
186. Hadi Kazemi, Fariborz Taherkhani, and Nasser M. Nasrabadi. Preference-Based Image Generation. In Proceedings of IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), 2020. doi: 10.1109/WACV45572.2020.9093406. — [doi:10.1109/wacv45572.2020.9093406](https://doi.org/10.1109/wacv45572.2020.9093406) [find](https://scholar.google.com/scholar?q=Hadi+Kazemi%2C+Fariborz+Taherkhani%2C+and+Nasser+M%2E+Nasrabadi%2E+Preference%2DBased+Image+Generation%2E+In+Proceedings+of+IEEE%2FCVF+Winter+Conference+on+Applications+of+Computer+Vision+%28WACV%29%2C+2020%2E+doi%3A+10%2E1109)
187. Changyeon Kim, Jongjin Park, Jinwoo Shin, Honglak Lee, Pieter Abbeel, and Kimin Lee. Preference Transformer: Modeling Human Preferences using Transformers for RL. In Proceedings of the International Conference on Learning Representations (ICLR), 2023. URL [https://openreview.net/forum?](https://openreview.net/forum?) id=Peot1SFDX0. [↗](https://openreview.net/forum?)
188. Kuno Kim, Shivam Garg, Kirankumar Shiragur, and Stefano Ermon. Reward Identification in Inverse Reinforcement Learning. In Proceedings of the International Conference on Machine Learning (ICML). PMLR, 2021. URL [https://proceedings.mlr.press/v139/kim21c.html](https://proceedings.mlr.press/v139/kim21c.html). [↗](https://proceedings.mlr.press/v139/kim21c.html)
189. Jan H. Kirchner, Logan Smith, Jacques Thibodeau, Kyle McDonell, and Laria Reynolds. Researching Alignment Research: Unsupervised Analysis, 2022. arXiv: 2206.02841. preprint. — [arXiv:2206.02841](https://arxiv.org/abs/2206.02841) [find](https://scholar.google.com/scholar?q=Jan+H%2E+Kirchner%2C+Logan+Smith%2C+Jacques+Thibodeau%2C+Kyle+McDonell%2C+and+Laria+Reynolds%2E+Researching+Alignment+Research%3A+Unsupervised+Analysis%2C+2022%2E+arXiv%3A+2206%2E02841%2E+preprint%2E+%E2%80%94+%5BarXiv%3A2206%2E02841%5D%28https)
190. Martin Klissarov, Pierluca D’Oro, Shagun Sodhani, Roberta Raileanu, Pierre-Luc Bacon, Pascal Vincent, Amy Zhang, and Mikael Henaff. Motif: Intrinsic Motivation from Artificial Intelligence Feedback. In Proceedings of the International Conference on Learning Representations (ICLR), 2024. URL https: /… [find](https://scholar.google.com/scholar?q=Martin+Klissarov%2C+Pierluca+D%E2%80%99Oro%2C+Shagun+Sodhani%2C+Roberta+Raileanu%2C+Pierre%2DLuc+Bacon%2C+Pascal+Vincent%2C+Amy+Zhang%2C+and+Mikael+Henaff%2E+Motif%3A+Intrinsic+Motivation+from+Artificial+Intelligence+Feedback%2E+I)
191. W. Bradley Knox. Learning from Human-generated Reward. PhD thesis, The University of Texas at Austin, 2012. URL [https://repositories.lib.utexas.edu/items/](https://repositories.lib.utexas.edu/items/) 20b9e8a1-a78d-4844-816f-3c0b0a4c848a. [↗](https://repositories.lib.utexas.edu/items/)
192. W. Bradley Knox and Peter Stone. Interactively shaping agents via human reinforcement: The TAMER framework. In Proceedings of the International Conference on Knowledge Capture (K-CAP). Association for Computing Machinery, 2009. doi: 10.1145/1597735.1597738. — [doi:10.1145/1597735.1597738](https://doi.org/10.1145/1597735.1597738) [find](https://scholar.google.com/scholar?q=W%2E+Bradley+Knox+and+Peter+Stone%2E+Interactively+shaping+agents+via+human+reinforcement%3A+The+TAMER+framework%2E+In+Proceedings+of+the+International+Conference+on+Knowledge+Capture+%28K%2DCAP%29%2E+Association+for)
193. W. Bradley Knox, Alessandro Allievi, Holger Banzhaf, Felix Schmitt, and Peter Stone. Reward (Mis)design for autonomous driving. Artificial Intelligence, 316:103829, 2023. doi: 10.1016/j.artint.2022.103829. — [doi:10.1016/j.artint.2022.103829](https://doi.org/10.1016/j.artint.2022.103829) [find](https://scholar.google.com/scholar?q=W%2E+Bradley+Knox%2C+Alessandro+Allievi%2C+Holger+Banzhaf%2C+Felix+Schmitt%2C+and+Peter+Stone%2E+Reward+%28Mis%29design+for+autonomous+driving%2E+Artificial+Intelligence%2C+316%3A103829%2C+2023%2E+doi%3A+10%2E1016%2Fj%2Eartint%2E2022%2E10)
194. W. Bradley Knox, Stephane Hatgis-Kessell, Serena Booth, Scott Niekum, Peter Stone, and Alessandro G. Allievi. Models of human preference for learning reward functions. Transactions on Machine Learning Research, 2024. ISSN 2835-8856. URL [https://openreview.net/forum?id=hpKJkVoThY](https://openreview.net/forum?id=hpKJkVoThY). [↗](https://openreview.net/forum?id=hpKJkVoThY)
195. Dingwen Kong and Lin Yang. Provably Feedback-Efficient Reinforcement Learning via Active Reward Learning. In Advances in Neural Information Processing Systems (NeurIPS), volume 35, 2022. URL [https://proceedings.neurips.cc/paper/2022/hash/](https://proceedings.neurips.cc/paper/2022/hash/) 476c289f685e27936aa089e9d53a4213-Abstract-Conference.html. [↗](https://proceedings.neurips.cc/paper/2022/hash/)
196. Andreas Köpf, Yannic Kilcher, Dimitri von Rütte, Sotiris Anagnostidis, Zhi Rui Tam, Keith Stevens, Abdullah Barhoum, Duc Nguyen, Oliver Stanley, Richárd Nagyfi, Shahul Es, Sameer Suri, David Glushkov, Arnav Dantuluri, Andrew Maguire, Christoph Schuhmann, Huu Nguyen, and Alexander Mattick. OpenAssist… — [OpenAssistant Conversations -- Democratizing Large Language Model Alignment](https://md.research.elamur.ai/free/en/papers/kopf-2023-openassistant-conversations-democratizing-large-language/)
197. Pallavi Koppol, Henny Admoni, and Reid Simmons. Iterative Interactive Reward Learning. In ICML 2020 Workshop on Participatory Approaches to Machine Learning, 2020. [find](https://scholar.google.com/scholar?q=Pallavi+Koppol%2C+Henny+Admoni%2C+and+Reid+Simmons%2E+Iterative+Interactive+Reward+Learning%2E+In+ICML+2020+Workshop+on+Participatory+Approaches+to+Machine+Learning%2C+2020%2E)
198. Anna Korba, Stéphan Clemencon, and Eric Sibony. A Learning Theory of Ranking Aggregation. In Proceedings of the International Conference on Artificial Intelligence and Statistics (AISTATS). PMLR, 2017. URL [https://proceedings.mlr.press/v54/korba17a.html](https://proceedings.mlr.press/v54/korba17a.html). [↗](https://proceedings.mlr.press/v54/korba17a.html)
199. Tomasz Korbak, Kejian Shi, Angelica Chen, Rasika Vinayak Bhalerao, Christopher Buckley, Jason Phang, Samuel R. Bowman, and Ethan Perez. [Pretraining Language Models with Human Preferences](https://md.research.elamur.ai/free/en/papers/korbak-2023-pretraining-language-models-human-preferences/). In Proceedings of the International Conference on Machine Learning (ICML). PMLR, 2023. URL [https://proceedings.…](https://proceedings.…) [↗](https://proceedings.…)
200. Dmitrii Krasheninnikov, Rohin Shah, and Herke van Hoof. Combining Reward Information from Multiple Sources, 2021. arXiv: 2103.12142. preprint. — [arXiv:2103.12142](https://arxiv.org/abs/2103.12142) [find](https://scholar.google.com/scholar?q=Dmitrii+Krasheninnikov%2C+Rohin+Shah%2C+and+Herke+van+Hoof%2E+Combining+Reward+Information+from+Multiple+Sources%2C+2021%2E+arXiv%3A+2103%2E12142%2E+preprint%2E+%E2%80%94+%5BarXiv%3A2103%2E12142%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2103%2E12142%29)
201. Samantha Krening and Karen M. Feigh. Interaction Algorithm Effect on Human Experience with Reinforcement Learning. ACM Transactions on Human-Robot Interaction, 7(2):16:1–16:22, 2018. doi: 10.1145/3277904. — [doi:10.1145/3277904](https://doi.org/10.1145/3277904) [find](https://scholar.google.com/scholar?q=Samantha+Krening+and+Karen+M%2E+Feigh%2E+Interaction+Algorithm+Effect+on+Human+Experience+with+Reinforcement+Learning%2E+ACM+Transactions+on+Human%2DRobot+Interaction%2C+7%282%29%3A16%3A1%E2%80%9316%3A22%2C+2018%2E+doi%3A+10%2E1145%2F3277)
202. Andras Kupcsik, David Hsu, and Wee Sun Lee. Learning Dynamic Robot-to-Human Object Handover from Human Feedback. In Antonio Bicchi and Wolfram Burgard (eds.), Robotics Research: Volume 1, pp. 161–176. Springer International Publishing, 2018. doi: 10.1007/978-3-319-51532-8_10. — [doi:10.1007/978-3-319-51532-8_10](https://doi.org/10.1007/978-3-319-51532-8_10) [find](https://scholar.google.com/scholar?q=Andras+Kupcsik%2C+David+Hsu%2C+and+Wee+Sun+Lee%2E+Learning+Dynamic+Robot%2Dto%2DHuman+Object+Handover+from+Human+Feedback%2E+In+Antonio+Bicchi+and+Wolfram+Burgard+%28eds%2E%29%2C+Robotics+Research%3A+Volume+1%2C+pp%2E+161%E2%80%93176%2E)
203. Minae Kwon, Sang Michael Xie, Kalesha Bullard, and Dorsa Sadigh. [Reward Design with Language Models](https://md.research.elamur.ai/free/en/papers/kwon-2023-reward-design-language-models/). In Proceedings of the International Conference on Learning Representations (ICLR), 2023. URL [https://openreview.net/forum?id=10uNUgI5Kl](https://openreview.net/forum?id=10uNUgI5Kl). [↗](https://openreview.net/forum?id=10uNUgI5Kl)
204. Nathan Lambert. Reward is not enough, reward-is-not-enough. 2021. [find](https://scholar.google.com/scholar?q=Nathan+Lambert%2E+Reward+is+not+enough%2C+reward%2Dis%2Dnot%2Denough%2E+2021%2E)
205. URL [https://robotic.substack.com/p/Nathan](https://robotic.substack.com/p/Nathan) Lambert, Thomas Krendl Gilbert, and Tom Zick. The History and Risks of Reinforcement Learning and Human Feedback, 2023. arXiv: 2310.13595. preprint. — [arXiv:2310.13595](https://arxiv.org/abs/2310.13595) [↗](https://robotic.substack.com/p/Nathan)
206. Nathan Lambert, Valentina Pyatkin, Jacob Morrison, L. J. Miranda, Bill Yuchen Lin, Khyathi Chandu, Nouha Dziri, Sachin Kumar, Tom Zick, Yejin Choi, Noah A. Smith, and Hannaneh Hajishirzi. Reward-Bench: Evaluating Reward Models for Language Modeling, 2024. arXiv: 2403.13787. preprint. [find](https://scholar.google.com/scholar?q=Nathan+Lambert%2C+Valentina+Pyatkin%2C+Jacob+Morrison%2C+L%2E+J%2E+Miranda%2C+Bill+Yuchen+Lin%2C+Khyathi+Chandu%2C+Nouha+Dziri%2C+Sachin+Kumar%2C+Tom+Zick%2C+Yejin+Choi%2C+Noah+A%2E+Smith%2C+and+Hannaneh+Hajishirzi%2E+Reward%2DBench)
207. Misha Laskin, Kimin Lee, Adam Stooke, Lerrel Pinto, Pieter Abbeel, and Aravind Srinivas. Reinforcement Learning with Augmented Data. In Advances in Neural Information Processing Systems (NeurIPS), volume 33. Curran Associates, Inc., 2020. URL [https://proceedings.neurips.cc/paper/2020/hash/](https://proceedings.neurips.cc/paper/2020/hash/) e615c82ab… [↗](https://proceedings.neurips.cc/paper/2020/hash/)
208. Tor Lattimore and Csaba Szepesvári. 10.1017/9781108571401. Bandit Algorithms. — [doi:10.1017/9781108571401](https://doi.org/10.1017/9781108571401) [find](https://scholar.google.com/scholar?q=Tor+Lattimore+and+Csaba+Szepesv%C3%A1ri%2E+10%2E1017%2F9781108571401%2E+Bandit+Algorithms%2E+%E2%80%94+%5Bdoi%3A10%2E1017%2F9781108571401%5D%28https%3A%2F%2Fdoi%2Eorg%2F10%2E1017%2F9781108571401%29)
209. Cambridge University Press, 2020. doi: [find](https://scholar.google.com/scholar?q=Cambridge+University+Press%2C+2020%2E+doi%3A)
210. Hoang Le, Cameron Voloshin, and Yisong Yue. Batch Policy Learning under Constraints. In Proceedings of the International Conference on Machine Learning (ICML). PMLR, 2019. URL [https://proceedings.mlr.press/v97/le19a.html](https://proceedings.mlr.press/v97/le19a.html). [↗](https://proceedings.mlr.press/v97/le19a.html)
211. Kimin Lee, Kibok Lee, Jinwoo Shin, and Honglak Lee. Network Randomization: A Simple Technique for Generalization in Deep Reinforcement Learning. In Proceedings of the International Conference on Learning Representations (ICLR), 2020. URL [https://openreview.net/forum?id=HJgcvJBFvB](https://openreview.net/forum?id=HJgcvJBFvB). [↗](https://openreview.net/forum?id=HJgcvJBFvB)
212. Kimin Lee, Laura Smith, Anca Dragan, and Pieter Abbeel. B-Pref: Benchmarking Preference-Based Reinforcement Learning. In Proceedings of the Neural Information Processing Systems Track on Datasets and Benchmarks (NeurIPS Datasets and Benchmarks), volume 1, 2021a. URL https://datasets-benchmarks-proce… [find](https://scholar.google.com/scholar?q=Kimin+Lee%2C+Laura+Smith%2C+Anca+Dragan%2C+and+Pieter+Abbeel%2E+B%2DPref%3A+Benchmarking+Preference%2DBased+Reinforcement+Learning%2E+In+Proceedings+of+the+Neural+Information+Processing+Systems+Track+on+Datasets+and+)
213. Kimin Lee, Laura M. Smith, and Pieter Abbeel. PEBBLE: Feedback-Efficient Interactive Reinforcement Learning via Relabeling Experience and Unsupervised Pre-training. In Proceedings of the International Conference on Machine Learning (ICML). PMLR, 2021b. URL [https://proceedings.mlr.press/v139/](https://proceedings.mlr.press/v139/) lee21i.… [↗](https://proceedings.mlr.press/v139/)
214. Kimin Lee, Hao Liu, Moonkyung Ryu, Olivia Watkins, Yuqing Du, Craig Boutilier, Pieter Abbeel, Mohammad Ghavamzadeh, and Shixiang Shane Gu. Aligning Text-to-Image Models using Human Feedback, 2023. arXiv: 2302.12192. preprint. — [arXiv:2302.12192](https://arxiv.org/abs/2302.12192) [find](https://scholar.google.com/scholar?q=Kimin+Lee%2C+Hao+Liu%2C+Moonkyung+Ryu%2C+Olivia+Watkins%2C+Yuqing+Du%2C+Craig+Boutilier%2C+Pieter+Abbeel%2C+Mohammad+Ghavamzadeh%2C+and+Shixiang+Shane+Gu%2E+Aligning+Text%2Dto%2DImage+Models+using+Human+Feedback%2C+2023%2E+arX)
215. Joel Lehman, Jeff Clune, Dusan Misevic, Christoph Adami, Lee Altenberg, Julie Beaulieu, Peter J. Bentley, Samuel Bernard, Guillaume Beslon, David M. Bryson, Nick Cheney, Patryk Chrabaszcz, Antoine Cully, Stephane Doncieux, Fred C. Dyer, Kai Olav Ellefsen, Robert Feldt, Stephan Fischer, Stephanie For… — [doi:10.1162/artl_a_00319](https://doi.org/10.1162/artl_a_00319) [find](https://scholar.google.com/scholar?q=Joel+Lehman%2C+Jeff+Clune%2C+Dusan+Misevic%2C+Christoph+Adami%2C+Lee+Altenberg%2C+Julie+Beaulieu%2C+Peter+J%2E+Bentley%2C+Samuel+Bernard%2C+Guillaume+Beslon%2C+David+M%2E+Bryson%2C+Nick+Cheney%2C+Patryk+Chrabaszcz%2C+Antoine+Cul)
216. Jan Leike. Why I’m excited about AI-assisted human feedback, 2022. URL [https://aligned.substack.com/p/ai-assisted-human-feedback](https://aligned.substack.com/p/ai-assisted-human-feedback). [↗](https://aligned.substack.com/p/ai-assisted-human-feedback)
217. Jan Leike, David Krueger, Tom Everitt, Miljan Martic, Vishal Maini, and Shane Legg. [Scalable agent alignment via reward modeling: A research direction](https://md.research.elamur.ai/free/en/papers/leike-2018-scalable-agent-alignment-reward-modeling/), 2018. arXiv: 1811.07871. preprint.
218. Junlong Li, Shichao Sun, Weizhe Yuan, Run-Ze Fan, Hai Zhao, and Pengfei Liu. [Generative Judge for Evaluating Alignment](https://md.research.elamur.ai/free/en/papers/li-2023-generative-judge-evaluating-alignment/). In Proceedings of the International Conference on Learning Representations (ICLR), 2024. URL [https://openreview.net/forum?id=gtkFw6sZGS](https://openreview.net/forum?id=gtkFw6sZGS). [↗](https://openreview.net/forum?id=gtkFw6sZGS)
219. Mengxi Li, Alper Canberk, Dylan P. Losey, and Dorsa Sadigh. Learning Human Objectives from Sequences of Physical Corrections. In Proceedings of the IEEE International Conference on Robotics and Automation (ICRA), 2021. doi: 10.1109/ICRA48506.2021.9560829. — [doi:10.1109/icra48506.2021.9560829](https://doi.org/10.1109/icra48506.2021.9560829) [find](https://scholar.google.com/scholar?q=Mengxi+Li%2C+Alper+Canberk%2C+Dylan+P%2E+Losey%2C+and+Dorsa+Sadigh%2E+Learning+Human+Objectives+from+Sequences+of+Physical+Corrections%2E+In+Proceedings+of+the+IEEE+International+Conference+on+Robotics+and+Automa)
220. Zihao Li, Zhuoran Yang, and Mengdi Wang. Reinforcement learning with Human Feedback: Learning Dynamic Choices via Pessimism. In ICML 2023 Workshop on Interactive Learning with Implicit Human Feedback, 2023. URL [https://openreview.net/forum?id=gxM2AUFMsK](https://openreview.net/forum?id=gxM2AUFMsK). [↗](https://openreview.net/forum?id=gxM2AUFMsK)
221. Xinran Liang, Katherine Shu, Kimin Lee, and Pieter Abbeel. Reward Uncertainty for Exploration in Preference-based Reinforcement Learning. In Proceedings of the International Conference on Learning Representations (ICLR), 2022. URL [https://openreview.net/forum?id=OWZVD-l-ZrC](https://openreview.net/forum?id=OWZVD-l-ZrC). [↗](https://openreview.net/forum?id=OWZVD-l-ZrC)
222. Sarah Lichtenstein and Paul Slovic (eds.). The Construction of Preference. Cambridge University Press, 2006. doi: 10.1017/CBO9780511618031. — [doi:10.1017/cbo9780511618031](https://doi.org/10.1017/cbo9780511618031) [find](https://scholar.google.com/scholar?q=Sarah+Lichtenstein+and+Paul+Slovic+%28eds%2E%29%2E+The+Construction+of+Preference%2E+Cambridge+University+Press%2C+2006%2E+doi%3A+10%2E1017%2FCBO9780511618031%2E+%E2%80%94+%5Bdoi%3A10%2E1017%2Fcbo9780511618031%5D%28https%3A%2F%2Fdoi%2Eorg%2F10%2E1017%2Fcbo)
223. Jinying Lin, Zhen Ma, Randy Gomez, Keisuke Nakamura, Bo He, and Guangliang Li. A Review on Interactive Reinforcement Learning From Human Social Feedback. IEEE Access, 8:120757–120765, 2020a. doi: 10.1109/ACCESS.2020.3006254. — [doi:10.1109/access.2020.3006254](https://doi.org/10.1109/access.2020.3006254) [find](https://scholar.google.com/scholar?q=Jinying+Lin%2C+Zhen+Ma%2C+Randy+Gomez%2C+Keisuke+Nakamura%2C+Bo+He%2C+and+Guangliang+Li%2E+A+Review+on+Interactive+Reinforcement+Learning+From+Human+Social+Feedback%2E+IEEE+Access%2C+8%3A120757%E2%80%93120765%2C+2020a%2E+doi%3A+10%2E1)
224. Stephanie Lin, Jacob Hilton, and Owain Evans. [TruthfulQA: Measuring How Models Mimic Human Falsehoods](https://md.research.elamur.ai/free/en/papers/lin-2022-truthfulqa-measuring-how-models-mimic/). In Proceedings of the Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) (ACL). Association for Computational Linguistics, 2022. doi: 10.18653/v1/2022.acl-long.…
225. Yijiong Lin, Jiancong Huang, Matthieu Zimmer, Yisheng Guan, Juan Rojas, and Paul Weng. Invariant Transform Experience Replay: Data Augmentation for Deep Reinforcement Learning. IEEE Robotics and Automation Letters, 5(4):6615–6622, 2020b. doi: 10.1109/LRA.2020.3013937. — [doi:10.1109/lra.2020.3013937](https://doi.org/10.1109/lra.2020.3013937) [find](https://scholar.google.com/scholar?q=Yijiong+Lin%2C+Jiancong+Huang%2C+Matthieu+Zimmer%2C+Yisheng+Guan%2C+Juan+Rojas%2C+and+Paul+Weng%2E+Invariant+Transform+Experience+Replay%3A+Data+Augmentation+for+Deep+Reinforcement+Learning%2E+IEEE+Robotics+and+Autom)
226. David Lindner and Mennatallah El-Assady. Humans are not Boltzmann Distributions: Challenges and Opportunities for Modelling Human Feedback and Interaction in Reinforcement Learning. In IJCAI-ECAI 2022 Workshop on Communication in Human-AI Interaction, 2022. [find](https://scholar.google.com/scholar?q=David+Lindner+and+Mennatallah+El%2DAssady%2E+Humans+are+not+Boltzmann+Distributions%3A+Challenges+and+Opportunities+for+Modelling+Human+Feedback+and+Interaction+in+Reinforcement+Learning%2E+In+IJCAI%2DECAI+2022)
227. David Lindner, Rohin Shah, Pieter Abbeel, and Anca Dragan. Learning What To Do by Simulating the Past. In Proceedings of the International Conference on Learning Representations (ICLR), 2021a. URL [https://openreview.net/forum?id=kBVJ2NtiY-](https://openreview.net/forum?id=kBVJ2NtiY-). [↗](https://openreview.net/forum?id=kBVJ2NtiY-)
228. David Lindner, Matteo Turchetta, Sebastian Tschiatschek, Kamil Ciosek, and Andreas Krause. Information Directed Reward Learning for Reinforcement Learning. In Advances in Neural Information Processing Systems (NeurIPS), volume 34. Curran Associates, Inc., 2021b. URL [https://proceedings.neurips](https://proceedings.neurips). cc/p… [↗](https://proceedings.neurips)
229. Hao Liu, Carmelo Sferrazza, and Pieter Abbeel. [Chain of Hindsight aligns Language Models with Feedback](https://md.research.elamur.ai/free/en/papers/liu-2023-chain-hindsight-aligns-language-models/). In Proceedings of the International Conference on Learning Representations (ICLR), 2024a. URL [https://openreview.net/forum?id=6xfe4IVcOu](https://openreview.net/forum?id=6xfe4IVcOu). [↗](https://openreview.net/forum?id=6xfe4IVcOu)
230. Runze Liu, Fengshuo Bai, Yali Du, and Yaodong Yang. Meta-Reward-Net: Implicitly Differentiable Reward Learning for Preference-based Reinforcement Learning. In Advances in Neural Information Processing Systems (NeurIPS), 2022. URL [https://openreview.net/forum?id=OZKBReUF-wX](https://openreview.net/forum?id=OZKBReUF-wX). [↗](https://openreview.net/forum?id=OZKBReUF-wX)
231. Runze Liu, Yali Du, Fengshuo Bai, Jiafei Lyu, and Xiu Li. Zero-shot Cross-task Preference Alignment for Offline RL via Optimal Transport. In NeurIPS 2023 Workshop Optimal Transport and Machine Learning, 2023a. URL [https://openreview.net/forum?id=fwXj1c6faX](https://openreview.net/forum?id=fwXj1c6faX). [↗](https://openreview.net/forum?id=fwXj1c6faX)
232. Tianqi Liu, Yao Zhao, Rishabh Joshi, Misha Khalman, Mohammad Saleh, Peter J. Liu, and Jialu Liu. [Statistical Rejection Sampling Improves Preference Optimization](https://md.research.elamur.ai/free/en/papers/liu-2023-statistical-rejection-sampling-improves-preference/). In Proceedings of the International Conference on Learning Representations (ICLR), 2024b. URL [https://openreview.net/forum?id=xbjSwwrQOe](https://openreview.net/forum?id=xbjSwwrQOe). [↗](https://openreview.net/forum?id=xbjSwwrQOe)
233. Yi Liu, Gaurav Datta, Ellen Novoseller, and Daniel S. Brown. Efficient Preference-Based Reinforcement Learning Using Learned Dynamics Models. In Proceedings of the IEEE International Conference on Robotics and Automation (ICRA), 2023b. doi: 10.1109/ICRA48891.2023.10161081. — [doi:10.1109/icra48891.2023.10161081](https://doi.org/10.1109/icra48891.2023.10161081) [find](https://scholar.google.com/scholar?q=Yi+Liu%2C+Gaurav+Datta%2C+Ellen+Novoseller%2C+and+Daniel+S%2E+Brown%2E+Efficient+Preference%2DBased+Reinforcement+Learning+Using+Learned+Dynamics+Models%2E+In+Proceedings+of+the+IEEE+International+Conference+on+Rob)
234. Yiheng Liu, Tianle Han, Siyuan Ma, Jiayue Zhang, Yuanyuan Yang, Jiaming Tian, Hao He, Antong Li, Mengshen He, Zhengliang Liu, Zihao Wu, Lin Zhao, Dajiang Zhu, Xiang Li, Ning Qiang, Dingang Shen, Tianming Liu, and Bao Ge. Summary of ChatGPT-Related research and perspective towards the future of large… — [Summary of ChatGPT-Related Research and Perspective Towards the Future of Large Language Models](https://md.research.elamur.ai/free/en/papers/liu-2023-summary-chatgpt-related-research-perspective/)
235. Dylan P. Losey and Marcia K. O’Malley. Including Uncertainty when Learning from Human Corrections. In Proceedings of the Conference on Robot Learning (CoRL). PMLR, 2018. URL [https://proceedings.mlr.press/v87/losey18a.html](https://proceedings.mlr.press/v87/losey18a.html). [↗](https://proceedings.mlr.press/v87/losey18a.html)
236. Dylan P. Losey, Andrea Bajcsy, Marcia K. O’Malley, and Anca D. Dragan. Physical interaction as communication: Learning robot objectives online from human corrections. The International Journal of Robotics Research, 41(1):20–44, 2022. doi: 10.1177/02783649211050958. — [doi:10.1177/02783649211050958](https://doi.org/10.1177/02783649211050958) [find](https://scholar.google.com/scholar?q=Dylan+P%2E+Losey%2C+Andrea+Bajcsy%2C+Marcia+K%2E+O%E2%80%99Malley%2C+and+Anca+D%2E+Dragan%2E+Physical+interaction+as+communication%3A+Learning+robot+objectives+online+from+human+corrections%2E+The+International+Journal+of+Robo)
237. R. Duncan Luce. Individual Choice Behavior: A Theoretical Analysis. John Wiley, 1959. [find](https://scholar.google.com/scholar?q=R%2E+Duncan+Luce%2E+Individual+Choice+Behavior%3A+A+Theoretical+Analysis%2E+John+Wiley%2C+1959%2E)
238. Jelena Luketina, Nantas Nardelli, Gregory Farquhar, Jakob Foerster, Jacob Andreas, Edward Grefenstette, Shimon Whiteson, and Tim Rocktäschel. [A Survey of Reinforcement Learning Informed by Natural Language](https://md.research.elamur.ai/free/en/papers/luketina-2019-survey-reinforcement-learning-informed-natural/). In Proceedings of the International Joint Conference on Artificial Intelligence (IJCAI). Inte…
239. Jianlan Luo, Perry Dong, Yuexiang Zhai, Yi Ma, and Sergey Levine. RLIF: Interactive Imitation Learning as Reinforcement Learning. In Proceedings of the International Conference on Learning Representations (ICLR), 2024. URL [https://openreview.net/forum?id=oLLZhbBSOU](https://openreview.net/forum?id=oLLZhbBSOU). [↗](https://openreview.net/forum?id=oLLZhbBSOU)
240. Chang Ma, Junlei Zhang, Zhihao Zhu, Cheng Yang, Yujiu Yang, Yaohui Jin, Zhenzhong Lan, Lingpeng Kong, and Junxian He. [AgentBoard: An Analytical Evaluation Board of Multi-turn LLM Agents](https://md.research.elamur.ai/free/en/papers/ma-2024-agentboard-analytical-evaluation-board-multi/). In ICLR 2024 Workshop on Large Language Model (LLM) Agents, 2024a. URL [https://openreview.net/](https://openreview.net/) forum?id=09Y7J22N9… [↗](https://openreview.net/)
241. Yecheng Jason Ma, William Liang, Guanzhi Wang, De-An Huang, Osbert Bastani, Dinesh Jayaraman, Yuke Zhu, Linxi Fan, and Anima Anandkumar. [Eureka: Human-Level Reward Design via Coding Large Language Models](https://md.research.elamur.ai/free/en/papers/ma-2023-eureka-human-level-reward-design/). In Proceedings of the International Conference on Learning Representations (ICLR), 2024b. URL h…
242. James MacGlashan, Mark K. Ho, Robert Loftin, Bei Peng, Guan Wang, David L. Roberts, Matthew E. Taylor, and Michael L. Littman. Interactive Learning from Policy-Dependent Human Feedback. In Proceedings of the International Conference on Machine Learning (ICML). PMLR, 2017. URL [https://proceedings.ml…](https://proceedings.ml…) [↗](https://proceedings.ml…)
243. Richard Maclin, Jude Shavlik, Lisa Torrey, Trevor Walker, and Edward Wild. Giving advice about preferred actions to reinforcement learners via knowledge-based kernel regression. In Proceedings of the AAAI Conference on Artificial Intelligence. AAAI Press, 2005. URL [https://aaai.org/papers/00819-AAAI…](https://aaai.org/papers/00819-AAAI…) [↗](https://aaai.org/papers/00819-AAAI…)
244. Monika Mandl, Alexander Felfernig, Erich Teppan, and Monika Schubert. Consumer decision making in knowledge-based recommendation. Journal of Intelligent Information Systems, 37(1):1–22, 2011. doi: 10.1007/s10844-010-0134-3. — [doi:10.1007/s10844-010-0134-3](https://doi.org/10.1007/s10844-010-0134-3) [find](https://scholar.google.com/scholar?q=Monika+Mandl%2C+Alexander+Felfernig%2C+Erich+Teppan%2C+and+Monika+Schubert%2E+Consumer+decision+making+in+knowledge%2Dbased+recommendation%2E+Journal+of+Intelligent+Information+Systems%2C+37%281%29%3A1%E2%80%9322%2C+2011%2E+doi%3A+10%2E)
245. Daniel Marta, Simon Holk, Christian Pek, Jana Tumova, and Iolanda Leite. Aligning Human Preferences with Baseline Objectives in Reinforcement Learning. In Proceedings of the IEEE International Conference on Robotics and Automation (ICRA), 2023. doi: 10.1109/ICRA48891.2023.10161261. — [doi:10.1109/icra48891.2023.10161261](https://doi.org/10.1109/icra48891.2023.10161261) [find](https://scholar.google.com/scholar?q=Daniel+Marta%2C+Simon+Holk%2C+Christian+Pek%2C+Jana+Tumova%2C+and+Iolanda+Leite%2E+Aligning+Human+Preferences+with+Baseline+Objectives+in+Reinforcement+Learning%2E+In+Proceedings+of+the+IEEE+International+Confere)
246. Pascal Massart and Élodie Nédélec. Risk bounds for statistical learning. The Annals of Statistics, 34(5): 2326–2366, 2006. doi: 10.1214/009053606000000786. — [doi:10.1214/009053606000000786](https://doi.org/10.1214/009053606000000786) [find](https://scholar.google.com/scholar?q=Pascal+Massart+and+%C3%89lodie+N%C3%A9d%C3%A9lec%2E+Risk+bounds+for+statistical+learning%2E+The+Annals+of+Statistics%2C+34%285%29%3A+2326%E2%80%932366%2C+2006%2E+doi%3A+10%2E1214%2F009053606000000786%2E+%E2%80%94+%5Bdoi%3A10%2E1214%2F009053606000000786%5D%28https%3A%2F%2Fd)
247. Lev E. McKinney, Yawen Duan, David Krueger, and Adam Gleave. On The Fragility of Learned Reward Functions. In NeurIPS 2022 Workshop on Deep Reinforcement Learning, 2022. URL [https://openreview.net/forum?id=9gj9vXfeS-y](https://openreview.net/forum?id=9gj9vXfeS-y). [↗](https://openreview.net/forum?id=9gj9vXfeS-y)
248. Shaunak A. Mehta and Dylan P. Losey. Unified Learning from Demonstrations, Corrections, and Preferences during Physical Human-Robot Interaction. ACM Transactions on Human-Robot Interaction, 2023. doi: 10.1145/3623384. — [doi:10.1145/3623384](https://doi.org/10.1145/3623384) [find](https://scholar.google.com/scholar?q=Shaunak+A%2E+Mehta+and+Dylan+P%2E+Losey%2E+Unified+Learning+from+Demonstrations%2C+Corrections%2C+and+Preferences+during+Physical+Human%2DRobot+Interaction%2E+ACM+Transactions+on+Human%2DRobot+Interaction%2C+2023%2E+doi%3A)
249. Viraj Mehta, Vikramjeet Das, Ojash Neopane, Yijia Dai, Ilija Bogunovic, Jeff Schneider, and Willie Neiswanger. Sample Efficient Reinforcement Learning from Human Feedback via Active Exploration, 2023. arXiv: 2312.00267. preprint. [find](https://scholar.google.com/scholar?q=Viraj+Mehta%2C+Vikramjeet+Das%2C+Ojash+Neopane%2C+Yijia+Dai%2C+Ilija+Bogunovic%2C+Jeff+Schneider%2C+and+Willie+Neiswanger%2E+Sample+Efficient+Reinforcement+Learning+from+Human+Feedback+via+Active+Exploration%2C+2023%2E)
250. Jorge Mendez, Shashank Shivkumar, and Eric Eaton. Lifelong Inverse Reinforcement Learning. In Advances in Neural Information Processing Systems (NIPS), volume 31. Curran Associates, Inc., 2018. URL [https://papers.nips.cc/paper/2018/hash/2d969e2cee8cfa07ce7ca0bb13c7a36d-Abstract.html](https://papers.nips.cc/paper/2018/hash/2d969e2cee8cfa07ce7ca0bb13c7a36d-Abstract.html). [↗](https://papers.nips.cc/paper/2018/hash/2d969e2cee8cfa07ce7ca0bb13c7a36d-Abstract.html)
251. Jacob Menick, Maja Trebacz, Vladimir Mikulik, John Aslanides, Francis Song, Martin Chadwick, Mia Glaese, Susannah Young, Lucy Campbell-Gillingham, Geoffrey Irving, and Nat McAleese. [Teaching language models to support answers with verified quotes](https://md.research.elamur.ai/free/en/papers/menick-2022-teaching-language-models-support-answers/), 2022. arXiv: 2203.11147. preprint.
252. Katherine Metcalf, Miguel Sarabia, Natalie Mackraz, and Barry-John Theobald. Sample-Efficient Preference-based Reinforcement Learning with Dynamics Aware Rewards. In Proceedings of the Conference on Robot Learning (CoRL). PMLR, 2023. URL [https://proceedings.mlr.press/v229/metcalf23a.html](https://proceedings.mlr.press/v229/metcalf23a.html). [↗](https://proceedings.mlr.press/v229/metcalf23a.html)
253. Yannick Metz, David Lindner, Raphaël Baur, Daniel A. Keim, and Mennatallah El-Assady. RLHF-Blender: A Configurable Interactive Interface for Learning from Diverse Human Feedback. In ICML 2023 Workshop on Interactive Learning with Implicit Human Feedback, 2023. URL [https://openreview.net/forum?id=](https://openreview.net/forum?id=) Jv… [↗](https://openreview.net/forum?id=)
254. Stephanie Milani, Anssi Kanervisto, Karolis Ramanauskas, Sander Schulhoff, Brandon Houghton, Sharada Mohanty, Byron Galbraith, Ke Chen, Yan Song, Tianze Zhou, Bingquan Yu, He Liu, Kai Guan, Yujing Hu, Tangjie Lv, Federico Malato, Florian Leopold, Amogh Raut, Ville Hautamäki, Andrew Melnik, Shu Ishid… [find](https://scholar.google.com/scholar?q=Stephanie+Milani%2C+Anssi+Kanervisto%2C+Karolis+Ramanauskas%2C+Sander+Schulhoff%2C+Brandon+Houghton%2C+Sharada+Mohanty%2C+Byron+Galbraith%2C+Ke+Chen%2C+Yan+Song%2C+Tianze+Zhou%2C+Bingquan+Yu%2C+He+Liu%2C+Kai+Guan%2C+Yujing+Hu%2C)
255. PMLR, 2022. URL [https://proceedings.mlr.press/v220/milani22a.html](https://proceedings.mlr.press/v220/milani22a.html). [↗](https://proceedings.mlr.press/v220/milani22a.html)
256. Stephanie Milani, Nicholay Topin, Manuela Veloso, and Fei Fang. Explainable Reinforcement Learning: A Survey and Comparative Review. ACM Computing Surveys, 2023. doi: 10.1145/3616864. — [doi:10.1145/3616864](https://doi.org/10.1145/3616864) [find](https://scholar.google.com/scholar?q=Stephanie+Milani%2C+Nicholay+Topin%2C+Manuela+Veloso%2C+and+Fei+Fang%2E+Explainable+Reinforcement+Learning%3A+A+Survey+and+Comparative+Review%2E+ACM+Computing+Surveys%2C+2023%2E+doi%3A+10%2E1145%2F3616864%2E+%E2%80%94+%5Bdoi%3A10%2E1145%2F3)
257. Smitha Milli and Anca D. Dragan. Literal or Pedagogic Human? Analyzing Human Model Misspecification in Objective Learning. In Proceedings of the Conference on Uncertainty in Artificial Intelligence (UAI). PMLR, 2020. URL [https://proceedings.mlr.press/v115/milli20a.html](https://proceedings.mlr.press/v115/milli20a.html). [↗](https://proceedings.mlr.press/v115/milli20a.html)
258. Sören Mindermann and Stuart Armstrong. Occam’ s razor is insufficient to infer the preferences of irrational agents. In Advances in Neural Information Processing Systems (NIPS), volume 31. Curran Associates, Inc., 2018. URL [https://proceedings.neurips.cc/paper/2018/hash/](https://proceedings.neurips.cc/paper/2018/hash/) d89a66c7c80a29b1bdbab0f2a1a9… [↗](https://proceedings.neurips.cc/paper/2018/hash/)
259. Sören Mindermann, Rohin Shah, Adam Gleave, and Dylan Hadfield-Menell. Active Inverse Reward Design. In ICML 2018 Workshop on Goals in Reinforcement Learning, 2018. [find](https://scholar.google.com/scholar?q=S%C3%B6ren+Mindermann%2C+Rohin+Shah%2C+Adam+Gleave%2C+and+Dylan+Hadfield%2DMenell%2E+Active+Inverse+Reward+Design%2E+In+ICML+2018+Workshop+on+Goals+in+Reinforcement+Learning%2C+2018%2E)
260. Abhilash Mishra. AI Alignment and Social Choice: Fundamental Limitations and Policy Implications, 2023. arXiv: 2310.16048. preprint. — [arXiv:2310.16048](https://arxiv.org/abs/2310.16048) [find](https://scholar.google.com/scholar?q=Abhilash+Mishra%2E+AI+Alignment+and+Social+Choice%3A+Fundamental+Limitations+and+Policy+Implications%2C+2023%2E+arXiv%3A+2310%2E16048%2E+preprint%2E+%E2%80%94+%5BarXiv%3A2310%2E16048%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2310%2E16048%29)
261. Volodymyr Mnih, Koray Kavukcuoglu, David Silver, Andrei A. Rusu, Joel Veness, Marc G. Bellemare, Alex Graves, Martin Riedmiller, Andreas K. Fidjeland, Georg Ostrovski, Stig Petersen, Charles Beattie, Amir Sadik, Ioannis Antonoglou, Helen King, Dharshan Kumaran, Daan Wierstra, Shane Legg, and Demis H… [find](https://scholar.google.com/scholar?q=Volodymyr+Mnih%2C+Koray+Kavukcuoglu%2C+David+Silver%2C+Andrei+A%2E+Rusu%2C+Joel+Veness%2C+Marc+G%2E+Bellemare%2C+Alex+Graves%2C+Martin+Riedmiller%2C+Andreas+K%2E+Fidjeland%2C+Georg+Ostrovski%2C+Stig+Petersen%2C+Charles+Beattie%2C+)
262. Ted Moskovitz, Aaditya K. Singh, D. J. Strouse, Tuomas Sandholm, Ruslan Salakhutdinov, Anca Dragan, and Stephen Marcus McAleer. Confronting Reward Model Overoptimization with Constrained RLHF. In Proceedings of the International Conference on Learning Representations (ICLR), 2024. URL https://openr… [find](https://scholar.google.com/scholar?q=Ted+Moskovitz%2C+Aaditya+K%2E+Singh%2C+D%2E+J%2E+Strouse%2C+Tuomas+Sandholm%2C+Ruslan+Salakhutdinov%2C+Anca+Dragan%2C+and+Stephen+Marcus+McAleer%2E+Confronting+Reward+Model+Overoptimization+with+Constrained+RLHF%2E+In+Proc)
263. Weiyan Mu and Shifeng Xiong. On Huber’s contaminated model. Journal of Complexity, 77:101745, 2023. doi: 10.1016/j.jco.2023.101745. — [doi:10.1016/j.jco.2023.101745](https://doi.org/10.1016/j.jco.2023.101745) [find](https://scholar.google.com/scholar?q=Weiyan+Mu+and+Shifeng+Xiong%2E+On+Huber%E2%80%99s+contaminated+model%2E+Journal+of+Complexity%2C+77%3A101745%2C+2023%2E+doi%3A+10%2E1016%2Fj%2Ejco%2E2023%2E101745%2E+%E2%80%94+%5Bdoi%3A10%2E1016%2Fj%2Ejco%2E2023%2E101745%5D%28https%3A%2F%2Fdoi%2Eorg%2F10%2E1016%2Fj%2Ejco%2E2023)
264. Remi Munos, Michal Valko, Daniele Calandriello, Mohammad Gheshlaghi Azar, Mark Rowland, Zhaohan Daniel Guo, Yunhao Tang, Matthieu Geist, Thomas Mesnard, Côme Fiegel, Andrea Michi, Marco Selvi, Sertan Girgin, Nikola Momchev, Olivier Bachem, Daniel J. Mankowitz, Doina Precup, and Bilal Piot. Nash Lear… [find](https://scholar.google.com/scholar?q=Remi+Munos%2C+Michal+Valko%2C+Daniele+Calandriello%2C+Mohammad+Gheshlaghi+Azar%2C+Mark+Rowland%2C+Zhaohan+Daniel+Guo%2C+Yunhao+Tang%2C+Matthieu+Geist%2C+Thomas+Mesnard%2C+C%C3%B4me+Fiegel%2C+Andrea+Michi%2C+Marco+Selvi%2C+Sertan+)
265. Vivek Myers, Erdem Biyik, Nima Anari, and Dorsa Sadigh. Learning Multimodal Rewards from Rankings. In Proceedings of the Conference on Robot Learning (CoRL). PMLR, 2021. URL [https://proceedings.mlr.press/v164/myers22a.html](https://proceedings.mlr.press/v164/myers22a.html). [↗](https://proceedings.mlr.press/v164/myers22a.html)
266. Vivek Myers, Erdem Bıyık, and Dorsa Sadigh. Active Reward Learning from Online Preferences. In Proceedings of the IEEE International Conference on Robotics and Automation (ICRA), 2023. doi: 10.1109/ICRA48891.2023.10160439. — [doi:10.1109/icra48891.2023.10160439](https://doi.org/10.1109/icra48891.2023.10160439) [find](https://scholar.google.com/scholar?q=Vivek+Myers%2C+Erdem+B%C4%B1y%C4%B1k%2C+and+Dorsa+Sadigh%2E+Active+Reward+Learning+from+Online+Preferences%2E+In+Proceedings+of+the+IEEE+International+Conference+on+Robotics+and+Automation+%28ICRA%29%2C+2023%2E+doi%3A+10%2E1109%2FIC)
267. Suraj Nair, Eric Mitchell, Kevin Chen, Brian Ichter, Silvio Savarese, and Chelsea Finn. Learning Language-Conditioned Robot Behavior from Offline Data and Crowd-Sourced Annotation. In Proceedings of the Conference on Robot Learning (CoRL). PMLR, 2021. URL [https://proceedings.mlr.press/v164/](https://proceedings.mlr.press/v164/) nair22a.… [↗](https://proceedings.mlr.press/v164/)
268. Anis Najar and Mohamed Chetouani. Reinforcement Learning With Human Advice: A Survey. Frontiers in Robotics and AI, 8:584075, 2021. doi: 10.3389/frobt.2021.584075. — [doi:10.3389/frobt.2021.584075](https://doi.org/10.3389/frobt.2021.584075) [find](https://scholar.google.com/scholar?q=Anis+Najar+and+Mohamed+Chetouani%2E+Reinforcement+Learning+With+Human+Advice%3A+A+Survey%2E+Frontiers+in+Robotics+and+AI%2C+8%3A584075%2C+2021%2E+doi%3A+10%2E3389%2Ffrobt%2E2021%2E584075%2E+%E2%80%94+%5Bdoi%3A10%2E3389%2Ffrobt%2E2021%2E584075%5D%28ht)
269. Reiichiro Nakano, Jacob Hilton, Suchir Balaji, Jeff Wu, Long Ouyang, Christina Kim, Christopher Hesse, Shantanu Jain, Vineet Kosaraju, William Saunders, Xu Jiang, Karl Cobbe, Tyna Eloundou, Gretchen Krueger, Kevin Button, Matthew Knight, Benjamin Chess, and John Schulman. WebGPT: Browser-assisted qu… — [WebGPT: Browser-assisted question-answering with human feedback](https://md.research.elamur.ai/free/en/papers/nakano-2021-webgpt-browser-assisted-question-answering/)
270. Sanmit Narvekar, Bei Peng, Matteo Leonetti, Jivko Sinapov, Matthew E. Taylor, and Peter Stone. Curriculum Learning for Reinforcement Learning Domains: A Framework and Survey. Journal of Machine Learning Research, 21(181):1–50, 2020. ISSN 1533-7928. URL [http://jmlr.org/papers/v21/20-212.html](http://jmlr.org/papers/v21/20-212.html). [↗](http://jmlr.org/papers/v21/20-212.html)
271. Gergely Neu and Csaba Szepesvári. Training parsers by inverse reinforcement learning. Machine Learning, 77(2):303–337, 2009. doi: 10.1007/s10994-009-5110-1. — [doi:10.1007/s10994-009-5110-1](https://doi.org/10.1007/s10994-009-5110-1) [find](https://scholar.google.com/scholar?q=Gergely+Neu+and+Csaba+Szepesv%C3%A1ri%2E+Training+parsers+by+inverse+reinforcement+learning%2E+Machine+Learning%2C+77%282%29%3A303%E2%80%93337%2C+2009%2E+doi%3A+10%2E1007%2Fs10994%2D009%2D5110%2D1%2E+%E2%80%94+%5Bdoi%3A10%2E1007%2Fs10994%2D009%2D5110%2D1%5D%28https%3A%2F%2Fd)
272. Andrew Y. Ng and Stuart J. Russell. Algorithms for Inverse Reinforcement Learning. In Proceedings of the International Conference on Machine Learning (ICML). Morgan Kaufmann Publishers Inc., 2000. [find](https://scholar.google.com/scholar?q=Andrew+Y%2E+Ng+and+Stuart+J%2E+Russell%2E+Algorithms+for+Inverse+Reinforcement+Learning%2E+In+Proceedings+of+the+International+Conference+on+Machine+Learning+%28ICML%29%2E+Morgan+Kaufmann+Publishers+Inc%2E%2C+2000%2E)
273. Andrew Y. Ng, Daishi Harada, and Stuart J. Russell. Policy Invariance Under Reward Transformations: Theory and Application to Reward Shaping. In Proceedings of the International Conference on Machine Learning (ICML). Morgan Kaufmann Publishers Inc., 1999. [find](https://scholar.google.com/scholar?q=Andrew+Y%2E+Ng%2C+Daishi+Harada%2C+and+Stuart+J%2E+Russell%2E+Policy+Invariance+Under+Reward+Transformations%3A+Theory+and+Application+to+Reward+Shaping%2E+In+Proceedings+of+the+International+Conference+on+Machine+)
274. Richard Ngo, Lawrence Chan, and Sören Mindermann. [The Alignment Problem from a Deep Learning Perspective](https://md.research.elamur.ai/free/en/papers/ngo-2022-alignment-problem-deep-learning-perspective/). In Proceedings of the International Conference on Learning Representations (ICLR), 2024. URL [https://openreview.net/forum?id=fh8EYKFKns](https://openreview.net/forum?id=fh8EYKFKns). [↗](https://openreview.net/forum?id=fh8EYKFKns)
275. Ellen Novoseller, Yibing Wei, Yanan Sui, Yisong Yue, and Joel Burdick. Dueling Posterior Sampling for Preference-Based Reinforcement Learning. In Proceedings of the Conference on Uncertainty in Artificial Intelligence (UAI). PMLR, 2020. URL [https://proceedings.mlr.press/v124/novoseller20a.html](https://proceedings.mlr.press/v124/novoseller20a.html). [↗](https://proceedings.mlr.press/v124/novoseller20a.html)
276. Ellen Novoseller, Vinicius G. Goecks, David Watkins, Josh Miller, and Nicholas R. Waytowich. DIP-RL: Demonstration-Inferred Preference Learning in Minecraft. In ICML 2023 Workshop The Many Facets of Preference-Based Learning, 2023. URL [https://openreview.net/forum?id=lpouLcLhX6](https://openreview.net/forum?id=lpouLcLhX6). [↗](https://openreview.net/forum?id=lpouLcLhX6)
277. OpenAI. Introducing ChatGPT, 2022. URL [https://openai.com/index/chatgpt/](https://openai.com/index/chatgpt/). (accessed 2024-05-28). [↗](https://openai.com/index/chatgpt/)
278. OpenAI. [GPT-4 Technical Report](https://md.research.elamur.ai/free/en/papers/openai-2023-gpt-4-technical-report/). Technical report, OpenAI, 2023. URL [https://cdn.openai.com/](https://cdn.openai.com/) papers/gpt-4.pdf. [↗](https://cdn.openai.com/)
279. Takayuki Osa, Joni Pajarinen, Gerhard Neumann, J. Andrew Bagnell, Pieter Abbeel, and Jan Peters. An Algorithmic Perspective on Imitation Learning. Foundations and Trends® in Robotics, 7(1-2):1–179, 2018. doi: 10.1561/2300000053. — [doi:10.1561/2300000053](https://doi.org/10.1561/2300000053) [find](https://scholar.google.com/scholar?q=Takayuki+Osa%2C+Joni+Pajarinen%2C+Gerhard+Neumann%2C+J%2E+Andrew+Bagnell%2C+Pieter+Abbeel%2C+and+Jan+Peters%2E+An+Algorithmic+Perspective+on+Imitation+Learning%2E+Foundations+and+Trends%C2%AE+in+Robotics%2C+7%281%2D2%29%3A1%E2%80%93179%2C+20)
280. Ian Osband, Zheng Wen, Seyed Mohammad Asghari, Vikranth Dwaracherla, Morteza Ibrahimi, Xiuyuan Lu, and Benjamin Van Roy. Epistemic Neural Networks. In Advances in Neural Information Processing Systems (NeurIPS), volume 36, 2023. URL [https://proceedings.neurips.cc/paper_files/paper/](https://proceedings.neurips.cc/paper_files/paper/) 2023/hash/07fbde9… [↗](https://proceedings.neurips.cc/paper_files/paper/)
281. Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, John Schulman, Jacob Hilton, Fraser Kelton, Luke Miller, Maddie Simens, Amanda Askell, Peter Welinder, Paul F. Christiano, Jan Leike, and Ryan Lowe. Training… — [Training language models to follow instructions with human feedback](https://md.research.elamur.ai/free/en/papers/ouyang-2022-training-language-models-follow-instructions/)
282. Cosmin Paduraru. Off-Policy Evaluation in Markov Decision Processes. PhD thesis, McGill University, 2013. URL [https://escholarship.mcgill.ca/concern/theses/p8418r74h](https://escholarship.mcgill.ca/concern/theses/p8418r74h). [↗](https://escholarship.mcgill.ca/concern/theses/p8418r74h)
283. Malayandi Palan, Gleb Shevchuk, Nicholas Charles Landolfi, and Dorsa Sadigh. Learning Reward Functions by Integrating Human Demonstrations and Preferences. In Proceedings of Robotics: Science and Systems (RSS), volume 15, 2019. URL [http://www.roboticsproceedings.org/rss15/p23.html](http://www.roboticsproceedings.org/rss15/p23.html). [↗](http://www.roboticsproceedings.org/rss15/p23.html)
284. Chanwoo Park, Mingyang Liu, Dingwen Kong, Kaiqing Zhang, and Asuman E. Ozdaglar. RLHF from Heterogeneous Feedback via Personalization and Preference Aggregation. In ICML 2024 Workshop on Aligning Reinforcement Learning Experimentalists and Theorists (ARLET), 2024. URL [https://openreview.net/forum?i…](https://openreview.net/forum?i…) [↗](https://openreview.net/forum?i…)
285. Jongjin Park, Younggyo Seo, Jinwoo Shin, Honglak Lee, Pieter Abbeel, and Kimin Lee. SURF: Semi-supervised Reward Learning with Data Augmentation for Feedback-efficient Preference-based Reinforcement Learning. In Proceedings of the International Conference on Learning Representations (ICLR), 2022. UR… [find](https://scholar.google.com/scholar?q=Jongjin+Park%2C+Younggyo+Seo%2C+Jinwoo+Shin%2C+Honglak+Lee%2C+Pieter+Abbeel%2C+and+Kimin+Lee%2E+SURF%3A+Semi%2Dsupervised+Reward+Learning+with+Data+Augmentation+for+Feedback%2Defficient+Preference%2Dbased+Reinforcement+L)
286. Robert Pinsler, Riad Akrour, Takayuki Osa, Jan Peters, and Gerhard Neumann. Sample and Feedback Efficient Hierarchical Reinforcement Learning from Human Preferences. In Proceedings of the IEEE International Conference on Robotics and Automation (ICRA), 2018. doi: 10.1109/ICRA.2018.8460907. — [doi:10.1109/icra.2018.8460907](https://doi.org/10.1109/icra.2018.8460907) [find](https://scholar.google.com/scholar?q=Robert+Pinsler%2C+Riad+Akrour%2C+Takayuki+Osa%2C+Jan+Peters%2C+and+Gerhard+Neumann%2E+Sample+and+Feedback+Efficient+Hierarchical+Reinforcement+Learning+from+Human+Preferences%2E+In+Proceedings+of+the+IEEE+Interna)
287. André Susano Pinto, Alexander Kolesnikov, Yuge Shi, Lucas Beyer, and Xiaohua Zhai. Tuning Computer Vision Models With Task Rewards. In Proceedings of the International Conference on Machine Learning (ICML). PMLR, 2023. URL [https://proceedings.mlr.press/v202/susano-pinto23a.html](https://proceedings.mlr.press/v202/susano-pinto23a.html). [↗](https://proceedings.mlr.press/v202/susano-pinto23a.html)
288. R. L. Plackett. The Analysis of Permutations. Journal of the Royal Statistical Society. Series C (Applied Statistics), 24(2):193–202, 1975. doi: 10.2307/2346567. — [doi:10.2307/2346567](https://doi.org/10.2307/2346567) [find](https://scholar.google.com/scholar?q=R%2E+L%2E+Plackett%2E+The+Analysis+of+Permutations%2E+Journal+of+the+Royal+Statistical+Society%2E+Series+C+%28Applied+Statistics%29%2C+24%282%29%3A193%E2%80%93202%2C+1975%2E+doi%3A+10%2E2307%2F2346567%2E+%E2%80%94+%5Bdoi%3A10%2E2307%2F2346567%5D%28https%3A%2F%2Fdoi%2Eor)
289. Alina Pommeranz, Joost Broekens, Pascal Wiggers, Willem-Paul Brinkman, and Catholijn M. Jonker. Designing interfaces for explicit preference elicitation: A user-centered investigation of preference representation and elicitation process. User Modeling and User-Adapted Interaction, 22(4):357–397, 201… — [doi:10.1007/s11257-011-9116-6](https://doi.org/10.1007/s11257-011-9116-6) [find](https://scholar.google.com/scholar?q=Alina+Pommeranz%2C+Joost+Broekens%2C+Pascal+Wiggers%2C+Willem%2DPaul+Brinkman%2C+and+Catholijn+M%2E+Jonker%2E+Designing+interfaces+for+explicit+preference+elicitation%3A+A+user%2Dcentered+investigation+of+preference+re)
290. Benjamin Poole and Minwoo Lee. Towards interactive reinforcement learning with intrinsic feedback. Neurocomputing, pp. 127628, 2024. doi: 10.1016/j.neucom.2024.127628. — [doi:10.1016/j.neucom.2024.127628](https://doi.org/10.1016/j.neucom.2024.127628) [find](https://scholar.google.com/scholar?q=Benjamin+Poole+and+Minwoo+Lee%2E+Towards+interactive+reinforcement+learning+with+intrinsic+feedback%2E+Neurocomputing%2C+pp%2E+127628%2C+2024%2E+doi%3A+10%2E1016%2Fj%2Eneucom%2E2024%2E127628%2E+%E2%80%94+%5Bdoi%3A10%2E1016%2Fj%2Eneucom%2E2024%2E127)
291. Doina Precup, Richard S. Sutton, and Satinder P. Singh. Eligibility Traces for Off-Policy Policy Evaluation. In Proceedings of the International Conference on Machine Learning (ICML). Morgan Kaufmann Publishers Inc., 2000. [find](https://scholar.google.com/scholar?q=Doina+Precup%2C+Richard+S%2E+Sutton%2C+and+Satinder+P%2E+Singh%2E+Eligibility+Traces+for+Off%2DPolicy+Policy+Evaluation%2E+In+Proceedings+of+the+International+Conference+on+Machine+Learning+%28ICML%29%2E+Morgan+Kaufmann+)
292. Erika Puiutta and Eric M. S. P. Veith. Explainable Reinforcement Learning: A Survey. In Proceedings of Machine Learning and Knowledge Extraction (CD-MAKE). Springer International Publishing, 2020. doi: 10.1007/978-3-030-57321-8_5. — [doi:10.1007/978-3-030-57321-8_5](https://doi.org/10.1007/978-3-030-57321-8_5) [find](https://scholar.google.com/scholar?q=Erika+Puiutta+and+Eric+M%2E+S%2E+P%2E+Veith%2E+Explainable+Reinforcement+Learning%3A+A+Survey%2E+In+Proceedings+of+Machine+Learning+and+Knowledge+Extraction+%28CD%2DMAKE%29%2E+Springer+International+Publishing%2C+2020%2E+doi)
293. Junqi Qian, Paul Weng, and Chenmien Tan. Learning Rewards to Optimize Global Performance Metrics in Deep Reinforcement Learning. In Proceedings of the International Conference on Autonomous Agents and Multiagent Systems (AAMAS). IFAAMAS, 2023. doi: 10.5555/3545946.3598864. — [doi:10.5555/3545946.3598864](https://doi.org/10.5555/3545946.3598864) [find](https://scholar.google.com/scholar?q=Junqi+Qian%2C+Paul+Weng%2C+and+Chenmien+Tan%2E+Learning+Rewards+to+Optimize+Global+Performance+Metrics+in+Deep+Reinforcement+Learning%2E+In+Proceedings+of+the+International+Conference+on+Autonomous+Agents+and)
294. Yunpeng Qing, Shunyu Liu, Jie Song, Huiqiong Wang, and Mingli Song. A Survey on Explainable Reinforcement Learning: Concepts, Algorithms, Challenges, 2023. arXiv: 2211.06665. preprint. — [arXiv:2211.06665](https://arxiv.org/abs/2211.06665) [find](https://scholar.google.com/scholar?q=Yunpeng+Qing%2C+Shunyu+Liu%2C+Jie+Song%2C+Huiqiong+Wang%2C+and+Mingli+Song%2E+A+Survey+on+Explainable+Reinforcement+Learning%3A+Concepts%2C+Algorithms%2C+Challenges%2C+2023%2E+arXiv%3A+2211%2E06665%2E+preprint%2E+%E2%80%94+%5BarXiv%3A2211%2E0)
295. Mattia Racca, Antti Oulasvirta, and Ville Kyrki. Teacher-Aware Active Robot Learning. In Proceedings of the ACM/IEEE International Conference on Human-Robot Interaction (HRI), 2019. doi: 10.1109/HRI.2019.8673300. — [doi:10.1109/hri.2019.8673300](https://doi.org/10.1109/hri.2019.8673300) [find](https://scholar.google.com/scholar?q=Mattia+Racca%2C+Antti+Oulasvirta%2C+and+Ville+Kyrki%2E+Teacher%2DAware+Active+Robot+Learning%2E+In+Proceedings+of+the+ACM%2FIEEE+International+Conference+on+Human%2DRobot+Interaction+%28HRI%29%2C+2019%2E+doi%3A+10%2E1109%2FHRI%2E2)
296. Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D. Manning, Stefano Ermon, and Chelsea Finn. [Direct Preference Optimization: Your Language Model is Secretly a Reward Model](https://md.research.elamur.ai/free/en/papers/rafailov-2023-direct-preference-optimization-language-model/). In Advances in Neural Information Processing Systems (NeurIPS), volume 36, 2023. URL [https://papers.nips.cc/](https://papers.nips.cc/) paper_f… [↗](https://papers.nips.cc/)
297. Matthew Rahtz, Vikrant Varma, Ramana Kumar, Zachary Kenton, Shane Legg, and Jan Leike. Safe Deep RL in 3D Environments using Human Feedback, 2022. arXiv: 2201.08102. preprint. — [arXiv:2201.08102](https://arxiv.org/abs/2201.08102) [find](https://scholar.google.com/scholar?q=Matthew+Rahtz%2C+Vikrant+Varma%2C+Ramana+Kumar%2C+Zachary+Kenton%2C+Shane+Legg%2C+and+Jan+Leike%2E+Safe+Deep+RL+in+3D+Environments+using+Human+Feedback%2C+2022%2E+arXiv%3A+2201%2E08102%2E+preprint%2E+%E2%80%94+%5BarXiv%3A2201%2E08102%5D%28htt)
298. Rajkumar Ramamurthy, Prithviraj Ammanabrolu, Kianté Brantley, Jack Hessel, Rafet Sifa, Christian Bauckhage, Hannaneh Hajishirzi, and Yejin Choi. [Is Reinforcement Learning (Not) for Natural Language Processing: Benchmarks, Baselines, and Building Blocks for Natural Language Policy Optimization](https://md.research.elamur.ai/free/en/papers/ramamurthy-2022-reinforcement-learning-natural-language-processing/). In Pr…
299. Alexandre Rame, Nino Vieillard, Leonard Hussenot, Robert Dadashi, Geoffrey Cideron, Olivier Bachem, and Johan Ferret. WARM: On the Benefits of Weight Averaged Reward Models. In Proceedings of the International Conference on Machine Learning (ICML), 2024. URL [https://openreview.net/forum?](https://openreview.net/forum?) id=s7RDnNUJ… [↗](https://openreview.net/forum?)
300. Siddharth Reddy, Anca Dragan, Sergey Levine, Shane Legg, and Jan Leike. Learning Human Objectives by Evaluating Hypothetical Behavior. In Proceedings of the International Conference on Machine Learning (ICML). PMLR, 2020. URL [https://proceedings.mlr.press/v119/reddy20a.html](https://proceedings.mlr.press/v119/reddy20a.html). [↗](https://proceedings.mlr.press/v119/reddy20a.html)
301. Kevin Regan and Craig Boutilier. Regret-based reward elicitation for Markov decision processes. In Proceedings of the Conference on Uncertainty in Artificial Intelligence (UAI). AUAI Press, 2009. URL [https://dl.acm.org/doi/10.5555/1795114.1795166](https://dl.acm.org/doi/10.5555/1795114.1795166). — [doi:10.5555/1795114.1795166](https://doi.org/10.5555/1795114.1795166) [↗](https://dl.acm.org/doi/10.5555/1795114.1795166)
302. Kevin Regan and Craig Boutilier. Robust online optimization of reward-uncertain MDPs. In Proceedings of the International Joint Conference on Artificial Intelligence (IJCAI). AAAI Press, 2011. doi: 10.5591/978-1-57735-516-8/IJCAI11-361. — [doi:10.5591/978-1-57735-516-8/ijcai11-361](https://doi.org/10.5591/978-1-57735-516-8/ijcai11-361) [find](https://scholar.google.com/scholar?q=Kevin+Regan+and+Craig+Boutilier%2E+Robust+online+optimization+of+reward%2Duncertain+MDPs%2E+In+Proceedings+of+the+International+Joint+Conference+on+Artificial+Intelligence+%28IJCAI%29%2E+AAAI+Press%2C+2011%2E+doi%3A+10)
303. Zhizhou Ren, Anji Liu, Yitao Liang, Jian Peng, and Jianzhu Ma. Efficient Meta Reinforcement Learning for Preference-based Fast Adaptation. In Advances in Neural Information Processing Systems (NeurIPS), volume 35, 2022. URL [https://papers.nips.cc/paper_files/paper/2022/hash/](https://papers.nips.cc/paper_files/paper/2022/hash/) 63b2b056f48653b7cff0d8d2… [↗](https://papers.nips.cc/paper_files/paper/2022/hash/)
304. Carl Orge Retzlaff, Srijita Das, Christabel Wayllace, Payam Mousavi, Mohammad Afshari, Tianpei Yang, Anna Saranti, Alessa Angerschmid, Matthew E. Taylor, and Andreas Holzinger. Human-in-the-Loop Reinforcement Learning: A Survey and Position on Requirements, Challenges, and Opportunities. Journal of… — [doi:10.1613/jair.1.15348](https://doi.org/10.1613/jair.1.15348) [find](https://scholar.google.com/scholar?q=Carl+Orge+Retzlaff%2C+Srijita+Das%2C+Christabel+Wayllace%2C+Payam+Mousavi%2C+Mohammad+Afshari%2C+Tianpei+Yang%2C+Anna+Saranti%2C+Alessa+Angerschmid%2C+Matthew+E%2E+Taylor%2C+and+Andreas+Holzinger%2E+Human%2Din%2Dthe%2DLoop+Reinf)
305. Daniel Russo and Benjamin Van Roy. Eluder Dimension and the Sample Complexity of Optimistic Exploration. In Advances in Neural Information Processing Systems (NIPS), volume 26. Curran Associates, Inc., 2013. URL [https://papers.nips.cc/paper/2013/hash/](https://papers.nips.cc/paper/2013/hash/) 41bfd20a38bb1b0bec75acf0845530a7-Abstract.html. [↗](https://papers.nips.cc/paper/2013/hash/)
306. John Rust. Optimal Replacement of GMC Bus Engines: An Empirical Model of Harold Zurcher. Econometrica, 55(5):999–1033, 1987. doi: 10.2307/1911259. — [doi:10.2307/1911259](https://doi.org/10.2307/1911259) [find](https://scholar.google.com/scholar?q=John+Rust%2E+Optimal+Replacement+of+GMC+Bus+Engines%3A+An+Empirical+Model+of+Harold+Zurcher%2E+Econometrica%2C+55%285%29%3A999%E2%80%931033%2C+1987%2E+doi%3A+10%2E2307%2F1911259%2E+%E2%80%94+%5Bdoi%3A10%2E2307%2F1911259%5D%28https%3A%2F%2Fdoi%2Eorg%2F10%2E2307%2F19112)
307. Mandy Ryan, Karen Gerard, Mabel Amaya-Amaya, and Ian J. Bateman (eds.). Using Discrete Choice Experiments to Value Health and Health Care, volume 11. Springer Netherlands, 2008. doi: 10.1007/978-1-4020-5753-3. — [doi:10.1007/978-1-4020-5753-3](https://doi.org/10.1007/978-1-4020-5753-3) [find](https://scholar.google.com/scholar?q=Mandy+Ryan%2C+Karen+Gerard%2C+Mabel+Amaya%2DAmaya%2C+and+Ian+J%2E+Bateman+%28eds%2E%29%2E+Using+Discrete+Choice+Experiments+to+Value+Health+and+Health+Care%2C+volume+11%2E+Springer+Netherlands%2C+2008%2E+doi%3A+10%2E1007%2F978%2D1%2D402)
308. Dorsa Sadigh, Anca Dragan, Shankar Sastry, and Sanjit Seshia. Active Preference-Based Learning of Reward Functions. In Proceedings of Robotics: Science and Systems (RSS), volume 13, 2017. URL [http://www.roboticsproceedings.org/rss13/p53.html](http://www.roboticsproceedings.org/rss13/p53.html). [↗](http://www.roboticsproceedings.org/rss13/p53.html)
309. Aadirupa Saha. Optimal Algorithms for Stochastic Contextual Preference Bandits. In Advances in Neural Information Processing Systems (NeurIPS), volume 34. Curran Associates, Inc., 2021. URL [https://proceedings.neurips.cc/paper/2021/hash/fc3cf452d3da8402bebb765225ce8c0e-Abstract.html](https://proceedings.neurips.cc/paper/2021/hash/fc3cf452d3da8402bebb765225ce8c0e-Abstract.html). [↗](https://proceedings.neurips.cc/paper/2021/hash/fc3cf452d3da8402bebb765225ce8c0e-Abstract.html)
310. Aadirupa Saha and Akshay Krishnamurthy. Efficient and Optimal Algorithms for Contextual Dueling Bandits under Realizability. In Proceedings of the International Conference on Algorithmic Learning Theory (ALT). PMLR, 2022. URL [https://proceedings.mlr.press/v167/saha22a.html](https://proceedings.mlr.press/v167/saha22a.html). [↗](https://proceedings.mlr.press/v167/saha22a.html)
311. Aadirupa Saha, Aldo Pacchiano, and Jonathan Lee. Dueling RL: Reinforcement Learning with Trajectory Preferences. In Proceedings of the International Conference on Artificial Intelligence and Statistics (AISTATS). PMLR, 2023. URL [https://proceedings.mlr.press/v206/saha23a.html](https://proceedings.mlr.press/v206/saha23a.html). [↗](https://proceedings.mlr.press/v206/saha23a.html)
312. William Saunders, Girish Sastry, Andreas Stuhlmüller, and Owain Evans. Trial without Error: Towards Safe Reinforcement Learning via Human Intervention. In Proceedings of the International Conference on Autonomous Agents and Multiagent Systems (AAMAS). IFAAMAS, 2018. URL [https://dl.acm.org/](https://dl.acm.org/) doi/10.55… — [doi:10.5555/3237383.3238074](https://doi.org/10.5555/3237383.3238074) [↗](https://dl.acm.org/)
313. William Saunders, Catherine Yeh, Jeff Wu, Steven Bills, Long Ouyang, Jonathan Ward, and Jan Leike. [Self-critiquing models for assisting human evaluators](https://md.research.elamur.ai/free/en/papers/saunders-2022-self-critiquing-models-assisting-human/), 2022. arXiv: 2206.05802. preprint.
314. Marc Schoenauer, Riad Akrour, Michele Sebag, and Jean-Christophe Souplet. Programming by Feedback. In Proceedings of the International Conference on Machine Learning (ICML). PMLR, 2014. URL [https://proceedings.mlr.press/v32/schoenauer14.html](https://proceedings.mlr.press/v32/schoenauer14.html). [↗](https://proceedings.mlr.press/v32/schoenauer14.html)
315. John Schulman. Reinforcement Learning from Human Feedback: Progress and Challenges, 2023. URL [https://www.youtube.com/watch?v=hhiLw5Q_UFg](https://www.youtube.com/watch?v=hhiLw5Q_UFg). [↗](https://www.youtube.com/watch?v=hhiLw5Q_UFg)
316. John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal Policy Optimization Algorithms, 2017. arXiv: 1707.06347. preprint. [find](https://scholar.google.com/scholar?q=John+Schulman%2C+Filip+Wolski%2C+Prafulla+Dhariwal%2C+Alec+Radford%2C+and+Oleg+Klimov%2E+Proximal+Policy+Optimization+Algorithms%2C+2017%2E+arXiv%3A+1707%2E06347%2E+preprint%2E)
317. Max Schwarzer, Ankesh Anand, Rishab Goel, R. Devon Hjelm, Aaron Courville, and Philip Bachman. Data-Efficient Reinforcement Learning with Self-Predictive Representations. In Proceedings of the International Conference on Learning Representations (ICLR), 2021. URL [https://openreview.net/forum?](https://openreview.net/forum?) id=uCQ… [↗](https://openreview.net/forum?)
318. Ayush Sekhari, Karthik Sridharan, Wen Sun, and Runzhe Wu. Contextual Bandits and Imitation Learning with Preference-Based Active Queries. In Advances in Neural Information Processing Systems (NeurIPS), volume 36, 2023. URL [https://proceedings.neurips.cc/paper_files/paper/2023/hash/](https://proceedings.neurips.cc/paper_files/paper/2023/hash/) 2567c95fd41459a98… [↗](https://proceedings.neurips.cc/paper_files/paper/2023/hash/)
319. Burr Settles. Active Learning. Morgan & Claypool Publishers, 2012. ISBN 978-1-60845-725-0. [find](https://scholar.google.com/scholar?q=Burr+Settles%2E+Active+Learning%2E+Morgan+%26+Claypool+Publishers%2C+2012%2E+ISBN+978%2D1%2D60845%2D725%2D0%2E)
320. Rohin Shah, Dmitrii Krasheninnikov, Jordan Alexander, Pieter Abbeel, and Anca Dragan. Preferences Implicit in the State of the World. In Proceedings of the International Conference on Learning Representations (ICLR), 2019. URL [https://openreview.net/forum?id=rkevMnRqYQ](https://openreview.net/forum?id=rkevMnRqYQ). [↗](https://openreview.net/forum?id=rkevMnRqYQ)
321. Rohin Shah, Pedro Freire, Neel Alex, Rachel Freedman, Dmitrii Krasheninnikov, Lawrence Chan, Michael D. Dennis, Pieter Abbeel, Anca Dragan, and Stuart Russell. Benefits of Assistance over Reward Learning, 2021a. URL [https://openreview.net/forum?id=DFIoGDZejIB](https://openreview.net/forum?id=DFIoGDZejIB). preprint. [↗](https://openreview.net/forum?id=DFIoGDZejIB)
322. Rohin Shah, Cody Wild, Steven H. Wang, Neel Alex, Brandon Houghton, William Guss, Sharada Mohanty, Anssi Kanervisto, Stephanie Milani, Nicholay Topin, Pieter Abbeel, Stuart Russell, and Anca Dragan. The MineRL BASALT Competition on Learning from Human Feedback, 2021b. arXiv: 2107.01969. preprint. — [arXiv:2107.01969](https://arxiv.org/abs/2107.01969) [find](https://scholar.google.com/scholar?q=Rohin+Shah%2C+Cody+Wild%2C+Steven+H%2E+Wang%2C+Neel+Alex%2C+Brandon+Houghton%2C+William+Guss%2C+Sharada+Mohanty%2C+Anssi+Kanervisto%2C+Stephanie+Milani%2C+Nicholay+Topin%2C+Pieter+Abbeel%2C+Stuart+Russell%2C+and+Anca+Dragan%2E+T)
323. Lingfeng Shen, Sihao Chen, Linfeng Song, Lifeng Jin, Baolin Peng, Haitao Mi, Daniel Khashabi, and Dong Yu. The Trickle-down Impact of Reward Inconsistency on RLHF. In Proceedings of the International Conference on Learning Representations (ICLR), 2024. URL [https://openreview.net/forum?](https://openreview.net/forum?) id=MeHmwCDifc… [↗](https://openreview.net/forum?)
324. Daniel Shin, Anca Dragan, and Daniel S. Brown. Benchmarks and Algorithms for Offline Preference-Based Reward Learning. Transactions on Machine Learning Research, 2023. ISSN 2835-8856. URL [https://openreview.net/forum?id=TGuXXlbKsn](https://openreview.net/forum?id=TGuXXlbKsn). [↗](https://openreview.net/forum?id=TGuXXlbKsn)
325. Pannaga Shivaswamy and Thorsten Joachims. Online structured prediction via coactive learning. In Proceedings of the International Conference on Machine Learning (ICML). Omnipress, 2012. URL [http://icml.cc/2012/papers/717.pdf](http://icml.cc/2012/papers/717.pdf). [↗](http://icml.cc/2012/papers/717.pdf)
326. Pannaga Shivaswamy and Thorsten Joachims. Coactive Learning. Journal of Artificial Intelligence Research, 53:1–40, 2015. doi: 10.1613/jair.4539. — [doi:10.1613/jair.4539](https://doi.org/10.1613/jair.4539) [find](https://scholar.google.com/scholar?q=Pannaga+Shivaswamy+and+Thorsten+Joachims%2E+Coactive+Learning%2E+Journal+of+Artificial+Intelligence+Research%2C+53%3A1%E2%80%9340%2C+2015%2E+doi%3A+10%2E1613%2Fjair%2E4539%2E+%E2%80%94+%5Bdoi%3A10%2E1613%2Fjair%2E4539%5D%28https%3A%2F%2Fdoi%2Eorg%2F10%2E1613%2Fjair%2E)
327. Umer Siddique, Abhinav Sinha, and Yongcan Cao. Fairness in Preference-based Reinforcement Learning. In ICML 2023 Workshop on The Many Facets of Preference-Based Learning (MFPL), 2023. URL [https://openreview.net/forum?id=ColATVnkEl](https://openreview.net/forum?id=ColATVnkEl). [↗](https://openreview.net/forum?id=ColATVnkEl)
328. David Silver, Satinder Singh, Doina Precup, and Richard S. Sutton. Reward is enough. Artificial Intelligence, 299:103535, 2021. doi: 10.1016/j.artint.2021.103535. — [doi:10.1016/j.artint.2021.103535](https://doi.org/10.1016/j.artint.2021.103535) [find](https://scholar.google.com/scholar?q=David+Silver%2C+Satinder+Singh%2C+Doina+Precup%2C+and+Richard+S%2E+Sutton%2E+Reward+is+enough%2E+Artificial+Intelligence%2C+299%3A103535%2C+2021%2E+doi%3A+10%2E1016%2Fj%2Eartint%2E2021%2E103535%2E+%E2%80%94+%5Bdoi%3A10%2E1016%2Fj%2Eartint%2E2021%2E103535%5D%28)
329. Avi Singh, Larry Yang, Chelsea Finn, and Sergey Levine. End-To-End Robotic Reinforcement Learning without Reward Engineering. In Proceedings of Robotics: Science and Systems (RSS), volume 15, 2019. URL [http://www.roboticsproceedings.org/rss15/p73.html](http://www.roboticsproceedings.org/rss15/p73.html). [↗](http://www.roboticsproceedings.org/rss15/p73.html)
330. Anand Siththaranjan, Cassidy Laidlaw, and Dylan Hadfield-Menell. Understanding Hidden Context in Preference Learning: Consequences for RLHF. In Proceedings of the International Conference on Learning Representations (ICLR), 2024. URL [https://openreview.net/forum?id=0tWTxYYPnW](https://openreview.net/forum?id=0tWTxYYPnW). [↗](https://openreview.net/forum?id=0tWTxYYPnW)
331. Joar Max Viktor Skalse and Alessandro Abate. The Reward Hypothesis is False. In NeurIPS 2022 Workshop on ML Safety, 2022. URL [https://openreview.net/forum?id=5l1NgpzAfH](https://openreview.net/forum?id=5l1NgpzAfH). [↗](https://openreview.net/forum?id=5l1NgpzAfH)
332. Joar Max Viktor Skalse and Alessandro Abate. Misspecification in Inverse Reinforcement Learning. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 37, 2023. doi: 10.1609/aaai.v37i12.26766. — [doi:10.1609/aaai.v37i12.26766](https://doi.org/10.1609/aaai.v37i12.26766) [find](https://scholar.google.com/scholar?q=Joar+Max+Viktor+Skalse+and+Alessandro+Abate%2E+Misspecification+in+Inverse+Reinforcement+Learning%2E+In+Proceedings+of+the+AAAI+Conference+on+Artificial+Intelligence%2C+volume+37%2C+2023%2E+doi%3A+10%2E1609%2Faaai%2Ev3)
333. Joar Max Viktor Skalse and Alessandro Abate. Quantifying the Sensitivity of Inverse Reinforcement Learning to Misspecification. In Proceedings of the International Conference on Learning Representations (ICLR), 2024. URL [https://openreview.net/forum?id=pz2E1Q9Wni](https://openreview.net/forum?id=pz2E1Q9Wni). [↗](https://openreview.net/forum?id=pz2E1Q9Wni)
334. Joar Max Viktor Skalse, Nikolaus Howe, Dmitrii Krasheninnikov, and David Krueger. Defining and Characterizing Reward Gaming. In Advances in Neural Information Processing Systems (NeurIPS), volume 35, 2022. URL [https://proceedings.neurips.cc/paper_files/paper/2022/hash/](https://proceedings.neurips.cc/paper_files/paper/2022/hash/) 3d719fee332caa23d5038b8a90e817… [↗](https://proceedings.neurips.cc/paper_files/paper/2022/hash/)
335. Joar Max Viktor Skalse, Matthew Farrugia-Roberts, Stuart Russell, Alessandro Abate, and Adam Gleave. Invariance in Policy Optimisation and Partial Identifiability in Reward Learning. In Proceedings of the International Conference on Machine Learning (ICML). PMLR, 2023. URL [https://proceedings.mlr](https://proceedings.mlr). p… [↗](https://proceedings.mlr)
336. Joar Max Viktor Skalse, Lucy Farnik, Sumeet Ramesh Motwani, Erik Jenner, Adam Gleave, and Alessandro Abate. STARC: A General Framework For Quantifying Differences Between Reward Functions. In Proceedings of the International Conference on Learning Representations (ICLR), 2024. URL https://openrevie… [find](https://scholar.google.com/scholar?q=Joar+Max+Viktor+Skalse%2C+Lucy+Farnik%2C+Sumeet+Ramesh+Motwani%2C+Erik+Jenner%2C+Adam+Gleave%2C+and+Alessandro+Abate%2E+STARC%3A+A+General+Framework+For+Quantifying+Differences+Between+Reward+Functions%2E+In+Proceedi)
337. Feifan Song, Bowen Yu, Minghao Li, Haiyang Yu, Fei Huang, Yongbin Li, and Houfeng Wang. [Preference Ranking Optimization for Human Alignment](https://md.research.elamur.ai/free/en/papers/song-2023-preference-ranking-optimization-human-alignment/). In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, 2024. doi: 10.1609/aaai.v38i17.29865.
338. Ziang Song, Tianle Cai, Jason D. Lee, and Weijie J. Su. Reward Collapse in Aligning Large Language Models: A Prompt-Aware Approach to Preference Rankings. In ICML 2023 Workshop on The Many Facets of Preference-Based Learning (MFPL), 2023. URL [https://openreview.net/forum?id=dpWxK6aqIK](https://openreview.net/forum?id=dpWxK6aqIK). [↗](https://openreview.net/forum?id=dpWxK6aqIK)
339. Taylor Sorensen, Jared Moore, Jillian Fisher, Mitchell L. Gordon, Niloofar Mireshghallah, Christopher Michael Rytting, Andre Ye, Liwei Jiang, Ximing Lu, Nouha Dziri, Tim Althoff, and Yejin Choi. Position: [A Roadmap to Pluralistic Alignment](https://md.research.elamur.ai/free/en/papers/sorensen-2024-roadmap-pluralistic-alignment/). In Proceedings of the International Conference on Machine L…
340. Nisan Stiennon, Long Ouyang, Jeffrey Wu, Daniel Ziegler, Ryan Lowe, Chelsea Voss, Alec Radford, Dario Amodei, and Paul F Christiano. Learning to summarize with human feedback. In Advances in Neural Information Processing Systems (NeurIPS), volume 33. Curran Associates, Inc., 2020. URL https://proce… [find](https://scholar.google.com/scholar?q=Nisan+Stiennon%2C+Long+Ouyang%2C+Jeffrey+Wu%2C+Daniel+Ziegler%2C+Ryan+Lowe%2C+Chelsea+Voss%2C+Alec+Radford%2C+Dario+Amodei%2C+and+Paul+F+Christiano%2E+Learning+to+summarize+with+human+feedback%2E+In+Advances+in+Neural+In)
341. Hiroaki Sugiyama, Toyomi Meguro, and Yasuhiro Minami. Preference-learning based inverse reinforcement learning for dialog control. In Proceedings of Interspeech. ISCA, 2012. doi: 10.21437/Interspeech.2012-72. — [doi:10.21437/interspeech.2012-72](https://doi.org/10.21437/interspeech.2012-72) [find](https://scholar.google.com/scholar?q=Hiroaki+Sugiyama%2C+Toyomi+Meguro%2C+and+Yasuhiro+Minami%2E+Preference%2Dlearning+based+inverse+reinforcement+learning+for+dialog+control%2E+In+Proceedings+of+Interspeech%2E+ISCA%2C+2012%2E+doi%3A+10%2E21437%2FInterspeech%2E)
342. Yanan Sui, Vincent Zhuang, Joel W. Burdick, and Yisong Yue. Multi-dueling Bandits with Dependent Arms. In Proceedings of the Conference on Uncertainty in Artificial Intelligence (UAI). AUAI Press, 2017. URL [http://auai.org/uai2017/proceedings/papers/155.pdf](http://auai.org/uai2017/proceedings/papers/155.pdf). [↗](http://auai.org/uai2017/proceedings/papers/155.pdf)
343. Yanan Sui, Masrour Zoghi, Katja Hofmann, and Yisong Yue. Advancements in Dueling Bandits. In Proceedings of the International Joint Conference on Artificial Intelligence (IJCAI). International Joint Conferences on Artificial Intelligence Organization, 2018. doi: 10.24963/ijcai.2018/776. — [doi:10.24963/ijcai.2018/776](https://doi.org/10.24963/ijcai.2018/776) [find](https://scholar.google.com/scholar?q=Yanan+Sui%2C+Masrour+Zoghi%2C+Katja+Hofmann%2C+and+Yisong+Yue%2E+Advancements+in+Dueling+Bandits%2E+In+Proceedings+of+the+International+Joint+Conference+on+Artificial+Intelligence+%28IJCAI%29%2E+International+Joint+C)
344. Theodore Sumers, Robert Hawkins, Mark K. Ho, Tom Griffiths, and Dylan Hadfield-Menell. How to talk so AI will learn: Instructions, descriptions, and autonomy. In Advances in Neural Information Processing Systems (NeurIPS), volume 35, 2022a. URL [https://proceedings.neurips.cc/paper_files/paper/](https://proceedings.neurips.cc/paper_files/paper/) 2022/… [↗](https://proceedings.neurips.cc/paper_files/paper/)
345. Theodore Sumers, Robert D. Hawkins, Mark K. Ho, Thomas L. Griffiths, and Dylan Hadfield-Menell. Linguistic communication as (inverse) reward design. In ACL Workshop on Learning with Natural Language Supervision, 2022b. URL [https://openreview.net/forum?id=Bh4u3ZDhsWq](https://openreview.net/forum?id=Bh4u3ZDhsWq). [↗](https://openreview.net/forum?id=Bh4u3ZDhsWq)
346. Theodore R. Sumers, Mark K. Ho, Robert D. Hawkins, Karthik Narasimhan, and Thomas L. Griffiths. Learning Rewards From Linguistic Feedback. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, 2021. doi: 10.1609/aaai.v35i7.16749. — [doi:10.1609/aaai.v35i7.16749](https://doi.org/10.1609/aaai.v35i7.16749) [find](https://scholar.google.com/scholar?q=Theodore+R%2E+Sumers%2C+Mark+K%2E+Ho%2C+Robert+D%2E+Hawkins%2C+Karthik+Narasimhan%2C+and+Thomas+L%2E+Griffiths%2E+Learning+Rewards+From+Linguistic+Feedback%2E+In+Proceedings+of+the+AAAI+Conference+on+Artificial+Intellige)
347. Theodore R. Sumers, Mark K. Ho, Robert D. Hawkins, and Thomas L. Griffiths. Show or tell? Exploring when (and why) teaching with language outperforms demonstration. Cognition, 232:105326, 2023. doi: 10.1016/j.cognition.2022.105326. — [doi:10.1016/j.cognition.2022.105326](https://doi.org/10.1016/j.cognition.2022.105326) [find](https://scholar.google.com/scholar?q=Theodore+R%2E+Sumers%2C+Mark+K%2E+Ho%2C+Robert+D%2E+Hawkins%2C+and+Thomas+L%2E+Griffiths%2E+Show+or+tell%3F+Exploring+when+%28and+why%29+teaching+with+language+outperforms+demonstration%2E+Cognition%2C+232%3A105326%2C+2023%2E+doi%3A+1)
348. Zhiqing Sun, Yikang Shen, Hongxin Zhang, Qinhong Zhou, Zhenfang Chen, David Daniel Cox, Yiming Yang, and Chuang Gan. [SALMON: Self-Alignment with Instructable Reward Models](https://md.research.elamur.ai/free/en/papers/sun-2023-salmon-self-alignment-instructable-reward/). In Proceedings of the International Conference on Learning Representations (ICLR), 2024. URL https://openreview. net/forum?id=x…
349. Richard S. Sutton and Andrew G. Barto. Reinforcement Learning: An Introduction. The MIT Press, second edition edition, 2018. ISBN 978-0-262-03924-6. [find](https://scholar.google.com/scholar?q=Richard+S%2E+Sutton+and+Andrew+G%2E+Barto%2E+Reinforcement+Learning%3A+An+Introduction%2E+The+MIT+Press%2C+second+edition+edition%2C+2018%2E+ISBN+978%2D0%2D262%2D03924%2D6%2E)
350. Gokul Swamy, Christoph Dann, Rahul Kidambi, Steven Wu, and Alekh Agarwal. A Minimaximalist Approach to Reinforcement Learning from Human Feedback. In Proceedings of the International Conference on Machine Learning (ICML). PMLR, 2024. URL [https://proceedings.mlr.press/v235/swamy24a](https://proceedings.mlr.press/v235/swamy24a). html. [↗](https://proceedings.mlr.press/v235/swamy24a)
351. Zhiwei Tang, Dmitry Rybin, and Tsung-Hui Chang. Zeroth-Order Optimization Meets Human Feedback: Provable Learning via Ranking Oracles. In Proceedings of the International Conference on Learning Representations (ICLR), 2024. URL [https://openreview.net/forum?id=TVDUVpgu9s](https://openreview.net/forum?id=TVDUVpgu9s). [↗](https://openreview.net/forum?id=TVDUVpgu9s)
352. Louis Leon Thurstone. A law of comparative judgment. Psychological Review, 34:273–286, 1927. doi: 10.1037/h0070288. — [doi:10.1037/h0070288](https://doi.org/10.1037/h0070288) [find](https://scholar.google.com/scholar?q=Louis+Leon+Thurstone%2E+A+law+of+comparative+judgment%2E+Psychological+Review%2C+34%3A273%E2%80%93286%2C+1927%2E+doi%3A+10%2E1037%2Fh0070288%2E+%E2%80%94+%5Bdoi%3A10%2E1037%2Fh0070288%5D%28https%3A%2F%2Fdoi%2Eorg%2F10%2E1037%2Fh0070288%29)
353. Jeremy Tien, Jerry Zhi-Yang He, Zackory Erickson, Anca Dragan, and Daniel S. Brown. A Study of Causal Confusion in Preference-Based Reward Learning. In ICML 2022 Workshop on Spurious Correlations, Invariance and Stability, 2022. URL [https://openreview.net/forum?id=WaZZ0Sw9fWf](https://openreview.net/forum?id=WaZZ0Sw9fWf). [↗](https://openreview.net/forum?id=WaZZ0Sw9fWf)
354. Jeremy Tien, Jerry Zhi-Yang He, Zackory Erickson, Anca Dragan, and Daniel S. Brown. Causal Confusion and Reward Misidentification in Preference-Based Reward Learning. In Proceedings of the International Conference on Learning Representations (ICLR), 2023. URL [https://openreview.net/forum?](https://openreview.net/forum?) id=R0Xxvr_… [↗](https://openreview.net/forum?)
355. Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, Dan Bikel, Lukas Blecher, Cristian Canton Ferrer, Moya Chen, Guillem Cucurull, David Esiobu, Jude Fernandes, Jeremy Fu, Wenyin Fu, Brian Fuller,… — [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://md.research.elamur.ai/free/en/papers/touvron-2023-llama-2-open-foundation-fine/)
356. Kenneth E. Train. Discrete Choice Methods with Simulation. Cambridge University Press, 2 edition, 2009. doi: 10.1017/CBO9780511805271. — [doi:10.1017/cbo9780511805271](https://doi.org/10.1017/cbo9780511805271) [find](https://scholar.google.com/scholar?q=Kenneth+E%2E+Train%2E+Discrete+Choice+Methods+with+Simulation%2E+Cambridge+University+Press%2C+2+edition%2C+2009%2E+doi%3A+10%2E1017%2FCBO9780511805271%2E+%E2%80%94+%5Bdoi%3A10%2E1017%2Fcbo9780511805271%5D%28https%3A%2F%2Fdoi%2Eorg%2F10%2E1017%2Fcbo97805)
357. Thi Ngoc Trang Tran, Alexander Felfernig, and Nava Tintarev. Humanized Recommender Systems: State-of-the-art and Research Issues. ACM Transactions on Interactive Intelligent Systems, 11(2):9:1–9:41, 2021. doi: 10.1145/3446906. — [doi:10.1145/3446906](https://doi.org/10.1145/3446906) [find](https://scholar.google.com/scholar?q=Thi+Ngoc+Trang+Tran%2C+Alexander+Felfernig%2C+and+Nava+Tintarev%2E+Humanized+Recommender+Systems%3A+State%2Dof%2Dthe%2Dart+and+Research+Issues%2E+ACM+Transactions+on+Interactive+Intelligent+Systems%2C+11%282%29%3A9%3A1%E2%80%939%3A41%2C+2)
358. Maegan Tucker, Kejun Li, Yisong Yue, and Aaron D. Ames. POLAR: Preference Optimization and Learning Algorithms for Robotics, 2022. arXiv: 2208.04404. preprint. — [arXiv:2208.04404](https://arxiv.org/abs/2208.04404) [find](https://scholar.google.com/scholar?q=Maegan+Tucker%2C+Kejun+Li%2C+Yisong+Yue%2C+and+Aaron+D%2E+Ames%2E+POLAR%3A+Preference+Optimization+and+Learning+Algorithms+for+Robotics%2C+2022%2E+arXiv%3A+2208%2E04404%2E+preprint%2E+%E2%80%94+%5BarXiv%3A2208%2E04404%5D%28https%3A%2F%2Farxiv%2Eorg%2Fa)
359. L. G. Valiant. A Theory of the Learnable. Communications of the ACM, 27(11):1134–1142, 1984. doi: 10.1145/1968.1972. — [doi:10.1145/1968.1972](https://doi.org/10.1145/1968.1972) [find](https://scholar.google.com/scholar?q=L%2E+G%2E+Valiant%2E+A+Theory+of+the+Learnable%2E+Communications+of+the+ACM%2C+27%2811%29%3A1134%E2%80%931142%2C+1984%2E+doi%3A+10%2E1145%2F1968%2E1972%2E+%E2%80%94+%5Bdoi%3A10%2E1145%2F1968%2E1972%5D%28https%3A%2F%2Fdoi%2Eorg%2F10%2E1145%2F1968%2E1972%29)
360. Peter Vamplew, Benjamin J. Smith, Johan Källström, Gabriel Ramos, Roxana Rădulescu, Diederik M. Roijers, Conor F. Hayes, Fredrik Heintz, Patrick Mannion, Pieter J. K. Libin, Richard Dazeley, and Cameron Foale. Scalar reward is not enough: A response to Silver, Singh, Precup and Sutton (2021). Autono… — [doi:10.1007/s10458-022-09575-5](https://doi.org/10.1007/s10458-022-09575-5) [find](https://scholar.google.com/scholar?q=Peter+Vamplew%2C+Benjamin+J%2E+Smith%2C+Johan+K%C3%A4llstr%C3%B6m%2C+Gabriel+Ramos%2C+Roxana+R%C4%83dulescu%2C+Diederik+M%2E+Roijers%2C+Conor+F%2E+Hayes%2C+Fredrik+Heintz%2C+Patrick+Mannion%2C+Pieter+J%2E+K%2E+Libin%2C+Richard+Dazeley%2C+and+Camer)
361. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Łukasz Kaiser, and Illia Polosukhin. [Attention is All you Need](https://md.research.elamur.ai/free/en/papers/vaswani-2017-attention-all-need/). In Advances in Neural Information Processing Systems (NIPS), volume 30. Curran Associates, Inc., 2017. URL [https://papers.nips.cc/paper/2017/hash/…](https://papers.nips.cc/paper/2017/hash/…) [↗](https://papers.nips.cc/paper/2017/hash/…)
362. Mudit Verma and Subbarao Kambhampati. Data Driven Reward Initialization for Preference based Reinforcement Learning. In AAAI 2023 Workshop on Representation Learning for Responsible Human-Centric AI, 2023a. [find](https://scholar.google.com/scholar?q=Mudit+Verma+and+Subbarao+Kambhampati%2E+Data+Driven+Reward+Initialization+for+Preference+based+Reinforcement+Learning%2E+In+AAAI+2023+Workshop+on+Representation+Learning+for+Responsible+Human%2DCentric+AI%2C+)
363. Mudit Verma and Subbarao Kambhampati. A State Augmentation based approach to Reinforcement Learning from Human Preferences. In AAAI 2023 Workshop on Representation Learning for Responsible Human-Centric AI, 2023b. [find](https://scholar.google.com/scholar?q=Mudit+Verma+and+Subbarao+Kambhampati%2E+A+State+Augmentation+based+approach+to+Reinforcement+Learning+from+Human+Preferences%2E+In+AAAI+2023+Workshop+on+Representation+Learning+for+Responsible+Human%2DCentr)
364. Mudit Verma and Katherine Metcalf. Hindsight PRIORs for Reward Learning from Human Preferences. In Proceedings of the International Conference on Learning Representations (ICLR), 2024. URL [https://openreview.net/forum?id=NLevOah0CJ](https://openreview.net/forum?id=NLevOah0CJ). [↗](https://openreview.net/forum?id=NLevOah0CJ)
365. Mudit Verma, Siddhant Bhambri, and Subbarao Kambhampati. Exploiting Unlabeled Data for Feedback Efficient Human Preference based Reinforcement Learning. In AAAI 2023 Workshop on Representation Learning for Responsible Human-Centric AI, 2023. [find](https://scholar.google.com/scholar?q=Mudit+Verma%2C+Siddhant+Bhambri%2C+and+Subbarao+Kambhampati%2E+Exploiting+Unlabeled+Data+for+Feedback+Efficient+Human+Preference+based+Reinforcement+Learning%2E+In+AAAI+2023+Workshop+on+Representation+Learnin)
366. Chaoqi Wang, Yibo Jiang, Chenghao Yang, Han Liu, and Yuxin Chen. Beyond Reverse KL: Generalizing Direct Preference Optimization with Diverse Divergence Constraints. In Proceedings of the International Conference on Learning Representations (ICLR), 2024a. URL [https://openreview.net/forum?](https://openreview.net/forum?) id=2cRzmWXK… [↗](https://openreview.net/forum?)
367. Haoran Wang, Qiuye Jin, Shiman Li, Siyu Liu, Manning Wang, and Zhijian Song. A comprehensive survey on deep active learning in medical image analysis. Medical Image Analysis, 95:103201, 2024b. doi: 10.1016/j.media.2024.103201. — [doi:10.1016/j.media.2024.103201](https://doi.org/10.1016/j.media.2024.103201) [find](https://scholar.google.com/scholar?q=Haoran+Wang%2C+Qiuye+Jin%2C+Shiman+Li%2C+Siyu+Liu%2C+Manning+Wang%2C+and+Zhijian+Song%2E+A+comprehensive+survey+on+deep+active+learning+in+medical+image+analysis%2E+Medical+Image+Analysis%2C+95%3A103201%2C+2024b%2E+doi%3A+10)
368. Yizhong Wang, Yeganeh Kordi, Swaroop Mishra, Alisa Liu, Noah A. Smith, Daniel Khashabi, and Hannaneh Hajishirzi. Self-Instruct: Aligning Language Models with Self-Generated Instructions. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers… [find](https://scholar.google.com/scholar?q=Yizhong+Wang%2C+Yeganeh+Kordi%2C+Swaroop+Mishra%2C+Alisa+Liu%2C+Noah+A%2E+Smith%2C+Daniel+Khashabi%2C+and+Hannaneh+Hajishirzi%2E+Self%2DInstruct%3A+Aligning+Language+Models+with+Self%2DGenerated+Instructions%2E+In+Proceeding)
369. Yuanhao Wang, Qinghua Liu, and Chi Jin. Is RLHF More Difficult than Standard RL? A Theoretical Perspective. In Advances in Neural Information Processing Systems (NeurIPS), volume 36, 2023b. URL [https://papers.nips.cc/paper_files/paper/2023/hash/](https://papers.nips.cc/paper_files/paper/2023/hash/) efb9629755e598c4f261c44aeb6fde5e-Abstract-Conference.h… [↗](https://papers.nips.cc/paper_files/paper/2023/hash/)
370. Yufei Wang, Zhanyi Sun, Jesse Zhang, Zhou Xian, Erdem Biyik, David Held, and Zackory Erickson. RL-VLM-F: Reinforcement Learning from Vision Language Foundation Model Feedback. In Proceedings of the International Conference on Machine Learning (ICML), 2024c. URL [https://openreview.net/forum?](https://openreview.net/forum?) id=YSoMm… [↗](https://openreview.net/forum?)
371. Zhilin Wang, Yi Dong, Jiaqi Zeng, Virginia Adams, Makesh Narsimhan Sreedhar, Daniel Egert, Olivier Delalleau, Jane Scowcroft, Neel Kant, Aidan Swope, and Oleksii Kuchaiev. [HelpSteer: Multi-attribute Helpfulness Dataset for SteerLM](https://md.research.elamur.ai/free/en/papers/wang-2024-helpsteer-multi-attribute-helpfulness-dataset/). In Proceedings of the Conference of the North American Chapter of th…
372. Zizhao Wang, Junyao Shi, Iretiayo Akinola, and Peter Allen. Maximizing BCI Human Feedback using Active Learning. In Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2020. doi: 10.1109/IROS45743.2020.9341669. — [doi:10.1109/iros45743.2020.9341669](https://doi.org/10.1109/iros45743.2020.9341669) [find](https://scholar.google.com/scholar?q=Zizhao+Wang%2C+Junyao+Shi%2C+Iretiayo+Akinola%2C+and+Peter+Allen%2E+Maximizing+BCI+Human+Feedback+using+Active+Learning%2E+In+Proceedings+of+the+IEEE%2FRSJ+International+Conference+on+Intelligent+Robots+and+Syste)
373. Garrett Warnell, Nicholas Waytowich, Vernon Lawhern, and Peter Stone. Deep TAMER: Interactive Agent Shaping in High-Dimensional State Spaces. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 32, 2018. doi: 10.1609/aaai.v32i1.11485. — [doi:10.1609/aaai.v32i1.11485](https://doi.org/10.1609/aaai.v32i1.11485) [find](https://scholar.google.com/scholar?q=Garrett+Warnell%2C+Nicholas+Waytowich%2C+Vernon+Lawhern%2C+and+Peter+Stone%2E+Deep+TAMER%3A+Interactive+Agent+Shaping+in+High%2DDimensional+State+Spaces%2E+In+Proceedings+of+the+AAAI+Conference+on+Artificial+Intell)
374. Paul Weng and Bruno Zanuttini. Interactive value iteration for Markov decision processes with unknown rewards. In Proceedings of the International Joint Conference on Artificial Intelligence (IJCAI). AAAI Press, 2013. URL [https://www.ijcai.org/Proceedings/13/Papers/355.pdf](https://www.ijcai.org/Proceedings/13/Papers/355.pdf). [↗](https://www.ijcai.org/Proceedings/13/Papers/355.pdf)
375. Nils Wilde and Javier Alonso-Mora. Do we use the Right Measure? Challenges in Evaluating Reward Learning Algorithms. In Proceedings of the Conference on Robot Learning (CoRL). PMLR, 2022. URL [https://proceedings.mlr.press/v205/wilde23a.html](https://proceedings.mlr.press/v205/wilde23a.html). [↗](https://proceedings.mlr.press/v205/wilde23a.html)
376. Nils Wilde, Dana Kulić, and Stephen L. Smith. Learning User Preferences in Robot Motion Planning Through Interaction. In Proceedings of the IEEE International Conference on Robotics and Automation (ICRA), 2018. doi: 10.1109/ICRA.2018.8460586. — [doi:10.1109/icra.2018.8460586](https://doi.org/10.1109/icra.2018.8460586) [find](https://scholar.google.com/scholar?q=Nils+Wilde%2C+Dana+Kuli%C4%87%2C+and+Stephen+L%2E+Smith%2E+Learning+User+Preferences+in+Robot+Motion+Planning+Through+Interaction%2E+In+Proceedings+of+the+IEEE+International+Conference+on+Robotics+and+Automation+%28IC)
377. Nils Wilde, Dana Kulić, and Stephen L. Smith. Active Preference Learning using Maximum Regret. In Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2020. doi: 10.1109/IROS45743.2020.9341530. — [doi:10.1109/iros45743.2020.9341530](https://doi.org/10.1109/iros45743.2020.9341530) [find](https://scholar.google.com/scholar?q=Nils+Wilde%2C+Dana+Kuli%C4%87%2C+and+Stephen+L%2E+Smith%2E+Active+Preference+Learning+using+Maximum+Regret%2E+In+Proceedings+of+the+IEEE%2FRSJ+International+Conference+on+Intelligent+Robots+and+Systems+%28IROS%29%2C+2020%2E+d)
378. Nils Wilde, Erdem Biyik, Dorsa Sadigh, and Stephen L. Smith. Learning Reward Functions from Scale Feedback. In Proceedings of the Conference on Robot Learning (CoRL). PMLR, 2021. URL [https://proceedings.mlr.press/v164/wilde22a.html](https://proceedings.mlr.press/v164/wilde22a.html). [↗](https://proceedings.mlr.press/v164/wilde22a.html)
379. Aaron Wilson, Alan Fern, and Prasad Tadepalli. A Bayesian Approach for Policy Learning from Trajectory Preference Queries. In Advances in Neural Information Processing Systems (NIPS), volume 25. Curran Associates, Inc., 2012. URL [https://proceedings.neurips.cc/paper/2012/hash/](https://proceedings.neurips.cc/paper/2012/hash/) 16c222aa19898e50589381… [↗](https://proceedings.neurips.cc/paper/2012/hash/)
380. Christian Wirth and Johannes Fürnkranz. EPMC: Every Visit Preference Monte Carlo for Reinforcement Learning. In Proceedings of the Asian Conference on Machine Learning (ACML). PMLR, 2013a. URL [https://proceedings.mlr.press/v29/Wirth13.html](https://proceedings.mlr.press/v29/Wirth13.html). [↗](https://proceedings.mlr.press/v29/Wirth13.html)
381. Christian Wirth and Johannes Fürnkranz. A Policy Iteration Algorithm for Learning from Preference-Based Feedback. In Advances in Intelligent Data Analysis (IDA). Springer, 2013b. doi: 10.1007/978-3-642-41398-8_37. — [doi:10.1007/978-3-642-41398-8_37](https://doi.org/10.1007/978-3-642-41398-8_37) [find](https://scholar.google.com/scholar?q=Christian+Wirth+and+Johannes+F%C3%BCrnkranz%2E+A+Policy+Iteration+Algorithm+for+Learning+from+Preference%2DBased+Feedback%2E+In+Advances+in+Intelligent+Data+Analysis+%28IDA%29%2E+Springer%2C+2013b%2E+doi%3A+10%2E1007%2F978%2D3%2D64)
382. Christian Wirth, Johannes Fürnkranz, and Gerhard Neumann. Model-Free Preference-Based Reinforcement Learning. In Proceedings of the AAAI Conference on Artificial Intelligence. AAAI Press, 2016. doi: 10.1609/aaai.v30i1.10269. [find](https://scholar.google.com/scholar?q=Christian+Wirth%2C+Johannes+F%C3%BCrnkranz%2C+and+Gerhard+Neumann%2E+Model%2DFree+Preference%2DBased+Reinforcement+Learning%2E+In+Proceedings+of+the+AAAI+Conference+on+Artificial+Intelligence%2E+AAAI+Press%2C+2016%2E+doi%3A+1)
383. Christian Wirth, Riad Akrour, Gerhard Neumann, and Johannes Fürnkranz. A Survey of Preference-Based Reinforcement Learning Methods. Journal of Machine Learning Research, 18(136):1–46, 2017. ISSN 1533-7928. URL [http://jmlr.org/papers/v18/16-634.html](http://jmlr.org/papers/v18/16-634.html). [↗](http://jmlr.org/papers/v18/16-634.html)
384. Fan Wu, Huseyin A. Inan, Arturs Backurs, Varun Chandrasekaran, Janardhan Kulkarni, and Robert Sim. Privately Aligning Language Models with Reinforcement Learning. In Proceedings of the International Conference on Learning Representations (ICLR), 2024. URL [https://openreview.net/forum?](https://openreview.net/forum?) id=3d0OmYTNui. [↗](https://openreview.net/forum?)
385. Huasen Wu and Xin Liu. Double Thompson Sampling for Dueling Bandits. In Advances in Neural Information Processing Systems (NIPS), volume 29. Curran Associates, Inc., 2016. URL [https://papers.nips.cc/paper_files/paper/2016/hash/9de6d14fff9806d4bcd1ef555be766cd-Abstract.html](https://papers.nips.cc/paper_files/paper/2016/hash/9de6d14fff9806d4bcd1ef555be766cd-Abstract.html). [↗](https://papers.nips.cc/paper_files/paper/2016/hash/9de6d14fff9806d4bcd1ef555be766cd-Abstract.html)
386. Jeff Wu, Long Ouyang, Daniel M. Ziegler, Nisan Stiennon, Ryan Lowe, Jan Leike, and Paul Christiano. [Recursively Summarizing Books with Human Feedback](https://md.research.elamur.ai/free/en/papers/wu-2021-recursively-summarizing-books-human-feedback/), 2021. arXiv: 2109.10862. preprint.
387. Runzhe Wu and Wen Sun. Making RL with Preference-based Feedback Efficient via Randomization. In Proceedings of the International Conference on Learning Representations (ICLR), 2024. URL [https://openreview.net/forum?id=Pe2lo3QOvo](https://openreview.net/forum?id=Pe2lo3QOvo). [↗](https://openreview.net/forum?id=Pe2lo3QOvo)
388. Tianhao Wu, Banghua Zhu, Ruoyu Zhang, Zhaojin Wen, Kannan Ramchandran, and Jiantao Jiao. Pairwise Proximal Policy Optimization: Harnessing Relative Feedback for LLM Alignment. In NeurIPS 2023 Foundation Models for Decision Making Workshop, 2023. URL [https://openreview.net/forum?id=](https://openreview.net/forum?id=) yQT406rH72. [↗](https://openreview.net/forum?id=)
389. Xingjiao Wu, Luwei Xiao, Yixuan Sun, Junhang Zhang, Tianlong Ma, and Liang He. A survey of human-in-the-loop for machine learning. Future Generation Computer Systems, 135:364–381, 2022. doi: 10.1016/j.future.2022.05.014. — [doi:10.1016/j.future.2022.05.014](https://doi.org/10.1016/j.future.2022.05.014) [find](https://scholar.google.com/scholar?q=Xingjiao+Wu%2C+Luwei+Xiao%2C+Yixuan+Sun%2C+Junhang+Zhang%2C+Tianlong+Ma%2C+and+Liang+He%2E+A+survey+of+human%2Din%2Dthe%2Dloop+for+machine+learning%2E+Future+Generation+Computer+Systems%2C+135%3A364%E2%80%93381%2C+2022%2E+doi%3A+10%2E1016%2Fj)
390. Blake Wulfe, Logan Michael Ellis, Jean Mercat, Rowan Thomas McAllister, and Adrien Gaidon. Dynamics-Aware Comparison of Learned Reward Functions. In Proceedings of the International Conference on Learning Representations (ICLR), 2022. URL [https://openreview.net/forum?id=CALFyKVs87](https://openreview.net/forum?id=CALFyKVs87). [↗](https://openreview.net/forum?id=CALFyKVs87)
391. Baicen Xiao, Qifan Lu, Bhaskar Ramasubramanian, Andrew Clark, Linda Bushnell, and Radha Poovendran. FRESH: Interactive Reward Shaping in High-Dimensional State Spaces using Human Feedback. In Proceedings of the International Conference on Autonomous Agents and Multiagent Systems (AAMAS). IFAAMAS, 20… — [doi:10.5555/3398761.3398935](https://doi.org/10.5555/3398761.3398935) [find](https://scholar.google.com/scholar?q=Baicen+Xiao%2C+Qifan+Lu%2C+Bhaskar+Ramasubramanian%2C+Andrew+Clark%2C+Linda+Bushnell%2C+and+Radha+Poovendran%2E+FRESH%3A+Interactive+Reward+Shaping+in+High%2DDimensional+State+Spaces+using+Human+Feedback%2E+In+Proceedi)
392. Annie Xie, Avi Singh, Sergey Levine, and Chelsea Finn. Few-Shot Goal Inference for Visuomotor Learning and Planning. In Proceedings of the Conference on Robot Learning (CoRL). PMLR, 2018. URL [https://proceedings.mlr.press/v87/xie18a.html](https://proceedings.mlr.press/v87/xie18a.html). [↗](https://proceedings.mlr.press/v87/xie18a.html)
393. Tianbao Xie, Siheng Zhao, Chen Henry Wu, Yitao Liu, Qian Luo, Victor Zhong, Yanchao Yang, and Tao Yu. Text2Reward: Reward Shaping with Language Models for Reinforcement Learning. In Proceedings of the International Conference on Learning Representations (ICLR), 2024. URL https://openreview. net/foru… [find](https://scholar.google.com/scholar?q=Tianbao+Xie%2C+Siheng+Zhao%2C+Chen+Henry+Wu%2C+Yitao+Liu%2C+Qian+Luo%2C+Victor+Zhong%2C+Yanchao+Yang%2C+and+Tao+Yu%2E+Text2Reward%3A+Reward+Shaping+with+Language+Models+for+Reinforcement+Learning%2E+In+Proceedings+of+the)
394. Wei Xiong, Hanze Dong, Chenlu Ye, Ziqi Wang, Han Zhong, Heng Ji, Nan Jiang, and Tong Zhang. [Iterative Preference Learning from Human Feedback: Bridging Theory and Practice for RLHF under KL-constraint](https://md.research.elamur.ai/free/en/papers/xiong-2023-iterative-preference-learning-human-feedback/). In ICLR 2024 Workshop on Mathematical and Empirical Understanding of Foundation Models, 2024. URL…
395. Jiazheng Xu, Xiao Liu, Yuchen Wu, Yuxuan Tong, Qinkai Li, Ming Ding, Jie Tang, and Yuxiao Dong. ImageReward: Learning and Evaluating Human Preferences for Text-to-Image Generation. In Advances in Neural Information Processing Systems (NeurIPS), volume 36, 2023. URL [https://papers.nips.cc/](https://papers.nips.cc/) paper_file… [↗](https://papers.nips.cc/)
396. Shusheng Xu, Wei Fu, Jiaxuan Gao, Wenjie Ye, Weilin Liu, Zhiyu Mei, Guangju Wang, Chao Yu, and Yi Wu. Is DPO Superior to PPO for LLM Alignment? A Comprehensive Study. In Proceedings of the International Conference on Machine Learning (ICML). PMLR, 2024. URL [https://proceedings.mlr](https://proceedings.mlr). press/v235/xu24h.… [↗](https://proceedings.mlr)
397. Yichong Xu, Ruosong Wang, Lin Yang, Aarti Singh, and Artur Dubrawski. Preference-based Reinforcement Learning with Finite-Time Guarantees. In Advances in Neural Information Processing Systems (NeurIPS), volume 33. Curran Associates, Inc., 2020. URL [https://proceedings.neurips.cc/paper/2020/hash/](https://proceedings.neurips.cc/paper/2020/hash/) d9d… [↗](https://proceedings.neurips.cc/paper/2020/hash/)
398. Wanqi Xue, Qingpeng Cai, Zhenghai Xue, Shuo Sun, Shuchang Liu, Dong Zheng, Peng Jiang, Kun Gai, and Bo An. PrefRec: Recommender Systems with Human Preferences for Reinforcing Long-term User Engagement. In Proceedings of the ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD). Associat… — [doi:10.1145/3580305.3599473](https://doi.org/10.1145/3580305.3599473) [find](https://scholar.google.com/scholar?q=Wanqi+Xue%2C+Qingpeng+Cai%2C+Zhenghai+Xue%2C+Shuo+Sun%2C+Shuchang+Liu%2C+Dong+Zheng%2C+Peng+Jiang%2C+Kun+Gai%2C+and+Bo+An%2E+PrefRec%3A+Recommender+Systems+with+Human+Preferences+for+Reinforcing+Long%2Dterm+User+Engagement)
399. Wanqi Xue, Bo An, Shuicheng Yan, and Zhongwen Xu. Reinforcement Learning from Diverse Human Preferences. In Proceedings of the International Joint Conference on Artificial Intelligence (IJCAI), volume 6, 2024. doi: 10.24963/ijcai.2024/586. — [doi:10.24963/ijcai.2024/586](https://doi.org/10.24963/ijcai.2024/586) [find](https://scholar.google.com/scholar?q=Wanqi+Xue%2C+Bo+An%2C+Shuicheng+Yan%2C+and+Zhongwen+Xu%2E+Reinforcement+Learning+from+Diverse+Human+Preferences%2E+In+Proceedings+of+the+International+Joint+Conference+on+Artificial+Intelligence+%28IJCAI%29%2C+volume)
400. Jingfeng Yang. Capability or Alignment? Respect the LLM Base Model’s Capability During Alignment, 2024. URL [https://jingfengyang.github.io/alignment](https://jingfengyang.github.io/alignment). [↗](https://jingfengyang.github.io/alignment)
401. Georgios N. Yannakakis and Héctor P. Martínez. Ratings are Overrated! Frontiers in ICT, 2:13, 2015. doi: 10.3389/fict.2015.00013. — [doi:10.3389/fict.2015.00013](https://doi.org/10.3389/fict.2015.00013) [find](https://scholar.google.com/scholar?q=Georgios+N%2E+Yannakakis+and+H%C3%A9ctor+P%2E+Mart%C3%ADnez%2E+Ratings+are+Overrated%21+Frontiers+in+ICT%2C+2%3A13%2C+2015%2E+doi%3A+10%2E3389%2Ffict%2E2015%2E00013%2E+%E2%80%94+%5Bdoi%3A10%2E3389%2Ffict%2E2015%2E00013%5D%28https%3A%2F%2Fdoi%2Eorg%2F10%2E3389%2Ffict%2E2015%2E0001)
402. Chenlu Ye, Wei Xiong, Yuheng Zhang, Nan Jiang, and Tong Zhang. A Theoretical Analysis of Nash Learning from Human Feedback under General KL-Regularized Preference, 2024. arXiv: 2402.07314. preprint. — [arXiv:2402.07314](https://arxiv.org/abs/2402.07314) [find](https://scholar.google.com/scholar?q=Chenlu+Ye%2C+Wei+Xiong%2C+Yuheng+Zhang%2C+Nan+Jiang%2C+and+Tong+Zhang%2E+A+Theoretical+Analysis+of+Nash+Learning+from+Human+Feedback+under+General+KL%2DRegularized+Preference%2C+2024%2E+arXiv%3A+2402%2E07314%2E+preprint%2E+%E2%80%94)
403. Tianhe Yu, Ted Xiao, Jonathan Tompson, Austin Stone, Su Wang, Anthony Brohan, Jaspiar Singh, Clayton Tan, Dee M, Jodilyn Peralta, Karol Hausman, Brian Ichter, and Fei Xia. Scaling Robot Learning with Semantically Imagined Experience. In Proceedings of Robotics: Science and Systems (RSS), volume 19,… [find](https://scholar.google.com/scholar?q=Tianhe+Yu%2C+Ted+Xiao%2C+Jonathan+Tompson%2C+Austin+Stone%2C+Su+Wang%2C+Anthony+Brohan%2C+Jaspiar+Singh%2C+Clayton+Tan%2C+Dee+M%2C+Jodilyn+Peralta%2C+Karol+Hausman%2C+Brian+Ichter%2C+and+Fei+Xia%2E+Scaling+Robot+Learning+with+)
404. Hongyi Yuan, Zheng Yuan, Chuanqi Tan, Wei Wang, Songfang Huang, and Fei Huang. RRHF: Rank Responses to Align Language Models with Human Feedback. In Advances in Neural Information Processing Systems (NeurIPS), volume 36, 2023. URL [https://proceedings.neurips.cc/paper_files/paper/](https://proceedings.neurips.cc/paper_files/paper/) 2023/hash/23e6f78bd… [↗](https://proceedings.neurips.cc/paper_files/paper/)
405. Yifu Yuan, Jianye Hao, Yi Ma, Zibin Dong, Hebin Liang, Jinyi Liu, Zhixin Feng, Kai Zhao, and Yan Zheng. Uni-RLHF: Universal Platform and Benchmark Suite for Reinforcement Learning with Diverse Human Feedback. In Proceedings of the International Conference on Learning Representations (ICLR), 2024. UR… [find](https://scholar.google.com/scholar?q=Yifu+Yuan%2C+Jianye+Hao%2C+Yi+Ma%2C+Zibin+Dong%2C+Hebin+Liang%2C+Jinyi+Liu%2C+Zhixin+Feng%2C+Kai+Zhao%2C+and+Yan+Zheng%2E+Uni%2DRLHF%3A+Universal+Platform+and+Benchmark+Suite+for+Reinforcement+Learning+with+Diverse+Human+F)
406. Yisong Yue and Thorsten Joachims. Interactively optimizing information retrieval systems as a dueling bandits problem. In Proceedings of the International Conference on Machine Learning (ICML). Association for Computing Machinery, 2009. doi: 10.1145/1553374.1553527. [find](https://scholar.google.com/scholar?q=Yisong+Yue+and+Thorsten+Joachims%2E+Interactively+optimizing+information+retrieval+systems+as+a+dueling+bandits+problem%2E+In+Proceedings+of+the+International+Conference+on+Machine+Learning+%28ICML%29%2E+Associ)
407. Huixin Zhan, Feng Tao, and Yongcan Cao. Human-Guided Robot Behavior Learning: A GAN-Assisted Preference-Based Reinforcement Learning Approach. IEEE Robotics and Automation Letters, 6(2):3545– 3552, 2021. doi: 10.1109/LRA.2021.3063927. — [doi:10.1109/lra.2021.3063927](https://doi.org/10.1109/lra.2021.3063927) [find](https://scholar.google.com/scholar?q=Huixin+Zhan%2C+Feng+Tao%2C+and+Yongcan+Cao%2E+Human%2DGuided+Robot+Behavior+Learning%3A+A+GAN%2DAssisted+Preference%2DBased+Reinforcement+Learning+Approach%2E+IEEE+Robotics+and+Automation+Letters%2C+6%282%29%3A3545%E2%80%93+3552%2C+20)
408. Wenhao Zhan, Masatoshi Uehara, Nathan Kallus, Jason D. Lee, and Wen Sun. Provable Offline Preference-Based Reinforcement Learning. In Proceedings of the International Conference on Learning Representations (ICLR), 2024a. URL [https://openreview.net/forum?id=tVMPfEGT2w](https://openreview.net/forum?id=tVMPfEGT2w). [↗](https://openreview.net/forum?id=tVMPfEGT2w)
409. Wenhao Zhan, Masatoshi Uehara, Wen Sun, and Jason D. Lee. Provable Reward-Agnostic Preference-Based Reinforcement Learning. In Proceedings of the International Conference on Learning Representations (ICLR), 2024b. URL [https://openreview.net/forum?id=yTBXeXdbMf](https://openreview.net/forum?id=yTBXeXdbMf). [↗](https://openreview.net/forum?id=yTBXeXdbMf)
410. David Zhang, Micah Carroll, Andreea Bobu, and Anca Dragan. Time-Efficient Reward Learning via Visually Assisted Cluster Ranking, 2022. arXiv: 2212.00169. preprint. — [arXiv:2212.00169](https://arxiv.org/abs/2212.00169) [find](https://scholar.google.com/scholar?q=David+Zhang%2C+Micah+Carroll%2C+Andreea+Bobu%2C+and+Anca+Dragan%2E+Time%2DEfficient+Reward+Learning+via+Visually+Assisted+Cluster+Ranking%2C+2022%2E+arXiv%3A+2212%2E00169%2E+preprint%2E+%E2%80%94+%5BarXiv%3A2212%2E00169%5D%28https%3A%2F%2Farxiv%2Eo)
411. Guoxi Zhang and Hisashi Kashima. Learning state importance for preference-based reinforcement learning. Machine Learning, 2023. doi: 10.1007/s10994-022-06295-5. — [doi:10.1007/s10994-022-06295-5](https://doi.org/10.1007/s10994-022-06295-5) [find](https://scholar.google.com/scholar?q=Guoxi+Zhang+and+Hisashi+Kashima%2E+Learning+state+importance+for+preference%2Dbased+reinforcement+learning%2E+Machine+Learning%2C+2023%2E+doi%3A+10%2E1007%2Fs10994%2D022%2D06295%2D5%2E+%E2%80%94+%5Bdoi%3A10%2E1007%2Fs10994%2D022%2D06295%2D5%5D%28http)
412. Han Zhang, Yu Lei, Lin Gui, Min Yang, Yulan He, Hui Wang, and Ruifeng Xu. CPPO: Continual Learning for Reinforcement Learning with Human Feedback. In Proceedings of the International Conference on Learning Representations (ICLR), 2024. URL [https://openreview.net/forum?id=86zAUE80pP](https://openreview.net/forum?id=86zAUE80pP). [↗](https://openreview.net/forum?id=86zAUE80pP)
413. Ruohan Zhang, Faraz Torabi, Garrett Warnell, and Peter Stone. Recent advances in leveraging human guidance for sequential decision-making tasks. Autonomous Agents and Multi-Agent Systems, 35(2):31, 2021. doi: 10.1007/s10458-021-09514-w. — [doi:10.1007/s10458-021-09514-w](https://doi.org/10.1007/s10458-021-09514-w) [find](https://scholar.google.com/scholar?q=Ruohan+Zhang%2C+Faraz+Torabi%2C+Garrett+Warnell%2C+and+Peter+Stone%2E+Recent+advances+in+leveraging+human+guidance+for+sequential+decision%2Dmaking+tasks%2E+Autonomous+Agents+and+Multi%2DAgent+Systems%2C+35%282%29%3A31%2C+20)
414. Tianjun Zhang, Fangchen Liu, Justin Wong, Pieter Abbeel, and Joseph E. Gonzalez. The Wisdom of Hindsight Makes Language Models Better Instruction Followers. In Proceedings of the International Conference on Machine Learning (ICML). PMLR, 2023a. URL [https://proceedings.mlr.press/v202/zhang23ab](https://proceedings.mlr.press/v202/zhang23ab). html. [↗](https://proceedings.mlr.press/v202/zhang23ab)
415. Yue Zhang, Yafu Li, Leyang Cui, Deng Cai, Lemao Liu, Tingchen Fu, Xinting Huang, Enbo Zhao, Yu Zhang, Yulong Chen, Longyue Wang, Anh Tuan Luu, Wei Bi, Freda Shi, and Shuming Shi. Siren’s Song in the AI Ocean: A Survey on Hallucination in Large Language Models, 2023b. [arXiv: 2309.01219](https://md.research.elamur.ai/free/en/papers/zhang-2023-siren-s-song-ai-ocean/). preprint.
416. Yao Zhao, Rishabh Joshi, Tianqi Liu, Misha Khalman, Mohammad Saleh, and Peter J. Liu. [SLiC-HF: Sequence Likelihood Calibration with Human Feedback](https://md.research.elamur.ai/free/en/papers/zhao-2023-slic-hf-sequence-likelihood-calibration/), 2023. arXiv: 2305.10425. preprint.
417. Rui Zheng, Wei Shen, Yuan Hua, Wenbin Lai, Shihan Dou, Yuhao Zhou, Zhiheng Xi, Xiao Wang, Haoran Huang, Tao Gui, Qi Zhang, and Xuanjing Huang. Improving Generalization of Alignment with Human Preferences through Group Invariant Learning. In Proceedings of the International Conference on Learning Rep… [find](https://scholar.google.com/scholar?q=Rui+Zheng%2C+Wei+Shen%2C+Yuan+Hua%2C+Wenbin+Lai%2C+Shihan+Dou%2C+Yuhao+Zhou%2C+Zhiheng+Xi%2C+Xiao+Wang%2C+Haoran+Huang%2C+Tao+Gui%2C+Qi+Zhang%2C+and+Xuanjing+Huang%2E+Improving+Generalization+of+Alignment+with+Human+Preferen)
418. Huiying Zhong, Zhun Deng, Weijie J. Su, Zhiwei Steven Wu, and Linjun Zhang. Provable Multi-Party Reinforcement Learning with Diverse Human Feedback, 2024a. arXiv: 2403.05006. preprint. — [arXiv:2403.05006](https://arxiv.org/abs/2403.05006) [find](https://scholar.google.com/scholar?q=Huiying+Zhong%2C+Zhun+Deng%2C+Weijie+J%2E+Su%2C+Zhiwei+Steven+Wu%2C+and+Linjun+Zhang%2E+Provable+Multi%2DParty+Reinforcement+Learning+with+Diverse+Human+Feedback%2C+2024a%2E+arXiv%3A+2403%2E05006%2E+preprint%2E+%E2%80%94+%5BarXiv%3A2403%2E0)
419. Yifan Zhong, Chengdong Ma, Xiaoyuan Zhang, Ziran Yang, Haojun Chen, Qingfu Zhang, Siyuan Qi, and Yaodong Yang. Panacea: Pareto Alignment via Preference Adaptation for LLMs. In Advances in Neural Information Processing Systems (NeurIPS), volume 37, 2024b. URL [https://proceedings.neurips.cc/](https://proceedings.neurips.cc/) paper_fil… [↗](https://proceedings.neurips.cc/)
420. Chunting Zhou, Pengfei Liu, Puxin Xu, Srinivasan Iyer, Jiao Sun, Yuning Mao, Xuezhe Ma, Avia Efrat, Ping Yu, Lili Yu, Susan Zhang, Gargi Ghosh, Mike Lewis, Luke Zettlemoyer, and Omer Levy. LIMA: Less Is More for Alignment. In Advances in Neural Information Processing Systems (NeurIPS), volume 36, 20… [find](https://scholar.google.com/scholar?q=Chunting+Zhou%2C+Pengfei+Liu%2C+Puxin+Xu%2C+Srinivasan+Iyer%2C+Jiao+Sun%2C+Yuning+Mao%2C+Xuezhe+Ma%2C+Avia+Efrat%2C+Ping+Yu%2C+Lili+Yu%2C+Susan+Zhang%2C+Gargi+Ghosh%2C+Mike+Lewis%2C+Luke+Zettlemoyer%2C+and+Omer+Levy%2E+LIMA%3A+Less+)
421. Wangchunshu Zhou and Ke Xu. Learning to Compare for Better Training and Evaluation of Open Domain Natural Language Generation Models. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 34, 2020. doi: 10.1609/aaai.v34i05.6521. — [doi:10.1609/aaai.v34i05.6521](https://doi.org/10.1609/aaai.v34i05.6521) [find](https://scholar.google.com/scholar?q=Wangchunshu+Zhou+and+Ke+Xu%2E+Learning+to+Compare+for+Better+Training+and+Evaluation+of+Open+Domain+Natural+Language+Generation+Models%2E+In+Proceedings+of+the+AAAI+Conference+on+Artificial+Intelligence%2C+)
422. Yifei Zhou, Andrea Zanette, Jiayi Pan, Sergey Levine, and Aviral Kumar. ArCHer: Training Language Model Agents via Hierarchical Multi-Turn RL. In Proceedings of the International Conference on Machine Learning (ICML). PMLR, 2024. URL [https://proceedings.mlr.press/v235/zhou24t.html](https://proceedings.mlr.press/v235/zhou24t.html). [↗](https://proceedings.mlr.press/v235/zhou24t.html)
423. Banghua Zhu, Michael Jordan, and Jiantao Jiao. Principled Reinforcement Learning with Human Feedback from Pairwise or K-wise Comparisons. In Proceedings of the International Conference on Machine Learning (ICML). PMLR, 2023a. URL [https://proceedings.mlr.press/v202/zhu23f.html](https://proceedings.mlr.press/v202/zhu23f.html). [↗](https://proceedings.mlr.press/v202/zhu23f.html)
424. Banghua Zhu, Hiteshi Sharma, Felipe Vieira Frujeri, Shi Dong, Chenguang Zhu, Michael I. Jordan, and Jiantao Jiao. Fine-Tuning Language Models with Advantage-Induced Policy Alignment, 2023b. arXiv: 2306.02231. preprint. — [arXiv:2306.02231](https://arxiv.org/abs/2306.02231) [find](https://scholar.google.com/scholar?q=Banghua+Zhu%2C+Hiteshi+Sharma%2C+Felipe+Vieira+Frujeri%2C+Shi+Dong%2C+Chenguang+Zhu%2C+Michael+I%2E+Jordan%2C+and+Jiantao+Jiao%2E+Fine%2DTuning+Language+Models+with+Advantage%2DInduced+Policy+Alignment%2C+2023b%2E+arXiv%3A+230)
425. Banghua Zhu, Michael Jordan, and Jiantao Jiao. Iterative Data Smoothing: Mitigating Reward Overfitting and Overoptimization in RLHF. In Proceedings of the International Conference on Machine Learning (ICML), 2024. URL [https://openreview.net/forum?id=WXg6MJo1FH](https://openreview.net/forum?id=WXg6MJo1FH). [↗](https://openreview.net/forum?id=WXg6MJo1FH)
426. Brian D. Ziebart, Andrew Maas, J. Andrew Bagnell, and Anind K. Dey. Maximum entropy inverse reinforcement learning. In Proceedings of the 23rd National Conference on Artificial Intelligence - Volume 3. AAAI Press, 2008. ISBN 978-1-57735-368-3. URL [http://www.aaai.org/Library/AAAI/2008/aaai08-227](http://www.aaai.org/Library/AAAI/2008/aaai08-227). ph… [↗](http://www.aaai.org/Library/AAAI/2008/aaai08-227)
427. Daniel M. Ziegler, Nisan Stiennon, Jeffrey Wu, Tom B. Brown, Alec Radford, Dario Amodei, Paul Christiano, and Geoffrey Irving. Fine-Tuning Language Models from Human Preferences, 2020. arXiv: 1909.08593. preprint. [find](https://scholar.google.com/scholar?q=Daniel+M%2E+Ziegler%2C+Nisan+Stiennon%2C+Jeffrey+Wu%2C+Tom+B%2E+Brown%2C+Alec+Radford%2C+Dario+Amodei%2C+Paul+Christiano%2C+and+Geoffrey+Irving%2E+Fine%2DTuning+Language+Models+from+Human+Preferences%2C+2020%2E+arXiv%3A+1909%2E085)

*Updated 2 November 2026*