---
title: "LearnLM: Improving Gemini for learning — paper by Team (2024)"
description: "Most chat assistants default to simply handing over information, whereas a good tutor guides learners… 25 references, cited by 23 papers here."
authors: ["LearnLM Team"]
year: 2024
venue: "arXiv (Cornell University)"
source: "https://arxiv.org/abs/2412.16429"
arxiv: "2412.16429"
licence: "CC BY 4.0 (arXiv)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/team-2024-learnlm-improving-gemini-learning/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/team-2024-learnlm-improving-gemini-learning/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2024](https://md.research.elamur.ai/free/en/papers/2024/page/11/) / LearnLM: Improving Gemini for learning

# LearnLM: Improving Gemini for learning

**Authors:** LearnLM Team · **Year:** 2024 · **Venue:** arXiv (Cornell University) · **Licence:** [CC BY 4.0 (arXiv)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://arxiv.org/abs/2412.16429) · [PDF](https://arxiv.org/pdf/2412.16429) · [arXiv 2412.16429](https://arxiv.org/abs/2412.16429) · [OpenAlex](https://openalex.org/W4405766242)

**Citations:** 4 citations (OpenAlex, 2026-10-14) · [23](https://md.research.elamur.ai/free/en/papers/team-2024-learnlm-improving-gemini-learning/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
Most chat assistants default to simply handing over information, whereas a good tutor guides learners toward working things out. Pedagogy varies widely across subjects, ages, cultures and products, so the LearnLM Team at Google does not fix one definition of good teaching. Instead, they treat the problem as **pedagogical instruction following**: each training conversation opens with a System Instruction that describes the teaching behaviour it demonstrates. Teachers and developers can then set the behaviour they want through prompts.

Their pedagogical data was mixed directly into Gemini's own post-training rather than applied as a separate fine-tune afterwards. This covered supervised demonstrations, human preference labels for a reward model, and RLHF. Co-training keeps the model's general abilities intact and makes it easier to stay in step with new Gemini versions.

In a scenario-based, side-by-side study, pedagogy experts clearly preferred the resulting LearnLM over GPT-4o, Claude 3.5 Sonnet and its own base model. This suggests that a general-purpose model can become a more controllable tutor without being tuned separately for each application.

## Key points
- LearnLM is built on gemini-1.5-pro-002 and has been made available in experimental form through Google AI Studio. Part of its improvements already went into Gemini 2.0.
- Training combines SFT with RLHF, using a reward model trained on turn-level labels of how well responses followed the instructions. The authors found RL much more effective than SFT for following subtle pedagogical instructions.
- The evaluation used 49 learning scenarios. 186 educators and experts role-played learners, and a separate pool of 248 experts judged conversation pairs on a seven-point scale.
- Average expert preference for LearnLM was +31% over GPT-4o, +11% over Claude 3.5 Sonnet and +13% over Gemini 1.5 Pro.
- In learners' explanations, LearnLM was praised for:
  - staying on topic,
  - challenging the learner,
  - not giving away answers.
  
  It was sometimes criticised on clarity and style. An appendix reports a feasibility study in medical education.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

📚 Shelves (5): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 19,503 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 18,805 works · [🗄️ Information Systems](https://md.research.elamur.ai/free/en/shelves/computer-science/information-systems/) · 1,888 works · [🩺 Medicine](https://md.research.elamur.ai/free/en/shelves/medicine/) · 288 works · [🩺 Health Informatics](https://md.research.elamur.ai/free/en/shelves/medicine/health-informatics/) · 227 works

## Cite

Cite as: LearnLM Team. 2024. LearnLM: Improving Gemini for learning. arXiv (Cornell University).

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@misc{team2024learnlm,
  title = {{LearnLM: Improving Gemini for learning}},
  author = {LearnLM Team},
  year = {2024},
  eprint = {2412.16429},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2412.16429}
}
```

- [← previous L2CEval: Evaluating Language-to-Code Generation Capabilities of Large Language Models](https://md.research.elamur.ai/free/en/papers/ni-2024-l2ceval-evaluating-language-code-generation/)
- [next → LLM-as-a-Judge & Reward Model: What They Can and Cannot Do](https://md.research.elamur.ai/free/en/papers/son-2024-llm-judge-reward-model-they/)
- [↑ back to papers of 2024](https://md.research.elamur.ai/free/en/papers/2024/page/11/)

## References

25 references, 6 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Irina Jurenka, Markus Kunesch, Kevin R McKee, Daniel Gillick, Shaojian Zhu, Sara Wiltberger, Shubham Milind Phal, Katherine Hermann, Daniel Kasenberg, Avishkar Bhoopchand, et al. [Towards responsible development of generative ai for education: An evaluation-driven approach](https://md.research.elamur.ai/free/en/papers/jurenka-2024-responsible-development-generative-ai-education/). arXiv preprint arXiv:2407.…
2. Daniel M Ziegler, Nisan Stiennon, Jeffrey Wu, Tom B Brown, Alec Radford, Dario Amodei, Paul Christiano, and Geoffrey Irving. Fine-tuning language models from human preferences. arXiv preprint arXiv:1909.08593, 2019. [find](https://scholar.google.com/scholar?q=Daniel+M+Ziegler%2C+Nisan+Stiennon%2C+Jeffrey+Wu%2C+Tom+B+Brown%2C+Alec+Radford%2C+Dario+Amodei%2C+Paul+Christiano%2C+and+Geoffrey+Irving%2E+Fine%2Dtuning+language+models+from+human+preferences%2E+arXiv+preprint+arXiv%3A19)
3. Gemini Team, Petko Georgiev, Ving Ian Lei, Ryan Burnell, Libin Bai, Anmol Gulati, Garrett Tanzer, Damien Vincent, Zhufeng Pan, Shibo Wang, et al. [Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context](https://md.research.elamur.ai/free/en/papers/gemini-team-2024-gemini-1-5-unlocking-multimodal/). arXiv preprint arXiv:2403.05530, 2024. 7 At the time of publication,…
4. . [4] Sundar Pichai, Demis Hassabis, and Koray Kavukcuoglu. Introducing gemini 2.0: our new ai model for the agentic era. [https://blog.google/technology/google-deepmind/](https://blog.google/technology/google-deepmind/) google-gemini-ai-update-december-2024/ , 2024. [↗](https://blog.google/technology/google-deepmind/)
5. Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. [Training language models to follow instructions with human feedback](https://md.research.elamur.ai/free/en/papers/ouyang-2022-training-language-models-follow-instructions/). Advances in neural information processing systems, 35: 27730–27744, 2022.
6. Ethan Mollick and Lilach Mollick. Assigning ai: Seven approaches for students, with prompts. arXiv preprint arXiv:2306.10052, 2023. — [arXiv:2306.10052](https://arxiv.org/abs/2306.10052) [find](https://scholar.google.com/scholar?q=Ethan+Mollick+and+Lilach+Mollick%2E+Assigning+ai%3A+Seven+approaches+for+students%2C+with+prompts%2E+arXiv+preprint+arXiv%3A2306%2E10052%2C+2023%2E+%E2%80%94+%5BarXiv%3A2306%2E10052%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2306%2E10052%29)
7. Bosi Wen, Pei Ke, Xiaotao Gu, Lindong Wu, Hao Huang, Jinfeng Zhou, Wenchuang Li, Binxin Hu, Wendy Gao, Jiaxin Xu, et al. Benchmarking complex instruction-following with multiple constraints composition. arXiv preprint arXiv:2407.03978, 2024. — [arXiv:2407.03978](https://arxiv.org/abs/2407.03978) [find](https://scholar.google.com/scholar?q=Bosi+Wen%2C+Pei+Ke%2C+Xiaotao+Gu%2C+Lindong+Wu%2C+Hao+Huang%2C+Jinfeng+Zhou%2C+Wenchuang+Li%2C+Binxin+Hu%2C+Wendy+Gao%2C+Jiaxin+Xu%2C+et+al%2E+Benchmarking+complex+instruction%2Dfollowing+with+multiple+constraints+compositio)
8. Jeffrey Zhou, Tianjian Lu, Swaroop Mishra, Siddhartha Brahma, Sujoy Basu, Yi Luan, Denny Zhou, and Le Hou. [Instruction-following evaluation for large language models](https://md.research.elamur.ai/free/en/papers/zhou-2023-instruction-following-evaluation-large-language/). arXiv preprint arXiv:2311.07911, 2023.
9. Yiwei Qin, Kaiqiang Song, Yebowen Hu, Wenlin Yao, Sangwoo Cho, Xiaoyang Wang, Xuansheng Wu, Fei Liu, Pengfei Liu, and Dong Yu. [Infobench: Evaluating instruction following ability in large language models](https://md.research.elamur.ai/free/en/papers/qin-2024-infobench-evaluating-instruction-following-ability/). arXiv preprint arXiv:2401.03601, 2024.
10. Lujain Ibrahim, Saffron Huang, Lama Ahmad, and Markus Anderljung. Beyond static ai evaluations: advancing human interaction evaluations for llm harms and risks. arXiv preprint arXiv:2405.10632, 2024. — [arXiv:2405.10632](https://arxiv.org/abs/2405.10632) [find](https://scholar.google.com/scholar?q=Lujain+Ibrahim%2C+Saffron+Huang%2C+Lama+Ahmad%2C+and+Markus+Anderljung%2E+Beyond+static+ai+evaluations%3A+advancing+human+interaction+evaluations+for+llm+harms+and+risks%2E+arXiv+preprint+arXiv%3A2405%2E10632%2C+2024%2E+)
11. Kevin R McKee. Human participants in AI research: Ethics and transparency in practice. IEEE Transactions on Technology and Society, 5(3):279–288, 2024. doi: 10.1109/TTS.2024.3446183. — [doi:10.1109/tts.2024.3446183](https://doi.org/10.1109/tts.2024.3446183) [find](https://scholar.google.com/scholar?q=Kevin+R+McKee%2E+Human+participants+in+AI+research%3A+Ethics+and+transparency+in+practice%2E+IEEE+Transactions+on+Technology+and+Society%2C+5%283%29%3A279%E2%80%93288%2C+2024%2E+doi%3A+10%2E1109%2FTTS%2E2024%2E3446183%2E+%E2%80%94+%5Bdoi%3A10%2E1109%2Ftt)
12. Andrew Gelman, John B Carlin, Hal S Stern, and Donald B Rubin. Bayesian data analysis. Chapman and Hall/CRC, 1995. [find](https://scholar.google.com/scholar?q=Andrew+Gelman%2C+John+B+Carlin%2C+Hal+S+Stern%2C+and+Donald+B+Rubin%2E+Bayesian+data+analysis%2E+Chapman+and+Hall%2FCRC%2C+1995%2E)
13. Wei-Lin Chiang Tianle Li, Anastasios Angelopoulos. Does style matter? disentangling style and substance in chatbot arena. [https://blog.lmarena.ai/blog/2024/style-control/](https://blog.lmarena.ai/blog/2024/style-control/) , August 2024. [↗](https://blog.lmarena.ai/blog/2024/style-control/)
14. Fred D Davis. Perceived usefulness, perceived ease of use and user acceptance of information technology. MIS quarterly, 1989. [find](https://scholar.google.com/scholar?q=Fred+D+Davis%2E+Perceived+usefulness%2C+perceived+ease+of+use+and+user+acceptance+of+information+technology%2E+MIS+quarterly%2C+1989%2E)
15. Susan T Fiske, Amy JC Cuddy, and Peter Glick. Universal dimensions of social cognition: Warmth and competence. Trends in cognitive sciences, 11(2):77–83, 2007. [find](https://scholar.google.com/scholar?q=Susan+T+Fiske%2C+Amy+JC+Cuddy%2C+and+Peter+Glick%2E+Universal+dimensions+of+social+cognition%3A+Warmth+and+competence%2E+Trends+in+cognitive+sciences%2C+11%282%29%3A77%E2%80%9383%2C+2007%2E)
16. Kevin R McKee, Xuechunzi Bai, and Susan T Fiske. Humans perceive warmth and competence in artificial intelligence. iScience, 26(8), 2023. doi: 10.1016/j.isci.2023.107256. — [doi:10.1016/j.isci.2023.107256](https://doi.org/10.1016/j.isci.2023.107256) [find](https://scholar.google.com/scholar?q=Kevin+R+McKee%2C+Xuechunzi+Bai%2C+and+Susan+T+Fiske%2E+Humans+perceive+warmth+and+competence+in+artificial+intelligence%2E+iScience%2C+26%288%29%2C+2023%2E+doi%3A+10%2E1016%2Fj%2Eisci%2E2023%2E107256%2E+%E2%80%94+%5Bdoi%3A10%2E1016%2Fj%2Eisci%2E2023%2E10)
17. Team Gemini, Rohan Anil, Sebastian Borgeaud, Yonghui Wu, Jean-Baptiste Alayrac, Jiahui Yu, Radu Soricut, Johan Schalkwyk, Andrew M Dai, Anja Hauth, et al. [Gemini: A family of highly capable multimodal models](https://md.research.elamur.ai/free/en/papers/gemini-team-2023-gemini-family-highly-capable-multimodal/). arXiv preprint arXiv:2312.11805, 2023.
18. Paul A Kirschner and Carl Hendrick. How learning happens: Seminal works in educational psychology and what they mean in practice. Routledge, 2020. [find](https://scholar.google.com/scholar?q=Paul+A+Kirschner+and+Carl+Hendrick%2E+How+learning+happens%3A+Seminal+works+in+educational+psychology+and+what+they+mean+in+practice%2E+Routledge%2C+2020%2E)
19. Gregory Kestin, Kelly Miller, Anna Klales, Timothy Milbourne, and Gregorio Ponti. Ai tutoring outperforms active learning. 2024. [find](https://scholar.google.com/scholar?q=Gregory+Kestin%2C+Kelly+Miller%2C+Anna+Klales%2C+Timothy+Milbourne%2C+and+Gregorio+Ponti%2E+Ai+tutoring+outperforms+active+learning%2E+2024%2E)
20. Rose E Wang, Ana T Ribeiro, Carly D Robinson, Susanna Loeb, and Dora Demszky. Tutor copilot: A human-ai approach for scaling real-time expertise. arXiv preprint arXiv:2410.03017, 2024. [find](https://scholar.google.com/scholar?q=Rose+E+Wang%2C+Ana+T+Ribeiro%2C+Carly+D+Robinson%2C+Susanna+Loeb%2C+and+Dora+Demszky%2E+Tutor+copilot%3A+A+human%2Dai+approach+for+scaling+real%2Dtime+expertise%2E+arXiv+preprint+arXiv%3A2410%2E03017%2C+2024%2E)
21. Hamsa Bastani, Osbert Bastani, Alp Sungu, Haosen Ge, Ozge Kabakcı, and Rei Mariman. Generative ai can harm learning. Available at SSRN, 4895486, 2024. [find](https://scholar.google.com/scholar?q=Hamsa+Bastani%2C+Osbert+Bastani%2C+Alp+Sungu%2C+Haosen+Ge%2C+Ozge+Kabakc%C4%B1%2C+and+Rei+Mariman%2E+Generative+ai+can+harm+learning%2E+Available+at+SSRN%2C+4895486%2C+2024%2E)
22. National ing in Education the age Association (NEA). of artificial intelligence. Teaching and learn-https://www.nea. org/resource-library/artificial-intelligence-education/ iv-teaching-and-learning-age-artificial-intelligence . Accessed: 2024-12-10. [find](https://scholar.google.com/scholar?q=National+ing+in+Education+the+age+Association+%28NEA%29%2E+of+artificial+intelligence%2E+Teaching+and+learn%2Dhttps%3A%2F%2Fwww%2Enea%2E+org%2Fresource%2Dlibrary%2Fartificial%2Dintelligence%2Deducation%2F+iv%2Dteaching%2Dand%2Dlearning%2Dag)
23. Kimberly Lomis, Pamela Jeffries, Anthony Palatta, Melanie Sage, Javaid Sheikh, Carl Sheperis, and Alison Whelan. Artificial intelligence for health professions educators. NAM perspectives, 2021, 2021. [find](https://scholar.google.com/scholar?q=Kimberly+Lomis%2C+Pamela+Jeffries%2C+Anthony+Palatta%2C+Melanie+Sage%2C+Javaid+Sheikh%2C+Carl+Sheperis%2C+and+Alison+Whelan%2E+Artificial+intelligence+for+health+professions+educators%2E+NAM+perspectives%2C+2021%2C+2021%2E)
24. Sanjay V Desai, Jesse Burk-Rafel, Kimberly D Lomis, Kelly Caverzagie, Judee Richardson, Celia Laird O’Brien, John Andrews, Kevin Heckman, David Henderson, Charles G Prober, et al. Precision education: the future of lifelong learning in medicine. Academic Medicine, pages 10–1097, 2023. [find](https://scholar.google.com/scholar?q=Sanjay+V+Desai%2C+Jesse+Burk%2DRafel%2C+Kimberly+D+Lomis%2C+Kelly+Caverzagie%2C+Judee+Richardson%2C+Celia+Laird+O%E2%80%99Brien%2C+John+Andrews%2C+Kevin+Heckman%2C+David+Henderson%2C+Charles+G+Prober%2C+et+al%2E+Precision+education%3A)
25. Kevin R McKee, Xuechunzi Bai, and Susan T Fiske. Warmth and competence in human-agent cooperation. Autonomous Agents and Multi-Agent Systems, 38(1):23, 2024. Contributions and Acknowledgments Core Contributors Abhinit Modi, Aditya Srikanth Veerubhotla, Aliya Rysbek, Andrea Huber, Brett Wiltshire, Br… [find](https://scholar.google.com/scholar?q=Kevin+R+McKee%2C+Xuechunzi+Bai%2C+and+Susan+T+Fiske%2E+Warmth+and+competence+in+human%2Dagent+cooperation%2E+Autonomous+Agents+and+Multi%2DAgent+Systems%2C+38%281%29%3A23%2C+2024%2E+Contributions+and+Acknowledgments+Core+Con)

*Updated 3 November 2026*