---
title: "Llama 2: Open Foundation and Fine-Tuned Chat Models (2023)"
description: "However, they cannot replace ready-made chat products like ChatGPT, BARD or Claude… 135 references, cited by 7,074 papers here."
authors: ["Hugo Touvron", "Louis Martin", "Kevin Stone", "Peter Albert", "Amjad Almahairi", "Yasmine Babaei", "Nikolay Bashlykov", "Soumya Batra", "Prajjwal Bhargava", "Shruti Bhosale", "Dan Bikel", "Lukas Blecher", "Cristian Canton Ferrer", "Moya Chen", "Guillem Cucurull", "David Esiobu", "Jude Fernandes", "Jeremy Fu", "Wenyin Fu", "Brian Fuller", "Cynthia Gao", "Vedanuj Goswami", "Naman Goyal", "Anthony Hartshorn", "Saghar Hosseini", "Rui Hou", "Hakan Inan", "Marcin Kardas", "Viktor Kerkez", "Madian Khabsa", "Isabel Kloumann", "Artem Korenev", "Punit Singh Koura", "Marie-Anne Lachaux", "Thibaut Lavril", "Jenya Lee", "Diana Liskovich", "Yinghai Lu", "Yuning Mao", "Xavier Martinet", "Todor Mihaylov", "Pushkar Mishra", "Igor Molybog", "Yixin Nie", "Andrew Poulton", "Jeremy Reizenstein", "Rashi Rungta", "Kalyan Saladi", "Alan Schelten", "Ruan Silva", "Eric Michael Smith", "Ranjan Subramanian", "Xiaoqing Ellen Tan", "Binh Tang", "Ross Taylor", "Adina Williams", "Jian Xiang Kuan", "Puxin Xu", "Zheng Yan", "Iliyan Zarov", "Yuchen Zhang", "Angela Fan", "Melanie Kambadur", "Sharan Narang", "Aurelien Rodriguez", "Robert Stojnic", "Sergey Edunov", "Thomas Scialom"]
year: 2023
venue: "arXiv:2307.09288"
source: "https://arxiv.org/abs/2307.09288"
doi: "10.48550/arxiv.2307.09288"
arxiv: "2307.09288"
licence: "arXiv — non-exclusive licence to distribute"
licence_url: "http://arxiv.org/licenses/nonexclusive-distrib/1.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/touvron-2023-llama-2-open-foundation-fine/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/touvron-2023-llama-2-open-foundation-fine/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2023](https://md.research.elamur.ai/free/en/papers/2023/) / Llama 2: Open Foundation and Fine-Tuned Chat Models

# Llama 2: Open Foundation and Fine-Tuned Chat Models

**Authors:** [Hugo Touvron](https://md.research.elamur.ai/free/en/authors/hugo-touvron/), [Louis Martin](https://md.research.elamur.ai/free/en/authors/louis-martin/), Kevin Stone, Peter Albert, [Amjad Almahairi](https://md.research.elamur.ai/free/en/authors/amjad-almahairi/), Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, [Prajjwal Bhargava](https://md.research.elamur.ai/free/en/authors/prajjwal-bhargava/), [Shruti Bhosale](https://md.research.elamur.ai/free/en/authors/shruti-bhosale/), Dan Bikel, Lukas Blecher, Cristian Canton Ferrer, [Moya Chen](https://md.research.elamur.ai/free/en/authors/moya-chen/), [Guillem Cucurull](https://md.research.elamur.ai/free/en/authors/guillem-cucurull/), David Esiobu, Jude Fernandes, Jeremy Fu, Wenyin Fu, Brian Fuller, Cynthia Gao, Vedanuj Goswami, [Naman Goyal](https://md.research.elamur.ai/free/en/authors/naman-goyal/), [Anthony Hartshorn](https://md.research.elamur.ai/free/en/authors/anthony-hartshorn/), Saghar Hosseini, [Rui Hou](https://md.research.elamur.ai/free/en/authors/rui-hou/), Hakan Inan, Marcin Kardas, Viktor Kerkez, [Madian Khabsa](https://md.research.elamur.ai/free/en/authors/madian-khabsa/), Isabel Kloumann, Artem Korenev, Punit Singh Koura, [Marie-Anne Lachaux](https://md.research.elamur.ai/free/en/authors/marie-anne-lachaux/), [Thibaut Lavril](https://md.research.elamur.ai/free/en/authors/thibaut-lavril/), Jenya Lee, Diana Liskovich, Yinghai Lu, [Yuning Mao](https://md.research.elamur.ai/free/en/authors/yuning-mao/), [Xavier Martinet](https://md.research.elamur.ai/free/en/authors/xavier-martinet/), [Todor Mihaylov](https://md.research.elamur.ai/free/en/authors/todor-mihaylov/), [Pushkar Mishra](https://md.research.elamur.ai/free/en/authors/pushkar-mishra/), Igor Molybog, [Yixin Nie](https://md.research.elamur.ai/free/en/authors/yixin-nie/), Andrew Poulton, Jeremy Reizenstein, [Rashi Rungta](https://md.research.elamur.ai/free/en/authors/rashi-rungta/), Kalyan Saladi, [Alan Schelten](https://md.research.elamur.ai/free/en/authors/alan-schelten/), Ruan Silva, [Eric Michael Smith](https://md.research.elamur.ai/free/en/authors/eric-michael-smith/), Ranjan Subramanian, [Xiaoqing Ellen Tan](https://md.research.elamur.ai/free/en/authors/xiaoqing-ellen-tan/), Binh Tang, Ross Taylor, [Adina Williams](https://md.research.elamur.ai/free/en/authors/adina-williams/), Jian Xiang Kuan, Puxin Xu, Zheng Yan, Iliyan Zarov, [Yuchen Zhang](https://md.research.elamur.ai/free/en/authors/yuchen-zhang/), [Angela Fan](https://md.research.elamur.ai/free/en/authors/angela-fan/), Melanie Kambadur, [Sharan Narang](https://md.research.elamur.ai/free/en/authors/sharan-narang/), [Aurelien Rodriguez](https://md.research.elamur.ai/free/en/authors/aurelien-rodriguez/), Robert Stojnic, [Sergey Edunov](https://md.research.elamur.ai/free/en/authors/sergey-edunov/) and [Thomas Scialom](https://md.research.elamur.ai/free/en/authors/thomas-scialom/) · **Year:** 2023 · **Venue:** arXiv:2307.09288 · **Licence:** [arXiv — non-exclusive licence to distribute](http://arxiv.org/licenses/nonexclusive-distrib/1.0/)

**Links:** [Publisher's page](https://arxiv.org/abs/2307.09288) · [PDF](https://arxiv.org/pdf/2307.09288) · [DOI 10.48550/arxiv.2307.09288](https://doi.org/10.48550/arxiv.2307.09288) · [arXiv 2307.09288](https://arxiv.org/abs/2307.09288) · [OpenAlex](https://openalex.org/W4384918448)

**Citations:** 2,595 citations (OpenAlex, 2026-10-14) · [7074](https://md.research.elamur.ai/free/en/papers/touvron-2023-llama-2-open-foundation-fine/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
The paper addresses the gap between open and closed large language models. Open LLMs such as BLOOM, LLaMa-1 or Falcon are already close to GPT-3 and Chinchilla in their base capabilities. However, they cannot replace ready-made chat products like ChatGPT, BARD or Claude, because those are carefully tuned to human preferences. The Meta team built and released the Llama 2 family, with models ranging from 7 to 70 billion parameters. It includes base models and the dialogue versions Llama 2-Chat. The base model was trained on an updated mix of publicly available data. It was then fine-tuned through supervised fine-tuning and several iterations of RLHF, which used rejection sampling and PPO. Separately, the authors worked on safety: they collected dedicated annotations, conducted red teaming and repeated evaluations. Llama 2-Chat mostly outperforms open chat models. In human evaluations of helpfulness and safety, it approaches some of the closed systems but still lags behind GPT-4. The value of the paper lies in the model being available for research and commercial use, and in the fine-tuning process being described in detail and openly.

## Key points
- Models with 7B, 13B and 70B parameters were released for public access in base and Chat versions. A 34B variant was trained but has not yet been released, because there was not enough time to test it sufficiently through red teaming.
- Compared with Llama 1, the amount of training data grew by about 40% and amounts to 2T tokens. The context was doubled to 4k tokens. The 34B and 70B models use grouped-query attention (GQA) to run faster during inference.
- The architecture is a standard transformer with RMSNorm, SwiGLU and RoPE. Training used AdamW and a cosine learning rate schedule. Data from Meta's products was not included in the corpus, and the authors tried to filter out sources with large amounts of personal information.
- Humans evaluated helpfulness on about 4 thousand prompts and safety on about 2 thousand adversarial prompts. In addition, GPT-4 compared the responses.
- Testing was conducted in English only. Developers are therefore advised to carry out their own safety testing before using Llama 2 in their applications.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

📚 Shelves (6): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🩺 Medicine](https://md.research.elamur.ai/free/en/shelves/medicine/) · 241 works · [🩺 Health Informatics](https://md.research.elamur.ai/free/en/shelves/medicine/health-informatics/) · 184 works · [🏥 Health Professions](https://md.research.elamur.ai/free/en/shelves/health-professions/) · 69 works · [🏥 Health Information Management](https://md.research.elamur.ai/free/en/shelves/health-professions/health-information-management/) · 50 works

💡 Concepts (2): [💡 Large language models](https://md.research.elamur.ai/free/en/concepts/large-language-models/) · 48 works · [💡 Fine-tuning](https://md.research.elamur.ai/free/en/concepts/fine-tuning/) · 27 works

## Cite

Cite as: Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, Dan Bikel, Lukas Blecher, Cristian Canton Ferrer, Moya Chen, Guillem Cucurull, David Esiobu, Jude Fernandes, Jeremy Fu, Wenyin Fu, Brian Fuller, Cynthia Gao, Vedanuj Goswami, Naman Goyal, Anthony Hartshorn, Saghar Hosseini, Rui Hou, Hakan Inan, Marcin Kardas, Viktor Kerkez, Madian Khabsa, Isabel Kloumann, Artem Korenev, Punit Singh Koura, Marie-Anne Lachaux, Thibaut Lavril, Jenya Lee, Diana Liskovich, Yinghai Lu, Yuning Mao, Xavier Martinet, Todor Mihaylov, Pushkar Mishra, Igor Molybog, Yixin Nie, Andrew Poulton, Jeremy Reizenstein, Rashi Rungta, Kalyan Saladi, Alan Schelten, Ruan Silva, Eric Michael Smith, Ranjan Subramanian, Xiaoqing Ellen Tan, Binh Tang, Ross Taylor, Adina Williams, Jian Xiang Kuan, Puxin Xu, Zheng Yan, Iliyan Zarov, Yuchen Zhang, Angela Fan, Melanie Kambadur, Sharan Narang, Aurelien Rodriguez, Robert Stojnic, Sergey Edunov and Thomas Scialom. 2023. Llama 2: Open Foundation and Fine-Tuned Chat Models. arXiv:2307.09288.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@misc{touvron2023llama,
  title = {{Llama 2: Open Foundation and Fine-Tuned Chat Models}},
  author = {Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, Dan Bikel, Lukas Blecher, Cristian Canton Ferrer, Moya Chen, Guillem Cucurull, David Esiobu, Jude Fernandes, Jeremy Fu, Wenyin Fu, Brian Fuller, Cynthia Gao, Vedanuj Goswami, Naman Goyal, Anthony Hartshorn, Saghar Hosseini, Rui Hou, Hakan Inan, Marcin Kardas, Viktor Kerkez, Madian Khabsa, Isabel Kloumann, Artem Korenev, Punit Singh Koura, Marie-Anne Lachaux, Thibaut Lavril, Jenya Lee, Diana Liskovich, Yinghai Lu, Yuning Mao, Xavier Martinet, Todor Mihaylov, Pushkar Mishra, Igor Molybog, Yixin Nie, Andrew Poulton, Jeremy Reizenstein, Rashi Rungta, Kalyan Saladi, Alan Schelten, Ruan Silva, Eric Michael Smith, Ranjan Subramanian, Xiaoqing Ellen Tan, Binh Tang, Ross Taylor, Adina Williams, Jian Xiang Kuan, Puxin Xu, Zheng Yan, Iliyan Zarov, Yuchen Zhang, Angela Fan, Melanie Kambadur, Sharan Narang, Aurelien Rodriguez, Robert Stojnic, Sergey Edunov and Thomas Scialom},
  year = {2023},
  doi = {10.48550/arxiv.2307.09288},
  eprint = {2307.09288},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2307.09288}
}
```

- [← previous GPT-4 Technical Report](https://md.research.elamur.ai/free/en/papers/openai-2023-gpt-4-technical-report/)
- [next → LLaMA: Open and Efficient Foundation Language Models](https://md.research.elamur.ai/free/en/papers/touvron-2023-llama-open-efficient-foundation-language/)
- [↑ back to papers of 2023](https://md.research.elamur.ai/free/en/papers/2023/)

## References

135 references, 64 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Daron Acemoglu and Pascual Restrepo. Artificial intelligence, automation, and work. In The economics of artificial intelligence: An agenda, pages 197–236. University of Chicago Press, 2018. [find](https://scholar.google.com/scholar?q=Daron+Acemoglu+and+Pascual+Restrepo%2E+Artificial+intelligence%2C+automation%2C+and+work%2E+In+The+economics+of+artificial+intelligence%3A+An+agenda%2C+pages+197%E2%80%93236%2E+University+of+Chicago+Press%2C+2018%2E)
2. Joshua Ainslie, James Lee-Thorp, Michiel de Jong, Yury Zemlyanskiy, Federico Lebrón, and Sumit Sanghai. [Gqa: Training generalized multi-query transformer models from multi-head checkpoints](https://md.research.elamur.ai/free/en/papers/ainslie-2023-gqa-training-generalized-multi-query/), 2023.
3. Ebtesam Almazrouei, Hamza Alobeidli, Abdulaziz Alshamsi, Alessandro Cappelli, Ruxandra Cojocaru, Merouane Debbah, Etienne Goffinet, Daniel Heslow, Julien Launay, Quentin Malartic, Badreddine Noune, Baptiste Pannier, and Guilherme Penedo. Falcon-40B: an open large language model with state-of-the-art… [find](https://scholar.google.com/scholar?q=Ebtesam+Almazrouei%2C+Hamza+Alobeidli%2C+Abdulaziz+Alshamsi%2C+Alessandro+Cappelli%2C+Ruxandra+Cojocaru%2C+Merouane+Debbah%2C+Etienne+Goffinet%2C+Daniel+Heslow%2C+Julien+Launay%2C+Quentin+Malartic%2C+Badreddine+Noune%2C+Ba)
4. Rohan Anil, Andrew M. Dai, Orhan Firat, Melvin Johnson, Dmitry Lepikhin, Alexandre Passos, Siamak Shakeri, Emanuel Taropa, Paige Bailey, Zhifeng Chen, Eric Chu, Jonathan H. Clark, Laurent El Shafey, Yanping Huang, Kathy Meier-Hellstern, Gaurav Mishra, Erica Moreira, Mark Omernick, Kevin Robinson, Se… — [PaLM 2 Technical Report](https://md.research.elamur.ai/free/en/papers/anil-2023-palm-2-technical-report/)
5. Amanda Askell, Yuntao Bai, Anna Chen, Dawn Drain, Deep Ganguli, Tom Henighan, Andy Jones, Nicholas Joseph, Ben Mann, Nova DasSarma, Nelson Elhage, Zac Hatfield-Dodds, Danny Hernandez, Jackson Kernion, Kamal Ndousse, Catherine Olsson, Dario Amodei, Tom Brown, Jack Clark, Sam McCandlish, and Chris Ola… [find](https://scholar.google.com/scholar?q=Amanda+Askell%2C+Yuntao+Bai%2C+Anna+Chen%2C+Dawn+Drain%2C+Deep+Ganguli%2C+Tom+Henighan%2C+Andy+Jones%2C+Nicholas+Joseph%2C+Ben+Mann%2C+Nova+DasSarma%2C+Nelson+Elhage%2C+Zac+Hatfield%2DDodds%2C+Danny+Hernandez%2C+Jackson+Kernion%2C)
6. Amanda Askell, Yuntao Bai, Anna Chen, Dawn Drain, Deep Ganguli, Tom Henighan, Andy Jones, Nicholas Joseph, Ben Mann, Nova DasSarma, et al. A general language assistant as a laboratory for alignment. arXiv preprint arXiv:2112.00861, 2021b. [find](https://scholar.google.com/scholar?q=Amanda+Askell%2C+Yuntao+Bai%2C+Anna+Chen%2C+Dawn+Drain%2C+Deep+Ganguli%2C+Tom+Henighan%2C+Andy+Jones%2C+Nicholas+Joseph%2C+Ben+Mann%2C+Nova+DasSarma%2C+et+al%2E+A+general+language+assistant+as+a+laboratory+for+alignment%2E+a)
7. Jacob Austin, Augustus Odena, Maxwell Nye, Maarten Bosma, Henryk Michalewski, David Dohan, Ellen Jiang, Carrie Cai, Michael Terry, Quoc Le, and Charles Sutton. [Program synthesis with large language models](https://md.research.elamur.ai/free/en/papers/austin-2021-program-synthesis-large-language-models/), 2021.
8. David Autor and Anna Salomons. Is automation labor-displacing? productivity growth, employment, and the labor share. Technical report, National Bureau of Economic Research, 2018. [find](https://scholar.google.com/scholar?q=David+Autor+and+Anna+Salomons%2E+Is+automation+labor%2Ddisplacing%3F+productivity+growth%2C+employment%2C+and+the+labor+share%2E+Technical+report%2C+National+Bureau+of+Economic+Research%2C+2018%2E)
9. Yuntao Bai, Andy Jones, Kamal Ndousse, Amanda Askell, Anna Chen, Nova DasSarma, Dawn Drain, Stanislav Fort, Deep Ganguli, Tom Henighan, et al. Training a helpful and harmless assistant with reinforcement learning from human feedback. arXiv preprint arXiv:2204.05862, 2022a. [find](https://scholar.google.com/scholar?q=Yuntao+Bai%2C+Andy+Jones%2C+Kamal+Ndousse%2C+Amanda+Askell%2C+Anna+Chen%2C+Nova+DasSarma%2C+Dawn+Drain%2C+Stanislav+Fort%2C+Deep+Ganguli%2C+Tom+Henighan%2C+et+al%2E+Training+a+helpful+and+harmless+assistant+with+reinforcem)
10. Yuntao Bai, Saurav Kadavath, Sandipan Kundu, Amanda Askell, Jackson Kernion, Andy Jones, Anna Chen, Anna Goldie, Azalia Mirhoseini, Cameron McKinnon, et al. [Constitutional ai: Harmlessness from ai feedback](https://md.research.elamur.ai/free/en/papers/bai-2022-constitutional-ai-harmlessness-ai-feedback/). arXiv preprint arXiv:2212.08073, 2022b.
11. April H Bailey, Adina Williams, and Andrei Cimpian. Based on billions of words on the internet, people= men. Science Advances, 8(13):eabm2463, 2022. [find](https://scholar.google.com/scholar?q=April+H+Bailey%2C+Adina+Williams%2C+and+Andrei+Cimpian%2E+Based+on+billions+of+words+on+the+internet%2C+people%3D+men%2E+Science+Advances%2C+8%2813%29%3Aeabm2463%2C+2022%2E)
12. Emily M Bender, Timnit Gebru, Angelina McMillan-Major, and Margaret Mitchell. On the dangers of stochastic parrots: Can language models be too big? In Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency, pages 610–623, 2021a. [find](https://scholar.google.com/scholar?q=Emily+M+Bender%2C+Timnit+Gebru%2C+Angelina+McMillan%2DMajor%2C+and+Margaret+Mitchell%2E+On+the+dangers+of+stochastic+parrots%3A+Can+language+models+be+too+big%3F+In+Proceedings+of+the+2021+ACM+Conference+on+Fairnes)
13. Emily M Bender, Timnit Gebru, Angelina McMillan-Major, and Shmargaret Shmitchell. On the dangers of stochastic parrots: Can language models be too big? In Proceedings of the 2021 ACM conference on fairness, accountability, and transparency, pages 610–623, 2021b. [find](https://scholar.google.com/scholar?q=Emily+M+Bender%2C+Timnit+Gebru%2C+Angelina+McMillan%2DMajor%2C+and+Shmargaret+Shmitchell%2E+On+the+dangers+of+stochastic+parrots%3A+Can+language+models+be+too+big%3F+In+Proceedings+of+the+2021+ACM+conference+on+fai)
14. A Stevie Bergman, Gavin Abercrombie, Shannon L Spruit, Dirk Hovy, Emily Dinan, Y-Lan Boureau, and Verena Rieser. Guiding the release of safer e2e conversational ai through value sensitive design. In Proceedings of the 23rd Annual Meeting of the Special Interest Group on Discourse and Dialogue, pages… [find](https://scholar.google.com/scholar?q=A+Stevie+Bergman%2C+Gavin+Abercrombie%2C+Shannon+L+Spruit%2C+Dirk+Hovy%2C+Emily+Dinan%2C+Y%2DLan+Boureau%2C+and+Verena+Rieser%2E+Guiding+the+release+of+safer+e2e+conversational+ai+through+value+sensitive+design%2E+In+P)
15. Shaily Bhatt, Sunipa Dev, Partha Talukdar, Shachi Dave, and Vinodkumar Prabhakaran. Re-contextualizing fairness in nlp: The case of india, 2022. [find](https://scholar.google.com/scholar?q=Shaily+Bhatt%2C+Sunipa+Dev%2C+Partha+Talukdar%2C+Shachi+Dave%2C+and+Vinodkumar+Prabhakaran%2E+Re%2Dcontextualizing+fairness+in+nlp%3A+The+case+of+india%2C+2022%2E)
16. Yonatan Bisk, Rowan Zellers, Jianfeng Gao, Yejin Choi, et al. [Piqa: Reasoning about physical commonsense in natural language](https://md.research.elamur.ai/free/en/papers/bisk-2019-piqa-reasoning-about-physical-commonsense/). In Proceedings of the AAAI conference on artificial intelligence, pages 7432–7439, 2020.
17. Su Lin Blodgett, Gilsinia Lopez, Alexandra Olteanu, Robert Sim, and Hanna Wallach. Stereotyping norwegian salmon: An inventory of pitfalls in fairness benchmark datasets. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Confe… [find](https://scholar.google.com/scholar?q=Su+Lin+Blodgett%2C+Gilsinia+Lopez%2C+Alexandra+Olteanu%2C+Robert+Sim%2C+and+Hanna+Wallach%2E+Stereotyping+norwegian+salmon%3A+An+inventory+of+pitfalls+in+fairness+benchmark+datasets%2E+In+Proceedings+of+the+59th+An)
18. Piotr Bojanowski, Edouard Grave, Armand Joulin, and Tomás Mikolov. [Enriching word vectors with subword information](https://md.research.elamur.ai/free/en/papers/bojanowski-2017-enriching-word-vectors-subword-information/). CoRR, abs/1607.04606, 2016. URL http://arxiv.org/abs/1607.04606. [↗](http://arxiv.org/abs/1607.04606)
19. Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel Ziegler, Jeffrey Wu, Clemens Winter, Chri… [find](https://scholar.google.com/scholar?q=Tom+Brown%2C+Benjamin+Mann%2C+Nick+Ryder%2C+Melanie+Subbiah%2C+Jared+D+Kaplan%2C+Prafulla+Dhariwal%2C+Arvind+Neelakantan%2C+Pranav+Shyam%2C+Girish+Sastry%2C+Amanda+Askell%2C+Sandhini+Agarwal%2C+Ariel+Herbert%2DVoss%2C+Gretchen)
20. Curran Associates, Inc., 2020. URL [https://proceedings.neurips.cc/paper_files/paper/2020/file/1457c0d6bfcb4967418bfb8ac142f64a-Paper.pdf](https://proceedings.neurips.cc/paper_files/paper/2020/file/1457c0d6bfcb4967418bfb8ac142f64a-Paper.pdf). [↗](https://proceedings.neurips.cc/paper_files/paper/2020/file/1457c0d6bfcb4967418bfb8ac142f64a-Paper.pdf)
21. Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde de Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, Alex Ray, Raul Puri, Gretchen Krueger, Michael Petrov, Heidy Khlaaf, Girish Sastry, Pamela Mishkin, Brooke Chan, Scott Gray, Nick Ryder, Mikhai… — [Evaluating Large Language Models Trained on Code](https://md.research.elamur.ai/free/en/papers/chen-2021-evaluating-large-language-models-trained/)
22. Wei-Lin Chiang, Zhuohan Li, Zi Lin, Ying Sheng, Zhanghao Wu, Hao Zhang, Lianmin Zheng, Siyuan Zhuang, Yonghao Zhuang, Joseph E. Gonzalez, Ion Stoica, and Eric P. Xing. Vicuna: An open-source chatbot impressing gpt-4 with 90%* chatgpt quality, March 2023. URL [https://lmsys.org/blog/2023-03-30-vicuna/…](https://lmsys.org/blog/2023-03-30-vicuna/…) [↗](https://lmsys.org/blog/2023-03-30-vicuna/…)
23. Eunsol Choi, He He, Mohit Iyyer, Mark Yatskar, Wen-tau Yih, Yejin Choi, Percy Liang, and Luke Zettlemoyer. [Quac: Question answering in context](https://md.research.elamur.ai/free/en/papers/choi-2018-quac-question-answering-context/). In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 2174–2184, 2018.
24. Aakanksha Chowdhery, Sharan Narang, Jacob Devlin, Maarten Bosma, Gaurav Mishra, Adam Roberts, Paul Barham, Hyung Won Chung, Charles Sutton, Sebastian Gehrmann, Parker Schuh, Kensen Shi, Sasha Tsvyashchenko, Joshua Maynez, Abhishek Rao, Parker Barnes, Yi Tay, Noam Shazeer, Vinodkumar Prabhakaran, Emi… — [PaLM: Scaling Language Modeling with Pathways](https://md.research.elamur.ai/free/en/papers/chowdhery-2022-palm-scaling-language-modeling-pathways/)
25. Paul F Christiano, Jan Leike, Tom Brown, Miljan Martic, Shane Legg, and Dario Amodei. Deep reinforcement learning from human preferences. Advances in neural information processing systems, 30, 2017. [find](https://scholar.google.com/scholar?q=Paul+F+Christiano%2C+Jan+Leike%2C+Tom+Brown%2C+Miljan+Martic%2C+Shane+Legg%2C+and+Dario+Amodei%2E+Deep+reinforcement+learning+from+human+preferences%2E+Advances+in+neural+information+processing+systems%2C+30%2C+2017%2E)
26. Hyung Won Chung, Le Hou, S. Longpre, Barret Zoph, Yi Tay, William Fedus, Eric Li, Xuezhi Wang, Mostafa Dehghani, Siddhartha Brahma, Albert Webson, Shixiang Shane Gu, Zhuyun Dai, Mirac Suzgun, Xinyun Chen, Aakanksha Chowdhery, Dasha Valter, Sharan Narang, Gaurav Mishra, Adams Wei Yu, Vincent Zhao, Ya… [find](https://scholar.google.com/scholar?q=Hyung+Won+Chung%2C+Le+Hou%2C+S%2E+Longpre%2C+Barret+Zoph%2C+Yi+Tay%2C+William+Fedus%2C+Eric+Li%2C+Xuezhi+Wang%2C+Mostafa+Dehghani%2C+Siddhartha+Brahma%2C+Albert+Webson%2C+Shixiang+Shane+Gu%2C+Zhuyun+Dai%2C+Mirac+Suzgun%2C+Xinyun+C)
27. Christopher Clark, Kenton Lee, Ming-Wei Chang, Tom Kwiatkowski, Michael Collins, and Kristina Toutanova. [Boolq: Exploring the surprising difficulty of natural yes/no questions](https://md.research.elamur.ai/free/en/papers/clark-2019-boolq-exploring-surprising-difficulty-natural/). arXiv preprint arXiv:1905.10044, 2019.
28. Elizabeth Clark, Tal August, Sofia Serrano, Nikita Haduong, Suchin Gururangan, and Noah A. Smith. All that’s ‘human’ is not gold: Evaluating human evaluation of generated text. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint… [find](https://scholar.google.com/scholar?q=Elizabeth+Clark%2C+Tal+August%2C+Sofia+Serrano%2C+Nikita+Haduong%2C+Suchin+Gururangan%2C+and+Noah+A%2E+Smith%2E+All+that%E2%80%99s+%E2%80%98human%E2%80%99+is+not+gold%3A+Evaluating+human+evaluation+of+generated+text%2E+In+Proceedings+of+the+5)
29. Peter Clark, Isaac Cowhey, Oren Etzioni, Tushar Khot, Ashish Sabharwal, Carissa Schoenick, and Oyvind Tafjord. [Think you have solved question answering? try arc, the ai2 reasoning challenge](https://md.research.elamur.ai/free/en/papers/clark-2018-think-have-solved-question-answering/). arXiv preprint arXiv:1803.05457, 2018.
30. Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, et al. [Training verifiers to solve math word problems](https://md.research.elamur.ai/free/en/papers/cobbe-2021-training-verifiers-solve-math-word/). arXiv preprint arXiv:2110.14168, 2021.
31. Jiawen Deng, Hao Sun, Zhexin Zhang, Jiale Cheng, and Minlie Huang. Recent advances towards safe, responsible, and moral dialogue systems: A survey. arXiv preprint arXiv:2302.09270, 2023. — [arXiv:2302.09270](https://arxiv.org/abs/2302.09270) [find](https://scholar.google.com/scholar?q=Jiawen+Deng%2C+Hao+Sun%2C+Zhexin+Zhang%2C+Jiale+Cheng%2C+and+Minlie+Huang%2E+Recent+advances+towards+safe%2C+responsible%2C+and+moral+dialogue+systems%3A+A+survey%2E+arXiv+preprint+arXiv%3A2302%2E09270%2C+2023%2E+%E2%80%94+%5BarXiv%3A2302)
32. Yuntian Deng, Anton Bakhtin, Myle Ott, Arthur Szlam, and Marc’Aurelio Ranzato. Residual energy-based models for text generation. In International Conference on Learning Representations, 2019. [find](https://scholar.google.com/scholar?q=Yuntian+Deng%2C+Anton+Bakhtin%2C+Myle+Ott%2C+Arthur+Szlam%2C+and+Marc%E2%80%99Aurelio+Ranzato%2E+Residual+energy%2Dbased+models+for+text+generation%2E+In+International+Conference+on+Learning+Representations%2C+2019%2E)
33. Jwala Dhamala, Tony Sun, Varun Kumar, Satyapriya Krishna, Yada Pruksachatkun, Kai-Wei Chang, and Rahul Gupta. BOLD: Dataset and metrics for measuring biases in open-ended language generation. In Proceedings of the 2021 ACM conference on fairness, accountability, and transparency, pages 862–872, 2021… [find](https://scholar.google.com/scholar?q=Jwala+Dhamala%2C+Tony+Sun%2C+Varun+Kumar%2C+Satyapriya+Krishna%2C+Yada+Pruksachatkun%2C+Kai%2DWei+Chang%2C+and+Rahul+Gupta%2E+BOLD%3A+Dataset+and+metrics+for+measuring+biases+in+open%2Dended+language+generation%2E+In+Proce)
34. Emily Dinan, Gavin Abercrombie, A Stevie Bergman, Shannon Spruit, Dirk Hovy, Y-Lan Boureau, and Verena Rieser. Anticipating safety issues in e2e conversational ai: Framework and tooling. arXiv preprint arXiv:2107.03451, 2021. — [arXiv:2107.03451](https://arxiv.org/abs/2107.03451) [find](https://scholar.google.com/scholar?q=Emily+Dinan%2C+Gavin+Abercrombie%2C+A+Stevie+Bergman%2C+Shannon+Spruit%2C+Dirk+Hovy%2C+Y%2DLan+Boureau%2C+and+Verena+Rieser%2E+Anticipating+safety+issues+in+e2e+conversational+ai%3A+Framework+and+tooling%2E+arXiv+preprin)
35. Jesse Dodge, Maarten Sap, Ana Marasović, William Agnew, Gabriel Ilharco, Dirk Groeneveld, Margaret Mitchell, and Matt Gardner. Documenting large webtext corpora: A case study on the colossal clean crawled corpus. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processi… [find](https://scholar.google.com/scholar?q=Jesse+Dodge%2C+Maarten+Sap%2C+Ana+Marasovi%C4%87%2C+William+Agnew%2C+Gabriel+Ilharco%2C+Dirk+Groeneveld%2C+Margaret+Mitchell%2C+and+Matt+Gardner%2E+Documenting+large+webtext+corpora%3A+A+case+study+on+the+colossal+clean+cra)
36. Jesse Dodge, Taylor Prewitt, Remi Tachet Des Combes, Erika Odmark, Roy Schwartz, Emma Strubell, Alexandra Sasha Luccioni, Noah A Smith, Nicole DeCario, and Will Buchanan. Measuring the carbon intensity of ai in cloud instances. arXiv preprint arXiv:2206.05229, 2022. — [arXiv:2206.05229](https://arxiv.org/abs/2206.05229) [find](https://scholar.google.com/scholar?q=Jesse+Dodge%2C+Taylor+Prewitt%2C+Remi+Tachet+Des+Combes%2C+Erika+Odmark%2C+Roy+Schwartz%2C+Emma+Strubell%2C+Alexandra+Sasha+Luccioni%2C+Noah+A+Smith%2C+Nicole+DeCario%2C+and+Will+Buchanan%2E+Measuring+the+carbon+intensit)
37. Nan Du, Yanping Huang, Andrew M Dai, Simon Tong, Dmitry Lepikhin, Yuanzhong Xu, Maxim Krikun, Yanqi Zhou, Adams Wei Yu, Orhan Firat, Barret Zoph, Liam Fedus, Maarten P Bosma, Zongwei Zhou, Tao Wang, Emma Wang, Kellie Webster, Marie Pellat, Kevin Robinson, Kathleen Meier-Hellstern, Toju Duke, Lucas D… — [GLaM: Efficient Scaling of Language Models with Mixture-of-Experts](https://md.research.elamur.ai/free/en/papers/du-2021-glam-efficient-scaling-language-models/)
38. Kawin Ethayarajh, Yejin Choi, and Swabha Swayamdipta. Understanding dataset difficulty with V-usable information. In Kamalika Chaudhuri, Stefanie Jegelka, Le Song, Csaba Szepesvari, Gang Niu, and Sivan Sabato, editors, Proceedings of the 39th International Conference on Machine Learning, volume 162… [find](https://scholar.google.com/scholar?q=Kawin+Ethayarajh%2C+Yejin+Choi%2C+and+Swabha+Swayamdipta%2E+Understanding+dataset+difficulty+with+V%2Dusable+information%2E+In+Kamalika+Chaudhuri%2C+Stefanie+Jegelka%2C+Le+Song%2C+Csaba+Szepesvari%2C+Gang+Niu%2C+and+Siva)
39. Prakhar Ganesh, Hongyan Chang, Martin Strobel, and Reza Shokri. On the impact of machine learning randomness on group fairness. In Proceedings of the 2023 ACM Conference on Fairness, Accountability, and Transparency, pages 1789–1800, 2023. [find](https://scholar.google.com/scholar?q=Prakhar+Ganesh%2C+Hongyan+Chang%2C+Martin+Strobel%2C+and+Reza+Shokri%2E+On+the+impact+of+machine+learning+randomness+on+group+fairness%2E+In+Proceedings+of+the+2023+ACM+Conference+on+Fairness%2C+Accountability%2C+a)
40. Deep Ganguli, Liane Lovitt, Jackson Kernion, Amanda Askell, Yuntao Bai, Saurav Kadavath, Ben Mann, Ethan Perez, Nicholas Schiefer, Kamal Ndousse, et al. [Red teaming language models to reduce harms: Methods, scaling behaviors, and lessons learned](https://md.research.elamur.ai/free/en/papers/ganguli-2022-red-teaming-language-models-reduce/). arXiv preprint arXiv:2209.07858, 2022.
41. Deep Ganguli, Amanda Askell, Nicholas Schiefer, Thomas Liao, Kamilė Lukošiūtė, Anna Chen, Anna Goldie, Azalia Mirhoseini, Catherine Olsson, Danny Hernandez, et al. [The capacity for moral self-correction in large language models](https://md.research.elamur.ai/free/en/papers/ganguli-2023-capacity-moral-self-correction-large/). arXiv preprint arXiv:2302.07459, 2023.
42. Leo Gao, Jonathan Tow, Stella Biderman, Sid Black, Anthony DiPofi, Charles Foster, Laurence Golding, Jeffrey Hsu, Kyle McDonell, Niklas Muennighoff, Jason Phang, Laria Reynolds, Eric Tang, Anish Thite, Ben Wang, Kevin Wang, and Andy Zou. A framework for few-shot language model evaluation, September… — [doi:10.5281/zenodo.5371628](https://doi.org/10.5281/zenodo.5371628) [find](https://scholar.google.com/scholar?q=Leo+Gao%2C+Jonathan+Tow%2C+Stella+Biderman%2C+Sid+Black%2C+Anthony+DiPofi%2C+Charles+Foster%2C+Laurence+Golding%2C+Jeffrey+Hsu%2C+Kyle+McDonell%2C+Niklas+Muennighoff%2C+Jason+Phang%2C+Laria+Reynolds%2C+Eric+Tang%2C+Anish+Thite)
43. Sebastian Gehrmann, Elizabeth Clark, and Thibault Sellam. Repairing the cracked foundation: A survey of obstacles in evaluation practices for generated text. Journal of Artificial Intelligence Research, 77:103–166, 2023. [find](https://scholar.google.com/scholar?q=Sebastian+Gehrmann%2C+Elizabeth+Clark%2C+and+Thibault+Sellam%2E+Repairing+the+cracked+foundation%3A+A+survey+of+obstacles+in+evaluation+practices+for+generated+text%2E+Journal+of+Artificial+Intelligence+Researc)
44. Fabrizio Gilardi, Meysam Alizadeh, and Maël Kubli. [Chatgpt outperforms crowd-workers for text-annotation tasks](https://md.research.elamur.ai/free/en/papers/gilardi-2023-chatgpt-outperforms-crowd-workers-text/). arXiv preprint arXiv:2303.15056, 2023.
45. Arnav Gudibande, Eric Wallace, Charlie Snell, Xinyang Geng, Hao Liu, Pieter Abbeel, Sergey Levine, and Dawn Song. [The false promise of imitating proprietary llms](https://md.research.elamur.ai/free/en/papers/gudibande-2023-false-promise-imitating-proprietary-llms/). arXiv preprint arXiv:2305.15717, 2023.
46. Udit Gupta, Mariam Elgamal, Gage Hills, Gu-Yeon Wei, Hsien-Hsin S Lee, David Brooks, and Carole-Jean Wu. Act: designing sustainable computer systems with an architectural carbon modeling tool. In Proceedings of the 49th Annual International Symposium on Computer Architecture, pages 784–799, 2022a. [find](https://scholar.google.com/scholar?q=Udit+Gupta%2C+Mariam+Elgamal%2C+Gage+Hills%2C+Gu%2DYeon+Wei%2C+Hsien%2DHsin+S+Lee%2C+David+Brooks%2C+and+Carole%2DJean+Wu%2E+Act%3A+designing+sustainable+computer+systems+with+an+architectural+carbon+modeling+tool%2E+In+Proc)
47. Udit Gupta, Young Guen Kim, Sylvia Lee, Jordan Tse, Hsien-Hsin Sean Lee, Gu-Yeon Wei, David Brooks, and Carole-Jean Wu. Chasing carbon: The elusive environmental footprint of computing. IEEE Micro, 2022b. [find](https://scholar.google.com/scholar?q=Udit+Gupta%2C+Young+Guen+Kim%2C+Sylvia+Lee%2C+Jordan+Tse%2C+Hsien%2DHsin+Sean+Lee%2C+Gu%2DYeon+Wei%2C+David+Brooks%2C+and+Carole%2DJean+Wu%2E+Chasing+carbon%3A+The+elusive+environmental+footprint+of+computing%2E+IEEE+Micro%2C+20)
48. Kilem L. Gwet. Handbook of inter-rater reliability: The definitive guide to measuring the extent of agreement among raters. Advanced Analytics, LLC, 2014. [find](https://scholar.google.com/scholar?q=Kilem+L%2E+Gwet%2E+Handbook+of+inter%2Drater+reliability%3A+The+definitive+guide+to+measuring+the+extent+of+agreement+among+raters%2E+Advanced+Analytics%2C+LLC%2C+2014%2E)
49. Kilem Li Gwet. Computing inter-rater reliability and its variance in the presence of high agreement. British Journal of Mathematical and Statistical Psychology, 61(1):29–48, 2008. [find](https://scholar.google.com/scholar?q=Kilem+Li+Gwet%2E+Computing+inter%2Drater+reliability+and+its+variance+in+the+presence+of+high+agreement%2E+British+Journal+of+Mathematical+and+Statistical+Psychology%2C+61%281%29%3A29%E2%80%9348%2C+2008%2E)
50. Thomas Hartvigsen, Saadia Gabriel, Hamid Palangi, Maarten Sap, Dipankar Ray, and Ece Kamar. [Toxigen: A large-scale machine-generated dataset for adversarial and implicit hate speech detection](https://md.research.elamur.ai/free/en/papers/hartvigsen-2022-toxigen-large-scale-machine-generated/). In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long…
51. Alex Havrilla. synthetic-instruct-gptj-pairwise. — [DeBERTa: Decoding-enhanced BERT with Disentangled Attention](https://md.research.elamur.ai/free/en/papers/he-2020-deberta-decoding-enhanced-bert-disentangled/) [https://huggingface.co/datasets/Dahoas/synthetic-instruct-gptj-pairwise](https://huggingface.co/datasets/Dahoas/synthetic-instruct-gptj-pairwise). Pengcheng He, Xiaodong Liu, Jianfeng Gao, and Weizhu Chen. Deberta: Decoding-enhanced bert with disentangled attention. arXiv preprint arXiv:2006.03654, 2020. [↗](https://huggingface.co/datasets/Dahoas/synthetic-instruct-gptj-pairwise)
52. Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Xiaodong Song, and Jacob Steinhardt. [Measuring massive multitask language understanding](https://md.research.elamur.ai/free/en/papers/hendrycks-2020-measuring-massive-multitask-language-understanding/). arXiv preprint arXiv:2009.03300, 2020.
53. Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardt. [Measuring mathematical problem solving with the math dataset](https://md.research.elamur.ai/free/en/papers/hendrycks-2021-measuring-mathematical-problem-solving-math/). arXiv preprint arXiv:2103.03874, 2021.
54. Jordan Hoffmann, Sebastian Borgeaud, Arthur Mensch, Elena Buchatskaya, Trevor Cai, Eliza Rutherford, Diego de Las Casas, Lisa Anne Hendricks, Johannes Welbl, Aidan Clark, et al. [Training compute-optimal large language models](https://md.research.elamur.ai/free/en/papers/hoffmann-2022-training-compute-optimal-large-language/). arXiv preprint arXiv:2203.15556, 2022.
55. Ari Holtzman, Jan Buys, Li Du, Maxwell Forbes, and Yejin Choi. [The curious case of neural text degeneration](https://md.research.elamur.ai/free/en/papers/holtzman-2019-curious-case-neural-text-degeneration/). In International Conference on Learning Representations, 2020. URL [https://openreview.net/forum?id=rygGQyrFvH](https://openreview.net/forum?id=rygGQyrFvH). [↗](https://openreview.net/forum?id=rygGQyrFvH)
56. Or Honovich, Thomas Scialom, Omer Levy, and Timo Schick. [Unnatural instructions: Tuning language models with (almost) no human labor](https://md.research.elamur.ai/free/en/papers/honovich-2023-unnatural-instructions-tuning-language-models/). arXiv preprint arXiv:2212.09689, 2022.
57. Saghar Hosseini, Hamid Palangi, and Ahmed Hassan Awadallah. [An empirical study of metrics to measure representational harms in pre-trained language models](https://md.research.elamur.ai/free/en/papers/hosseini-2023-empirical-study-metrics-measure-representational/). arXiv preprint arXiv:2301.09211, 2023.
58. Fan Huang, Haewoon Kwak, and Jisun An. Is chatgpt better than human annotators? potential and limitations of chatgpt in explaining implicit hate speech. arXiv preprint arXiv:2302.07736, 2023. [find](https://scholar.google.com/scholar?q=Fan+Huang%2C+Haewoon+Kwak%2C+and+Jisun+An%2E+Is+chatgpt+better+than+human+annotators%3F+potential+and+limitations+of+chatgpt+in+explaining+implicit+hate+speech%2E+arXiv+preprint+arXiv%3A2302%2E07736%2C+2023%2E)
59. Clayton Hutto and Eric Gilbert. Vader: A parsimonious rule-based model for sentiment analysis of social media text. In Proceedings of the international AAAI conference on web and social media, volume 8, pages 216–225, 2014. [find](https://scholar.google.com/scholar?q=Clayton+Hutto+and+Eric+Gilbert%2E+Vader%3A+A+parsimonious+rule%2Dbased+model+for+sentiment+analysis+of+social+media+text%2E+In+Proceedings+of+the+international+AAAI+conference+on+web+and+social+media%2C+volume+)
60. Mandar Joshi, Eunsol Choi, Daniel S Weld, and Luke Zettlemoyer. [Triviaqa: A large scale distantly supervised challenge dataset for reading comprehension](https://md.research.elamur.ai/free/en/papers/joshi-2017-triviaqa-large-scale-distantly-supervised/). arXiv preprint arXiv:1705.03551, 2017.
61. Jared Kaplan, Sam McCandlish, Tom Henighan, Tom B Brown, Benjamin Chess, Rewon Child, Scott Gray, Alec Radford, Jeffrey Wu, and Dario Amodei. [Scaling laws for neural language models](https://md.research.elamur.ai/free/en/papers/kaplan-2020-scaling-laws-neural-language-models/). arXiv preprint arXiv:2001.08361, 2020.
62. James Kirkpatrick, Razvan Pascanu, Neil Rabinowitz, Joel Veness, Guillaume Desjardins, Andrei A Rusu, Kieran Milan, John Quan, Tiago Ramalho, Agnieszka Grabska-Barwinska, et al. Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences, 114(13):3521–3526,… [find](https://scholar.google.com/scholar?q=James+Kirkpatrick%2C+Razvan+Pascanu%2C+Neil+Rabinowitz%2C+Joel+Veness%2C+Guillaume+Desjardins%2C+Andrei+A+Rusu%2C+Kieran+Milan%2C+John+Quan%2C+Tiago+Ramalho%2C+Agnieszka+Grabska%2DBarwinska%2C+et+al%2E+Overcoming+catastrophi)
63. Andreas Köpf, Yannic Kilcher, Dimitri von Rütte, Sotiris Anagnostidis, Zhi-Rui Tam, Keith Stevens, Abdullah Barhoum, Nguyen Minh Duc, Oliver Stanley, Richárd Nagyfi, et al. Openassistant conversations– democratizing large language model alignment. arXiv preprint arXiv:2304.[07327, 2023](https://md.research.elamur.ai/free/en/papers/kopf-2023-openassistant-conversations-democratizing-large-language/).
64. Tomasz Korbak, Kejian Shi, Angelica Chen, Rasika Bhalerao, Christopher L Buckley, Jason Phang, Samuel R Bowman, and Ethan Perez. [Pretraining language models with human preferences](https://md.research.elamur.ai/free/en/papers/korbak-2023-pretraining-language-models-human-preferences/). arXiv preprint arXiv:2302.08582, 2023.
65. Taku Kudo and John Richardson. [Sentencepiece: A simple and language independent subword tokenizer and detokenizer for neural text processing](https://md.research.elamur.ai/free/en/papers/kudo-2018-sentencepiece-simple-language-independent-subword/), 2018.
66. Sachin Kumar, Vidhisha Balachandran, Lucille Njoo, Antonios Anastasopoulos, and Yulia Tsvetkov. [Language generation models can cause harm: So what can we do about it? an actionable survey](https://md.research.elamur.ai/free/en/papers/kumar-2023-language-generation-models-can-cause/). arXiv preprint arXiv:2210.07700, 2022.
67. Tom Kwiatkowski, Jennimaria Palomaki, Olivia Redfield, Michael Collins, Ankur Parikh, Chris Alberti, Danielle Epstein, Illia Polosukhin, Jacob Devlin, Kenton Lee, et al. [Natural questions: a benchmark for question answering research](https://md.research.elamur.ai/free/en/papers/kwiatkowski-2019-natural-questions-benchmark-question-answering/). Transactions of the Association for Computational Linguistics, 7:4…
68. Nathan Lambert, Lewis Tunstall, Nazneen Rajani, and Tristan Thrush. Huggingface h4 stack exchange preference dataset. 2023. URL [https://huggingface.co/datasets/HuggingFaceH4/stack-exchange-preferences](https://huggingface.co/datasets/HuggingFaceH4/stack-exchange-preferences). [↗](https://huggingface.co/datasets/HuggingFaceH4/stack-exchange-preferences)
69. Katherine Lee, Daphne Ippolito, Andrew Nystrom, Chiyuan Zhang, Douglas Eck, Chris Callison-Burch, and Nicholas Carlini. Deduplicating training data makes language models better. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics. Association for Computational… [find](https://scholar.google.com/scholar?q=Katherine+Lee%2C+Daphne+Ippolito%2C+Andrew+Nystrom%2C+Chiyuan+Zhang%2C+Douglas+Eck%2C+Chris+Callison%2DBurch%2C+and+Nicholas+Carlini%2E+Deduplicating+training+data+makes+language+models+better%2E+In+Proceedings+of+the+)
70. Kevin Lee and Shubho Sengupta. Introducing the ai research supercluster — meta’s cutting-edge ai supercomputer for ai research, 2022. URL [https://ai.facebook.com/blog/ai-rsc/](https://ai.facebook.com/blog/ai-rsc/). [↗](https://ai.facebook.com/blog/ai-rsc/)
71. Stephanie Lin, Jacob Hilton, and Owain Evans. [Truthfulqa: Measuring how models mimic human falsehoods](https://md.research.elamur.ai/free/en/papers/lin-2022-truthfulqa-measuring-how-models-mimic/). arXiv preprint arXiv:2109.07958, 2021.
72. Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. [Roberta: A robustly optimized bert pretraining approach](https://md.research.elamur.ai/free/en/papers/liu-2019-roberta-robustly-optimized-bert-pretraining/). arXiv preprint arXiv:1907.11692, 2019.
73. Shayne Longpre, Le Hou, Tu Vu, Albert Webson, Hyung Won Chung, Yi Tay, Denny Zhou, Quoc V Le, Barret Zoph, Jason Wei, et al. [The flan collection: Designing data and methods for effective instruction tuning](https://md.research.elamur.ai/free/en/papers/longpre-2023-flan-collection-designing-data-methods/). arXiv preprint arXiv:2301.13688, 2023.
74. Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization. arXiv preprint arXiv:1711.05101, 2017. [find](https://scholar.google.com/scholar?q=Ilya+Loshchilov+and+Frank+Hutter%2E+Decoupled+weight+decay+regularization%2E+arXiv+preprint+arXiv%3A1711%2E05101%2C+2017%2E)
75. Aman Madaan, Niket Tandon, Prakhar Gupta, Skyler Hallinan, Luyu Gao, Sarah Wiegreffe, Uri Alon, Nouha Dziri, Shrimai Prabhumoye, Yiming Yang, et al. Self-refine: Iterative refinement with self-feedback. arXiv preprint arXiv:2303.17651, 2023. [find](https://scholar.google.com/scholar?q=Aman+Madaan%2C+Niket+Tandon%2C+Prakhar+Gupta%2C+Skyler+Hallinan%2C+Luyu+Gao%2C+Sarah+Wiegreffe%2C+Uri+Alon%2C+Nouha+Dziri%2C+Shrimai+Prabhumoye%2C+Yiming+Yang%2C+et+al%2E+Self%2Drefine%3A+Iterative+refinement+with+self%2Dfeedbac)
76. Grégoire Mialon, Roberto Dessì, Maria Lomeli, Christoforos Nalmpantis, Ram Pasunuru, Roberta Raileanu, Baptiste Rozière, Timo Schick, Jane Dwivedi-Yu, Asli Celikyilmaz, et al. [Augmented language models: a survey](https://md.research.elamur.ai/free/en/papers/mialon-2023-augmented-language-models-survey/). arXiv preprint arXiv:2302.07842, 2023.
77. Todor Mihaylov, Peter Clark, Tushar Khot, and Ashish Sabharwal. Can a suit of armor conduct electricity? a new dataset for open book question answering. arXiv preprint arXiv:1809.02789, 2018. [find](https://scholar.google.com/scholar?q=Todor+Mihaylov%2C+Peter+Clark%2C+Tushar+Khot%2C+and+Ashish+Sabharwal%2E+Can+a+suit+of+armor+conduct+electricity%3F+a+new+dataset+for+open+book+question+answering%2E+arXiv+preprint+arXiv%3A1809%2E02789%2C+2018%2E)
78. Margaret Mitchell, Simone Wu, Andrew Zaldivar, Parker Barnes, Lucy Vasserman, Ben Hutchinson, Elena Spitzer, Inioluwa Deborah Raji, and Timnit Gebru. Model cards for model reporting. CoRR, abs/1810.03993, 2018. URL http://arxiv.org/abs/1810.03993. [↗](http://arxiv.org/abs/1810.03993)
79. MosaicML NLP Team et al. Introducing mpt-7b: A new standard for open-source, commercially usable llms, 2023. [find](https://scholar.google.com/scholar?q=MosaicML+NLP+Team+et+al%2E+Introducing+mpt%2D7b%3A+A+new+standard+for+open%2Dsource%2C+commercially+usable+llms%2C+2023%2E)
80. Reiichiro Nakano, Jacob Hilton, Suchir Balaji, Jeff Wu, Lonbrown Ouyanbrown, Christina Kim, Christopher Hesse, Shantanu Jain, Vineet Kosaraju, William Saunders, Xu Jiang, Karl Cobbe, Tyna Eloundou, Gretchen Krueger, Kevin Button, Matthew Knight, Benjamin Chess, and John Schulman. Webgpt: Browser-ass… — [WebGPT: Browser-assisted question-answering with human feedback](https://md.research.elamur.ai/free/en/papers/nakano-2021-webgpt-browser-assisted-question-answering/)
81. Cuong V. Nguyen, Alessandro Achille, Michael Lam, Tal Hassner, Vijay Mahadevan, and Stefano Soatto. Toward understanding catastrophic forgetting in continual learning. arXiv preprint arXiv:1908.01091, 2019. — [arXiv:1908.01091](https://arxiv.org/abs/1908.01091) [find](https://scholar.google.com/scholar?q=Cuong+V%2E+Nguyen%2C+Alessandro+Achille%2C+Michael+Lam%2C+Tal+Hassner%2C+Vijay+Mahadevan%2C+and+Stefano+Soatto%2E+Toward+understanding+catastrophic+forgetting+in+continual+learning%2E+arXiv+preprint+arXiv%3A1908%2E01091%2C)
82. OpenAI. [GPT-4 technical report](https://md.research.elamur.ai/free/en/papers/openai-2023-gpt-4-technical-report/). CoRR, abs/2303.08774, 2023. doi: 10.48550/arXiv.2303.08774. URL https://doi.org/10.48550/arXiv.2303.08774. [↗](https://doi.org/10.48550/arXiv.2303.08774)
83. Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. [Training language models to follow instructions with human feedback](https://md.research.elamur.ai/free/en/papers/ouyang-2022-training-language-models-follow-instructions/). Advances in Neural Information Processing Systems, 35:27730–27744, 2022.
84. David Patterson, Joseph Gonzalez, Quoc Le, Chen Liang, Lluis-Miquel Munguia, Daniel Rothchild, David So, Maud Texier, and Jeff Dean. [Carbon emissions and large neural network training](https://md.research.elamur.ai/free/en/papers/patterson-2021-carbon-emissions-large-neural-network/). arXiv preprint arXiv:2104.10350, 2021.
85. Guilherme Penedo, Quentin Malartic, Daniel Hesslow, Ruxandra Cojocaru, Alessandro Cappelli, Hamza Alobeidli, Baptiste Pannier, Ebtesam Almazrouei, and Julien Launay. The refinedweb dataset for falcon llm: Outperforming curated corpora with web data, and web data only, 2023. [find](https://scholar.google.com/scholar?q=Guilherme+Penedo%2C+Quentin+Malartic%2C+Daniel+Hesslow%2C+Ruxandra+Cojocaru%2C+Alessandro+Cappelli%2C+Hamza+Alobeidli%2C+Baptiste+Pannier%2C+Ebtesam+Almazrouei%2C+and+Julien+Launay%2E+The+refinedweb+dataset+for+falcon+)
86. Reiner Pope, Sholto Douglas, Aakanksha Chowdhery, Jacob Devlin, James Bradbury, Anselm Levskaya, Jonathan Heek, Kefan Xiao, Shivani Agrawal, and Jeff Dean. Efficiently scaling transformer inference, 2022. [find](https://scholar.google.com/scholar?q=Reiner+Pope%2C+Sholto+Douglas%2C+Aakanksha+Chowdhery%2C+Jacob+Devlin%2C+James+Bradbury%2C+Anselm+Levskaya%2C+Jonathan+Heek%2C+Kefan+Xiao%2C+Shivani+Agrawal%2C+and+Jeff+Dean%2E+Efficiently+scaling+transformer+inference%2C+2)
87. Jack W. Rae, Sebastian Borgeaud, Trevor Cai, Katie Millican, Jordan Hoffmann, Francis Song, John Aslanides, Sarah Henderson, Roman Ring, Susannah Young, Eliza Rutherford, Tom Hennigan, Jacob Menick, Albin Cassirer, Richard Powell, George van den Driessche, Lisa Anne Hendricks, Maribeth Rauh, Po-Sen… [find](https://scholar.google.com/scholar?q=Jack+W%2E+Rae%2C+Sebastian+Borgeaud%2C+Trevor+Cai%2C+Katie+Millican%2C+Jordan+Hoffmann%2C+Francis+Song%2C+John+Aslanides%2C+Sarah+Henderson%2C+Roman+Ring%2C+Susannah+Young%2C+Eliza+Rutherford%2C+Tom+Hennigan%2C+Jacob+Menick%2C+A)
88. Pranav Rajpurkar, Robin Jia, and Percy Liang. Know what you don’t know: Unanswerable questions for squad. arXiv preprint arXiv:1806.[03822, 2018](https://md.research.elamur.ai/free/en/papers/rajpurkar-2018-know-what-you-don-t/).
89. Vinay Venkatesh Ramasesh, Aitor Lewkowycz, and Ethan Dyer. Effect of scale on catastrophic forgetting in neural networks. In International Conference on Learning Representations, 2021. [find](https://scholar.google.com/scholar?q=Vinay+Venkatesh+Ramasesh%2C+Aitor+Lewkowycz%2C+and+Ethan+Dyer%2E+Effect+of+scale+on+catastrophic+forgetting+in+neural+networks%2E+In+International+Conference+on+Learning+Representations%2C+2021%2E)
90. Stephen Roller, Y-Lan Boureau, Jason Weston, Antoine Bordes, Emily Dinan, Angela Fan, David Gunning, Da Ju, Margaret Li, Spencer Poff, et al. Open-domain conversational agents: Current progress, open problems, and future directions. arXiv preprint arXiv:2006.12442, 2020. — [arXiv:2006.12442](https://arxiv.org/abs/2006.12442) [find](https://scholar.google.com/scholar?q=Stephen+Roller%2C+Y%2DLan+Boureau%2C+Jason+Weston%2C+Antoine+Bordes%2C+Emily+Dinan%2C+Angela+Fan%2C+David+Gunning%2C+Da+Ju%2C+Margaret+Li%2C+Spencer+Poff%2C+et+al%2E+Open%2Ddomain+conversational+agents%3A+Current+progress%2C+open+)
91. Keisuke Sakaguchi, Ronan Le Bras, Chandra Bhagavatula, and Yejin Choi. Winogrande: An adversarial winograd schema challenge at scale. Communications of the ACM, 64(9):99–106, 2021. [find](https://scholar.google.com/scholar?q=Keisuke+Sakaguchi%2C+Ronan+Le+Bras%2C+Chandra+Bhagavatula%2C+and+Yejin+Choi%2E+Winogrande%3A+An+adversarial+winograd+schema+challenge+at+scale%2E+Communications+of+the+ACM%2C+64%289%29%3A99%E2%80%93106%2C+2021%2E)
92. Maarten Sap, Hannah Rashkin, Derek Chen, Ronan LeBras, and Yejin Choi. [Socialiqa: Commonsense reasoning about social interactions](https://md.research.elamur.ai/free/en/papers/sap-2019-socialiqa-commonsense-reasoning-about-social/). arXiv preprint arXiv:1904.09728, 2019.
93. Teven Le Scao, Angela Fan, Christopher Akiki, Ellie Pavlick, Suzana Ilić, Daniel Hesslow, Roman Castagné, Alexandra Sasha Luccioni, François Yvon, Matthias Gallé, et al. Bloom: A 176b-parameter open-access multilingual language model. arXiv preprint arXiv:2211.05100, 2022. [find](https://scholar.google.com/scholar?q=Teven+Le+Scao%2C+Angela+Fan%2C+Christopher+Akiki%2C+Ellie+Pavlick%2C+Suzana+Ili%C4%87%2C+Daniel+Hesslow%2C+Roman+Castagn%C3%A9%2C+Alexandra+Sasha+Luccioni%2C+Fran%C3%A7ois+Yvon%2C+Matthias+Gall%C3%A9%2C+et+al%2E+Bloom%3A+A+176b%2Dparameter+open%2Da)
94. Timo Schick, Jane Dwivedi-Yu, Roberto Dessì, Roberta Raileanu, Maria Lomeli, Luke Zettlemoyer, Nicola Cancedda, and Thomas Scialom. [Toolformer: Language models can teach themselves to use tools](https://md.research.elamur.ai/free/en/papers/schick-2023-toolformer-language-models-can-teach/). arXiv preprint arXiv:2302.04761, 2023.
95. John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347, 2017. [find](https://scholar.google.com/scholar?q=John+Schulman%2C+Filip+Wolski%2C+Prafulla+Dhariwal%2C+Alec+Radford%2C+and+Oleg+Klimov%2E+Proximal+policy+optimization+algorithms%2E+arXiv+preprint+arXiv%3A1707%2E06347%2C+2017%2E)
96. Thomas Scialom, Paul-Alexis Dray, Sylvain Lamprier, Benjamin Piwowarski, and Jacopo Staiano. Discriminative adversarial search for abstractive summarization. In Hal Daumé III and Aarti Singh, editors, Proceedings of the 37th International Conference on Machine Learning, volume 119 of Proceedings of… [find](https://scholar.google.com/scholar?q=Thomas+Scialom%2C+Paul%2DAlexis+Dray%2C+Sylvain+Lamprier%2C+Benjamin+Piwowarski%2C+and+Jacopo+Staiano%2E+Discriminative+adversarial+search+for+abstractive+summarization%2E+In+Hal+Daum%C3%A9+III+and+Aarti+Singh%2C+editors%2C)
97. Thomas Scialom, Paul-Alexis Dray, Sylvain Lamprier, Benjamin Piwowarski, and Jacopo Staiano. Coldgans: Taming language gans with cautious sampling strategies. Advances in Neural Information Processing Systems, 33:18978–18989, 2020b. [find](https://scholar.google.com/scholar?q=Thomas+Scialom%2C+Paul%2DAlexis+Dray%2C+Sylvain+Lamprier%2C+Benjamin+Piwowarski%2C+and+Jacopo+Staiano%2E+Coldgans%3A+Taming+language+gans+with+cautious+sampling+strategies%2E+Advances+in+Neural+Information+Processing)
98. Rico Sennrich, Barry Haddow, and Alexandra Birch. [Neural machine translation of rare words with subword units](https://md.research.elamur.ai/free/en/papers/sennrich-2016-neural-machine-translation-rare-words/), 2016.
99. Uri Shaham, Elad Segal, Maor Ivgi, Avia Efrat, Ori Yoran, Adi Haviv, Ankit Gupta, Wenhan Xiong, Mor Geva, Jonathan Berant, and Omer Levy. [SCROLLS: Standardized CompaRison over long language sequences](https://md.research.elamur.ai/free/en/papers/shaham-2022-scrolls-standardized-comparison-over-long/). In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 12…
100. Noam Shazeer. Fast transformer decoding: One write-head is all you need, 2019. [find](https://scholar.google.com/scholar?q=Noam+Shazeer%2E+Fast+transformer+decoding%3A+One+write%2Dhead+is+all+you+need%2C+2019%2E)
101. Noam Shazeer. Glu variants improve transformer, 2020. [find](https://scholar.google.com/scholar?q=Noam+Shazeer%2E+Glu+variants+improve+transformer%2C+2020%2E)
102. Mohammad Shoeybi, Mostofa Patwary, Raul Puri, Patrick LeGresley, Jared Casper, and Bryan Catanzaro. Megatron-lm: Training multi-billion parameter language models using model parallelism, 2019. [find](https://scholar.google.com/scholar?q=Mohammad+Shoeybi%2C+Mostofa+Patwary%2C+Raul+Puri%2C+Patrick+LeGresley%2C+Jared+Casper%2C+and+Bryan+Catanzaro%2E+Megatron%2Dlm%3A+Training+multi%2Dbillion+parameter+language+models+using+model+parallelism%2C+2019%2E)
103. Ilia Shumailov, Zakhar Shumaylov, Yiren Zhao, Yarin Gal, Nicolas Papernot, and Ross Anderson. [The curse of recursion: Training on generated data makes models forget](https://md.research.elamur.ai/free/en/papers/shumailov-2023-curse-recursion-training-generated-data/). arXiv preprint arxiv:2305.17493, 2023.
104. Eric Michael Smith and Adina Williams. Hi, my name is martha: Using names to measure and mitigate bias in generative dialogue models. arXiv preprint arXiv:2109.03300, 2021. — [arXiv:2109.03300](https://arxiv.org/abs/2109.03300) [find](https://scholar.google.com/scholar?q=Eric+Michael+Smith+and+Adina+Williams%2E+Hi%2C+my+name+is+martha%3A+Using+names+to+measure+and+mitigate+bias+in+generative+dialogue+models%2E+arXiv+preprint+arXiv%3A2109%2E03300%2C+2021%2E+%E2%80%94+%5BarXiv%3A2109%2E03300%5D%28https%3A)
105. Eric Michael Smith, Melissa Hall, Melanie Kambadur, Eleonora Presani, and Adina Williams. “i’m sorry to hear that”: Finding new biases in language models with a holistic descriptor dataset. In Proceedings of the 2022 [Conference on Empirical Methods in Natural Language Processing, pages 9180–9211, 20…](https://md.research.elamur.ai/free/en/papers/smith-2022-i-m-sorry-hear-that/)
106. Irene Solaiman, Zeerak Talat, William Agnew, Lama Ahmad, Dylan Baker, Su Lin Blodgett, Hal Daumé III, Jesse Dodge, Ellie Evans, Sara Hooker, et al. Evaluating the social impact of generative ai systems in systems and society. arXiv preprint arXiv:2306.05949, 2023. — [arXiv:2306.05949](https://arxiv.org/abs/2306.05949) [find](https://scholar.google.com/scholar?q=Irene+Solaiman%2C+Zeerak+Talat%2C+William+Agnew%2C+Lama+Ahmad%2C+Dylan+Baker%2C+Su+Lin+Blodgett%2C+Hal+Daum%C3%A9+III%2C+Jesse+Dodge%2C+Ellie+Evans%2C+Sara+Hooker%2C+et+al%2E+Evaluating+the+social+impact+of+generative+ai+system)
107. Nisan Stiennon, Long Ouyang, Jeff Wu, Daniel M. Ziegler, Ryan Lowe, Chelsea Voss, Alec Radford, Dario Amodei, and Paul Christiano. Learning to summarize from human feedback. In NeurIPS, 2020. [find](https://scholar.google.com/scholar?q=Nisan+Stiennon%2C+Long+Ouyang%2C+Jeff+Wu%2C+Daniel+M%2E+Ziegler%2C+Ryan+Lowe%2C+Chelsea+Voss%2C+Alec+Radford%2C+Dario+Amodei%2C+and+Paul+Christiano%2E+Learning+to+summarize+from+human+feedback%2E+In+NeurIPS%2C+2020%2E)
108. Jianlin Su, Yu Lu, Shengfeng Pan, Ahmed Murtadha, Bo Wen, and Yunfeng Liu. [Roformer: Enhanced transformer with rotary position embedding](https://md.research.elamur.ai/free/en/papers/su-2021-roformer-enhanced-transformer-rotary-position/), 2022.
109. Mirac Suzgun, Nathan Scales, Nathanael Schärli, Sebastian Gehrmann, Yi Tay, Hyung Won Chung, Aakanksha Chowdhery, Quoc V Le, Ed H Chi, Denny Zhou, et al. [Challenging big-bench tasks and whether chain-of-thought can solve them](https://md.research.elamur.ai/free/en/papers/suzgun-2023-challenging-big-bench-tasks-whether/). arXiv preprint arXiv:2210.09261, 2022.
110. Gabriel Synnaeve, Jonas Gehring, Zeming Lin, Daniel Haziza, Nicolas Usunier, Danielle Rothermel, Vegard Mella, Da Ju, Nicolas Carion, Laura Gustafson, et al. Growing up together: Structured exploration for large action spaces. 2019. [find](https://scholar.google.com/scholar?q=Gabriel+Synnaeve%2C+Jonas+Gehring%2C+Zeming+Lin%2C+Daniel+Haziza%2C+Nicolas+Usunier%2C+Danielle+Rothermel%2C+Vegard+Mella%2C+Da+Ju%2C+Nicolas+Carion%2C+Laura+Gustafson%2C+et+al%2E+Growing+up+together%3A+Structured+exploratio)
111. Yarden Tal, Inbal Magar, and Roy Schwartz. Fewer errors, but more stereotypes? the effect of model size on gender bias. In Proceedings of the 4th Workshop on Gender Bias in Natural Language Processing (GeBNLP), pages 112–120, Seattle, Washington, July 2022. Association for Computational Linguistics.… [find](https://scholar.google.com/scholar?q=Yarden+Tal%2C+Inbal+Magar%2C+and+Roy+Schwartz%2E+Fewer+errors%2C+but+more+stereotypes%3F+the+effect+of+model+size+on+gender+bias%2E+In+Proceedings+of+the+4th+Workshop+on+Gender+Bias+in+Natural+Language+Processing)
112. Alon Talmor, Jonathan Herzig, Nicholas Lourie, and Jonathan Berant. [Commonsenseqa: A question answering challenge targeting commonsense knowledge](https://md.research.elamur.ai/free/en/papers/talmor-2019-commonsenseqa-question-answering-challenge-targeting/). arXiv preprint arXiv:1811.00937, 2018.
113. Rohan Taori, Ishaan Gulrajani, Tianyi Zhang, Yann Dubois, Xuechen Li, Carlos Guestrin, Percy Liang, and Tatsunori B. Hashimoto. Stanford alpaca: An instruction-following llama model. [https://github.com/tatsu-lab/stanford_alpaca](https://github.com/tatsu-lab/stanford_alpaca), 2023. [↗](https://github.com/tatsu-lab/stanford_alpaca)
114. Ross Taylor, Marcin Kardas, Guillem Cucurull, Thomas Scialom, Anthony Hartshorn, Elvis Saravia, Andrew Poulton, Viktor Kerkez, and Robert Stojnic. [Galactica: A large language model for science](https://md.research.elamur.ai/free/en/papers/taylor-2022-galactica-large-language-model-science/). arXiv preprint arXiv:2211.09085, 2022.
115. Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, Aur’elien Rodriguez, Armand Joulin, Edouard Grave, and Guillaume Lample. [Llama: Open and efficient foundation language models](https://md.research.elamur.ai/free/en/papers/touvron-2023-llama-open-efficient-foundation-language/). arXiv preprin…
116. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. [Attention is all you need](https://md.research.elamur.ai/free/en/papers/vaswani-2017-attention-all-need/), 2017.
117. Oriol Vinyals, Igor Babuschkin, Wojciech M Czarnecki, Michaël Mathieu, Andrew Dudzik, Junyoung Chung, David H Choi, Richard Powell, Timo Ewalds, Petko Georgiev, et al. Grandmaster level in starcraft ii using multi-agent reinforcement learning. Nature, 575(7782):350–354, 2019. [find](https://scholar.google.com/scholar?q=Oriol+Vinyals%2C+Igor+Babuschkin%2C+Wojciech+M+Czarnecki%2C+Micha%C3%ABl+Mathieu%2C+Andrew+Dudzik%2C+Junyoung+Chung%2C+David+H+Choi%2C+Richard+Powell%2C+Timo+Ewalds%2C+Petko+Georgiev%2C+et+al%2E+Grandmaster+level+in+starcraft+i)
118. Yizhong Wang, Yeganeh Kordi, Swaroop Mishra, Alisa Liu, Noah A Smith, Daniel Khashabi, and Hannaneh Hajishirzi. Self-instruct: Aligning language model with self generated instructions. arXiv preprint arXiv:2212.10560, 2022. [find](https://scholar.google.com/scholar?q=Yizhong+Wang%2C+Yeganeh+Kordi%2C+Swaroop+Mishra%2C+Alisa+Liu%2C+Noah+A+Smith%2C+Daniel+Khashabi%2C+and+Hannaneh+Hajishirzi%2E+Self%2Dinstruct%3A+Aligning+language+model+with+self+generated+instructions%2E+arXiv+preprint+)
119. Michael Webb. The impact of artificial intelligence on the labor market. Available at SSRN 3482150, 2019. [find](https://scholar.google.com/scholar?q=Michael+Webb%2E+The+impact+of+artificial+intelligence+on+the+labor+market%2E+Available+at+SSRN+3482150%2C+2019%2E)
120. Jason Wei, Maarten Bosma, Vincent Zhao, Kelvin Guu, Adams Wei Yu, Brian Lester, Nan Du, Andrew M Dai, and Quoc V Le. [Finetuned language models are zero-shot learners](https://md.research.elamur.ai/free/en/papers/wei-2021-finetuned-language-models-zero-shot/). In International Conference on Learning Representations, 2021.
121. Jason Wei, Maarten Bosma, Vincent Zhao, Kelvin Guu, Adams Wei Yu, Brian Lester, Nan Du, Andrew M. Dai, and Quoc V Le. [Finetuned language models are zero-shot learners](https://md.research.elamur.ai/free/en/papers/wei-2021-finetuned-language-models-zero-shot/). In International Conference on Learning Representations, 2022a. URL [https://openreview.net/forum?id=gEZrGCozdqR](https://openreview.net/forum?id=gEZrGCozdqR). [↗](https://openreview.net/forum?id=gEZrGCozdqR)
122. Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. [Chain-of-thought prompting elicits reasoning in large language models](https://md.research.elamur.ai/free/en/papers/wei-2022-chain-thought-prompting-elicits-reasoning/). Advances in Neural Information Processing Systems, 35:24824–24837, 2022b.
123. Laura Weidinger, John Mellor, Maribeth Rauh, Conor Griffin, Jonathan Uesato, Po-Sen Huang, Myra Cheng, Mia Glaese, Borja Balle, Atoosa Kasirzadeh, et al. [Ethical and social risks of harm from language models](https://md.research.elamur.ai/free/en/papers/weidinger-2021-ethical-social-risks-harm-language/). arXiv preprint arXiv:2112.04359, 2021.
124. Johannes Welbl, Amelia Glaese, Jonathan Uesato, Sumanth Dathathri, John Mellor, Lisa Anne Hendricks, Kirsty Anderson, Pushmeet Kohli, Ben Coppin, and Po-Sen Huang. [Challenges in detoxifying language models](https://md.research.elamur.ai/free/en/papers/welbl-2021-challenges-detoxifying-language-models/), 2021.
125. Carole-Jean Wu, Ramya Raghavendra, Udit Gupta, Bilge Acun, Newsha Ardalani, Kiwan Maeng, Gloria Chang, Fiona Aga, Jinshi Huang, Charles Bai, et al. Sustainable ai: Environmental implications, challenges and opportunities. Proceedings of Machine Learning and Systems, 4:795–813, 2022. [find](https://scholar.google.com/scholar?q=Carole%2DJean+Wu%2C+Ramya+Raghavendra%2C+Udit+Gupta%2C+Bilge+Acun%2C+Newsha+Ardalani%2C+Kiwan+Maeng%2C+Gloria+Chang%2C+Fiona+Aga%2C+Jinshi+Huang%2C+Charles+Bai%2C+et+al%2E+Sustainable+ai%3A+Environmental+implications%2C+challeng)
126. Jing Xu, Da Ju, Margaret Li, Y-Lan Boureau, Jason Weston, and Emily Dinan. [Recipes for safety in open-domain chatbots](https://md.research.elamur.ai/free/en/papers/xu-2020-recipes-safety-open-domain-chatbots/), 2021.
127. Rowan Zellers, Ari Holtzman, Yonatan Bisk, Ali Farhadi, and Yejin Choi. Hellaswag: Can a machine really finish your sentence? arXiv preprint arXiv:1905.07830, 2019a. [find](https://scholar.google.com/scholar?q=Rowan+Zellers%2C+Ari+Holtzman%2C+Yonatan+Bisk%2C+Ali+Farhadi%2C+and+Yejin+Choi%2E+Hellaswag%3A+Can+a+machine+really+finish+your+sentence%3F+arXiv+preprint+arXiv%3A1905%2E07830%2C+2019a%2E)
128. Rowan Zellers, Ari Holtzman, Hannah Rashkin, Yonatan Bisk, Ali Farhadi, Franziska Roesner, and Yejin Choi. [Defending against neural fake news](https://md.research.elamur.ai/free/en/papers/zellers-2019-defending-against-neural-fake-news/). Advances in neural information processing systems, 32, 2019b.
129. Biao Zhang and Rico Sennrich. Root mean square layer normalization, 2019. [find](https://scholar.google.com/scholar?q=Biao+Zhang+and+Rico+Sennrich%2E+Root+mean+square+layer+normalization%2C+2019%2E)
130. Susan Zhang, Stephen Roller, Naman Goyal, Mikel Artetxe, Moya Chen, Shuohui Chen, Christopher Dewan, Mona Diab, Xian Li, Xi Victoria Lin, et al. [Opt: Open pre-trained transformer language models](https://md.research.elamur.ai/free/en/papers/zhang-2022-opt-open-pre-trained-transformer/). arXiv preprint arXiv:2205.01068, 2022.
131. Yanli Zhao, Andrew Gu, Rohan Varma, Liang Luo, Chien-Chin Huang, Min Xu, Less Wright, Hamid Shojanazeri, Myle Ott, Sam Shleifer, Alban Desmaison, Can Balioglu, Bernard Nguyen, Geeta Chauhan, Yuchen Hao, and Shen Li. Pytorch fsdp: Experiences on scaling fully sharded data parallel, 2023. [find](https://scholar.google.com/scholar?q=Yanli+Zhao%2C+Andrew+Gu%2C+Rohan+Varma%2C+Liang+Luo%2C+Chien%2DChin+Huang%2C+Min+Xu%2C+Less+Wright%2C+Hamid+Shojanazeri%2C+Myle+Ott%2C+Sam+Shleifer%2C+Alban+Desmaison%2C+Can+Balioglu%2C+Bernard+Nguyen%2C+Geeta+Chauhan%2C+Yuchen+Ha)
132. Wanjun Zhong, Ruixiang Cui, Yiduo Guo, Yaobo Liang, Shuai Lu, Yanlin Wang, Amin Saied, Weizhu Chen, and Nan Duan. [Agieval: A human-centric benchmark for evaluating foundation models](https://md.research.elamur.ai/free/en/papers/zhong-2024-agieval-human-centric-benchmark-evaluating/). arXiv preprint arXiv:2304.06364, 2023.
133. Chunting Zhou, Pengfei Liu, Puxin Xu, Srini Iyer, Jiao Sun, Yuning Mao, Xuezhe Ma, Avia Efrat, Ping Yu, Lili Yu, Susan Zhang, Gargi Ghosh, Mike Lewis, Luke Zettlemoyer, and Omer Levy. Lima: Less is more for alignment. arXiv preprint arXiv:2305.11206, 2023. [find](https://scholar.google.com/scholar?q=Chunting+Zhou%2C+Pengfei+Liu%2C+Puxin+Xu%2C+Srini+Iyer%2C+Jiao+Sun%2C+Yuning+Mao%2C+Xuezhe+Ma%2C+Avia+Efrat%2C+Ping+Yu%2C+Lili+Yu%2C+Susan+Zhang%2C+Gargi+Ghosh%2C+Mike+Lewis%2C+Luke+Zettlemoyer%2C+and+Omer+Levy%2E+Lima%3A+Less+is+mo)
134. Yongchao Zhou, Andrei Ioan Muresanu, Ziwen Han, Keiran Paster, Silviu Pitis, Harris Chan, and Jimmy Ba. [Large language models are human-level prompt engineers](https://md.research.elamur.ai/free/en/papers/zhou-2022-large-language-models-human-level/). In The Eleventh International Conference on Learning Representations, 2022.
135. Terry Yue Zhuo, Yujin Huang, Chunyang Chen, and Zhenchang Xing. Exploring ai ethics of chatgpt: A diagnostic analysis. arXiv preprint arXiv:2301.[12867, 2023](https://md.research.elamur.ai/free/en/papers/zhuo-2023-red-teaming-chatgpt-jailbreaking-bias/).

*Updated 30 October 2026*