---
title: "BigScience Episode – Workshop on Challenges & Perspectives"
description: "BigScience Episode – Workshop on Challenges & Perspectives in Creating Large Language Models 2022 — accepted papers: 2 papers in…"
canonical: "https://research.elamur.ai/free/en/venues/bigscience/2022/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/venues/bigscience/2022/)

[home](https://md.research.elamur.ai/free/en/) / [venues](https://md.research.elamur.ai/free/en/venues/) / [BigScience Episode – Workshop on…](https://md.research.elamur.ai/free/en/venues/bigscience/) / 2022

# BigScience Episode – Workshop on Challenges & Perspectives in Creating Large Language Models 2022 — accepted papers

BigScience Episode – Workshop on Challenges & Perspectives in Creating Large Language Models 2022 is the 2022 edition of the BigScience Episode – Workshop on Challenges & Perspectives in Creating Large Language Models, held in virtual+Dublin, May 2022 (publisher: Association for Computational Linguistics). The ACL Anthology lists 12 papers of it; our library holds 2 with our summaries. Below: the most cited papers, every accepted paper, and ours with their topics, authors and institutions.

📍 virtual+Dublin · 🗓 May 2022

[**12** papers in the proceedings](#all) · [**2** works in our library](https://md.research.elamur.ai/free/en/venues/bigscience/2022/#works) · [**951** citations of these works](https://md.research.elamur.ai/free/en/venues/bigscience/2022/#works)

## Most cited papers

By their OpenAlex citations; a paper with our summary links to its page here, the others to the original.

1. **[GPT-NeoX-20B: An Open-Source Autoregressive Language Model](https://md.research.elamur.ai/free/en/papers/black-2022-gpt-neox-20b-open-source/)**Sidney Black, Stella Biderman, Eric Hallahan et al.*404 citations · our summary*
2. **[You reap what you sow: On the Challenges of Bias Evaluation Under Multilingual Settings](https://md.research.elamur.ai/free/en/papers/talat-2022-you-reap-what-you-sow/)**Zeerak Talat, Aurélie Névéol, Stella Biderman et al.*59 citations · our summary*
3. **[Pipelines for Social Bias Testing of Large Language Models ↗](https://aclanthology.org/2022.bigscience-1.6/)**Debora Nozza, Federico Bianchi, Dirk Hovy*30 citations*
4. **[Lifelong Pretraining: Continually Adapting Language Models to Emerging Corpora ↗](https://aclanthology.org/2022.bigscience-1.1/)**Xisen Jin, Dejiao Zhang, Henghui Zhu et al.*26 citations*
5. **[UNIREX: A Unified Learning Framework for Language Model Rationale Extraction ↗](https://aclanthology.org/2022.bigscience-1.5/)**Aaron Chan, Maziar Sanjabi, Lambert Mathias et al.*19 citations*
6. **[A Holistic Assessment of the Carbon Footprint of Noor, a Very Large Arabic Language Model ↗](https://aclanthology.org/2022.bigscience-1.8/)**Imad Lakim, Ebtesam Almazrouei, Ibrahim Abualhaol et al.*18 citations*
7. **[Foundation Models of Scientific Knowledge for Chemistry: Opportunities, Challenges and Lessons Learned ↗](https://aclanthology.org/2022.bigscience-1.12/)**Sameera Horawalavithana, Ellyn Ayton, Shivam Sharma et al.*18 citations*
8. **[Emergent Structures and Training Dynamics in Large Language Models ↗](https://aclanthology.org/2022.bigscience-1.11/)**Ryan Teehan, Miruna Clinciu, Oleg Serikov et al.*7 citations*
9. **[Entities, Dates, and Languages: Zero-Shot on Historical Texts with T0 ↗](https://aclanthology.org/2022.bigscience-1.7/)**Francesco De Toni, Christopher Akiki, Javier De La Rosa et al.*7 citations*
10. **[Dataset Debt in Biomedical Language Modeling ↗](https://aclanthology.org/2022.bigscience-1.10/)**Jason Fries, Natasha Seelam, Gabriel Altay et al.*5 citations*

## All accepted papers (12)

The most cited first; 2 with our summary link to their page here, the others to the original.

1. **[GPT-NeoX-20B: An Open-Source Autoregressive Language Model](https://md.research.elamur.ai/free/en/papers/black-2022-gpt-neox-20b-open-source/)**Sidney Black, Stella Biderman, Eric Hallahan et al.*404 citations · our summary*
2. **[You reap what you sow: On the Challenges of Bias Evaluation Under Multilingual Settings](https://md.research.elamur.ai/free/en/papers/talat-2022-you-reap-what-you-sow/)**Zeerak Talat, Aurélie Névéol, Stella Biderman et al.*59 citations · our summary*
3. **[Pipelines for Social Bias Testing of Large Language Models ↗](https://aclanthology.org/2022.bigscience-1.6/)**Debora Nozza, Federico Bianchi, Dirk Hovy*30 citations*
4. **[Lifelong Pretraining: Continually Adapting Language Models to Emerging Corpora ↗](https://aclanthology.org/2022.bigscience-1.1/)**Xisen Jin, Dejiao Zhang, Henghui Zhu et al.*26 citations*
5. **[UNIREX: A Unified Learning Framework for Language Model Rationale Extraction ↗](https://aclanthology.org/2022.bigscience-1.5/)**Aaron Chan, Maziar Sanjabi, Lambert Mathias et al.*19 citations*
6. **[A Holistic Assessment of the Carbon Footprint of Noor, a Very Large Arabic Language Model ↗](https://aclanthology.org/2022.bigscience-1.8/)**Imad Lakim, Ebtesam Almazrouei, Ibrahim Abualhaol et al.*18 citations*
7. **[Foundation Models of Scientific Knowledge for Chemistry: Opportunities, Challenges and Lessons Learned ↗](https://aclanthology.org/2022.bigscience-1.12/)**Sameera Horawalavithana, Ellyn Ayton, Shivam Sharma et al.*18 citations*
8. **[Emergent Structures and Training Dynamics in Large Language Models ↗](https://aclanthology.org/2022.bigscience-1.11/)**Ryan Teehan, Miruna Clinciu, Oleg Serikov et al.*7 citations*
9. **[Entities, Dates, and Languages: Zero-Shot on Historical Texts with T0 ↗](https://aclanthology.org/2022.bigscience-1.7/)**Francesco De Toni, Christopher Akiki, Javier De La Rosa et al.*7 citations*
10. **[Dataset Debt in Biomedical Language Modeling ↗](https://aclanthology.org/2022.bigscience-1.10/)**Jason Fries, Natasha Seelam, Gabriel Altay et al.*5 citations*
11. **[Diverse Lottery Tickets Boost Ensemble from a Single Pretrained Model ↗](https://aclanthology.org/2022.bigscience-1.4/)**Sosuke Kobayashi, Shun Kiyono, Jun Suzuki et al.*4 citations*
12. **[Using ASR-Generated Text for Spoken Language Modeling ↗](https://aclanthology.org/2022.bigscience-1.2/)**Nicolas Hervé, Valentin Pelloin, Benoit Favre et al.*2 citations*

## Papers with our summary in our library (2)

The most cited first (OpenAlex citations plus our library's).

1. **[GPT-NeoX-20B: An Open-Source Autoregressive Language Model](https://md.research.elamur.ai/free/en/papers/black-2022-gpt-neox-20b-open-source/) [BigScience Episode – Workshop on Challenges & Perspectives in Creating Large Language Models](https://md.research.elamur.ai/free/en/venues/bigscience/)**Sidney Black, Stella Biderman, Eric Hallahan, et al.*2022 · 821 citations*
2. **[You reap what you sow: On the Challenges of Bias Evaluation Under Multilingual Settings](https://md.research.elamur.ai/free/en/papers/talat-2022-you-reap-what-you-sow/) [BigScience Episode – Workshop on Challenges & Perspectives in Creating Large Language Models](https://md.research.elamur.ai/free/en/venues/bigscience/)**Zeerak Talat, Aurélie Névéol, Stella Biderman, et al.*2022 · 130 citations*

## Top topics

Topics of its works in our library, with the number of works on each.

[Language models & pretraining (1)](https://md.research.elamur.ai/free/en/topics/language-models/) · [Multilingual & cross-lingual NLP (1)](https://md.research.elamur.ai/free/en/topics/multilingual-nlp/)

## Fields

[Arts and Humanities (2)](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · [Computer Science (2)](https://md.research.elamur.ai/free/en/shelves/computer-science/) · [Social Sciences (1)](https://md.research.elamur.ai/free/en/shelves/social-sciences/)

## Top authors

Authors with a page who published here most often: their works here, and the citations of those works.

[🧑‍🔬**Stella Biderman**2 works here · 951 citations](https://md.research.elamur.ai/free/en/authors/stella-biderman/) · [🧑‍🔬**Horace He**1 work here · 821 citations](https://md.research.elamur.ai/free/en/authors/horace-he/) · [🧑‍🔬**Jason Phang**1 work here · 821 citations](https://md.research.elamur.ai/free/en/authors/jason-phang/) · [🧑‍🔬**Kyle McDonell**1 work here · 821 citations](https://md.research.elamur.ai/free/en/authors/kyle-mcdonell/) · [🧑‍🔬**Leo Gao**1 work here · 821 citations](https://md.research.elamur.ai/free/en/authors/leo-gao/) · [🧑‍🔬**Michael Pieler**1 work here · 821 citations](https://md.research.elamur.ai/free/en/authors/michael-pieler/) · [🧑‍🔬**Quentin Anthony**1 work here · 821 citations](https://md.research.elamur.ai/free/en/authors/quentin-anthony/) · [🧑‍🔬**Arjun Subramonian**1 work here · 130 citations](https://md.research.elamur.ai/free/en/authors/arjun-subramonian/) · [🧑‍🔬**Aurelie Neveol**1 work here · 130 citations](https://md.research.elamur.ai/free/en/authors/aurelie-neveol/) · [🧑‍🔬**Dragomir Radev**1 work here · 130 citations](https://md.research.elamur.ai/free/en/authors/dragomir-radev/) · [🧑‍🔬**Manan Dey**1 work here · 130 citations](https://md.research.elamur.ai/free/en/authors/manan-dey/) · [🧑‍🔬**Maraim Masoud**1 work here · 130 citations](https://md.research.elamur.ai/free/en/authors/maraim-masoud/)

## Top institutions

Institutions on the bylines of its works here, by the number of works.

[🏢**Google** Mountain View · United States 1 work here](https://md.research.elamur.ai/free/en/companies/google/) · [🔬**Centre National de la Recherche Scientifique** Paris · France 1 work here](https://md.research.elamur.ai/free/en/labs/centre-national-de-la-recherche-scientifique/) · [🏢**Amazon** Seattle · United States 1 work here](https://md.research.elamur.ai/free/en/companies/amazon/) · [🎓**University of California, Los Angeles** Los Angeles · United States 1 work here](https://md.research.elamur.ai/free/en/universities/university-of-california-los-angeles/) · [🎓**National University of Singapore** Singapore · Singapore 1 work here](https://md.research.elamur.ai/free/en/universities/national-university-of-singapore/) · [🎓**Simon Fraser University** Burnaby · Canada 1 work here](https://md.research.elamur.ai/free/en/universities/simon-fraser-university/) · [🎓**Trinity College Dublin** Dublin · Ireland 1 work here](https://md.research.elamur.ai/free/en/universities/trinity-college-dublin/) · [🎓**Université Paris-Saclay** Gif-sur-Yvette · France 1 work here](https://md.research.elamur.ai/free/en/universities/universite-paris-saclay/) · [🎓**Heriot-Watt University** Edinburgh · United Kingdom 1 work here](https://md.research.elamur.ai/free/en/universities/heriot-watt-university/) · [🎓**Yale University** New Haven · United States 1 work here](https://md.research.elamur.ai/free/en/universities/yale-university/) · [🔬**Beijing Academy of Artificial Intelligence** Beijing · China 1 work here](https://md.research.elamur.ai/free/en/labs/beijing-academy-of-artificial-intelligence/) · [🏢**Nvidia** Santa Clara · United States 1 work here](https://md.research.elamur.ai/free/en/companies/nvidia/)

Place and dates from the event's site or the ACL Anthology; works and citations from OpenAlex and our library.

- [↑ back to BigScience Episode – Workshop on Challenges & Perspectives in Creating Large Language Models](https://md.research.elamur.ai/free/en/venues/bigscience/)