---
title: "IAI Group at CheckThat! 2024 — Aarnes 2024"
description: "The paper describes the University of Stavanger IAI group's entry to the CLEF 2024 CheckThat! 0.848 F1. 31 references, cited by 1 paper here."
authors: ["Peter R\\oysland Aarnes", "Vinay Setty", "Petra Galu\\vs\\vcakova"]
year: 2024
venue: "arXiv:2408.01118"
source: "https://arxiv.org/abs/2408.01118"
doi: "10.48550/arxiv.2408.01118"
arxiv: "2408.01118"
licence: "CC BY-SA 4.0 (arXiv)"
licence_url: "http://creativecommons.org/licenses/by-sa/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/aarnes-2024-iai-group-checkthat-2024-transformer/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/aarnes-2024-iai-group-checkthat-2024-transformer/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2024](https://md.research.elamur.ai/free/en/papers/2024/page/5/) / IAI Group at CheckThat! 2024: Transformer Models and Data Augmentation for Checkworthy Claim Detection

# IAI Group at CheckThat! 2024: Transformer Models and Data Augmentation for Checkworthy Claim Detection

**Authors:** Peter R\\oysland Aarnes, Vinay Setty and Petra Galu\\vs\\vcakova · **Year:** 2024 · **Venue:** arXiv:2408.01118 · **Licence:** [CC BY-SA 4.0 (arXiv)](http://creativecommons.org/licenses/by-sa/4.0/)

**Links:** [Publisher's page](https://arxiv.org/abs/2408.01118) · [PDF](https://arxiv.org/pdf/2408.01118) · [DOI 10.48550/arxiv.2408.01118](https://doi.org/10.48550/arxiv.2408.01118) · [arXiv 2408.01118](https://arxiv.org/abs/2408.01118) · [OpenAlex](https://openalex.org/W4406301873)

**Citations:** 3 citations (OpenAlex, 2026-10-14) · 1 in our library, of the 140,884 papers whose bibliographies we read

## Summary
The paper describes the University of Stavanger IAI group's entry to the CLEF 2024 CheckThat! Lab Task 1. The task is a binary decision: does a sentence deserve fact-checking? The data cover English political debate transcripts and Dutch and Arabic tweets. The team compared two families of models. The first was fine-tuned encoder models (RoBERTa-Large, XLM-RoBERTa). The second was generative LLMs (GPT-3.5, GPT-4, Mistral 7b), used either through few-shot chain-of-thought prompting or through fine-tuning. They also tested several data tricks:
- cross-lingual training with machine-translated data;
- tweet normalization;
- LLM-based style transfer from debate style to tweet style.

Their final scores were:
- **Arabic:** first place, by translating the Arabic test set into English and labelling it with an English-fine-tuned GPT-3.5.
- **Dutch:** third place, with GPT-4 prompting.
- **English:** ninth place, with a plainly fine-tuned RoBERTa-Large.

Every system scored much lower on the hidden test set than on the development test split. The paper is useful mainly as an honest record of which augmentation choices helped, which did not, and how unreliable development estimates can be under domain shift.

## Key points
- **Training set sizes** were highly unequal and skewed toward the "No" class:
  - English: 22,501 sentences
  - Arabic: 7,333
  - Dutch: 995
- **Final F1 scores:**
  - English: 0.753 (RoBERTa-Large on the untouched organizer data)
  - Dutch: 0.718 (GPT-4 with few-shot CoT)
  - Arabic: 0.569 (translate to English, then fine-tuned GPT-3.5)
- **A better Dutch run went unsubmitted.** A GPT-3.5 fine-tuned on Spanish, Arabic and Dutch data reached 0.781 on the Dutch test set. That would have topped the leaderboard.
- **Translation tool:** Google Translate, accessed through the deep-translator library, moved data between English, Dutch, Arabic and Spanish.
- **Two preprocessing steps did not help:**
  - TweetNormalizer lowered results.
  - Style transfer with gpt-3.5-turbo-0125 (debate sentences rewritten as Arabic-style tweets) gave no gain.
- **A misleading self-check:** the team hand-labelled 10% of the Arabic test data before gold labels were released. The chosen model scored 0.848 F1 on that sample but only 0.569 on the full set.

## Interesting ideas
- Instead of building a strong classifier in a weaker language, translate the input into the language where the best model already exists and classify it there; this reuses one well-trained component across languages.
- The authors suspect TweetNormalizer hurt because it collapsed every distinct @username into a generic @USER token, deleting identifiers that carried signal; normalization can silently throw away useful information.
- Rewriting training data with an LLM so its genre matches the target domain (debate prose into tweets) is a cheap way to make in-domain training data, but here it gave no benefit, so such data must be tested before it is trusted.
- A small hand-labelled sample of the test set picked the "best" model, yet the full test set reversed the picture; small in-house checks under distribution shift can mislead badly.
- The paper recommends choosing checkpoints by macro-averaged F1 rather than accuracy, so the rare positive class counts as much as the dominant one in imbalanced data.

## For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

- **I1** (back): Non-English input could be routed through translation into English, so the existing English reader handles meaning, with the translation step logged to keep the chain explainable.
- **I2** (tokens): The tokenizer should keep handles, names and similar identifiers as distinct, typed tokens rather than flattening them, so later stages can still link them to entities.
- **I3** (tree): Restyled treebank sentences could serve as candidate silver data for gparse on new genres, but they should be admitted only after an A/B comparison against the plain data.
- **I4** (learn): Evaluations should report how stable rankings are across several held-out slices and include negative controls, rather than trusting a single small sample.
- **I5** (learn): When training detectors for rare phenomena (for example irony or particular speech acts), select models by macro-F1 so frequent labels do not mask failures on rare ones.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

Fields: [Linguistics & languages](https://md.research.elamur.ai/free/en/shelves/linguistics/)

Topics: [🐪 Semitic languages](https://md.research.elamur.ai/free/en/topics/semitic-languages/)

## Institutions

Also on the byline: [University of Stavanger](https://ror.org/02qte9q33).

## Cite

Cite as: Peter R\\oysland Aarnes, Vinay Setty and Petra Galu\\vs\\vcakova. 2024. IAI Group at CheckThat! 2024: Transformer Models and Data Augmentation for Checkworthy Claim Detection. arXiv:2408.01118.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@misc{aarnes2024iai,
  title = {{IAI Group at CheckThat! 2024: Transformer Models and Data Augmentation for Checkworthy Claim Detection}},
  author = {Peter Roysland Aarnes, Vinay Setty and Petra Galuvsvcakova},
  year = {2024},
  doi = {10.48550/arxiv.2408.01118},
  eprint = {2408.01118},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2408.01118}
}
```

- [← previous How BERT Speaks Shakespearean English? Evaluating Historical Bias in Contextual Language Models](https://md.research.elamur.ai/free/en/papers/cuscito-2024-bert-speaks-shakespearean-english-evaluating/)
- [next → MsBERT: A New Model for the Reconstruction of Lacunae in Hebrew Manuscripts](https://md.research.elamur.ai/free/en/papers/shmidman-2024-msbert-new-model-reconstruction-lacunae/)
- [↑ back to papers of 2024](https://md.research.elamur.ai/free/en/papers/2024/page/5/)

## References

31 references, 7 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. L. Konstantinovskiy, O. Price, M. Babakar, A. Zubiaga, Toward automated factchecking: Developing an annotation schema and benchmark for consistent automated claim detection, Digital Threats 2 (2021) 1–16. [find](https://scholar.google.com/scholar?q=L%2E+Konstantinovskiy%2C+O%2E+Price%2C+M%2E+Babakar%2C+A%2E+Zubiaga%2C+Toward+automated+factchecking%3A+Developing+an+annotation+schema+and+benchmark+for+consistent+automated+claim+detection%2C+Digital+Threats+2+%282021%29+1)
2. N. Hassan, F. Arslan, C. Li, M. Tremayne, Toward automated fact-checking: Detecting check-worthy factual claims by claimbuster, in: Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 2017, pp. 1803–1812. [find](https://scholar.google.com/scholar?q=N%2E+Hassan%2C+F%2E+Arslan%2C+C%2E+Li%2C+M%2E+Tremayne%2C+Toward+automated+fact%2Dchecking%3A+Detecting+check%2Dworthy+factual+claims+by+claimbuster%2C+in%3A+Proceedings+of+the+23rd+ACM+SIGKDD+International+Conference+on+Knowl)
3. F. Alam, S. Shaar, F. Dalvi, H. Sajjad, A. Nikolov, H. Mubarak, G. Da San Martino, A. Abdelali, N. Durrani, K. Darwish, A. Al-Homaid, W. Zaghouani, T. Caselli, G. Danoe, F. Stolk, B. Bruntink, P. Nakov, Fighting the covid-19 infodemic: Modeling the perspective of journalists, fact-checkers, social media platforms, policy makers, and the society, in: Findings of the Association for Computational Linguistics: EMNLP 2021, 2021, pp. 611–649. [find](https://scholar.google.com/scholar?q=F%2E+Alam%2C+S%2E+Shaar%2C+F%2E+Dalvi%2C+H%2E+Sajjad%2C+A%2E+Nikolov%2C+H%2E+Mubarak%2C+G%2E+Da+San+Martino%2C+A%2E+Abdelali%2C+N%2E+Durrani%2C+K%2E+Darwish%2C+A%2E+Al%2DHomaid%2C+W%2E+Zaghouani%2C+T%2E+Caselli%2C+G%2E+Danoe%2C+F%2E+Stolk%2C+B%2E+Bruntink%2C+P%2E+Nako)
4. A. Barrón-Cedeño, F. Alam, T. Chakraborty, T. Elsayed, P. Nakov, P. Przybyła, J. M. Struß, F. Haouari, M. Hasanain, F. Ruggeri, X. Song, R. Suwaileh, The CLEF-2024 CheckThat! Lab: Check-worthiness, subjectivity, persuasion, roles, authorities, and adversarial robustness, in: Advances in Information Retrieval, 2024, pp. 449–458. [find](https://scholar.google.com/scholar?q=A%2E+Barr%C3%B3n%2DCede%C3%B1o%2C+F%2E+Alam%2C+T%2E+Chakraborty%2C+T%2E+Elsayed%2C+P%2E+Nakov%2C+P%2E+Przyby%C5%82a%2C+J%2E+M%2E+Stru%C3%9F%2C+F%2E+Haouari%2C+M%2E+Hasanain%2C+F%2E+Ruggeri%2C+X%2E+Song%2C+R%2E+Suwaileh%2C+The+CLEF%2D2024+CheckThat%21+Lab%3A+Check%2Dworthiness%2C+su)
5. I. Siles, P. J. Boczkowski, Making sense of the newspaper crisis: A critical assessment of existing research and an agenda for future work, New Media & Society 14 (2012) 1375–1394. [find](https://scholar.google.com/scholar?q=I%2E+Siles%2C+P%2E+J%2E+Boczkowski%2C+Making+sense+of+the+newspaper+crisis%3A+A+critical+assessment+of+existing+research+and+an+agenda+for+future+work%2C+New+Media+%26+Society+14+%282012%29+1375%E2%80%931394%2E)
6. S. Vosoughi, D. Roy, S. Aral, The spread of true and false news online, Science 359 (2018) 1146–1151. [find](https://scholar.google.com/scholar?q=S%2E+Vosoughi%2C+D%2E+Roy%2C+S%2E+Aral%2C+The+spread+of+true+and+false+news+online%2C+Science+359+%282018%29+1146%E2%80%931151%2E)
7. H. Allcott, M. Gentzkow, Social media and fake news in the 2016 election, Journal of Economic Perspectives 31 (2017) 211–236. [find](https://scholar.google.com/scholar?q=H%2E+Allcott%2C+M%2E+Gentzkow%2C+Social+media+and+fake+news+in+the+2016+election%2C+Journal+of+Economic+Perspectives+31+%282017%29+211%E2%80%93236%2E)
8. A. Bovet, H. A. Makse, Influence of fake news in twitter during the 2016 us presidential election, Nature Communications 10 (2019) 7. [find](https://scholar.google.com/scholar?q=A%2E+Bovet%2C+H%2E+A%2E+Makse%2C+Influence+of+fake+news+in+twitter+during+the+2016+us+presidential+election%2C+Nature+Communications+10+%282019%29+7%2E)
9. T. Pavlov, G. Mirceva, Covid-19 fake news detection by using bert and roberta models, in: 2022 45th jubilee international convention on information, communication and electronic technology (MIPRO), 2022, pp. 312–316. [find](https://scholar.google.com/scholar?q=T%2E+Pavlov%2C+G%2E+Mirceva%2C+Covid%2D19+fake+news+detection+by+using+bert+and+roberta+models%2C+in%3A+2022+45th+jubilee+international+convention+on+information%2C+communication+and+electronic+technology+%28MIPRO%29%2C+20)
10. Z. Guo, M. Schlichtkrull, A. Vlachos, A survey on automated fact-checking, Transactions of the Association for Computational Linguistics 10 (2022) 178–206. [our card](https://md.research.elamur.ai/free/en/papers/guo-2022-survey-automated-fact-checking/)
11. F. Arslan, N. Hassan, C. Li, M. Tremayne, A benchmark dataset of check-worthy factual claims, 2020. [find](https://scholar.google.com/scholar?q=F%2E+Arslan%2C+N%2E+Hassan%2C+C%2E+Li%2C+M%2E+Tremayne%2C+A+benchmark+dataset+of+check%2Dworthy+factual+claims%2C+2020%2E)
12. T. Elsayed, P. Nakov, A. Barrón-Cedeño, M. Hasanain, R. Suwaileh, G. D. S. Martino, P. Atanasova, Overview of the clef-2019 checkthat!: Automatic identification and verification of claims, 2019. [our card](https://md.research.elamur.ai/free/en/papers/elsayed-2021-overview-clef-2019-checkthat-automatic/)
13. A. Gupta, V. Srikumar, X-fact: A new benchmark dataset for multilingual fact checking, in: Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 2: Short Papers), 2021, pp. 675–682. [find](https://scholar.google.com/scholar?q=A%2E+Gupta%2C+V%2E+Srikumar%2C+X%2Dfact%3A+A+new+benchmark+dataset+for+multilingual+fact+checking%2C+in%3A+Proceedings+of+the+59th+Annual+Meeting+of+the+Association+for+Computational+Linguistics+and+the+11th+Internat)
14. X. Zeng, A. S. Abumansour, A. Zubiaga, Automated fact-checking: A survey, 2021. [find](https://scholar.google.com/scholar?q=X%2E+Zeng%2C+A%2E+S%2E+Abumansour%2C+A%2E+Zubiaga%2C+Automated+fact%2Dchecking%3A+A+survey%2C+2021%2E)
15. P. Nakov, A. Barrón-Cedeño, G. Da San Martino, F. Alam, R. Míguez, T. Caselli, M. Kutlu, W. Zaghouani, C. Li, S. Shaar, H. Mubarak, A. Nikolov, Y. S. Kartal, J. Beltrán, Overview of the CLEF-2022 CheckThat! lab task 1 on identifying relevant claims in tweets, in: Working Notes of CLEF 2022—Conference and Labs of the Evaluation Forum, 2022. [find](https://scholar.google.com/scholar?q=P%2E+Nakov%2C+A%2E+Barr%C3%B3n%2DCede%C3%B1o%2C+G%2E+Da+San+Martino%2C+F%2E+Alam%2C+R%2E+M%C3%ADguez%2C+T%2E+Caselli%2C+M%2E+Kutlu%2C+W%2E+Zaghouani%2C+C%2E+Li%2C+S%2E+Shaar%2C+H%2E+Mubarak%2C+A%2E+Nikolov%2C+Y%2E+S%2E+Kartal%2C+J%2E+Beltr%C3%A1n%2C+Overview+of+the+CLEF%2D2022+Chec)
16. F. Alam, A. Barrón-Cedeño, G. S. Cheema, G. K. Shahi, S. Hakimov, M. Hasanain, C. Li, R. Míguez, H. Mubarak, W. Zaghouani, P. Nakov, Overview of the clef-2023 checkthat! lab task 1 on check-worthiness of multimodal and multigenre content (2023). [find](https://scholar.google.com/scholar?q=F%2E+Alam%2C+A%2E+Barr%C3%B3n%2DCede%C3%B1o%2C+G%2E+S%2E+Cheema%2C+G%2E+K%2E+Shahi%2C+S%2E+Hakimov%2C+M%2E+Hasanain%2C+C%2E+Li%2C+R%2E+M%C3%ADguez%2C+H%2E+Mubarak%2C+W%2E+Zaghouani%2C+P%2E+Nakov%2C+Overview+of+the+clef%2D2023+checkthat%21+lab+task+1+on+check%2Dworthiness)
17. S. K. N. Mingzhe Du, Sujatha Das Gollapalli, NUS-IDS at CheckThat! 2022: identifying check-worthiness of tweets using CheckthaT5, in: N. Faggioli, Guglielmo andd Ferro, A. Hanbury, M. Potthast (Eds.), Working Notes of CLEF 2022 - Conference and Labs of the Evaluation Forum, CLEF 2022, 2022. [find](https://scholar.google.com/scholar?q=S%2E+K%2E+N%2E+Mingzhe+Du%2C+Sujatha+Das+Gollapalli%2C+NUS%2DIDS+at+CheckThat%21+2022%3A+identifying+check%2Dworthiness+of+tweets+using+CheckthaT5%2C+in%3A+N%2E+Faggioli%2C+Guglielmo+andd+Ferro%2C+A%2E+Hanbury%2C+M%2E+Potthast+%28Eds%2E%29%2C)
18. L. Xue, N. Constant, A. Roberts, M. Kale, R. Al-Rfou, A. Siddhant, A. Barua, C. Raffel, mt5: A massively multilingual pre-trained text-to-text transformer, 2021. [our card](https://md.research.elamur.ai/free/en/papers/xue-2021-mt5-massively-multilingual-pre-trained/)
19. M. Sawinski, K. Węcel, E. Księżniak, M. Stróżyna, W. Lewoniewski, P. Stolarski, W. Abramowicz, Openfact at checkthat! 2023: Head-to-head gpt vs. bert - a comparative study of transformers language models for the detection of check-worthy claims, CLEF 2023, 2023. [find](https://scholar.google.com/scholar?q=M%2E+Sawinski%2C+K%2E+W%C4%99cel%2C+E%2E+Ksi%C4%99%C5%BCniak%2C+M%2E+Str%C3%B3%C5%BCyna%2C+W%2E+Lewoniewski%2C+P%2E+Stolarski%2C+W%2E+Abramowicz%2C+Openfact+at+checkthat%21+2023%3A+Head%2Dto%2Dhead+gpt+vs%2E+bert+%2D+a+comparative+study+of+transformers+language+mod)
20. M. Abdul-Mageed, A. Elmadany, E. M. B. Nagoudi, Arbert & marbert: Deep bidirectional transformers for arabic, in: Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), 2021, pp. 7088–7105. [our card](https://md.research.elamur.ai/free/en/papers/abdul-mageed-2021-arbert-marbert-deep-bidirectional-transformers/)
21. H. T. Sadouk, F. Sebbak, H. E. Zekiri, Es-vrai at checkthat! 2023: Analyzing checkworthiness in multimodal and multigenre contents through fusion and sampling approaches (2023). [find](https://scholar.google.com/scholar?q=H%2E+T%2E+Sadouk%2C+F%2E+Sebbak%2C+H%2E+E%2E+Zekiri%2C+Es%2Dvrai+at+checkthat%21+2023%3A+Analyzing+checkworthiness+in+multimodal+and+multigenre+contents+through+fusion+and+sampling+approaches+%282023%29%2E)
22. S. Henning, W. Beluch, A. Fraser, A. Friedrich, A survey of methods for addressing class imbalance in deep-learning based natural language processing, in: Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics, 2023, pp. 523–540. [find](https://scholar.google.com/scholar?q=S%2E+Henning%2C+W%2E+Beluch%2C+A%2E+Fraser%2C+A%2E+Friedrich%2C+A+survey+of+methods+for+addressing+class+imbalance+in+deep%2Dlearning+based+natural+language+processing%2C+in%3A+Proceedings+of+the+17th+Conference+of+the+Eur)
23. L. Biewald, Experiment tracking with weights and biases}, 2020. [find](https://scholar.google.com/scholar?q=L%2E+Biewald%2C+Experiment+tracking+with+weights+and+biases%7D%2C+2020%2E)
24. D. Q. Nguyen, T. Vu, A. Tuan Nguyen, Bertweet: A pre-trained language model for english tweets, in: Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, 2020, pp. 9–14. [our card](https://md.research.elamur.ai/free/en/papers/nguyen-2020-bertweet-pre-trained-language-model/)
25. T. Kocmi, E. Avramidis, R. Bawden, O. Bojar, A. Dvorkovich, C. Federmann, M. Fishel, M. Freitag, T. Gowda, R. Grundkiewicz, B. Haddow, P. Koehn, B. Marie, C. Monz, M. Morishita, K. Murray, M. Nagata, T. Nakazawa, M. Popel, M. Popović, M. Shmatova, Findings of the 2023 conference on machine translation (WMT23): LLMs are here but not quite there yet, in: Proceedings of the Eighth Conference on Machine Translation, WMT’23, 2023, pp. 1–42. [find](https://scholar.google.com/scholar?q=T%2E+Kocmi%2C+E%2E+Avramidis%2C+R%2E+Bawden%2C+O%2E+Bojar%2C+A%2E+Dvorkovich%2C+C%2E+Federmann%2C+M%2E+Fishel%2C+M%2E+Freitag%2C+T%2E+Gowda%2C+R%2E+Grundkiewicz%2C+B%2E+Haddow%2C+P%2E+Koehn%2C+B%2E+Marie%2C+C%2E+Monz%2C+M%2E+Morishita%2C+K%2E+Murray%2C+M%2E+Nagata%2C+)
26. OpenAI, J. Achiam, S. Adler, S. Agarwal, L. Ahmad, I. Akkaya, F. L. Aleman, D. Almeida, J. Altenschmidt, S. Altman, S. Anadkat, R. Avila, I. Babuschkin, S. Balaji, V. Balcom, P. Baltescu, H. Bao, M. Bavarian, J. Belgum, I. Bello, J. Berdine, G. Bernadett-Shapiro, C. Berner, L. Bogdonoff, O. Boiko, M. Boyd, A.-L. Brakman, G. Brockman, T. Brooks, M. Brundage, K. Button, T. Cai, R. Campbell, A. Cann, B. Carey, C. Carlson, R. Carmichael, B. Chan, C. Chang, F. Chantzis, D. Chen, S. Chen, R. Chen, J. Chen, M. Chen, B. Chess, C. Cho, C. Chu, H. W. Chung, D. Cummings, J. Currier, Y. Dai, C. Decareaux, T. Degry, N. Deutsch, D. Deville, A. Dhar, D. Dohan, S. Dowling, S. Dunning, A. Ecoffet, A. Eleti, T. Eloundou, D. Farhi, L. Fedus, N. Felix, S. P. Fishman, J. Forte, I. Fulford, L. Gao, E. Georges, C. Gibson, V. Goel, T. Gogineni, G. Goh, R. Gontijo-Lopes, J. Gordon, M. Grafstein, S. Gray, R. Greene, J. Gross, S. S. Gu, Y. Guo, C. Hallacy, J. Han, J. Harris, Y. He, M. Heaton, J. Heidecke, C. Hesse, A. Hickey, W. Hickey, P. Hoeschele, B. Houghton, K. Hsu, S. Hu, X. Hu, J. Huizinga, S. Jain, S. Jain, J. Jang, A. Jiang, R. Jiang, H. Jin, D. Jin, S. Jomoto, B. Jonn, H. Jun, T. Kaftan, . Kaiser, A. Kamali, I. Kanitscheider, N. S. Keskar, T. Khan, L. Kilpatrick, J. W. Kim, C. Kim, Y. Kim, J. H. Kirchner, J. Kiros, M. Knight, D. Kokotajlo, . Kondraciuk, A. Kondrich, A. Konstantinidis, K. Kosic, G. Krueger, V. Kuo, M. Lampe, I. Lan, T. Lee, J. Leike, J. Leung, D. Levy, C. M. Li, R. Lim, M. Lin, S. Lin, M. Litwin, T. Lopez, R. Lowe, P. Lue, A. Makanju, K. Malfacini, S. Manning, T. Markov, Y. Markovski, B. Martin, K. Mayer, A. Mayne, B. McGrew, S. M. McKinney, C. McLeavey, P. McMillan, J. McNeil, D. Medina, A. Mehta, J. Menick, L. Metz, A. Mishchenko, P. Mishkin, V. Monaco, E. Morikawa, D. Mossing, T. Mu, M. Murati, O. Murk, D. Mély, A. Nair, R. Nakano, R. Nayak, A. Neelakantan, R. Ngo, H. Noh, L. Ouyang, C. O’Keefe, J. Pachocki, A. Paino, J. Palermo, A. Pantuliano, G. Parascandolo, J. Parish, E. Parparita, A. Passos, M. Pavlov, A. Peng, A. Perelman, F. d. A. B. Peres, M. Petrov, H. P. d. O. Pinto, Michael, Pokorny, M. Pokrass, V. H. Pong, T. Powell, A. Power, B. Power, E. Proehl, R. Puri, A. Radford, J. Rae, A. Ramesh, C. Raymond, F. Real, K. Rimbach, C. Ross, B. Rotsted, H. Roussez, N. Ryder, M. Saltarelli, T. Sanders, S. Santurkar, G. Sastry, H. Schmidt, D. Schnurr, J. Schulman, D. Selsam, K. Sheppard, T. Sherbakov, J. Shieh, S. Shoker, P. Shyam, S. Sidor, E. Sigler, M. Simens, J. Sitkin, K. Slama, I. Sohl, B. Sokolowsky, Y. Song, N. Staudacher, F. P. Such, N. Summers, I. Sutskever, J. Tang, N. Tezak, M. B. Thompson, P. Tillet, A. Tootoonchian, E. Tseng, P. Tuggle, N. Turley, J. Tworek, J. F. C. Uribe, A. Vallone, A. Vijayvergiya, C. Voss, C. Wainwright, J. J. Wang, A. Wang, B. Wang, J. Ward, J. Wei, C. J. Weinmann, A. Welihinda, P. Welinder, J. Weng, L. Weng, M. Wiethoff, D. Willner, C. Winter, S. Wolrich, H. Wong, L. Workman, S. Wu, J. Wu, M. Wu, K. Xiao, T. Xu, S. Yoo, K. Yu, Q. Yuan, W. Zaremba, R. Zellers, C. Zhang, M. Zhang, S. Zhao, T. Zheng, J. Zhuang, W. Zhuk, B. Zoph, Gpt-4 technical report, 2024. [our card](https://md.research.elamur.ai/free/en/papers/openai-2023-gpt-4-technical-report/)
27. A. Q. Jiang, A. Sablayrolles, A. Mensch, C. Bamford, D. S. Chaplot, D. de las Casas, F. Bressand, G. Lengyel, G. Lample, L. Saulnier, L. R. Lavaud, M.-A. Lachaux, P. Stock, T. L. Scao, T. Lavril, T. Wang, T. Lacroix, W. E. Sayed, Mistral 7b, 2023. [find](https://scholar.google.com/scholar?q=A%2E+Q%2E+Jiang%2C+A%2E+Sablayrolles%2C+A%2E+Mensch%2C+C%2E+Bamford%2C+D%2E+S%2E+Chaplot%2C+D%2E+de+las+Casas%2C+F%2E+Bressand%2C+G%2E+Lengyel%2C+G%2E+Lample%2C+L%2E+Saulnier%2C+L%2E+R%2E+Lavaud%2C+M%2E%2DA%2E+Lachaux%2C+P%2E+Stock%2C+T%2E+L%2E+Scao%2C+T%2E+Lavril%2C+T%2E+W)
28. L. Ouyang, J. Wu, X. Jiang, D. Almeida, C. L. Wainwright, P. Mishkin, C. Zhang, S. Agarwal, K. Slama, A. Ray, J. Schulman, J. Hilton, F. Kelton, L. Miller, M. Simens, A. Askell, P. Welinder, P. Christiano, J. Leike, R. Lowe, Training language models to follow instructions with human feedback, 2022. [our card](https://md.research.elamur.ai/free/en/papers/ouyang-2022-training-language-models-follow-instructions/)
29. A. Conneau, K. Khandelwal, N. Goyal, V. Chaudhary, G. Wenzek, F. Guzmán, E. Grave, M. Ott, L. Zettlemoyer, V. Stoyanov, Unsupervised cross-lingual representation learning at scale, 2020. [find](https://scholar.google.com/scholar?q=A%2E+Conneau%2C+K%2E+Khandelwal%2C+N%2E+Goyal%2C+V%2E+Chaudhary%2C+G%2E+Wenzek%2C+F%2E+Guzm%C3%A1n%2C+E%2E+Grave%2C+M%2E+Ott%2C+L%2E+Zettlemoyer%2C+V%2E+Stoyanov%2C+Unsupervised+cross%2Dlingual+representation+learning+at+scale%2C+2020%2E)
30. Y. Liu, M. Ott, N. Goyal, J. Du, M. Joshi, D. Chen, O. Levy, M. Lewis, L. Zettlemoyer, V. Stoyanov, Roberta: A robustly optimized bert pretraining approach, 2019. [find](https://scholar.google.com/scholar?q=Y%2E+Liu%2C+M%2E+Ott%2C+N%2E+Goyal%2C+J%2E+Du%2C+M%2E+Joshi%2C+D%2E+Chen%2C+O%2E+Levy%2C+M%2E+Lewis%2C+L%2E+Zettlemoyer%2C+V%2E+Stoyanov%2C+Roberta%3A+A+robustly+optimized+bert+pretraining+approach%2C+2019%2E)
31. A. J. Becker, Automated fact-checking of podcasts, 2023. [find](https://scholar.google.com/scholar?q=A%2E+J%2E+Becker%2C+Automated+fact%2Dchecking+of+podcasts%2C+2023%2E)