---
title: "Kenji-Endo: a BabyLM @EVALITA — Scozzaro 2026"
description: "The paper asks whether a compact Italian language model trained on modest data can compete with… 33 references, cited by 2 papers here."
authors: ["Calogero Jerik Scozzaro", "Matteo Rinaldi", "Gianluca Mittone", "Marco Antonio Stranisci"]
year: 2026
venue: "Proceedings of the Ninth Evaluation Campaign of Natural Language Processing and Speech Tools for Italian. Final Workshop (EVALITA 2026)"
source: "https://aclanthology.org/2026.evalita-1.4/"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/scozzaro-2026-kenji-endo-babylm-evalita/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/scozzaro-2026-kenji-endo-babylm-evalita/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2026](https://md.research.elamur.ai/free/en/papers/2026/page/8/) / Kenji-Endo: a BabyLM @EVALITA

# Kenji-Endo: a BabyLM @EVALITA

**Authors:** [Calogero Jerik Scozzaro](https://md.research.elamur.ai/free/en/authors/calogero-jerik-scozzaro/), [Matteo Rinaldi](https://md.research.elamur.ai/free/en/authors/matteo-rinaldi/), Gianluca Mittone and [Marco Antonio Stranisci](https://md.research.elamur.ai/free/en/authors/marco-antonio-stranisci/) · **Year:** 2026 · **Venue:** [Proceedings of the Ninth Evaluation Campaign of Natural Language Processing and Speech Tools for Italian. Final Workshop (EVALITA 2026)](https://md.research.elamur.ai/free/en/venues/evalita/) ([Evaluation Campaign of Natural Language Processing and Speech Tools for Italian. Final Workshop 2026](https://md.research.elamur.ai/free/en/venues/evalita/2026/)) · **Publisher:** [CEUR Workshop Proceedings](https://md.research.elamur.ai/free/en/publishers/ceur-workshop-proceedings/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2026.evalita-1.4/) · [PDF](https://aclanthology.org/2026.evalita-1.4.pdf) · [OpenAlex](https://openalex.org/W7216643159)

**Citations:** 0 citations (OpenAlex, 2026-10-14) · [2](https://md.research.elamur.ai/free/en/papers/scozzaro-2026-kenji-endo-babylm-evalita/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
The paper asks whether a compact Italian language model trained on modest data can compete with much larger LLMs on real evaluation benchmarks. The authors built the Kenji-Endo corpus for pretraining. It mixes child language (CHILDES), spoken transcriptions (Kiparla, VoLip), tweets, legal texts (Eurlex, Gazzetta Ufficiale), easy-to-read news (DueParole), and books, forums and Wikipedia from TestiMole. Documents were filtered for readability using De Mauro's Nuovo Vocabolario di Base. Three decoder models were pretrained from scratch: a dense model, a Mixture-of-Experts model, and an ALiBi variant trained with the in-house Matformer library on consumer hardware. The models were adapted to classification either by fine-tuning with a classification head or by prompting. They were then entered in four EVALITA 2026 tasks. On certain subtasks the results were strong: the models came second on DeSegMa-IT's Subtask A and topped the ranking for Subtask 1 of IMPOLS. Performance dropped sharply on pragmatically abstract or noisy inputs. The work matters because it shows that small, cheaply trained models remain a practical option for Italian NLP.

## Key points
- **Corpus:** after readability filtering (third-quartile threshold per text type), the paper reports 184,000 documents and 83.8M tokens. The introduction gives a rough figure of 135M tokens, so the reported sizes do not fully agree.
- **Models:**
  - Kenji-Endo vanilla: about 130M parameters, Qwen3ForCausalLM, 12 layers, hidden size 768.
  - Kenji-Endo MoE: about 240M parameters, Qwen2MoeForCausalLM, 7 experts with top-1 routing plus one shared expert.
  - Kenji-Endo AliBi: trained in 8 hours on a single RTX 3090 with the Muon optimizer.
- **Training setup:** the vanilla and MoE models were pretrained on a single Grace Hopper (H100) node of the HPC4AI cluster, using bfloat16, the xFFL framework, a 1024-token context and 10 epochs.
- **IMPOLS:** best F1 on Subtask 1 (0.9496, versus 0.9089 for the runner-up). Subtask 2 fell to 0.2226 because the model predicted only a single class.
- **FadeIT and MultiPRIDE:** FadeIT Subtask A ranked 7th of 15 runs with micro F1 47.37 (baseline 20.84), but the system detected none of 14 out of 19 fallacy types. On MultiPRIDE Subtask B it scored F1 0.7489, below the 0.8981 baseline.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [University of Turin](https://md.research.elamur.ai/free/en/universities/university-of-turin/) — University, Turin, Italy · 33 works · 6,637 citations

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 16,039 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 15,891 works

## Cite

Cite as: Calogero Jerik Scozzaro, Matteo Rinaldi, Gianluca Mittone and Marco Antonio Stranisci. 2026. Kenji-Endo: a BabyLM @EVALITA. Proceedings of the Ninth Evaluation Campaign of Natural Language Processing and Speech Tools for Italian. Final Workshop (EVALITA 2026).

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{scozzaro2026kenji,
  title = {{Kenji-Endo: a BabyLM @EVALITA}},
  author = {Calogero Jerik Scozzaro, Matteo Rinaldi, Gianluca Mittone and Marco Antonio Stranisci},
  booktitle = {Proceedings of the Ninth Evaluation Campaign of Natural Language Processing and Speech Tools for Italian. Final Workshop (EVALITA 2026)},
  year = {2026},
  url = {https://aclanthology.org/2026.evalita-1.4/}
}
```

- [← previous KCIF: Knowledge-Conditioned Instruction Following](https://md.research.elamur.ai/free/en/papers/murthy-2026-kcif-knowledge-conditioned-instruction-following/)
- [next → KG-CRAFT: Knowledge Graph-based Contrastive Reasoning with LLMs for Enhancing Automated Fact-checking](https://md.research.elamur.ai/free/en/papers/lourenco-2026-kg-craft-knowledge-graph-based/)
- [↑ back to papers of 2026](https://md.research.elamur.ai/free/en/papers/2026/page/8/)

## References

33 references, 6 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. J. Kaplan, S. McCandlish, T. Henighan, T. B. Brown, B. Chess, R. Child, S. Gray, A. Radford, J. Wu, D. Amodei, [Scaling laws for neural language models](https://md.research.elamur.ai/free/en/papers/kaplan-2020-scaling-laws-neural-language-models/), arXiv preprint arXiv:2001.08361 (2020).
2. J. Devlin, M.-W. Chang, K. Lee, K. Toutanova, [Bert: Pre-training of deep bidirectional transformers for language understanding](https://md.research.elamur.ai/free/en/papers/devlin-2019-bert-pre-training-deep-bidirectional/), in: Proceedings of the 2019 conference of the North American chapter of the association for computational linguistics: human language technologies, volume 1 (long and short…
3. A. Radford, J. Wu, R. Child, D. Luan, D. Amodei, I. Sutskever, et al., Language models are unsupervised multitask learners, OpenAI blog 1 (2019) 9. [find](https://scholar.google.com/scholar?q=A%2E+Radford%2C+J%2E+Wu%2C+R%2E+Child%2C+D%2E+Luan%2C+D%2E+Amodei%2C+I%2E+Sutskever%2C+et+al%2E%2C+Language+models+are+unsupervised+multitask+learners%2C+OpenAI+blog+1+%282019%29+9%2E)
4. A. Meta, The llama 4 herd: The beginning of a new era of natively multimodal ai innovation, [https://ai.meta](https://ai.meta). com/blog/llama-4-multimodal-intelligence/, checked on 4 (2025) 2025. [↗](https://ai.meta)
5. N. Shazeer, A. Mirhoseini, K. Maziarz, A. Davis, Q. Le, G. Hinton, J. Dean, Outrageously large neural networks: The sparsely-gated mixture-of-experts layer, in: International Conference on Learning Representations, 2017. [find](https://scholar.google.com/scholar?q=N%2E+Shazeer%2C+A%2E+Mirhoseini%2C+K%2E+Maziarz%2C+A%2E+Davis%2C+Q%2E+Le%2C+G%2E+Hinton%2C+J%2E+Dean%2C+Outrageously+large+neural+networks%3A+The+sparsely%2Dgated+mixture%2Dof%2Dexperts+layer%2C+in%3A+International+Conference+on+Learning+Re)
6. D. Dai, C. Deng, C. Zhao, R. Xu, H. Gao, D. Chen, J. Li, W. Zeng, X. Yu, et al., Deepseekmoe: Towards ultimate expert specialization in mixture-of-experts language models, arXiv preprint arXiv:2401.06066 (2024). [find](https://scholar.google.com/scholar?q=D%2E+Dai%2C+C%2E+Deng%2C+C%2E+Zhao%2C+R%2E+Xu%2C+H%2E+Gao%2C+D%2E+Chen%2C+J%2E+Li%2C+W%2E+Zeng%2C+X%2E+Yu%2C+et+al%2E%2C+Deepseekmoe%3A+Towards+ultimate+expert+specialization+in+mixture%2Dof%2Dexperts+language+models%2C+arXiv+preprint+arXiv%3A2401%2E06)
7. C. Raffel, N. Shazeer, A. Roberts, K. Lee, S. Narang, M. Matena, Y. Zhou, W. Li, P. J. Liu, [Exploring the limits of transfer learning with a unified text-to-text transformer](https://md.research.elamur.ai/free/en/papers/raffel-2019-exploring-limits-transfer-learning-unified/), Journal of machine learning research 21 (2020) 1–67.
8. L. Soldaini, R. Kinney, A. Bhagia, D. Schwenk, D. Atkinson, R. Authur, B. Bogin, K. Chandu, J. Dumas, Y. Elazar, et al., Dolma: An open corpus of three trillion tokens for language model pretraining research, in: Proceedings of the 62nd annual meeting of the association for computational linguistics… [find](https://scholar.google.com/scholar?q=L%2E+Soldaini%2C+R%2E+Kinney%2C+A%2E+Bhagia%2C+D%2E+Schwenk%2C+D%2E+Atkinson%2C+R%2E+Authur%2C+B%2E+Bogin%2C+K%2E+Chandu%2C+J%2E+Dumas%2C+Y%2E+Elazar%2C+et+al%2E%2C+Dolma%3A+An+open+corpus+of+three+trillion+tokens+for+language+model+pretraining+r)
9. M. C. Rillig, M. Ågerstrand, M. Bi, K. A. Gould, U. Sauerland, Risks and benefits of large language models for the environment, Environmental science & technology 57 (2023) 3464–3466. [find](https://scholar.google.com/scholar?q=M%2E+C%2E+Rillig%2C+M%2E+%C3%85gerstrand%2C+M%2E+Bi%2C+K%2E+A%2E+Gould%2C+U%2E+Sauerland%2C+Risks+and+benefits+of+large+language+models+for+the+environment%2C+Environmental+science+%26+technology+57+%282023%29+3464%E2%80%933466%2E)
10. I. Lee, T. Berg-Kirkpatrick, Readability≠ learnability: Rethinking the role of simplicity in training small language models, in: Second Conference on Language Modeling, ???? [find](https://scholar.google.com/scholar?q=I%2E+Lee%2C+T%2E+Berg%2DKirkpatrick%2C+Readability%E2%89%A0+learnability%3A+Rethinking+the+role+of+simplicity+in+training+small+language+models%2C+in%3A+Second+Conference+on+Language+Modeling%2C+%3F%3F%3F%3F)
11. A. Warstadt, A. Mueller, L. Choshen, E. Wilcox, C. Zhuang, J. Ciro, R. Mosquera, B. Paranjabe, A. Williams, T. Linzen, et al., Findings of the babylm challenge: Sample-efficient pretraining on developmentally plausible corpora, in: Proceedings of the BabyLM challenge at the 27th conference on comput… [find](https://scholar.google.com/scholar?q=A%2E+Warstadt%2C+A%2E+Mueller%2C+L%2E+Choshen%2C+E%2E+Wilcox%2C+C%2E+Zhuang%2C+J%2E+Ciro%2C+R%2E+Mosquera%2C+B%2E+Paranjabe%2C+A%2E+Williams%2C+T%2E+Linzen%2C+et+al%2E%2C+Findings+of+the+babylm+challenge%3A+Sample%2Defficient+pretraining+on+develop)
12. M. Y. Hu, A. Mueller, C. Ross, A. Williams, T. Linzen, C. Zhuang, R. Cotterell, L. Choshen, A. Warstadt, E. G. Wilcox, Findings of the second babylm challenge: Sample-efficient pretraining on developmentally plausible corpora, in: The 2nd BabyLM Challenge at the 28th Conference on Computational Natu… [find](https://scholar.google.com/scholar?q=M%2E+Y%2E+Hu%2C+A%2E+Mueller%2C+C%2E+Ross%2C+A%2E+Williams%2C+T%2E+Linzen%2C+C%2E+Zhuang%2C+R%2E+Cotterell%2C+L%2E+Choshen%2C+A%2E+Warstadt%2C+E%2E+G%2E+Wilcox%2C+Findings+of+the+second+babylm+challenge%3A+Sample%2Defficient+pretraining+on+developm)
13. L. G. G. Charpentier, L. Choshen, R. Cotterell, M. O. Gul, M. Y. Hu, J. Liu, J. Jumelet, T. Linzen, A. Mueller, C. Ross, et al., Proceedings of the first babylm workshop, in: Proceedings of the First BabyLM Workshop, 2025. [find](https://scholar.google.com/scholar?q=L%2E+G%2E+G%2E+Charpentier%2C+L%2E+Choshen%2C+R%2E+Cotterell%2C+M%2E+O%2E+Gul%2C+M%2E+Y%2E+Hu%2C+J%2E+Liu%2C+J%2E+Jumelet%2C+T%2E+Linzen%2C+A%2E+Mueller%2C+C%2E+Ross%2C+et+al%2E%2C+Proceedings+of+the+first+babylm+workshop%2C+in%3A+Proceedings+of+the+First+)
14. L. Capone, A. Suozzi, G. E. Lebani, A. Lenci, Bambi goes to school: Evaluating italian babylms with invalsi-ita (2025). [find](https://scholar.google.com/scholar?q=L%2E+Capone%2C+A%2E+Suozzi%2C+G%2E+E%2E+Lebani%2C+A%2E+Lenci%2C+Bambi+goes+to+school%3A+Evaluating+italian+babylms+with+invalsi%2Dita+%282025%29%2E)
15. Z. Shen, A. Joshi, R.-C. Chen, Bambino-lm:(bilingual-) human-inspired continual pre-training of babylm, in: Proceedings of the Workshop on Cognitive Modeling and Computational Linguistics, 2024, pp. 1–7. [find](https://scholar.google.com/scholar?q=Z%2E+Shen%2C+A%2E+Joshi%2C+R%2E%2DC%2E+Chen%2C+Bambino%2Dlm%3A%28bilingual%2D%29+human%2Dinspired+continual+pre%2Dtraining+of+babylm%2C+in%3A+Proceedings+of+the+Workshop+on+Cognitive+Modeling+and+Computational+Linguistics%2C+2024%2C+pp%2E+1)
16. J. Jumelet, A. Fourtassi, A. Haga, B. Bunzeck, B. Shandilya, D. Galvan-Sosa, F. G. Haznitrama, F. Padovani, F. Meyer, H. Hu, et al., Babybabellm: A multilingual benchmark of developmentally plausible training data, arXiv preprint arXiv:2510.10159 (2025). [find](https://scholar.google.com/scholar?q=J%2E+Jumelet%2C+A%2E+Fourtassi%2C+A%2E+Haga%2C+B%2E+Bunzeck%2C+B%2E+Shandilya%2C+D%2E+Galvan%2DSosa%2C+F%2E+G%2E+Haznitrama%2C+F%2E+Padovani%2C+F%2E+Meyer%2C+H%2E+Hu%2C+et+al%2E%2C+Babybabellm%3A+A+multilingual+benchmark+of+developmentally+plausible+)
17. F. Cutugno, A. Miaschi, A. P. Aprosio, G. Rambelli, L. Siciliani, M. A. Stranisci, Evalita 2026: Overview of the 9th evaluation campaign of natural language processing and speech tools for italian, in: Proceedings of the Ninth Evaluation Campaign of Natural Language Processing and Speech Tools for I… [find](https://scholar.google.com/scholar?q=F%2E+Cutugno%2C+A%2E+Miaschi%2C+A%2E+P%2E+Aprosio%2C+G%2E+Rambelli%2C+L%2E+Siciliani%2C+M%2E+A%2E+Stranisci%2C+Evalita+2026%3A+Overview+of+the+9th+evaluation+campaign+of+natural+language+processing+and+speech+tools+for+italian%2C+in)
18. G. Puccetti, A. Pedrotti, A. Esuli, Desegma-it at evalita 2026: Overview of the detection and segmentation of machine generated text in italian task, 2026. [find](https://scholar.google.com/scholar?q=G%2E+Puccetti%2C+A%2E+Pedrotti%2C+A%2E+Esuli%2C+Desegma%2Dit+at+evalita+2026%3A+Overview+of+the+detection+and+segmentation+of+machine+generated+text+in+italian+task%2C+2026%2E)
19. C. Ferrando, L. Draetta, M. Madeddu, M. Sosto, V. Patti, P. Rosso, C. Bosco, J. Mata, E. Gualda, Multipride at evalita 2026: Overview of the multilingual automatic detection of slur reclamation in the lgbtq+ context task, in: Proceedings of the Ninth Evaluation Campaign of Natural Language Processin… [find](https://scholar.google.com/scholar?q=C%2E+Ferrando%2C+L%2E+Draetta%2C+M%2E+Madeddu%2C+M%2E+Sosto%2C+V%2E+Patti%2C+P%2E+Rosso%2C+C%2E+Bosco%2C+J%2E+Mata%2C+E%2E+Gualda%2C+Multipride+at+evalita+2026%3A+Overview+of+the+multilingual+automatic+detection+of+slur+reclamation+in+the)
20. A. Ramponi, S. Tonelli, [FadeIT at EVALITA 2026: Overview of the fallacy detection in italian social media texts task](https://md.research.elamur.ai/free/en/papers/ramponi-2026-fadeit-evalita-2026-overview-fallacy/), in: Proceedings of the Ninth Evaluation Campaign of Natural Language Processing and Speech Tools for Italian. Final Workshop (EVALITA 2026), CEUR.org, Bari, Italy, 2026.
21. E. Goria, C. Mauri, Il corpus KIParla: una nuova risorsa per lo studio dell’italiano parlato, volume 2, CLUB- Circolo linguistico dell’Università di Bologna, Bologna, 2018, pp. 96–116. URL: [https://hdl.handle.net/2318/1689340](https://hdl.handle.net/2318/1689340). [↗](https://hdl.handle.net/2318/1689340)
22. M. Voghera, C. Iacobini, F. Cutugno, R. Savy, I. Alfano, A. De Rosa, Il VoLIP: una risorsa per lo studio della variazione nel parlato della lingua italiana, in: Actes du XXVIIe Congrès international de linguistique et de philologie romanes, ELiPhi, Strasbourg, 2016, pp. 1651–1663. [find](https://scholar.google.com/scholar?q=M%2E+Voghera%2C+C%2E+Iacobini%2C+F%2E+Cutugno%2C+R%2E+Savy%2C+I%2E+Alfano%2C+A%2E+De+Rosa%2C+Il+VoLIP%3A+una+risorsa+per+lo+studio+della+variazione+nel+parlato+della+lingua+italiana%2C+in%3A+Actes+du+XXVIIe+Congr%C3%A8s+international+d)
23. M. A. Stranisci, S. Frenda, M. Lai, O. Araque, A. T. Cignarella, V. Basile, C. Bosco, V. Patti, O-dang! the ontology of dangerous speech messages, in: I. Kernerman, S. Carvalho, C. A. Iglesias, R. Sprugnoli (Eds.), Proceedings of the 2nd Workshop on Sentiment Analysis and Linguistic Linked Data, Eur… [find](https://scholar.google.com/scholar?q=M%2E+A%2E+Stranisci%2C+S%2E+Frenda%2C+M%2E+Lai%2C+O%2E+Araque%2C+A%2E+T%2E+Cignarella%2C+V%2E+Basile%2C+C%2E+Bosco%2C+V%2E+Patti%2C+O%2Ddang%21+the+ontology+of+dangerous+speech+messages%2C+in%3A+I%2E+Kernerman%2C+S%2E+Carvalho%2C+C%2E+A%2E+Iglesias%2C+R%2E+Spr)
24. A. Ramponi, C. Casula, DiatopIt: A corpus of social media posts for the study of diatopic language variation in Italy, in: Tenth Workshop on NLP for Similar Languages, Varieties and Dialects (VarDial 2023), Association for Computational Linguistics, Dubrovnik, Croatia, 2023, pp. 187–199. URL: https:… [find](https://scholar.google.com/scholar?q=A%2E+Ramponi%2C+C%2E+Casula%2C+DiatopIt%3A+A+corpus+of+social+media+posts+for+the+study+of+diatopic+language+variation+in+Italy%2C+in%3A+Tenth+Workshop+on+NLP+for+Similar+Languages%2C+Varieties+and+Dialects+%28VarDial+)
25. E. Federici, M. Ferraretto, N. Landro, Gazzetta Ufficiale: A dataset of legislative texts, public and private acts, 2024. URL: [https://huggingface.co/datasets/mii-llm/gazzetta-ufficiale](https://huggingface.co/datasets/mii-llm/gazzetta-ufficiale). [↗](https://huggingface.co/datasets/mii-llm/gazzetta-ufficiale)
26. M. Rinaldi, R. Varvara, V. Patti, Testimole-conversational: A 30-billion-word italian discussion board corpus (1996[-2024) for language modeling and sociolinguistic research, 2026](https://md.research.elamur.ai/free/en/papers/rinaldi-2026-testimole-conversational-30-billion-word/). URL: https://arxiv.org/abs/2602.14819. arXiv:2602.14819 . [↗](https://arxiv.org/abs/2602.14819)
27. M. Aldinucci, S. Rabellino, M. Pironti, F. Spiga, P. Viviani, M. Drocco, M. Guerzoni, G. Boella, M. Mellia, P. Margara, I. Drago, R. Marturano, G. Marchetto, E. Piccolo, S. Bagnasco, S. Lusso, S. Vallero, G. Attardi, A. Barchiesi, A. Colla, F. Galeazzi, HPC4AI: an ai-on-demand federated platform end… — [doi:10.1145/3203217.3205340](https://doi.org/10.1145/3203217.3205340) [find](https://scholar.google.com/scholar?q=M%2E+Aldinucci%2C+S%2E+Rabellino%2C+M%2E+Pironti%2C+F%2E+Spiga%2C+P%2E+Viviani%2C+M%2E+Drocco%2C+M%2E+Guerzoni%2C+G%2E+Boella%2C+M%2E+Mellia%2C+P%2E+Margara%2C+I%2E+Drago%2C+R%2E+Marturano%2C+G%2E+Marchetto%2C+E%2E+Piccolo%2C+S%2E+Bagnasco%2C+S%2E+Lusso%2C+S%2E+Vall)
28. I. Colonnelli, R. Birke, G. Malenza, G. Mittone, A. Mulone, J. Galjaard, L. Y. Chen, S. Bassini, G. Scipione, J. Martinovič, V. Vondrák, M. Aldinucci, Cross-Facility Federated Learning, Procedia Computer Science 240 (2024) 3–12. doi: 10.1016/j.procs.2024.07.003 . — [doi:10.1016/j.procs.2024.07.003](https://doi.org/10.1016/j.procs.2024.07.003) [find](https://scholar.google.com/scholar?q=I%2E+Colonnelli%2C+R%2E+Birke%2C+G%2E+Malenza%2C+G%2E+Mittone%2C+A%2E+Mulone%2C+J%2E+Galjaard%2C+L%2E+Y%2E+Chen%2C+S%2E+Bassini%2C+G%2E+Scipione%2C+J%2E+Martinovi%C4%8D%2C+V%2E+Vondr%C3%A1k%2C+M%2E+Aldinucci%2C+Cross%2DFacility+Federated+Learning%2C+Procedia+Compu)
29. NVIDIA Developer, NVIDIA Hopper Architecture In Depth, 2023. URL: [https://developer.nvidia.com/blog/nvidia-hopper-architecture-in-depth/](https://developer.nvidia.com/blog/nvidia-hopper-architecture-in-depth/), accessed: 2026-01-07. [↗](https://developer.nvidia.com/blog/nvidia-hopper-architecture-in-depth/)
30. O. Press, N. A. Smith, M. Lewis, Train short, test long: Attention with linear biases enables input length extrapolation, arXiv preprint arXiv:2108.12409 (2022). URL: https://arxiv.org/abs/2108.12409. arXiv:2108.12409 . [↗](https://arxiv.org/abs/2108.12409)
31. J. Su, M. Ahmed, Y. Lu, S. Pan, W. Bo, Y. Liu, [Roformer: Enhanced transformer with rotary position embedding](https://md.research.elamur.ai/free/en/papers/su-2021-roformer-enhanced-transformer-rotary-position/), Neurocomputing 568 (2024) 127063.
32. K. Jordan, Y. Jin, V. Boza, Y. Jiacheng, F. Cesista, L. Newhouse, J. Bernstein, Muon: An optimizer for hidden layers in neural networks, 2024. URL: [https://kellerjordan.github.io/posts/muon/](https://kellerjordan.github.io/posts/muon/). [↗](https://kellerjordan.github.io/posts/muon/)
33. A. Ramponi, A. Daffara, S. Tonelli, Fine-grained fallacy detection with human label variation, in: L. Chiruzzo, A. Ritter, L. Wang (Eds.), Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1… [find](https://scholar.google.com/scholar?q=A%2E+Ramponi%2C+A%2E+Daffara%2C+S%2E+Tonelli%2C+Fine%2Dgrained+fallacy+detection+with+human+label+variation%2C+in%3A+L%2E+Chiruzzo%2C+A%2E+Ritter%2C+L%2E+Wang+%28Eds%2E%29%2C+Proceedings+of+the+2025+Conference+of+the+Nations+of+the+Amer)

*Updated 1 November 2026*