--- title: "Michael Matena — author profile and papers" description: "Michael Matena, University of North Carolina at Chapel Hill: 2 papers on Elamur Research (2019–2021), h-index 3 — affiliations, topics, co-authors and…" canonical: "https://research.elamur.ai/free/en/authors/michael-matena/" --- [Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/authors/michael-matena/) [home](https://md.research.elamur.ai/free/en/) / [authors](https://md.research.elamur.ai/free/en/authors/) / Michael Matena # Michael Matena Michael Matena works on transfer learning for natural language processing and on ways of combining neural networks that have already been trained. Their papers here run from large controlled comparisons of how language models are pre-trained and adapted to a probabilistic treatment of parameter averaging. Both lines ask how knowledge held in one trained model can be reused for, or joined with, another. The most cited of these papers is [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://md.research.elamur.ai/free/en/papers/raffel-2019-exploring-limits-transfer-learning-unified/) (2019). It recasts translation, classification, question answering and summarization alike as mapping one text to another, so that one model, loss and set of hyperparameters can serve every task. The authors used this setting not to propose a new method but to compare existing choices carefully: pre-training objectives, Transformer variants, unlabeled corpora, fine-tuning schemes and multi-task training. They also built the C4 corpus, a cleaned web crawl, for the experiments. The work was a joint effort with [Colin Raffel](https://md.research.elamur.ai/free/en/authors/colin-raffel/), [Noam Shazeer](https://md.research.elamur.ai/free/en/authors/noam-shazeer/), [Adam Roberts](https://md.research.elamur.ai/free/en/authors/adam-roberts/), [Katherine Lee](https://md.research.elamur.ai/free/en/authors/katherine-lee/) and others. The later paper, [Merging Models with Fisher-Weighted Averaging](https://md.research.elamur.ai/free/en/papers/matena-2021-merging-models-fisher-weighted-averaging/) (2021), written with Raffel, looks again at averaging the weights of models that share an architecture and initialization. It treats merging as a search for parameters that approximately maximize the joint likelihood under each model's posterior. In this view, simple averaging corresponds to assuming an isotropic Gaussian. A Laplace approximation with diagonal Fisher information replaces that assumption, so each parameter is averaged with weights given by its Fisher values. Matena's current institution is the [University of North Carolina at Chapel Hill](https://md.research.elamur.ai/free/en/universities/university-of-north-carolina-at-chapel-hill/), where their bylines appear from 2021. Michael Matena has 2 papers in our library, in 2019–2021; often with [Colin Raffel](https://md.research.elamur.ai/free/en/authors/colin-raffel/), [Adam Roberts](https://md.research.elamur.ai/free/en/authors/adam-roberts/), [Katherine Lee](https://md.research.elamur.ai/free/en/authors/katherine-lee/). 2 papers in our library · 2019–2021 [OpenAlex](https://openalex.org/A5063203090) · [**h-index 3**](https://openalex.org/authors/A5063203090) · [7 works](https://openalex.org/works?filter=author.id:A5063203090) · [3,709 citations](https://openalex.org/works?filter=author.id:A5063203090&sort=cited_by_count:desc) OpenAlex, 2026-10-14 [📄 2 works in our library](https://md.research.elamur.ai/free/en/authors/michael-matena/#papers) · [🔗 7,577 citations in our library](https://md.research.elamur.ai/free/en/authors/michael-matena/cited-by/) · [⭐ Paper Star rank #293](https://md.research.elamur.ai/free/en/paper-stars/page/3/#star-293) The profile is the library's match of the name; two people with one name are not always told apart. Elsewhere: [Google Scholar](https://scholar.google.com/citations?view_op=search_authors&mauthors=Michael+Matena) · [Semantic Scholar](https://www.semanticscholar.org/search?q=Michael+Matena) · [dblp](https://dblp.org/search?q=Michael+Matena) · [ACL Anthology](https://aclanthology.org/search/?q=Michael+Matena) ## Affiliations [🎓**University of North Carolina at Chapel Hill** Chapel Hill · United States 2021–2026 · 3 papers*current*](https://md.research.elamur.ai/free/en/universities/university-of-north-carolina-at-chapel-hill/) From the bylines of their papers (OpenAlex, 2026-10-14), ORCID and Wikidata; institutions on papers are matched by machine. 1 more institution on single bylines are not listed. ## Works by year ### 2020s [**2021** 1 paper](https://md.research.elamur.ai/free/en/papers/matena-2021-merging-models-fisher-weighted-averaging/) ### 2010s [**2019** 1 paper](https://md.research.elamur.ai/free/en/papers/raffel-2019-exploring-limits-transfer-learning-unified/) ## Papers The most cited first (OpenAlex and our library). 1. **[Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://md.research.elamur.ai/free/en/papers/raffel-2019-exploring-limits-transfer-learning-unified/)** *(2019 · 7,433 citations)* The authors from Google propose a unified view of transfer learning in NLP. They cast every language task as the conversion of input text into output text: translation, classification, question answering, summarization. This makes it possible to use a single model, a single loss function and the same hyperparameters… 2. **[Merging Models with Fisher-Weighted Averaging](https://md.research.elamur.ai/free/en/papers/matena-2021-merging-models-fisher-weighted-averaging/)** *(2021 · 190 citations)* Merging, or combining models by averaging their parameters, works only for networks with a shared architecture and initialization. ## Co-authors here 8 co-authors with a page, the most papers together first. - [Colin Raffel](https://md.research.elamur.ai/free/en/authors/colin-raffel/) ×2 - [Adam Roberts](https://md.research.elamur.ai/free/en/authors/adam-roberts/) - [Katherine Lee](https://md.research.elamur.ai/free/en/authors/katherine-lee/) - [Noam Shazeer](https://md.research.elamur.ai/free/en/authors/noam-shazeer/) - [Peter J. Liu](https://md.research.elamur.ai/free/en/authors/peter-j-liu/) - [Sharan Narang](https://md.research.elamur.ai/free/en/authors/sharan-narang/) - [Wei Li](https://md.research.elamur.ai/free/en/authors/wei-li/) - [Yanqi Zhou](https://md.research.elamur.ai/free/en/authors/yanqi-zhou/) ## Cited by Papers of our library cite their works 7,577 times (of the 140,884 papers whose bibliographies we read); [594 citing papers with a page here →](https://md.research.elamur.ai/free/en/authors/michael-matena/cited-by/).