--- title: "Wei-Ning Hsu — author profile and papers" description: "Wei-Ning Hsu: 2 papers on Elamur Research (2021), h-index 33 — affiliations, topics, co-authors and who cites them in our library." canonical: "https://research.elamur.ai/free/en/authors/wei-ning-hsu/" --- [Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/authors/wei-ning-hsu/) [home](https://md.research.elamur.ai/free/en/) / [authors](https://md.research.elamur.ai/free/en/authors/) / Wei-Ning Hsu # Wei-Ning Hsu Wei-Ning Hsu works on learning from speech without transcriptions. Their papers here, both from 2021, ask how a model can discover the units and structure of spoken language from raw audio alone, and how such systems ought to be evaluated. The approach is self-supervised. Transformer encoders are trained on unlabelled recordings, with discrete targets obtained by clustering acoustic features. The work sits where [Speech & spoken language](https://md.research.elamur.ai/free/en/topics/speech/) meets [Language models & pretraining](https://md.research.elamur.ai/free/en/topics/language-models/). The most cited of these is [HuBERT](https://md.research.elamur.ai/free/en/papers/hsu-2021-hubert-self-supervised-speech-representation/). It starts from the observation that speech lacks a ready set of discrete units and has unknown unit boundaries. To get around this, it groups audio frames offline, at first with k-means on MFCC features, and uses the resulting labels as pseudo-targets. An encoder in the style of wav2vec 2.0 must predict those labels for masked spans of audio. The clustering is then redone on the model's own learned features. The second paper, [On Generative Spoken Language Modeling from Raw Audio](https://md.research.elamur.ai/free/en/papers/lakhotia-2021-generative-spoken-language-modeling-raw/), defines the task of learning both the sounds and the linguistic structure of a language with no text at all. Its central contribution is a suite of metrics for encoding and generation. Generated speech is passed through a pretrained recognizer and scored with text-style measures, and those scores are checked against human ratings and cheaper zero-shot probes. Taken together, the two papers move from building speech representations toward generating spoken language. The more recent strand is concerned with comparable evaluation of such generative systems. Hsu's bylines list Meta in Menlo Park (2020–2025) and, earlier, the Massachusetts Institute of Technology, including its Computer Science and Artificial Intelligence Laboratory, from 2016. Johns Hopkins University appears from 2021, and National Taiwan University in 2015. Frequent co-authors here include [Kushal Lakhotia](https://md.research.elamur.ai/free/en/authors/kushal-lakhotia/), [Abdelrahman Mohamed](https://md.research.elamur.ai/free/en/authors/abdelrahman-mohamed/) and [Benjamin Bolte](https://md.research.elamur.ai/free/en/authors/benjamin-bolte/). The generative work also involved [Emmanuel Dupoux](https://md.research.elamur.ai/free/en/authors/emmanuel-dupoux/) and Yossi Adi. 2 papers on this site · 2021 Wei-Ning Hsu has 2 papers on this site, in 2021, most of them on [Language models & pretraining](https://md.research.elamur.ai/free/en/topics/language-models/), [Speech & spoken language](https://md.research.elamur.ai/free/en/topics/speech/); often with [Abdelrahman Mohamed](https://md.research.elamur.ai/free/en/authors/abdelrahman-mohamed/), [Benjamin Bolte](https://md.research.elamur.ai/free/en/authors/benjamin-bolte/), [Kushal Lakhotia](https://md.research.elamur.ai/free/en/authors/kushal-lakhotia/). [OpenAlex](https://openalex.org/A5051950818) · [ORCID](https://orcid.org/0000-0001-5546-5217) · [**h-index 33**](https://openalex.org/authors/A5051950818) · [115 works](https://openalex.org/works?filter=author.id:A5051950818) · [7 221 citations](https://openalex.org/works?filter=author.id:A5051950818&sort=cited_by_count:desc) OpenAlex, 2026-10-14 [2 works on this site](https://md.research.elamur.ai/free/en/authors/wei-ning-hsu/#papers) · [1 035 citations in our library](https://md.research.elamur.ai/free/en/authors/wei-ning-hsu/cited-by/) · 4 more works in our library The profile is the library's match of the name; two people with one name are not always told apart. Elsewhere: [Google Scholar](https://scholar.google.com/citations?view_op=search_authors&mauthors=Wei-Ning+Hsu) · [Semantic Scholar](https://www.semanticscholar.org/search?q=Wei-Ning+Hsu) · [dblp](https://dblp.org/search?q=Wei-Ning+Hsu) · [ACL Anthology](https://aclanthology.org/search/?q=Wei-Ning+Hsu) ## Affiliations - [Johns Hopkins University](https://md.research.elamur.ai/free/en/universities/johns-hopkins-university/) · Baltimore, United States —2021–2026 on 3 papers - [Meta (United States)](https://md.research.elamur.ai/free/en/companies/meta/) · Menlo Park, United States —2020–2025 on 29 papers - [MIT Computer Science and Artificial Intelligence Laboratory](https://md.research.elamur.ai/free/en/labs/mit-computer-science-and-artificial-intelligence-laboratory/) · Cambridge, United States —2016–2023 on 13 papers - [Meta (Israel)](https://md.research.elamur.ai/free/en/companies/meta-israel/) · Tel Aviv, Israel —2021–2023 on 4 papers - [Meta (United Kingdom)](https://md.research.elamur.ai/free/en/companies/meta-united-kingdom/) · London, United Kingdom —2021–2023 on 2 papers - [École des hautes études en sciences sociales](https://md.research.elamur.ai/free/en/labs/ecole-des-hautes-etudes-en-sciences-sociales/) · Paris, France —2021–2022 on 4 papers - [Bar-Ilan University](https://md.research.elamur.ai/free/en/universities/bar-ilan-university/) · Ramat Gan, Israel —2021–2022 on 2 papers - [Massachusetts Institute of Technology](https://md.research.elamur.ai/free/en/universities/massachusetts-institute-of-technology/) · Cambridge, United States —2016–2021 on 25 papers - [National Taiwan University](https://md.research.elamur.ai/free/en/universities/national-taiwan-university/) · Taipei, Taiwan —2015 on 2 papers From the bylines of their papers (OpenAlex, 2026-10-14), ORCID and Wikidata; institutions on papers are matched by machine. 5 more institutions on single bylines are not listed. ## Topics [Language models & pretraining 1](https://md.research.elamur.ai/free/en/topics/language-models/) · [Speech & spoken language 1](https://md.research.elamur.ai/free/en/topics/speech/) ## Works by year ### 2020s [**2021** 2 papers](https://md.research.elamur.ai/free/en/authors/wei-ning-hsu/year/2021/) ## Papers The most cited first (OpenAlex and our library). 1. **[HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Units](https://md.research.elamur.ai/free/en/papers/hsu-2021-hubert-self-supervised-speech-representation/)** *(2021 · 3 441 citations)* Unsupervised speech representation learning is harder than in text or vision for three reasons. Each utterance contains many sounds, there is no ready-made inventory of discrete units, and unit boundaries are unknown. HuBERT (Hidden-Unit BERT) handles this by first clustering acoustic frames offline, starting with… 2. **[On Generative Spoken Language Modeling from Raw Audio](https://md.research.elamur.ai/free/en/papers/lakhotia-2021-generative-spoken-language-modeling-raw/)** *(2021 · 165 citations)* The paper introduces Generative Spoken Language Modeling (GSLM): learning both the sound system and the linguistic structure of a language directly from raw audio, without transcripts or labels. ## Also in our library 4 more works of theirs are in our library without a page here yet; they will appear as we publish them. ## Co-authors here 10 co-authors with a page, the most papers together first. - [Abdelrahman Mohamed](https://md.research.elamur.ai/free/en/authors/abdelrahman-mohamed/) ×2 - [Benjamin Bolte](https://md.research.elamur.ai/free/en/authors/benjamin-bolte/) ×2 - [Kushal Lakhotia](https://md.research.elamur.ai/free/en/authors/kushal-lakhotia/) ×2 - [Alexei Baevski](https://md.research.elamur.ai/free/en/authors/alexei-baevski/) - [Emmanuel Dupoux](https://md.research.elamur.ai/free/en/authors/emmanuel-dupoux/) - [Eugene Kharitonov](https://md.research.elamur.ai/free/en/authors/eugene-kharitonov/) - [Jade Copet](https://md.research.elamur.ai/free/en/authors/jade-copet/) - [Ruslan Salakhutdinov](https://md.research.elamur.ai/free/en/authors/ruslan-salakhutdinov/) - [Yao-Hung Hubert Tsai](https://md.research.elamur.ai/free/en/authors/yao-hung-hubert-tsai/) - [Yossi Adi](https://md.research.elamur.ai/free/en/authors/yossi-adi/) ## Cited by Papers of our library cite their works 1 035 times (of the 140 884 papers whose bibliographies we read); [57 citing papers with a page here →](https://md.research.elamur.ai/free/en/authors/wei-ning-hsu/cited-by/).