---
title: "What Do Self-Supervised Speech Models Know About Words?"
description: "This study looks at how word-level information is represented across the layers of ten S3Ms. 100 references, cited by 17 papers here."
authors: ["Ankita Pasad", "Chung-Ming Chien", "Shane Settle", "Karen Livescu"]
year: 2024
venue: "Transactions of the Association for Computational Linguistics"
source: "https://aclanthology.org/2024.tacl-1.21/"
doi: "10.1162/tacl_a_00656"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/pasad-2024-what-do-self-supervised-speech/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/pasad-2024-what-do-self-supervised-speech/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2024](https://md.research.elamur.ai/free/en/papers/2024/page/6/) / What Do Self-Supervised Speech Models Know About Words?

# What Do Self-Supervised Speech Models Know About Words?

**Authors:** [Ankita Pasad](https://md.research.elamur.ai/free/en/authors/ankita-pasad/), Chung-Ming Chien, Shane Settle and [Karen Livescu](https://md.research.elamur.ai/free/en/authors/karen-livescu/) · **Year:** 2024 · **Venue:** [Transactions of the Association for Computational Linguistics](https://md.research.elamur.ai/free/en/venues/tacl/) ([Transactions of the Association for Computational Linguistics (TACL) 2024](https://md.research.elamur.ai/free/en/venues/tacl/2024/)) · **Publisher:** [The MIT Press](https://md.research.elamur.ai/free/en/publishers/the-mit-press/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2024.tacl-1.21/) · [PDF](https://aclanthology.org/2024.tacl-1.21.pdf) · [DOI 10.1162/tacl\_a\_00656](https://doi.org/10.1162/tacl_a_00656) · [OpenAlex](https://openalex.org/W4394773771)

**Citations:** 34 citations (OpenAlex, 2026-10-14) · [17](https://md.research.elamur.ai/free/en/papers/pasad-2024-what-do-self-supervised-speech/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary

Self-supervised speech models (S3Ms) do well on many speech tasks, but it is still unclear what they learn about units larger than phones. This study looks at how word-level information is represented across the layers of ten S3Ms. The models differ in training objective, input modality (speech-only or visually grounded) and size.

Trained probing classifiers can blur interpretation, so the authors avoid them. Instead, they use projection-weighted CCA to relate word-segment representations to four kinds of linguistic vectors:
- word identity,
- pronunciation, via acoustically grounded word embeddings,
- part-of-speech features,
- semantic attributes.

They also run three training-free evaluations: acoustic word discrimination, word segmentation and spoken sentence similarity.

The main findings are:
- Frames inside a word segment are not equally useful, and identity cues cluster near the segment's middle.
- Objective and size determine which layers hold linguistic content, usually intermediate ones.
- Visually grounded models beat the speech-only models they were initialized from on several tasks.
- Simple methods on frozen features beat earlier benchmark results.

These insights can guide decisions about layer selection, distillation and adaptation for downstream tasks.

## Key points

- The CCA targets are one-hot word IDs (500 dims), AGWEs trained on LibriSpeech (128), POS attributes derived from PTB (45) and semantic attributes from SemCor (41).
- FaST-VGS+ and VG-HuBERT outperform wav2vec2-Base and HuBERT-Base on discrimination, segmentation and semantic similarity.
- HuBERT and WavLM consistently rank above wav2vec2. Model rankings change between the LibriSpeech and Buckeye evaluation domains, while layer-wise trends stay stable.
- A parameter-free segmentation algorithm beats more complex unsupervised approaches. A single-layer RNN trained on little labeled data reaches near-perfect word discrimination.
- The paper appeared in TACL vol. 12 (2024), pp. 372–391. Code is available at github.com/ankitapasad/layerwise-analysis.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Toyota Technological Institute at Chicago](https://md.research.elamur.ai/free/en/universities/toyota-technological-institute-at-chicago/) — University, Chicago, United States · 33 works · 13,261 citations

📚 Shelves (5): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 16,039 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 15,891 works · [🔊 Signal Processing](https://md.research.elamur.ai/free/en/shelves/computer-science/signal-processing/) · 452 works

## Cite

Cite as: Ankita Pasad, Chung-Ming Chien, Shane Settle and Karen Livescu. 2024. What Do Self-Supervised Speech Models Know About Words?. Transactions of the Association for Computational Linguistics.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@article{pasad2024what,
  title = {{What Do Self-Supervised Speech Models Know About Words?}},
  author = {Ankita Pasad, Chung-Ming Chien, Shane Settle and Karen Livescu},
  journal = {Transactions of the Association for Computational Linguistics},
  year = {2024},
  doi = {10.1162/tacl\_a\_00656},
  url = {https://doi.org/10.1162/tacl_a_00656}
}
```

- [← previous VariErr NLI: Separating Annotation Error from Human Label Variation](https://md.research.elamur.ai/free/en/papers/genzel-2024-varierr-nli-separating-annotation-error/)
- [next → XCodeEval: An Execution-based Large Scale Multilingual Multitask Benchmark for Code Understanding, Generation, Translation and Retrieval](https://md.research.elamur.ai/free/en/papers/khan-2024-xcodeeval-execution-based-large-scale/)
- [↑ back to papers of 2024](https://md.research.elamur.ai/free/en/papers/2024/page/6/)

## References

100 references, 17 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Badr M. Abdullah, Mohammed Maqsood Shaik, Bernd Möbius, and Dietrich Klakow. 2023. An information-theoretic analysis of self-supervised discrete representations of speech. In Interspeech. https://doi.org/10.21437/Interspeech.2023-2131 — [doi:10.21437/interspeech.2023-2131](https://doi.org/10.21437/interspeech.2023-2131) [↗](https://doi.org/10.21437/Interspeech.2023-2131)
2. Yonatan Belinkov. 2022. [Probing classifiers: Promises, shortcomings, and advances](https://md.research.elamur.ai/free/en/papers/belinkov-2022-probing-classifiers-promises-shortcomings-advances/). Computational Linguistics. https://doi.org/10.1162/coli_a_00422 [↗](https://doi.org/10.1162/coli_a_00422)
3. Triantafyllos Afouras, Joon Son Chung, and Andrew Zisserman. 2018. LRS3-TED: A large-scale dataset for visual speech recognition. arXiv preprint arXiv:1809.00496. [find](https://scholar.google.com/scholar?q=Triantafyllos+Afouras%2C+Joon+Son+Chung%2C+and+Andrew+Zisserman%2E+2018%2E+LRS3%2DTED%3A+A+large%2Dscale+dataset+for+visual+speech+recognition%2E+arXiv+preprint+arXiv%3A1809%2E00496%2E)
4. Yonatan Belinkov and James Glass. 2019. [Analysis methods in neural language processing: A survey](https://md.research.elamur.ai/free/en/papers/belinkov-2019-analysis-methods-neural-language-processing/). Transactions of the Association for Computational Linguistics (TACL). https://doi.org/10.1162/tacl a 00254 [↗](https://doi.org/10.1162/tacl)
5. Robin Algayres, Tristan Ricoul, Julien Karadayi, Hugo Laurençon, Salah Zaiem, Abdelrahman Mohamed, Benoı̂t Sagot, and Emmanuel Dupoux. 2022. DP-Parse: Finding word boundaries from raw speech with an instance lexicon. Transactions of the Association for Computational Linguistics (TACL). https://doi… [find](https://scholar.google.com/scholar?q=Robin+Algayres%2C+Tristan+Ricoul%2C+Julien+Karadayi%2C+Hugo+Laurenc%CC%A7on%2C+Salah+Zaiem%2C+Abdelrahman+Mohamed%2C+Beno%C4%B1%CC%82t+Sagot%2C+and+Emmanuel+Dupoux%2E+2022%2E+DP%2DParse%3A+Finding+word+boundaries+from+raw+speech+with+an+)
6. Saurabhchand Bhati, Jesús Villalba, Piotr Żelasko, Laureano Moro-Velazquez, and Najim Dehak. 2021. Segmental contrastive predictive coding for unsupervised word segmentation. In Interspeech. https://doi.org/10.21437/Interspeech.2021-1874 [↗](https://doi.org/10.21437/Interspeech.2021-1874)
7. Robin Algayres, Mohamed Zaiem, Benoı̂t Sagot, and Emmanuel Dupoux. 2020. Evaluating the reliability of acoustic speech embeddings. In Interspeech. https://doi.org/10.21437/Interspeech.2020-2362 [↗](https://doi.org/10.21437/Interspeech.2020-2362)
8. Michael A. Carlin, Samuel Thomas, Aren Jansen, and Hynek Hermansky. 2011. Rapid evaluation of speech representations for spoken term discovery. In Interspeech. https://doi.org/10.21437/Interspeech.2011-304 — [doi:10.21437/interspeech.2011-304](https://doi.org/10.21437/interspeech.2011-304) [↗](https://doi.org/10.21437/Interspeech.2011-304)
9. Heng-Jui Chang, Shu-wen Yang, and Hung-yi Lee. 2022. DistilHuBERT: Speech representation learning by layer-wise distillation of hidden-unit BERT. In IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). https://doi.org/10 .1109/ICASSP43922.2022.9747490 of deep bidirectio… [find](https://scholar.google.com/scholar?q=Heng%2DJui+Chang%2C+Shu%2Dwen+Yang%2C+and+Hung%2Dyi+Lee%2E+2022%2E+DistilHuBERT%3A+Speech+representation+learning+by+layer%2Dwise+distillation+of+hidden%2Dunit+BERT%2E+In+IEEE+International+Conference+on+Acoustics%2C+Speech+)
10. Ewan Dunbar, Julien Karadayi, Mathieu Bernard, Xuan-Nga Cao, Robin Algayres, Lucas Ondel, Laurent Besacier, Sakriani Sakti, and Emmanuel Dupoux. 2020. The zero resource speech challenge 2020: Discovering discrete subword and word units. In Interspeech. https://doi.org/10.21437/Interspeech.2020-2743 [↗](https://doi.org/10.21437/Interspeech.2020-2743)
11. Guoguo Chen, Shuzhou Chai, Guanbo Wang, Jiayu Du, Wei-Qiang Zhang, Chao Weng, Dan Su, Daniel Povey, Jan Trmal, Junbo Zhang, Mingjie Jin, Sanjeev Khundanpur, Shinji Watanabe, Shuaijiang Zhao, Wei Zou, Xiangang Li, Xuchen Yao, Yongqing Wang, Yujun Wang, Zhao You, and Zhiyong Yan. 2021. GigaSpeech: An… [find](https://scholar.google.com/scholar?q=Guoguo+Chen%2C+Shuzhou+Chai%2C+Guanbo+Wang%2C+Jiayu+Du%2C+Wei%2DQiang+Zhang%2C+Chao+Weng%2C+Dan+Su%2C+Daniel+Povey%2C+Jan+Trmal%2C+Junbo+Zhang%2C+Mingjie+Jin%2C+Sanjeev+Khundanpur%2C+Shinji+Watanabe%2C+Shuaijiang+Zhao%2C+Wei+Zou%2C+)
12. Zhiyun Fan, Meng Li, Shiyu Zhou, and Bo Xu. 2021. Exploring wav2vec 2.0 on speaker verification and language identification. In Interspeech. [find](https://scholar.google.com/scholar?q=Zhiyun+Fan%2C+Meng+Li%2C+Shiyu+Zhou%2C+and+Bo+Xu%2E+2021%2E+Exploring+wav2vec+2%2E0+on+speaker+verification+and+language+identification%2E+In+Interspeech%2E)
13. Manaal Faruqui and Chris Dyer. 2014. Community evaluation and exchange of word vectors at wordvectors. org. In Association for Computational Linguistics (ACL): System Demonstrations. https://doi.org/10.3115/v1/P14-5004 [↗](https://doi.org/10.3115/v1/P14-5004)
14. Sanyuan Chen, Chengyi Wang, Zhengyang Chen, Yu Wu, Shujie Liu, Zhuo Chen, Jinyu Li, Naoyuki Kanda, Takuya Yoshioka, Xiong Xiao, Jian Wu, Long Zhou, Shuo Ren, Yanmin Qian, Yao Qian, Jian Wu, Michael Zeng, Xiangzhan Yu, and Furu Wei. 2022. WavLM: Large-scale self-supervised pre-training for full stack… [find](https://scholar.google.com/scholar?q=Sanyuan+Chen%2C+Chengyi+Wang%2C+Zhengyang+Chen%2C+Yu+Wu%2C+Shujie+Liu%2C+Zhuo+Chen%2C+Jinyu+Li%2C+Naoyuki+Kanda%2C+Takuya+Yoshioka%2C+Xiong+Xiao%2C+Jian+Wu%2C+Long+Zhou%2C+Shuo+Ren%2C+Yanmin+Qian%2C+Yao+Qian%2C+Jian+Wu%2C+Michael+Ze)
15. Manaal Faruqui, Yulia Tsvetkov, Pushpendre Rastogi, and Chris Dyer. 2016. [Problems with evaluation of word embeddings using word similarity tasks](https://md.research.elamur.ai/free/en/papers/faruqui-2016-problems-evaluation-word-embeddings-word/). In 1st Workshop on Evaluating Vector-Space Representations for NLP. https://doi.org/10.18653/v1/W16-2506 [↗](https://doi.org/10.18653/v1/W16-2506)
16. Hyeong-Seok Choi, Juheon Lee, Wansoo Kim, Jie Lee, Hoon Heo, and Kyogu Lee. 2021. Neural analysis and synthesis: Reconstructing speech from self-supervised representations. In Advances in Neural Information Processing Systems (NeurIPS). [find](https://scholar.google.com/scholar?q=Hyeong%2DSeok+Choi%2C+Juheon+Lee%2C+Wansoo+Kim%2C+Jie+Lee%2C+Hoon+Heo%2C+and+Kyogu+Lee%2E+2021%2E+Neural+analysis+and+synthesis%3A+Reconstructing+speech+from+self%2Dsupervised+representations%2E+In+Advances+in+Neural+Infor)
17. Chi-Luen Feng, Po-chun Hsu, and Hung-yi Lee. 2022. Silence is sweeter than speech: Self-supervised model using silence to store speaker information. arXiv preprint arXiv:2205.03759. — [arXiv:2205.03759](https://arxiv.org/abs/2205.03759) [find](https://scholar.google.com/scholar?q=Chi%2DLuen+Feng%2C+Po%2Dchun+Hsu%2C+and+Hung%2Dyi+Lee%2E+2022%2E+Silence+is+sweeter+than+speech%3A+Self%2Dsupervised+model+using+silence+to+store+speaker+information%2E+arXiv+preprint+arXiv%3A2205%2E03759%2E+%E2%80%94+%5BarXiv%3A2205%2E0375)
18. Tzeviya Sylvia Fuchs and Yedid Hoshen. 2023. Unsupervised word segmentation using temporal gradient pseudo-labels. In IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP). [find](https://scholar.google.com/scholar?q=Tzeviya+Sylvia+Fuchs+and+Yedid+Hoshen%2E+2023%2E+Unsupervised+word+segmentation+using+temporal+gradient+pseudo%2Dlabels%2E+In+IEEE+International+Conference+on+Acoustics%2C+Speech%2C+and+Signal+Processing+%28ICASSP%29)
19. Alexis Conneau and Douwe Kiela. 2018. [SentEval: An evaluation toolkit for universal sentence representations](https://md.research.elamur.ai/free/en/papers/conneau-2018-senteval-evaluation-toolkit-universal-sentence/). In International Conference on Language Resources and Evaluation (LREC).
20. Santiago Cuervo, Maciej Grabias, Jan Chorowski, Grzegorz Ciesielski, Adrian Łańcucki, Paweł Rychlikowski, and Ricard Marxer. 2022. Contrastive prediction strategies for unsupervised segmentation and categorization of phonemes and words. In International Conference on Acoustics, Speech and Signal Pr… [find](https://scholar.google.com/scholar?q=Santiago+Cuervo%2C+Maciej+Grabias%2C+Jan+Chorowski%2C+Grzegorz+Ciesielski%2C+Adrian+%C5%81an%CC%81cucki%2C+Pawe%C5%82+Rychlikowski%2C+and+Ricard+Marxer%2E+2022%2E+Contrastive+prediction+strategies+for+unsupervised+segmentation+and+)
21. John J. Godfrey, Edward C. Holliman, and Jane McDaniel. 1992. Switchboard: Telephone speech corpus for research and development. In IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP). https://doi.org/10.1109/ICASSP.1992.225858 [↗](https://doi.org/10.1109/ICASSP.1992.225858)
22. Anmol Gulati, James Qin, Chung-Cheng Chiu, Niki Parmar, Yu Zhang, Jiahui Yu, Wei Han, Shibo Wang, Zhengdong Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. [find](https://scholar.google.com/scholar?q=Anmol+Gulati%2C+James+Qin%2C+Chung%2DCheng+Chiu%2C+Niki+Parmar%2C+Yu+Zhang%2C+Jiahui+Yu%2C+Wei+Han%2C+Shibo+Wang%2C+Zhengdong+Jacob+Devlin%2C+Ming%2DWei+Chang%2C+Kenton+Lee%2C+and+Kristina+Toutanova%2E+2019%2E)
23. BERT: Pre-training Zhang, Yonghui Wu, and Ruoming Pang. 2020. [Conformer: Convolution-augmented transformer for speech recognition](https://md.research.elamur.ai/free/en/papers/gulati-2020-conformer-convolution-augmented-transformer-speech/). In Interspeech. https://doi.org/10.21437/Interspeech.2020-3015 [↗](https://doi.org/10.21437/Interspeech.2020-3015)
24. Aren Jansen, Samuel Thomas, and Hynek Hermansky. 2013. Weak top-down constraints for unsupervised acoustic model training. In IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP). https://doi.org/10.1109/ICASSP.2013.6639241 [↗](https://doi.org/10.1109/ICASSP.2013.6639241)
25. David Harwath and James Glass. 2017. Learning word-like units from joint audio-visual analysis. In Association for Computational Linguistics (ACL). https://doi.org/10 .18653/v1/P17-1047 [find](https://scholar.google.com/scholar?q=David+Harwath+and+James+Glass%2E+2017%2E+Learning+word%2Dlike+units+from+joint+audio%2Dvisual+analysis%2E+In+Association+for+Computational+Linguistics+%28ACL%29%2E+https%3A%2F%2Fdoi%2Eorg%2F10+%2E18653%2Fv1%2FP17%2D1047)
26. Hang Ji, Tanvina Patel, and Odette Scharenborg. 2022. Predicting within and across language phoneme recognition performance of self-supervised learning speech pre-trained models. arXiv preprint arXiv:2206.12489. — [arXiv:2206.12489](https://arxiv.org/abs/2206.12489) [find](https://scholar.google.com/scholar?q=Hang+Ji%2C+Tanvina+Patel%2C+and+Odette+Scharenborg%2E+2022%2E+Predicting+within+and+across+language+phoneme+recognition+performance+of+self%2Dsupervised+learning+speech+pre%2Dtrained+models%2E+arXiv+preprint+arXiv%3A)
27. Wanjia He, Weiran Wang, and Karen Livescu. 2017. Multi-view recurrent neural acoustic word embeddings. In International Conference on Learning Representations (ICLR). [find](https://scholar.google.com/scholar?q=Wanjia+He%2C+Weiran+Wang%2C+and+Karen+Livescu%2E+2017%2E+Multi%2Dview+recurrent+neural+acoustic+word+embeddings%2E+In+International+Conference+on+Learning+Representations+%28ICLR%29%2E)
28. Jacob Kahn, Morgane Rivière, Weiyi Zheng, Evgeny Kharitonov, Qiantong Xu, Pierre-Emmanuel Mazaré, Julien Karadayi, Vitaliy Liptchinsky, Ronan Collobert, Christian Fuegen, Tatiana Likhomanenko, Gabriel Synnaeve, Armand Joulin, Abdelrahman Mohamed, and Emmanuel Dupoux. 2020. Libri-Light: A benchmark… [find](https://scholar.google.com/scholar?q=Jacob+Kahn%2C+Morgane+Rivie%CC%80re%2C+Weiyi+Zheng%2C+Evgeny+Kharitonov%2C+Qiantong+Xu%2C+Pierre%2DEmmanuel+Mazare%CC%81%2C+Julien+Karadayi%2C+Vitaliy+Liptchinsky%2C+Ronan+Collobert%2C+Christian+Fuegen%2C+Tatiana+Likhomanenko%2C+Gabri)
29. John Hewitt and Percy Liang. 2019. Designing and interpreting probes with control tasks. In Empirical Methods in Natural Language Processing (EMNLP). https://doi.org/10 .18653/v1/D19-1275 [find](https://scholar.google.com/scholar?q=John+Hewitt+and+Percy+Liang%2E+2019%2E+Designing+and+interpreting+probes+with+control+tasks%2E+In+Empirical+Methods+in+Natural+Language+Processing+%28EMNLP%29%2E+https%3A%2F%2Fdoi%2Eorg%2F10+%2E18653%2Fv1%2FD19%2D1275)
30. Harold Hotelling. 1936. Relations between two sets of variates. Biometrika. https://doi.org/10.1093/biomet/28.3-4.321 — [doi:10.1093/biomet/28.3-4.321](https://doi.org/10.1093/biomet/28.3-4.321) [↗](https://doi.org/10.1093/biomet/28.3-4.321)
31. Wei-Ning Hsu, David Harwath, Christopher Song, and James Glass. 2021a. Text-free image-to-speech synthesis using learned segmental units. In Association for Computational Linguistics (ACL). [find](https://scholar.google.com/scholar?q=Wei%2DNing+Hsu%2C+David+Harwath%2C+Christopher+Song%2C+and+James+Glass%2E+2021a%2E+Text%2Dfree+image%2Dto%2Dspeech+synthesis+using+learned+segmental+units%2E+In+Association+for+Computational+Linguistics+%28ACL%29%2E)
32. Herman Kamper. 2022. Word segmentation on discovered phone units with dynamic programming and self-supervised scoring. IEEE/ACM Transactions on Audio, Speech, and Language Processing (TASLP). https://doi.org/10.1109/TASLP.2022.3229264 — [doi:10.1109/taslp.2022.3229264](https://doi.org/10.1109/taslp.2022.3229264) [↗](https://doi.org/10.1109/TASLP.2022.3229264)
33. Wei-Ning Hsu, Anuroop Sriram, Alexei Baevski, Tatiana Likhomanenko, Qiantong Xu, Vineel Pratap, Jacob Kahn, Ann Lee, Ronan Collobert, Gabriel Synnaeve, and Michael Auli. 2021b. Robust wav2vec 2.0: Analyzing domain shift in self-supervised pre-training. In Interspeech. [find](https://scholar.google.com/scholar?q=Wei%2DNing+Hsu%2C+Anuroop+Sriram%2C+Alexei+Baevski%2C+Tatiana+Likhomanenko%2C+Qiantong+Xu%2C+Vineel+Pratap%2C+Jacob+Kahn%2C+Ann+Lee%2C+Ronan+Collobert%2C+Gabriel+Synnaeve%2C+and+Michael+Auli%2E+2021b%2E+Robust+wav2vec+2%2E0%3A+Ana)
34. Herman Kamper, Weiran Wang, and Karen Livescu. 2016. Deep convolutional acoustic word embeddings using word-pair side information. In IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP). https://doi.org/10.1109/ICASSP.2016.7472619 [↗](https://doi.org/10.1109/ICASSP.2016.7472619)
35. Wei-Ning Hsu, Yao-Hung Hubert Tsai, Benjamin Bolte, Ruslan Salakhutdinov, and Abdelrahman Mohamed. 2021c. HuBERT: How much can a bad teacher benefit asr pre-training? In IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP). [find](https://scholar.google.com/scholar?q=Wei%2DNing+Hsu%2C+Yao%2DHung+Hubert+Tsai%2C+Benjamin+Bolte%2C+Ruslan+Salakhutdinov%2C+and+Abdelrahman+Mohamed%2E+2021c%2E+HuBERT%3A+How+much+can+a+bad+teacher+benefit+asr+pre%2Dtraining%3F+In+IEEE+International+Conference+)
36. Eesung Kim, Jae-Jin Jeon, Hyeji Seo, and Hoon Kim. 2022. Automatic pronunciation assessment using self-supervised speech representation learning. In Interspeech. https://doi.org/10.21437/Interspeech.2022-10245 — [doi:10.21437/interspeech.2022-10245](https://doi.org/10.21437/interspeech.2022-10245) [↗](https://doi.org/10.21437/Interspeech.2022-10245)
37. Yushi Hu, Shane Settle, and Karen Livescu. 2020. Multilingual jointly trained acoustic and written word embeddings. In Interspeech. https://doi.org/10.21437/Interspeech.2020-2828 [↗](https://doi.org/10.21437/Interspeech.2020-2828)
38. Simon Kornblith, Mohammad Norouzi, Honglak Lee, and Geoffrey Hinton. 2019. [Similarity of neural network representations revisited](https://md.research.elamur.ai/free/en/papers/kornblith-2019-similarity-neural-network-representations-revisited/). In International Conference on Machine Learning (ICML).
39. Dongseong Hwang, Khe Chai Sim, Zhouyuan Huo, and Trevor Strohman. 2022. Pseudo label is better than human label. In Interspeech. https://doi.org/10.21437/Interspeech.2022-11034 [↗](https://doi.org/10.21437/Interspeech.2022-11034)
40. Felix Kreuk, Joseph Keshet, and Yossi Adi. 2020. Self-supervised contrastive learning for unsupervised phoneme segmentation. In Interspeech. https://doi.org/10.21437/Interspeech.2020-2398 large annotated corpus of English: The Penn treebank. Computational Linguistics. https://doi.org/10.21236/ADA2… — [doi:10.21236/ada273556](https://doi.org/10.21236/ada273556) [↗](https://doi.org/10.21437/Interspeech.2020-2398)
41. Kushal Lakhotia, Eugene Kharitonov, Wei-Ning Hsu, Yossi Adi, Adam Polyak, Benjamin Bolte, Tu-Anh Nguyen, Jade Copet, Alexei Baevski, Abdelrahman Mohamed, and Emmanuel Dupoux. 2021. [On generative spoken language modeling from raw audio](https://md.research.elamur.ai/free/en/papers/lakhotia-2021-generative-spoken-language-modeling-raw/). Transactions of the Association for Computational Linguistics (T…
42. Michael McAuliffe, Michaela Socolof, Sarah Mihuc, Michael Wagner, and Morgan Sonderegger. 2017. Montreal forced aligner: Trainable text-speech alignment using kaldi. In Interspeech. https://doi.org/10 .21437/Interspeech.2017-1386 [find](https://scholar.google.com/scholar?q=Michael+McAuliffe%2C+Michaela+Socolof%2C+Sarah+Mihuc%2C+Michael+Wagner%2C+and+Morgan+Sonderegger%2E+2017%2E+Montreal+forced+aligner%3A+Trainable+text%2Dspeech+alignment+using+kaldi%2E+In+Interspeech%2E+https%3A%2F%2Fdoi%2Eorg%2F10)
43. Danny Merkx, Stefan L. Frank, and Mirjam Ernestus. 2021. Semantic sentence similarity: Size does not always matter. In Interspeech. https://doi.org/10.21437/Interspeech.2021-1464 [↗](https://doi.org/10.21437/Interspeech.2021-1464)
44. Keith Levin, Katharine Henry, Aren Jansen, and Karen Livescu. 2013. Fixed-dimensional acoustic embeddings of variable-length segments in low-resource settings. In IEEE Automatic Speech Recognition and Understanding Workshop (ASRU). https://doi.org/10.1109/ASRU.2013.6707765 — [doi:10.1109/asru.2013.6707765](https://doi.org/10.1109/asru.2013.6707765) [↗](https://doi.org/10.1109/ASRU.2013.6707765)
45. Danny Merkx, Sebastiaan Scholten, Stefan L. Frank, Mirjam Ernestus, and OdetteScharenborg. 2023. Modelling human word learning and recognition using visually grounded speech. Cognitive Computation. https://doi.org/10.1007/s12559-022-10059-7 — [doi:10.1007/s12559-022-10059-7](https://doi.org/10.1007/s12559-022-10059-7) [↗](https://doi.org/10.1007/s12559-022-10059-7)
46. Yuanchao Li, Yumnah Mohamied, Peter Bell, and Catherine Lai. 2023a. Exploration of a self-supervised speech model: A study on emotional corpora. In IEEE Spoken Language Technology Workshop (SLT). [find](https://scholar.google.com/scholar?q=Yuanchao+Li%2C+Yumnah+Mohamied%2C+Peter+Bell%2C+and+Catherine+Lai%2E+2023a%2E+Exploration+of+a+self%2Dsupervised+speech+model%3A+A+study+on+emotional+corpora%2E+In+IEEE+Spoken+Language+Technology+Workshop+%28SLT%29%2E)
47. George A. Miller, Claudia Leacock, Randee Tengi, and Ross T. Bunker. 1993. [A semantic concordance](https://md.research.elamur.ai/free/en/papers/miller-1993-semantic-concordance/). In Human Language Technology. https://doi.org/10.3115/1075671 .1075742 [↗](https://doi.org/10.3115/1075671)
48. Zhengyang Li, Thomas Graave, Jing Liu, Timo Lohrenz, Siegfried Kunzmann, and Tim Fingscheidt. 2023b. Parameter-efficient cross-language transfer learning for a language-modular audiovisual speech recognition. In IEEE Automatic Speech Recognition and Understanding Workshop (ASRU). [find](https://scholar.google.com/scholar?q=Zhengyang+Li%2C+Thomas+Graave%2C+Jing+Liu%2C+Timo+Lohrenz%2C+Siegfried+Kunzmann%2C+and+Tim+Fingscheidt%2E+2023b%2E+Parameter%2Defficient+cross%2Dlanguage+transfer+learning+for+a+language%2Dmodular+audiovisual+speech+reco)
49. Oli Liu, Hao Tang, and Sharon Goldwater. 2023. Self-supervised predictive coding models encode speaker and phonetic information in orthogonal subspaces. In Interspeech. https://doi.org/10.21437/Interspeech.2023 -871 — [doi:10.21437/interspeech.2023](https://doi.org/10.21437/interspeech.2023) [↗](https://doi.org/10.21437/Interspeech.2023)
50. Loren Lugosch, Mirco Ravanelli, Patrick Ignoto, Vikrant Singh Tomar, and Yoshua Bengio. 2019. Speech model pre-training for end-to-end spoken language understanding. In Interspeech. https://doi.org/10.21437/Interspeech.2019-2396 [↗](https://doi.org/10.21437/Interspeech.2019-2396)
51. Abdelrahman Mohamed, Hung-yi Lee, Lasse Borgholt, Jakob D. Havtorn, Joakim Edin, Christian Igel, Katrin Kirchhoff, Shang-Wen Li, Karen Livescu, Lars Maaløe, Tara N. Sainath, and Shinji Watanabe. 2022. Self-supervised speech representation learning: A review. IEEE Journal of Selected Topics in Signal… [find](https://scholar.google.com/scholar?q=Abdelrahman+Mohamed%2C+Hung%2Dyi+Lee%2C+Lasse+Borgholt%2C+Jakob+D%2E+Havtorn%2C+Joakim+Edin%2C+Christian+Igel%2C+Katrin+Kirchhoff%2C+Shang%2DWen+Li%2C+Karen+Livescu%2C+Lars+Maal%C3%B8e%2C+Tara+N%2E+Sainath%2C+and+Shinji+Watanabe%2E+2022%2E)
52. Ari Morcos, Maithra Raghu, and Samy Bengio. 2018. Insights on representational similarity in neural networks with canonical correlation. In Advances in Neural Information Processing Systems (NeurIPS). [find](https://scholar.google.com/scholar?q=Ari+Morcos%2C+Maithra+Raghu%2C+and+Samy+Bengio%2E+2018%2E+Insights+on+representational+similarity+in+neural+networks+with+canonical+correlation%2E+In+Advances+in+Neural+Information+Processing+Systems+%28NeurIPS%29%2E)
53. Tu Anh Nguyen, Maureen De Seyssel, Robin Algayres, Patricia Rozé, Ewan Dunbar, and Emmanuel Dupoux. 2022. Are word boundaries useful for unsupervised language learning? arXiv preprint arXiv:2210.02956. — [arXiv:2210.02956](https://arxiv.org/abs/2210.02956) [find](https://scholar.google.com/scholar?q=Tu+Anh+Nguyen%2C+Maureen+De+Seyssel%2C+Robin+Algayres%2C+Patricia+Roze%CC%81%2C+Ewan+Dunbar%2C+and+Emmanuel+Dupoux%2E+2022%2E+Are+word+boundaries+useful+for+unsupervised+language+learning%3F+arXiv+preprint+arXiv%3A2210%2E0295)
54. Danni Ma, Neville Ryant, and Mark Liberman. 2021. Probing acoustic representations for phonetic properties. In IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP). https://doi.org/10 .1109/ICASSP39728.2021.9414776 [find](https://scholar.google.com/scholar?q=Danni+Ma%2C+Neville+Ryant%2C+and+Mark+Liberman%2E+2021%2E+Probing+acoustic+representations+for+phonetic+properties%2E+In+IEEE+International+Conference+on+Acoustics%2C+Speech%2C+and+Signal+Processing+%28ICASSP%29%2E+https)
55. Tu Anh Nguyen, Maureen de Seyssel, Patricia Rozé, Morgane Rivière, Evgeny Kharitonov, Alexei Baevski, Ewan Dunbar, and Emmanuel Dupoux. 2020. [find](https://scholar.google.com/scholar?q=Tu+Anh+Nguyen%2C+Maureen+de+Seyssel%2C+Patricia+Roze%CC%81%2C+Morgane+Rivie%CC%80re%2C+Evgeny+Kharitonov%2C+Alexei+Baevski%2C+Ewan+Dunbar%2C+and+Emmanuel+Dupoux%2E+2020%2E)
56. The Zero Resource Speech Benchmark 2021: Metrics and baselines for Mitchell Marcus, Beatrice Santorini, and Mary Ann Marcinkiewicz. 1993. Building a unsupervised spoken language modeling. In NeurIPS Workshop on Self-Supervised Learning for Speech and Audio Processing. language modeling. In AAAI Work… [find](https://scholar.google.com/scholar?q=The+Zero+Resource+Speech+Benchmark+2021%3A+Metrics+and+baselines+for+Mitchell+Marcus%2C+Beatrice+Santorini%2C+and+Mary+Ann+Marcinkiewicz%2E+1993%2E+Building+a+unsupervised+spoken+language+modeling%2E+In+NeurIPS+W)
57. Tu Anh Nguyen, Eugene Kharitonov, Jade Copet, Yossi Adi, Wei-Ning Hsu, Ali Elkahky, Paden Tomasello, Robin Algayres, Benoit Sagot, Abdelrahman Mohamed, and Emmanuel Dupoux. 2023. [Generative spoken dialogue language modeling](https://md.research.elamur.ai/free/en/papers/nguyen-2023-generative-spoken-dialogue-language-modeling/). Transactions of the Association for Computational Linguistics (TACL). https…
58. Puyuan Peng and David Harwath. 2022b. Word discovery in visually grounded, self-supervised speech models. In Interspeech. [find](https://scholar.google.com/scholar?q=Puyuan+Peng+and+David+Harwath%2E+2022b%2E+Word+discovery+in+visually+grounded%2C+self%2Dsupervised+speech+models%2E+In+Interspeech%2E)
59. Puyuan Peng, Herman Kamper, and Karen Livescu. 2020. A correspondence variational autoencoder for unsupervised acoustic word embeddings. In NeurIPS Workshop on Self-Supervised Learning for Speech and Audio Processing. [find](https://scholar.google.com/scholar?q=Puyuan+Peng%2C+Herman+Kamper%2C+and+Karen+Livescu%2E+2020%2E+A+correspondence+variational+autoencoder+for+unsupervised+acoustic+word+embeddings%2E+In+NeurIPS+Workshop+on+Self%2DSupervised+Learning+for+Speech+and+)
60. Shruti Palaskar, Vikas Raunak, and Florian Metze. 2019. Learned in speech recognition: Contextual acoustic word embeddings. In IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP). https://doi.org/10.1109/ICASSP.2019 .8683868 — [doi:10.1109/icassp.2019](https://doi.org/10.1109/icassp.2019) [↗](https://doi.org/10.1109/ICASSP.2019)
61. Jeffrey Pennington, Richard Socher, and Christopher D. Manning. 2014. [GloVe: Global vectors for word representation](https://md.research.elamur.ai/free/en/papers/pennington-2014-glove-global-vectors-word-representation/). In Empirical Methods in Natural Language Processing (EN-MLP). https://doi.org/10.3115/v1/D14-1162 [↗](https://doi.org/10.3115/v1/D14-1162)
62. Vassil Panayotov, Guoguo Chen, Daniel Povey, and Sanjeev Khudanpur. 2015. LibriSpeech: An ASR corpus based on public domain audio books. In IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP). https://doi.org/10.1109/ICASSP.2015.7178964 — [doi:10.1109/icassp.2015.7178964](https://doi.org/10.1109/icassp.2015.7178964) [↗](https://doi.org/10.1109/ICASSP.2015.7178964)
63. Mark A. Pitt, Keith Johnson, Elizabeth Hume, Scott Kiesling, and William Raymond. 2005. The Buckeye corpus of conversational speech: Labeling conventions and a test of transcriber reliability. Speech Communication. https://doi.org/10.1016/j .specom.2004.09.001 — [doi:10.1016/j](https://doi.org/10.1016/j) [find](https://scholar.google.com/scholar?q=Mark+A%2E+Pitt%2C+Keith+Johnson%2C+Elizabeth+Hume%2C+Scott+Kiesling%2C+and+William+Raymond%2E+2005%2E+The+Buckeye+corpus+of+conversational+speech%3A+Labeling+conventions+and+a+test+of+transcriber+reliability%2E+Speech+)
64. Ankita Pasad, Ju-Chieh Chou, and Karen Livescu. 2021. Layer-wise analysis of a self-supervised speech representation model. In IEEE Automatic Speech Recognition and Understanding Workshop (ASRU). https://doi.org/10 .1109/ASRU51503.2021.9688093 [find](https://scholar.google.com/scholar?q=Ankita+Pasad%2C+Ju%2DChieh+Chou%2C+and+Karen+Livescu%2E+2021%2E+Layer%2Dwise+analysis+of+a+self%2Dsupervised+speech+representation+model%2E+In+IEEE+Automatic+Speech+Recognition+and+Understanding+Workshop+%28ASRU%29%2E+http)
65. Archiki Prasad and Preethi Jyothi. 2020. How accents confound: Probing for accent information in end-to-end speech recognition systems. In Association for Computational Linguistics (ACL). https://doi.org/10.18653/v1/2020.acl-main.345 [↗](https://doi.org/10.18653/v1/2020.acl-main.345)
66. Ankita Pasad, Bowen Shi, and Karen Livescu. 2023. Comparative layer-wise analysis of self-supervised speech models. In IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP). https://doi.org/10.1109/ICASSP49357.2023 .10096149 — [doi:10.1109/icassp49357.2023](https://doi.org/10.1109/icassp49357.2023) [↗](https://doi.org/10.1109/ICASSP49357.2023)
67. Maithra Raghu, Justin Gilmer, Jason Yosinski, and Jascha Sohl-Dickstein. 2017. SVCCA: Singular vector canonical correlation analysis for deep learning dynamics and interpretability. In Neural Information Processing Systems (NIPS). [find](https://scholar.google.com/scholar?q=Maithra+Raghu%2C+Justin+Gilmer%2C+Jason+Yosinski%2C+and+Jascha+Sohl%2DDickstein%2E+2017%2E+SVCCA%3A+Singular+vector+canonical+correlation+analysis+for+deep+learning+dynamics+and+interpretability%2E+In+Neural+Informat)
68. Okko Räsänen, Unto K. Laine, and Toomas Altosaar. 2011. Blind segmentation of speech using non-linear filtering methods. Speech Technologies. https://doi.org/10.5772/16433 — [doi:10.5772/16433](https://doi.org/10.5772/16433) [↗](https://doi.org/10.5772/16433)
69. Ankita Pasad, Felix Wu, Suwon Shon, Karen Livescu, and Kyu J. Han. 2022. On the use of external data for spoken named entity recognition. In North American Chapter of the Association for Computational Linguistics (NAACL). https://doi.org/10.18653/v1/2022.naacl-main.53 [↗](https://doi.org/10.18653/v1/2022.naacl-main.53)
70. Abhilasha Ravichander, Yonatan Belinkov, and Eduard Hovy. 2021. [Probing the probing paradigm: Does probing accuracy entail task relevance?](https://md.research.elamur.ai/free/en/papers/ravichander-2021-probing-probing-paradigm-does-probing/) In European Chapter of the Association for Computational Linguistics (EACL). https://doi.org/10 .18653/v1/2021.eacl-main.295
71. Puyuan Peng and David Harwath. 2022a. Self-supervised representation learning for speech using visual grounding and masked Ramon Sanabria, Hao Tang, and Sharon Goldwater. 2021. On the difficulty of segmenting words with attention. In Second Workshop on Insights from Negative Results in NLP. https://… [find](https://scholar.google.com/scholar?q=Puyuan+Peng+and+David+Harwath%2E+2022a%2E+Self%2Dsupervised+representation+learning+for+speech+using+visual+grounding+and+masked+Ramon+Sanabria%2C+Hao+Tang%2C+and+Sharon+Goldwater%2E+2021%2E+On+the+difficulty+of+se)
72. Bowen Shi, Shane Settle, and Karen Livescu. 2021. Whole-word segmental speech recognition with acoustic word embeddings. In IEEE Spoken Language Technology Workshop (SLT). https://doi.org/10.1109/SLT48900.2021.9383578 [↗](https://doi.org/10.1109/SLT48900.2021.9383578)
73. Ramon Sanabria, Hao Tang, and Sharon Goldwater. 2023. Analyzing acoustic word embeddings from pre-trained self-supervised speech models. In IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP). https://doi.org/10 .1109/ICASSP49357.2023.10096099 [find](https://scholar.google.com/scholar?q=Ramon+Sanabria%2C+Hao+Tang%2C+and+Sharon+Goldwater%2E+2023%2E+Analyzing+acoustic+word+embeddings+from+pre%2Dtrained+self%2Dsupervised+speech+models%2E+In+IEEE+International+Conference+on+Acoustics%2C+Speech%2C+and+Sign)
74. Suwon Shon, Siddhant Arora, Chyi-Jiunn Lin, Ankita Pasad, Felix Wu, Roshan Sharma, Wei-Lun Wu, Hung-Yi Lee, Karen Livescu, and Shinji Watanabe. 2023. SLUE Phase-2: A benchmark suite of diverse spoken language understanding tasks. In Association for Computational Linguistics (ACL). https://doi.org/10… [find](https://scholar.google.com/scholar?q=Suwon+Shon%2C+Siddhant+Arora%2C+Chyi%2DJiunn+Lin%2C+Ankita+Pasad%2C+Felix+Wu%2C+Roshan+Sharma%2C+Wei%2DLun+Wu%2C+Hung%2DYi+Lee%2C+Karen+Livescu%2C+and+Shinji+Watanabe%2E+2023%2E+SLUE+Phase%2D2%3A+A+benchmark+suite+of+diverse+spoken+)
75. Naomi Saphra and Adam Lopez. 2019. [Understanding learning dynamics of language models with SVCCA](https://md.research.elamur.ai/free/en/papers/saphra-2019-understanding-learning-dynamics-language-models/). In North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT).
76. Shane Settle, Kartik Audhkhasi, Karen Livescu, and Michael Picheny. 2019. Acoustically grounded word embeddings for improved acoustics-to-word speech recognition. In IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP). https://doi.org/10.1109/ICASSP.2019 .8682903 — [doi:10.1109/icassp.2019](https://doi.org/10.1109/icassp.2019) [↗](https://doi.org/10.1109/ICASSP.2019)
77. Shane Settle and Karen Livescu. 2016. Discriminative acoustic word embeddings: Recurrent neural network-based approaches. In IEEE Spoken Language Technology Workshop (SLT). https://doi.org/10.1109/SLT.2016.7846310 [↗](https://doi.org/10.1109/SLT.2016.7846310)
78. Suwon Shon, Ankita Pasad, Felix Wu, Pablo Brusco, Yoav Artzi, Karen Livescu, and Kyu J. Han. 2022. SLUE: New benchmark tasks for spoken language understanding evaluation on natural speech. In IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP). https://doi.org/10 .1109… [find](https://scholar.google.com/scholar?q=Suwon+Shon%2C+Ankita+Pasad%2C+Felix+Wu%2C+Pablo+Brusco%2C+Yoav+Artzi%2C+Karen+Livescu%2C+and+Kyu+J%2E+Han%2E+2022%2E+SLUE%3A+New+benchmark+tasks+for+spoken+language+understanding+evaluation+on+natural+speech%2E+In+IEEE+Int)
79. Louis ten Bosch and Bert Cranen. 2007. A computational model for unsupervised word discovery. In Interspeech. https://doi.org/10 .21437/Interspeech.2007-429 [find](https://scholar.google.com/scholar?q=Louis+ten+Bosch+and+Bert+Cranen%2E+2007%2E+A+computational+model+for+unsupervised+word+discovery%2E+In+Interspeech%2E+https%3A%2F%2Fdoi%2Eorg%2F10+%2E21437%2FInterspeech%2E2007%2D429)
80. Ian Tenney, Dipanjan Das, and Ellie Pavlick. 2019. [BERT rediscovers the classical NLP pipeline](https://md.research.elamur.ai/free/en/papers/tenney-2019-bert-rediscovers-classical-nlp-pipeline/). In Association for Computational Linguistics (ACL). https://doi.org/10 .18653/v1/P19-1452
81. Jui Shah, Yaman Kumar Singla, Changyou Chen, and Rajiv Ratn Shah. 2021. What all do audio transformer models hear? Probing acoustic representations for language delivery and its structure. In IEEE International Conference on Data Mining Workshops (ICDMW). [find](https://scholar.google.com/scholar?q=Jui+Shah%2C+Yaman+Kumar+Singla%2C+Changyou+Chen%2C+and+Rajiv+Ratn+Shah%2E+2021%2E+What+all+do+audio+transformer+models+hear%3F+Probing+acoustic+representations+for+language+delivery+and+its+structure%2E+In+IEEE+Int)
82. Gaofei Shen, Afra Alishahi, Arianna Bisazza, and Grzegorz Chrupała. 2023. Wave to syntax: Probing spoken language models for syntax. In Interspeech. https://doi.org/10 .21437/Interspeech.2023-679 [find](https://scholar.google.com/scholar?q=Gaofei+Shen%2C+Afra+Alishahi%2C+Arianna+Bisazza%2C+and+Grzegorz+Chrupa%C5%82a%2E+2023%2E+Wave+to+syntax%3A+Probing+spoken+language+models+for+syntax%2E+In+Interspeech%2E+https%3A%2F%2Fdoi%2Eorg%2F10+%2E21437%2FInterspeech%2E2023%2D679)
83. Hsiang-Sheng Tsai, Heng-Jui Chang, Wen-Chin Huang, Zili Huang, Kushal Lakhotia, Shu-wen Yang, Shuyan Dong, Andy T. Liu, Cheng-I Jeff Lai, Jiatong Shi, Xuankai Chang, Phil Hall, Hsuan-Jui Chen, Shang-Wen Li, Shinji Watanabe, Abdelrahman Mohamed, and Hung-yi Lee. 2022. SUPERB-SG: Enhanced speech proce… [find](https://scholar.google.com/scholar?q=Hsiang%2DSheng+Tsai%2C+Heng%2DJui+Chang%2C+Wen%2DChin+Huang%2C+Zili+Huang%2C+Kushal+Lakhotia%2C+Shu%2Dwen+Yang%2C+Shuyan+Dong%2C+Andy+T%2E+Liu%2C+Cheng%2DI+Jeff+Lai%2C+Jiatong+Shi%2C+Xuankai+Chang%2C+Phil+Hall%2C+Hsuan%2DJui+Chen%2C+Shang%2DW)
84. Bowen Shi, Wei-Ning Hsu, Kushal Lakhotia, and Abdelrahman Mohamed. 2022. Learning audio-visual speech representation by masked multimodal cluster prediction. In International Conference on Learning Representations (ICLR). [find](https://scholar.google.com/scholar?q=Bowen+Shi%2C+Wei%2DNing+Hsu%2C+Kushal+Lakhotia%2C+and+Abdelrahman+Mohamed%2E+2022%2E+Learning+audio%2Dvisual+speech+representation+by+masked+multimodal+cluster+prediction%2E+In+International+Conference+on+Learning+Re)
85. Yulia Tsvetkov, Manaal Faruqui, and Chris Dyer. 2016. Correlation-based intrinsic evaluation of word vector representations. In 1st Workshop on Evaluating Vector-Space Representations for NLP. https://doi.org/10.18653/v1/W16-2520 [↗](https://doi.org/10.18653/v1/W16-2520)
86. Yulia Tsvetkov, Manaal Faruqui, Wang Ling, Guillaume Lample, and Chris Dyer. 2015. Evaluation of word vector representations by subspace alignment. In Empirical Methods in Natural Language Processing (EMNLP). https://doi.org/10.18653/v1/D15-1243 [↗](https://doi.org/10.18653/v1/D15-1243)
87. Benjamin van Niekerk, Leanne Nortje, Matthew Baas, and Herman Kamper. 2021. Analyzing speaker information in self-supervised models to improve zero-resource speech processing. In Interspeech. https://doi.org/10.21437/Interspeech.2021-1182 — [doi:10.21437/interspeech.2021-1182](https://doi.org/10.21437/interspeech.2021-1182) [↗](https://doi.org/10.21437/Interspeech.2021-1182)
88. Changhan Wang, Morgane Riviere, Ann Lee, Anne Wu, Chaitanya Talnikar, Daniel Haziza, Mary Williamson, Juan Pino, and Emmanuel Dupoux. 2021. [VoxPopuli: A large-scale multilingual speech corpus for representation learning, semi-supervised learning and interpretation](https://md.research.elamur.ai/free/en/papers/wang-2021-voxpopuli-large-scale-multilingual-speech/). In Association for Computational L…
89. Felix Wu, Kwangyoun Kim, Shinji Watanabe, Kyu J. Han, Ryan McDonald, Kilian Q. Weinberger, and Yoav Artzi. 2023. Wav2seq: Pre-training speech-to-text encoder-decoder models using pseudo languages. In IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP). https://doi.org/… [find](https://scholar.google.com/scholar?q=Felix+Wu%2C+Kwangyoun+Kim%2C+Shinji+Watanabe%2C+Kyu+J%2E+Han%2C+Ryan+McDonald%2C+Kilian+Q%2E+Weinberger%2C+and+Yoav+Artzi%2E+2023%2E+Wav2seq%3A+Pre%2Dtraining+speech%2Dto%2Dtext+encoder%2Ddecoder+models+using+pseudo+languages%2E+In+)
90. Lisa Van Staden and Herman Kamper. 2021. A comparison of self-supervised speech representations as input features for unsupervised acoustic word embeddings. In IEEE Spoken Language Technology Workshop (SLT). https://doi.org/10.1109/SLT48900.2021.9383625 [↗](https://doi.org/10.1109/SLT48900.2021.9383625)
91. Shuo Xie, Jiahao Qiu, Ankita Pasad, Li Du, Qing Qu, and Hongyuan Mei. 2022. Hidden state variability of pretrained language models can guide computation reduction for transfer learning. In Findings of Empirical Methods in Natural Language Processing (EMNLP). https://doi.org/10.18653 /v1/2022.finding… [find](https://scholar.google.com/scholar?q=Shuo+Xie%2C+Jiahao+Qiu%2C+Ankita+Pasad%2C+Li+Du%2C+Qing+Qu%2C+and+Hongyuan+Mei%2E+2022%2E+Hidden+state+variability+of+pretrained+language+models+can+guide+computation+reduction+for+transfer+learning%2E+In+Findings+of)
92. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. [Attention is all you need](https://md.research.elamur.ai/free/en/papers/vaswani-2017-attention-all-need/). Advances in Neural Information Processing Systems (NIPS).
93. Pauli Virtanen, Ralf Gommers, Travis E. Oliphant, Matt Haberland, Tyler Reddy, David Cournapeau, Evgeni Burovski, Pearu Peterson, Warren Weckesser, Jonathan Bright, Stéfan J. van der Walt, Matthew Brett, Joshua Wilson, K. Jarrod Millman, Nikolay Mayorov, Andrew R. J. Nelson, Eric Jones, Robert Kern… — [SciPy 1.0--Fundamental Algorithms for Scientific Computing in Python](https://md.research.elamur.ai/free/en/papers/virtanen-2019-scipy-1-0-fundamental-algorithms/)
94. Gene-Ping Yang, Yue Gu, Qingming Tang, Dongsu Du, and Yuzong Liu. 2023a. On-device constrained self-supervised speech representation learning for keyword spotting via knowledge distillation. In Interspeech. [find](https://scholar.google.com/scholar?q=Gene%2DPing+Yang%2C+Yue+Gu%2C+Qingming+Tang%2C+Dongsu+Du%2C+and+Yuzong+Liu%2E+2023a%2E+On%2Ddevice+constrained+self%2Dsupervised+speech+representation+learning+for+keyword+spotting+via+knowledge+distillation%2E+In+Inters)
95. Mu Yang, Ram CMC Shekar, Okim Kang, and John HL Hansen. 202b. What can an accent identifier learn? Probing phonetic and prosodic information in a wav2vec2-based accent identification model. In Interspeech. [find](https://scholar.google.com/scholar?q=Mu+Yang%2C+Ram+CMC+Shekar%2C+Okim+Kang%2C+and+John+HL+Hansen%2E+202b%2E+What+can+an+accent+identifier+learn%3F+Probing+phonetic+and+prosodic+information+in+a+wav2vec2%2Dbased+accent+identification+model%2E+In+Intersp)
96. Shu-wen Yang, Po-Han Chi, Yung-Sung Chuang, Cheng-I Jeff Lai, Kushal Lakhotia, Yist Y. Lin, Andy T. Liu, Jiatong Shi, Xuankai Chang, Guan-Ting Lin, Tzu-Hsien Huang, Wei-Cheng Tseng, Ko-tik Lee, Da-Rong Liu, Zili Huang, Shuyan Dong, Shang-Wen Li, Shinji Watanabe, Abdelrahman Mohamed, and Hung-yi Lee.… [find](https://scholar.google.com/scholar?q=Shu%2Dwen+Yang%2C+Po%2DHan+Chi%2C+Yung%2DSung+Chuang%2C+Cheng%2DI+Jeff+Lai%2C+Kushal+Lakhotia%2C+Yist+Y%2E+Lin%2C+Andy+T%2E+Liu%2C+Jiatong+Shi%2C+Xuankai+Chang%2C+Guan%2DTing+Lin%2C+Tzu%2DHsien+Huang%2C+Wei%2DCheng+Tseng%2C+Ko%2Dtik+Lee%2C+Da%2DRon)
97. Elena Voita, Rico Sennrich, and Ivan Titov. 2019. [The bottom-up evolution of representations in the transformer: A study with machine translation and language modeling objectives](https://md.research.elamur.ai/free/en/papers/voita-2019-bottom-up-evolution-representations-transformer/). In North American Chapter of the Association for Computational Linguistics (NAACL). https://doi.org/10.18653/v1/D19-144… [↗](https://doi.org/10.18653/v1/D19-144…)
98. Salah Zaiem, Robin Algayres, Titouan Parcollet, Slim Essid, and Mirco Ravanelli. 2023a. Fine-tuning strategies for faster inference using speech self-supervised models: A comparative study. In IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP). https://doi.org/10.1109… — [arXiv:2308.14456](https://arxiv.org/abs/2308.14456) [find](https://scholar.google.com/scholar?q=Salah+Zaiem%2C+Robin+Algayres%2C+Titouan+Parcollet%2C+Slim+Essid%2C+and+Mirco+Ravanelli%2E+2023a%2E+Fine%2Dtuning+strategies+for+faster+inference+using+speech+self%2Dsupervised+models%3A+A+comparative+study%2E+In+IEEE+In)
99. Jian Zhu, Zuoyu Tian, Yadong Liu, Cong Zhang, and Chia-wen Lo. 2022. Bootstrapping meaning through listening: Unsupervised learning of spoken sentence embeddings. In Findings of Empirical Methods in Natural Language Processing (EMNLP). https://doi.org/10 .18653/v1/2022.findings-emnlp.81 [find](https://scholar.google.com/scholar?q=Jian+Zhu%2C+Zuoyu+Tian%2C+Yadong+Liu%2C+Cong+Zhang%2C+and+Chia%2Dwen+Lo%2E+2022%2E+Bootstrapping+meaning+through+listening%3A+Unsupervised+learning+of+spoken+sentence+embeddings%2E+In+Findings+of+Empirical+Methods+in+N)
100. Salah Zaiem, Youcef Kemiche, Titouan Parcollet, Slim Essid, and Mirco Ravanelli. 2023b. Speech self-supervised representations benchmarking: A case for larger probing heads. arXiv preprint [find](https://scholar.google.com/scholar?q=Salah+Zaiem%2C+Youcef+Kemiche%2C+Titouan+Parcollet%2C+Slim+Essid%2C+and+Mirco+Ravanelli%2E+2023b%2E+Speech+self%2Dsupervised+representations+benchmarking%3A+A+case+for+larger+probing+heads%2E+arXiv+preprint)

*Updated 1 November 2026*