---
title: "Difference in Task Performance on Sparse Speech"
description: "They examine how performance on different tasks changes as representations become sparser. 41 references."
authors: ["Wenjie Peng", "Chen Chen", "Thomas Hain"]
year: 2026
venue: "Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)"
source: "https://aclanthology.org/2026.acl-long.1303/"
doi: "10.18653/v1/2026.acl-long.1303"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/peng-2026-difference-task-performance-sparse-speech/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/peng-2026-difference-task-performance-sparse-speech/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2026](https://md.research.elamur.ai/free/en/papers/2026/page/27/) / Difference in Task Performance on Sparse Speech Representations

# Difference in Task Performance on Sparse Speech Representations

**Authors:** Wenjie Peng, [Chen Chen](https://md.research.elamur.ai/free/en/authors/chen-chen/) and [Thomas Hain](https://md.research.elamur.ai/free/en/authors/thomas-hain/) · **Year:** 2026 · **Venue:** [Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)](https://md.research.elamur.ai/free/en/venues/acl/) ([ACL 2026](https://md.research.elamur.ai/free/en/venues/acl/2026/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2026.acl-long.1303/) · [PDF](https://aclanthology.org/2026.acl-long.1303.pdf) · [DOI 10.18653/v1/2026.acl-long.1303](https://doi.org/10.18653/v1/2026.acl-long.1303) · [OpenAlex](https://openalex.org/W7166870203)

**Citations:** 0 citations (OpenAlex, 2026-10-14)

## Summary
Self-supervised speech models such as WavLM, HuBERT and wav2vec 2.0 produce general-purpose representations, but it is not well understood why downstream tasks benefit from them to different degrees. Drawing on the no-free-lunch theorem and on how speech is produced by a few slowly moving articulators, the authors use sparsity as an inductive bias. They examine how performance on different tasks changes as representations become sparser. They train k-sparse autoencoders (the TopK variant of Gao et al., 2025) on features from three SSL models, using twenty configurations of latent dimensionality and k. Each configuration is evaluated on six SUPERB tasks: SE, SID, SER, PR, ASR and SF. To interpret the results, they apply the Information Bottleneck framework, with a Bernoulli-based mutual information estimate computed from latent activation patterns. Tasks respond differently to sparsity, and their best sparsity levels differ. PR and ASR benefit most from retaining label-relevant information, while SID and SER gain from stronger compression. This matters for designing a single sparse representation learner that serves many speech tasks.

## Key points
- Autoencoders were trained on base-size WavLM, HuBERT and wav2vec 2.0 features and tested on six SUPERB tasks, which cover five of the benchmark's six domains.
- The choice of SSL feature matters little for most tasks, with PR as the clear exception. HuBERT behaves much like WavLM, while wav2vec 2.0 deviates, notably on SER.
- Mutual information was measured against labels on the dev sets: 1251 speaker classes, 4 emotion classes, 39 phone classes and 2969 word classes. Increasing k always raises this information. PR and ASR perform best at the largest k, while SID and SER peak at smaller k, which means sparser representations.
- To detect inactive latents, the authors use a batch-ratio threshold τ. An ablation with a 1536-dimensional SAE at k=32, trained on LibriSpeech train-clean-100, found τ=0.9999 best, with a PER of 9.98 on dev-clean.
- Limitations:
  - Only base-size SSL models were used.
  - Data are mostly English.
  - The mutual information measure ignores activation magnitudes.
  - Input quality can partly shift the optimal k.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [University of Sheffield](https://md.research.elamur.ai/free/en/universities/university-of-sheffield/) — University, Sheffield, United Kingdom · 113 works · 21,542 citations

📚 Shelves (5): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 16,039 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 15,891 works · [🔊 Signal Processing](https://md.research.elamur.ai/free/en/shelves/computer-science/signal-processing/) · 452 works

## Cite

Cite as: Wenjie Peng, Chen Chen and Thomas Hain. 2026. Difference in Task Performance on Sparse Speech Representations. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers).

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{peng2026difference,
  title = {{Difference in Task Performance on Sparse Speech Representations}},
  author = {Wenjie Peng, Chen Chen and Thomas Hain},
  booktitle = {Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2026},
  doi = {10.18653/v1/2026.acl-long.1303},
  url = {https://doi.org/10.18653/v1/2026.acl-long.1303}
}
```

- [← previous Diff4TST: Masked Diffusion Language Model for Text Style Transfer](https://md.research.elamur.ai/free/en/papers/ma-2026-diff4tst-masked-diffusion-language-model/)
- [next → Differentially Private De-identification of Dutch Clinical Notes: A Comparative Evaluation](https://md.research.elamur.ai/free/en/papers/miranda-2026-differentially-private-de-identification-dutch/)
- [↑ back to papers of 2026](https://md.research.elamur.ai/free/en/papers/2026/page/27/)

## References

41 references, 6 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Alexei Baevski, Wei-Ning Hsu, Qiantong Xu, Arun Babu, Jiatao Gu, and Michael Auli. 2022. Data2vec: A general framework for self-supervised learning in speech, vision and language. In Proceedings of the 39th International Conference on Machine Learning, pages 1298–1312. PMLR. [find](https://scholar.google.com/scholar?q=Alexei+Baevski%2C+Wei%2DNing+Hsu%2C+Qiantong+Xu%2C+Arun+Babu%2C+Jiatao+Gu%2C+and+Michael+Auli%2E+2022%2E+Data2vec%3A+A+general+framework+for+self%2Dsupervised+learning+in+speech%2C+vision+and+language%2E+In+Proceedings+of+th)
2. Alexei Baevski, Yuhao Zhou, Abdelrahman Mohamed, and Michael Auli. 2020. [wav2vec 2.0: A framework for self-supervised learning of speech representations](https://md.research.elamur.ai/free/en/papers/baevski-2020-wav2vec-2-0-framework-self/). In Proceedings of the 34th Advances in Neural Information Processing Systems, 33:12449–12460.
3. Yoshua Bengio, Aaron Courville, and Pascal Vincent. 2013. [Representation learning: A review and new perspectives](https://md.research.elamur.ai/free/en/papers/bengio-2012-representation-learning-review-new-perspectives/). IEEE Transactions on Pattern Analysis and Machine Intelligence, 35(8):1798–1828.
4. Trenton Bricken, Adly Templeton, Joshua Batson, Brian Chen, Adam Jermyn, Tom Conerly, Nick Turner, Cem Anil, Carson Denison, Amanda Askell, Robert Lasenby, Yifan Wu, Shauna Kravec, Nicholas Schiefer, Tim Maxwell, Nicholas Joseph, Zac Hatfield-Dodds, Alex Tamkin, Karina Nguyen, and 6 others. 2023. To… [find](https://scholar.google.com/scholar?q=Trenton+Bricken%2C+Adly+Templeton%2C+Joshua+Batson%2C+Brian+Chen%2C+Adam+Jermyn%2C+Tom+Conerly%2C+Nick+Turner%2C+Cem+Anil%2C+Carson+Denison%2C+Amanda+Askell%2C+Robert+Lasenby%2C+Yifan+Wu%2C+Shauna+Kravec%2C+Nicholas+Schiefer%2C+)
5. Transformer Circuits Thread. Https://transformer-circuits.pub/2023/monosemantic-features/index.html. [find](https://scholar.google.com/scholar?q=Transformer+Circuits+Thread%2E+Https%3A%2F%2Ftransformer%2Dcircuits%2Epub%2F2023%2Fmonosemantic%2Dfeatures%2Findex%2Ehtml%2E)
6. Christoph Feichtenhofer. 2022. Masked autoencoders that listen. Advances in Neural Information Processing Systems, 35:28708–28720. [find](https://scholar.google.com/scholar?q=Christoph+Feichtenhofer%2E+2022%2E+Masked+autoencoders+that+listen%2E+Advances+in+Neural+Information+Processing+Systems%2C+35%3A28708%E2%80%9328720%2E)
7. Sanyuan Chen, Chengyi Wang, Zhengyang Chen, Yu Wu, Shujie Liu, Zhuo Chen, Jinyu Li, Naoyuki Kanda, Takuya Yoshioka, Xiong Xiao, and 1 others. 2022. Wavlm: Large-scale self-supervised pre-training for full stack speech processing. IEEE Journal of Selected Topics in Signal Processing, 16(6):1505–1518. [find](https://scholar.google.com/scholar?q=Sanyuan+Chen%2C+Chengyi+Wang%2C+Zhengyang+Chen%2C+Yu+Wu%2C+Shujie+Liu%2C+Zhuo+Chen%2C+Jinyu+Li%2C+Naoyuki+Kanda%2C+Takuya+Yoshioka%2C+Xiong+Xiao%2C+and+1+others%2E+2022%2E+Wavlm%3A+Large%2Dscale+self%2Dsupervised+pre%2Dtraining+for+)
8. Shuai Huang and Trac D. Tran. 2019. Sparse signal recovery via generalized entropy functions minimization. IEEE Transactions on Signal Processing, 67(5):1322–1337. [find](https://scholar.google.com/scholar?q=Shuai+Huang+and+Trac+D%2E+Tran%2E+2019%2E+Sparse+signal+recovery+via+generalized+entropy+functions+minimization%2E+IEEE+Transactions+on+Signal+Processing%2C+67%285%29%3A1322%E2%80%931337%2E)
9. Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey Hinton. 2020. [A simple framework for contrastive learning of visual representations](https://md.research.elamur.ai/free/en/papers/chen-2020-simple-framework-contrastive-learning-visual/). In Proceedings of the 37th International Conference on Machine Learning, pages 1597–1607. PMLR.
10. Thomas M. Cover and Joy A. Thomas. 2006. Elements of Information Theory (Wiley Series in Telecommunications and Signal Processing). Wiley-Interscience, USA. [find](https://scholar.google.com/scholar?q=Thomas+M%2E+Cover+and+Joy+A%2E+Thomas%2E+2006%2E+Elements+of+Information+Theory+%28Wiley+Series+in+Telecommunications+and+Signal+Processing%29%2E+Wiley%2DInterscience%2C+USA%2E)
11. Jun Deng, Zixing Zhang, Erik Marchi, and Björn Schuller. 2013. Sparse autoencoder-based feature transfer learning for speech emotion recognition. In Proceedings of the 2013 Humaine Association Conference on Affective Computing and Intelligent Interaction, pages 511–516. IEEE. [find](https://scholar.google.com/scholar?q=Jun+Deng%2C+Zixing+Zhang%2C+Erik+Marchi%2C+and+Bj%C3%B6rn+Schuller%2E+2013%2E+Sparse+autoencoder%2Dbased+feature+transfer+learning+for+speech+emotion+recognition%2E+In+Proceedings+of+the+2013+Humaine+Association+Confere)
12. Li Deng. 1999. Computational models for speech production. In Computational models of speech pattern processing, pages 199–213. Springer. [find](https://scholar.google.com/scholar?q=Li+Deng%2E+1999%2E+Computational+models+for+speech+production%2E+In+Computational+models+of+speech+pattern+processing%2C+pages+199%E2%80%93213%2E+Springer%2E)
13. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. [Bert: Pre-training of deep bidirectional transformers for language understanding](https://md.research.elamur.ai/free/en/papers/devlin-2019-bert-pre-training-deep-bidirectional/). In Proceedings of the 2019 conference of the North American chapter of the association for computational linguistics: human language technologies,…
14. D.L. Donoho. 2006. Compressed sensing. IEEE Transactions on Information Theory, 52(4):1289–1306. [find](https://scholar.google.com/scholar?q=D%2EL%2E+Donoho%2E+2006%2E+Compressed+sensing%2E+IEEE+Transactions+on+Information+Theory%2C+52%284%29%3A1289%E2%80%931306%2E)
15. Leo Gao, Tom Dupre la Tour, Henk Tillman, Gabriel Goh, Rajan Troll, Alec Radford, Ilya Sutskever, Jan Leike, and Jeffrey Wu. 2025. Scaling and evaluating sparse autoencoders. In Proceedings of the 13th International Conference on Learning Representations. [find](https://scholar.google.com/scholar?q=Leo+Gao%2C+Tom+Dupre+la+Tour%2C+Henk+Tillman%2C+Gabriel+Goh%2C+Rajan+Troll%2C+Alec+Radford%2C+Ilya+Sutskever%2C+Jan+Leike%2C+and+Jeffrey+Wu%2E+2025%2E+Scaling+and+evaluating+sparse+autoencoders%2E+In+Proceedings+of+the+13t)
16. Selen Hande Kabil and Herve Bourlard. 2022. From undercomplete to sparse overcomplete autoencoders to improve lf-mmi based speech recognition. In Interspeech 2022, pages 1061–1065. [find](https://scholar.google.com/scholar?q=Selen+Hande+Kabil+and+Herve+Bourlard%2E+2022%2E+From+undercomplete+to+sparse+overcomplete+autoencoders+to+improve+lf%2Dmmi+based+speech+recognition%2E+In+Interspeech+2022%2C+pages+1061%E2%80%931065%2E)
17. Wei-Ning Hsu, Benjamin Bolte, Yao-Hung Hubert Tsai, Kushal Lakhotia, Ruslan Salakhutdinov, and Abdelrahman Mohamed. 2021. [Hubert: Self-supervised speech representation learning by masked prediction of hidden units](https://md.research.elamur.ai/free/en/papers/hsu-2021-hubert-self-supervised-speech-representation/). IEEE/ACM Transactions on Audio, Speech, and Language Processing, 29:3451–3460.
18. Po-Yao Huang, Hu Xu, Juncheng Li, Alexei Baevski, Michael Auli, Wojciech Galuba, Florian Metze, and Robert Huben, Hoagy Cunningham, Logan Riggs Smith, Aidan Ewart, and Lee Sharkey. 2024. Sparse autoencoders find highly interpretable features in language models. In Proceedings of the 12th Internation… [find](https://scholar.google.com/scholar?q=Po%2DYao+Huang%2C+Hu+Xu%2C+Juncheng+Li%2C+Alexei+Baevski%2C+Michael+Auli%2C+Wojciech+Galuba%2C+Florian+Metze%2C+and+Robert+Huben%2C+Hoagy+Cunningham%2C+Logan+Riggs+Smith%2C+Aidan+Ewart%2C+and+Lee+Sharkey%2E+2024%2E+Sparse+autoen)
19. Diederik P. Kingma and Jimmy Ba. 2015. Adam: A method for stochastic optimisation. International Conference on Learning Representations. [find](https://scholar.google.com/scholar?q=Diederik+P%2E+Kingma+and+Jimmy+Ba%2E+2015%2E+Adam%3A+A+method+for+stochastic+optimisation%2E+International+Conference+on+Learning+Representations%2E)
20. Tom Lieberum, Senthooran Rajamanoharan, Arthur Conmy, Lewis Smith, Nicolas Sonnerat, Vikrant Varma, Janos Kramar, Anca Dragan, Rohin Shah, and Neel Nanda. 2024. Gemma scope: Open sparse autoencoders everywhere all at once on gemma 2. In Proceedings of the 7th BlackboxNLP Workshop: Analyzing and Inte… [find](https://scholar.google.com/scholar?q=Tom+Lieberum%2C+Senthooran+Rajamanoharan%2C+Arthur+Conmy%2C+Lewis+Smith%2C+Nicolas+Sonnerat%2C+Vikrant+Varma%2C+Janos+Kramar%2C+Anca+Dragan%2C+Rohin+Shah%2C+and+Neel+Nanda%2E+2024%2E+Gemma+scope%3A+Open+sparse+autoencoders+e)
21. Alexander H Liu, Heng-Jui Chang, Michael Auli, Wei-Ning Hsu, and Jim Glass. 2023. Dinosr: Self-distillation and online clustering for self-supervised speech representation learning. Advances in Neural Information Processing Systems, 36:58346–58362. [find](https://scholar.google.com/scholar?q=Alexander+H+Liu%2C+Heng%2DJui+Chang%2C+Michael+Auli%2C+Wei%2DNing+Hsu%2C+and+Jim+Glass%2E+2023%2E+Dinosr%3A+Self%2Ddistillation+and+online+clustering+for+self%2Dsupervised+speech+representation+learning%2E+Advances+in+Neural)
22. Yizhou Lu, Mingkun Huang, Xinghua Qu, Pengfei Wei, and Zejun Ma. 2022. Language adaptive cross-lingual speech representation learning with sparse sharing sub-networks. In Proceedings of the 47th International Conference on Acoustics, Speech and Signal Processing, pages 6882–6886. IEEE. [find](https://scholar.google.com/scholar?q=Yizhou+Lu%2C+Mingkun+Huang%2C+Xinghua+Qu%2C+Pengfei+Wei%2C+and+Zejun+Ma%2E+2022%2E+Language+adaptive+cross%2Dlingual+speech+representation+learning+with+sparse+sharing+sub%2Dnetworks%2E+In+Proceedings+of+the+47th+Inter)
23. Alireza Makhzani and Brendan Frey. 2014. K-sparse autoencoders. International Conference on Learning Representations. [find](https://scholar.google.com/scholar?q=Alireza+Makhzani+and+Brendan+Frey%2E+2014%2E+K%2Dsparse+autoencoders%2E+International+Conference+on+Learning+Representations%2E)
24. Abdelrahman Mohamed, Hung-yi Lee, Lasse Borgholt, Jakob D Havtorn, Joakim Edin, Christian Igel, Katrin Kirchhoff, Shang-Wen Li, Karen Livescu, Lars Maaløe, and 1 others. 2022. Self-supervised speech representation learning: A review. IEEE Journal of Selected Topics in Signal Processing, 16(6):1179–… [find](https://scholar.google.com/scholar?q=Abdelrahman+Mohamed%2C+Hung%2Dyi+Lee%2C+Lasse+Borgholt%2C+Jakob+D+Havtorn%2C+Joakim+Edin%2C+Christian+Igel%2C+Katrin+Kirchhoff%2C+Shang%2DWen+Li%2C+Karen+Livescu%2C+Lars+Maal%C3%B8e%2C+and+1+others%2E+2022%2E+Self%2Dsupervised+speech+r)
25. Arsha Nagrani, Joon Son Chung, and Andrew Zisserman. 2017. Voxceleb: A large-scale speaker identification dataset. In Interspeech 2017, pages 2616– 2620. [find](https://scholar.google.com/scholar?q=Arsha+Nagrani%2C+Joon+Son+Chung%2C+and+Andrew+Zisserman%2E+2017%2E+Voxceleb%3A+A+large%2Dscale+speaker+identification+dataset%2E+In+Interspeech+2017%2C+pages+2616%E2%80%93+2620%2E)
26. Bruno A Olshausen and David J Field. 1997. Sparse coding with an overcomplete basis set: A strategy employed by v1? Vision Research, 37(23):3311– 3325. [find](https://scholar.google.com/scholar?q=Bruno+A+Olshausen+and+David+J+Field%2E+1997%2E+Sparse+coding+with+an+overcomplete+basis+set%3A+A+strategy+employed+by+v1%3F+Vision+Research%2C+37%2823%29%3A3311%E2%80%93+3325%2E)
27. Vassil Panayotov, Guoguo Chen, Daniel Povey, and Sanjeev Khudanpur. 2015. Librispeech: An asr corpus based on public domain audio books. In Proceedings of the 40th International Conference on Acoustics, Speech and Signal Processing, pages 5206–5210. [find](https://scholar.google.com/scholar?q=Vassil+Panayotov%2C+Guoguo+Chen%2C+Daniel+Povey%2C+and+Sanjeev+Khudanpur%2E+2015%2E+Librispeech%3A+An+asr+corpus+based+on+public+domain+audio+books%2E+In+Proceedings+of+the+40th+International+Conference+on+Acoustic)
28. Ankita Pasad, Ju-Chieh Chou, and Karen Livescu. 2021. Layer-wise analysis of a self-supervised speech representation model. In 2021 IEEE Automatic Speech Recognition and Understanding Workshop, pages 914–921. IEEE. [find](https://scholar.google.com/scholar?q=Ankita+Pasad%2C+Ju%2DChieh+Chou%2C+and+Karen+Livescu%2E+2021%2E+Layer%2Dwise+analysis+of+a+self%2Dsupervised+speech+representation+model%2E+In+2021+IEEE+Automatic+Speech+Recognition+and+Understanding+Workshop%2C+pages+)
29. Ankita Pasad, Bowen Shi, and Karen Livescu. 2023. Comparative layer-wise analysis of self-supervised speech models. In In Proceedings of the 48th International Conference on Acoustics, Speech and Signal Processing, pages 1–5. IEEE. [find](https://scholar.google.com/scholar?q=Ankita+Pasad%2C+Bowen+Shi%2C+and+Karen+Livescu%2E+2023%2E+Comparative+layer%2Dwise+analysis+of+self%2Dsupervised+speech+models%2E+In+In+Proceedings+of+the+48th+International+Conference+on+Acoustics%2C+Speech+and+Sign)
30. Giancarlo Pastor, Inmaculada Mora-Jiménez, Riku Jäntti, and Antonio J. Caamaño. 2013. Sparsity-based criteria for entropy measures. In International Symposium on Wireless Communication Systems. [find](https://scholar.google.com/scholar?q=Giancarlo+Pastor%2C+Inmaculada+Mora%2DJim%C3%A9nez%2C+Riku+J%C3%A4ntti%2C+and+Antonio+J%2E+Caama%C3%B1o%2E+2013%2E+Sparsity%2Dbased+criteria+for+entropy+measures%2E+In+International+Symposium+on+Wireless+Communication+Systems%2E)
31. Aaron Van Den Oord, Oriol Vinyals, and Koray Kavukcuoglu. 2017. Neural discrete representation learning. Advances in Neural Information Processing Systems, 30. [find](https://scholar.google.com/scholar?q=Aaron+Van+Den+Oord%2C+Oriol+Vinyals%2C+and+Koray+Kavukcuoglu%2E+2017%2E+Neural+discrete+representation+learning%2E+Advances+in+Neural+Information+Processing+Systems%2C+30%2E)
32. Shu-wen Yang, Po-Han Chi, Yung-Sung Chuang, Cheng-I Jeff Lai, Kushal Lakhotia, Yist Y. Lin, Andy T. Liu, Jiatong Shi, Xuankai Chang, Guan-Ting Lin, Tzu-Hsien Huang, Wei-Cheng Tseng, Ko tik Lee, Da-Rong Liu, Zili Huang, Shuyan Dong, Shang-Wen Li, Shinji Watanabe, Abdelrahman Mohamed, and Hung yi Lee.… [find](https://scholar.google.com/scholar?q=Shu%2Dwen+Yang%2C+Po%2DHan+Chi%2C+Yung%2DSung+Chuang%2C+Cheng%2DI+Jeff+Lai%2C+Kushal+Lakhotia%2C+Yist+Y%2E+Lin%2C+Andy+T%2E+Liu%2C+Jiatong+Shi%2C+Xuankai+Chang%2C+Guan%2DTing+Lin%2C+Tzu%2DHsien+Huang%2C+Wei%2DCheng+Tseng%2C+Ko+tik+Lee%2C+Da%2DRon)
33. Guangyan Zhang, Ying Qin, Daxin Tan, and Tan Lee. 2021. Applying the Information Bottleneck Principle to Prosodic Representation Learning. In Interspeech 2021, pages 3156–3160. [find](https://scholar.google.com/scholar?q=Guangyan+Zhang%2C+Ying+Qin%2C+Daxin+Tan%2C+and+Tan+Lee%2E+2021%2E+Applying+the+Information+Bottleneck+Principle+to+Prosodic+Representation+Learning%2E+In+Interspeech+2021%2C+pages+3156%E2%80%933160%2E)
34. Junyi Peng, Rongzhi Gu, Ladislav Mošner, Oldrich Plchot, Lukas Burget, and Jan Černocký. 2022. Learnable sparse filterbank for speaker verification. In Interspeech 2022, pages 5110–5114. [find](https://scholar.google.com/scholar?q=Junyi+Peng%2C+Rongzhi+Gu%2C+Ladislav+Mo%C5%A1ner%2C+Oldrich+Plchot%2C+Lukas+Burget%2C+and+Jan+C%CC%8Cernock%C3%BD%2E+2022%2E+Learnable+sparse+filterbank+for+speaker+verification%2E+In+Interspeech+2022%2C+pages+5110%E2%80%935114%2E)
35. Senthooran Rajamanoharan, Arthur Conmy, Lewis Smith, Tom Lieberum, Vikrant Varma, János Kramár, Rohin Shah, and Neel Nanda. 2024. [Improving dictionary learning with gated sparse autoencoders](https://md.research.elamur.ai/free/en/papers/rajamanoharan-2024-improving-dictionary-learning-gated-sparse/). arXiv preprint arXiv:2404.16014.
36. Shai Shalev-Shwartz and Shai Ben-David. 2014. Understanding machine learning: From theory to algorithms. Cambridge University Press. [find](https://scholar.google.com/scholar?q=Shai+Shalev%2DShwartz+and+Shai+Ben%2DDavid%2E+2014%2E+Understanding+machine+learning%3A+From+theory+to+algorithms%2E+Cambridge+University+Press%2E)
37. Shijing Si, Jianzong Wang, Huiming Sun, Jianhan Wu, Chuanyao Zhang, Xiaoyang Qu, Ning Cheng, Lei Chen, and Jing Xiao. 2021. Variational Information Bottleneck for Effective Low-Resource Audio Classification. In Interspeech 2021, pages 591–595. [find](https://scholar.google.com/scholar?q=Shijing+Si%2C+Jianzong+Wang%2C+Huiming+Sun%2C+Jianhan+Wu%2C+Chuanyao+Zhang%2C+Xiaoyang+Qu%2C+Ning+Cheng%2C+Lei+Chen%2C+and+Jing+Xiao%2E+2021%2E+Variational+Information+Bottleneck+for+Effective+Low%2DResource+Audio+Classifi)
38. Berrak Sisman, Mingyang Zhang, and Haizhou Li. 2018. A voice conversion framework with tandem feature sparse representation and speaker-adapted wavenet vocoder. In Interspeech 2018, pages 1978–1982. [find](https://scholar.google.com/scholar?q=Berrak+Sisman%2C+Mingyang+Zhang%2C+and+Haizhou+Li%2E+2018%2E+A+voice+conversion+framework+with+tandem+feature+sparse+representation+and+speaker%2Dadapted+wavenet+vocoder%2E+In+Interspeech+2018%2C+pages+1978%E2%80%931982%2E)
39. Themos Stafylakis, Anna Silnova, Johan Rohdin, Oldřich Plchot, and Lukáš Burget. 2024. Challenging margin-based speaker embedding extractors by using the variational information bottleneck. In Interspeech 2024, pages 3220–3224. [find](https://scholar.google.com/scholar?q=Themos+Stafylakis%2C+Anna+Silnova%2C+Johan+Rohdin%2C+Oldr%CC%8Cich+Plchot%2C+and+Luk%C3%A1%C5%A1+Burget%2E+2024%2E+Challenging+margin%2Dbased+speaker+embedding+extractors+by+using+the+variational+information+bottleneck%2E+In+Inters)
40. Naftali Tishby, Fernando C Pereira, and William Bialek. 2000. The information bottleneck method. arXiv preprint physics/0004057. [find](https://scholar.google.com/scholar?q=Naftali+Tishby%2C+Fernando+C+Pereira%2C+and+William+Bialek%2E+2000%2E+The+information+bottleneck+method%2E+arXiv+preprint+physics%2F0004057%2E)
41. Naftali Tishby and Noga Zaslavsky. 2015. Deep learning and the information bottleneck principle. In 2015 IEEE Information Theory Workshop, pages 1–5. IEEE. [find](https://scholar.google.com/scholar?q=Naftali+Tishby+and+Noga+Zaslavsky%2E+2015%2E+Deep+learning+and+the+information+bottleneck+principle%2E+In+2015+IEEE+Information+Theory+Workshop%2C+pages+1%E2%80%935%2E+IEEE%2E)

*Updated 1 November 2026*