---
title: "Probing Discrete Speech Tokens of Spoken — Naber 2026"
description: "Many spoken language models handle speech as sequences of discrete tokens."
authors: ["Sven Naber", "Julia Koch", "Pranav Singh", "Alberto Saponaro", "Ioanna Karagianni", "Ngoc Thang Vu"]
year: 2026
venue: "Proceedings of the Fifteenth Language Resources and Evaluation Conference"
source: "https://aclanthology.org/2026.lrec-1.184/"
doi: "10.63317/4d4kttgdydde"
licence: "CC BY-NC 4.0 (ELRA, LREC proceedings)"
licence_url: "https://creativecommons.org/licenses/by-nc/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/papers/naber-2026-probing-discrete-speech-tokens-spoken/"
---
[Elamur Research](https://md.research.elamur.ai/) · [this page in HTML](https://research.elamur.ai/free/papers/naber-2026-probing-discrete-speech-tokens-spoken/)

[home](https://md.research.elamur.ai/) / [papers](https://md.research.elamur.ai/free/papers/) / [2026](https://md.research.elamur.ai/free/papers/2026/page/4/) / Probing Discrete Speech Tokens of Spoken Language Models

# Probing Discrete Speech Tokens of Spoken Language Models

**Authors:** Sven Naber, Julia Koch, Pranav Singh, Alberto Saponaro, Ioanna Karagianni, [Ngoc Thang Vu](https://md.research.elamur.ai/free/authors/ngoc-thang-vu/) · **Year:** 2026 · **Venue:** Proceedings of the Fifteenth Language Resources and Evaluation Conference · **Licence:** [CC BY-NC 4.0 (ELRA, LREC proceedings)](https://creativecommons.org/licenses/by-nc/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2026.lrec-1.184/) · [PDF](https://aclanthology.org/2026.lrec-1.184.pdf) · [DOI 10.63317/4d4kttgdydde](https://doi.org/10.63317/4d4kttgdydde) · [OpenAlex](https://openalex.org/W7160022726)

**Citations:** 0 citations (openalex-snapshot · 2026-10-14)

## Summary
Many spoken language models handle speech as sequences of discrete tokens. It is still unclear which phonetic and speaker properties these tokens carry, and whether that information survives once the model generates new tokens. The authors propose a model-agnostic probing framework that gathers three layers of evidence. First, Jensen–Shannon divergence compares token distributions between attribute groups. Second, logistic-regression classifiers measure how well an attribute can be recovered from token identities. Third, a token–phoneme co-occurrence analysis, built on Montreal Forced Alignment, shows how attribute groups realize phones differently. They apply it to CosyVoice2 and SparkTTS on LibriTTS-R and VCTK, comparing three token sets: tokenizer output from real audio, zero-shot generations conditioned on reference speech, and generations from text alone. In both systems, gender is clearly decodable from tokenizer and zero-shot tokens but essentially absent from text-only output. CosyVoice2 encodes gender consistently across stages and corpora, while SparkTTS encodes it more weakly and links it to pauses and prosody. Valence, arousal and dominance are captured poorly. The framework offers an interpretable basis for comparing tokenizers and generation pipelines, which matters for controllable speech synthesis.

## Key points
- Two pipelines are probed, both built on the Qwen2.5-0.5B language model. CosyVoice2 uses semantic tokens at 25 tokens/s. SparkTTS (BiCodec) uses semantic tokens at 50 tokens/s plus a fixed-length set of global speaker tokens.
- The divergence probe compares the JSD between high and low attribute groups against a baseline of 20 random label shuffles. Only tokens occurring at least 50 times in the corpus are included.
- The classifiers use three feature modes: BOW counts, SHARE (L1-normalized proportions) and SET (binary presence). They are trained on 80/20 speaker-disjoint splits, and tokens below 0.002% of training occurrences are filtered out.
- Gender is strongly recoverable from tokenizer and reference-conditioned (zero-shot) tokens, while text-only generations perform roughly at chance level. CosyVoice2 shows recurring gender-marker tokens across datasets and high tokenizer–zero-shot similarity.
- According to the authors, both the framework's implementation and the complete set of analysis results have been released to the public.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

Fields: [Linguistics & languages](https://md.research.elamur.ai/free/shelves/linguistics/)

## Cite

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{naber2026probing,
  title = {{Probing Discrete Speech Tokens of Spoken Language Models}},
  author = {Naber, Sven and Koch, Julia and Singh, Pranav and Saponaro, Alberto and Karagianni, Ioanna and Vu, Ngoc Thang},
  booktitle = {Proceedings of the Fifteenth Language Resources and Evaluation Conference},
  year = {2026},
  doi = {10.63317/4d4kttgdydde},
  url = {https://doi.org/10.63317/4d4kttgdydde}
}
```

- [← previous Predict and Fix: A Unified Model for Translation Quality Estimation and Post-Editing](https://md.research.elamur.ai/free/papers/modrzejewski-2026-predict-fix-unified-model-translation/)
- [next → Probing the Dynamics of Syntactic Ability Acquisition Throughout LLM Pretraining](https://md.research.elamur.ai/free/papers/matsuda-2026-probing-dynamics-syntactic-ability-acquisition/)
- [↑ back to papers of 2026](https://md.research.elamur.ai/free/papers/2026/page/4/)

## References

28 references, 5 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Siddhant Arora, Kai-Wei Chang, Chung-Ming Chien, Yifan Peng, Haibin Wu, Yossi Adi, Emmanuel Dupoux, Hung-Yi Lee, Karen Livescu, and Shinji Watanabe. 2025. On the landscape of spoken language models: A comprehensive survey. [find](https://scholar.google.com/scholar?q=Siddhant+Arora%2C+Kai%2DWei+Chang%2C+Chung%2DMing+Chien%2C+Yifan+Peng%2C+Haibin+Wu%2C+Yossi+Adi%2C+Emmanuel+Dupoux%2C+Hung%2DYi+Lee%2C+Karen+Livescu%2C+and+Shinji+Watanabe%2E+2025%2E+On+the+landscape+of+spoken+language+models%3A+A)
2. Alexei Baevski, Henry Zhou, Abdelrahman Mohamed, and Michael Auli. 2020. wav2vec 2.0: a framework for self-supervised learning of speech representations. In Proceedings of the 34th International Conference on Neural Information Processing Systems, NIPS ’20, Red Hook, NY, USA. Curran Associates Inc. [find](https://scholar.google.com/scholar?q=Alexei+Baevski%2C+Henry+Zhou%2C+Abdelrahman+Mohamed%2C+and+Michael+Auli%2E+2020%2E+wav2vec+2%2E0%3A+a+framework+for+self%2Dsupervised+learning+of+speech+representations%2E+In+Proceedings+of+the+34th+International+Confe) [our card](https://md.research.elamur.ai/free/papers/baevski-2020-wav2vec-2-0-framework-self/)
3. Heng-Jui Chang, Hongyu Gong, Changhan Wang, James Glass, and Yu-An Chung. 2025. DC-Spin: A Speaker-invariant Speech Tokenizer for Spoken Language Models. In Interspeech 2025, pages 5723–5727. [find](https://scholar.google.com/scholar?q=Heng%2DJui+Chang%2C+Hongyu+Gong%2C+Changhan+Wang%2C+James+Glass%2C+and+Yu%2DAn+Chung%2E+2025%2E+DC%2DSpin%3A+A+Speaker%2Dinvariant+Speech+Tokenizer+for+Spoken+Language+Models%2E+In+Interspeech+2025%2C+pages+5723%E2%80%935727%2E)
4. Yunfei Chu, Jin Xu, Qian Yang, Haojie Wei, Xipin Wei, Zhifang Guo, Yichong Leng, Yuanjun Lv, Jinzheng He, Junyang Lin, Chang Zhou, and Jingren Zhou. 2024. Qwen2-audio technical report. [find](https://scholar.google.com/scholar?q=Yunfei+Chu%2C+Jin+Xu%2C+Qian+Yang%2C+Haojie+Wei%2C+Xipin+Wei%2C+Zhifang+Guo%2C+Yichong+Leng%2C+Yuanjun+Lv%2C+Jinzheng+He%2C+Junyang+Lin%2C+Chang+Zhou%2C+and+Jingren+Zhou%2E+2024%2E+Qwen2%2Daudio+technical+report%2E) [our card](https://md.research.elamur.ai/free/papers/chu-2024-qwen2-audio-technical-report/)
5. Yu-An Chung, Wei-Ning Hsu, Hao Tang, and James Glass. 2019. An Unsupervised Autoregressive Model for Speech Representation Learning. In Interspeech 2019, pages 146–150. [find](https://scholar.google.com/scholar?q=Yu%2DAn+Chung%2C+Wei%2DNing+Hsu%2C+Hao+Tang%2C+and+James+Glass%2E+2019%2E+An+Unsupervised+Autoregressive+Model+for+Speech+Representation+Learning%2E+In+Interspeech+2019%2C+pages+146%E2%80%93150%2E)
6. Zhihao Du, Qian Chen, Shiliang Zhang, Kai Hu, Heng Lu, Yexin Yang, Hangrui Hu, Siqi Zheng, Yue Gu, Ziyang Ma, Zhifu Gao, and Zhijie Yan. 2024a. Cosyvoice: A scalable multilingual zero-shot text-to-speech synthesizer based on supervised semantic tokens. ArXiv, abs/2407.05407. [find](https://scholar.google.com/scholar?q=Zhihao+Du%2C+Qian+Chen%2C+Shiliang+Zhang%2C+Kai+Hu%2C+Heng+Lu%2C+Yexin+Yang%2C+Hangrui+Hu%2C+Siqi+Zheng%2C+Yue+Gu%2C+Ziyang+Ma%2C+Zhifu+Gao%2C+and+Zhijie+Yan%2E+2024a%2E+Cosyvoice%3A+A+scalable+multilingual+zero%2Dshot+text%2Dto%2Dspe)
7. Zhihao Du, Yuxuan Wang, Qian Chen, Xian Shi, Xiang Lv, Tianyu Zhao, Zhifu Gao, Yexin Yang, Changfeng Gao, Hui Wang, Fan Yu, Huadai Liu, Zhengyan Sheng, Yue Gu, Chong Deng, Wen Wang, Shiliang Zhang, Zhijie Yan, and Jingren Zhou. 2024b. Cosyvoice 2: Scalable streaming speech synthesis with large langu… [find](https://scholar.google.com/scholar?q=Zhihao+Du%2C+Yuxuan+Wang%2C+Qian+Chen%2C+Xian+Shi%2C+Xiang+Lv%2C+Tianyu+Zhao%2C+Zhifu+Gao%2C+Yexin+Yang%2C+Changfeng+Gao%2C+Hui+Wang%2C+Fan+Yu%2C+Huadai+Liu%2C+Zhengyan+Sheng%2C+Yue+Gu%2C+Chong+Deng%2C+Wen+Wang%2C+Shiliang+Zhang%2C+Zh)
8. Alexandre Défossez, Laurent Mazaré, Manu Orsini, Amélie Royer, Patrick Pérez, Hervé Jégou, Edouard Grave, and Neil Zeghidour. 2024. Moshi: a speech-text foundation model for real-time dialogue. [find](https://scholar.google.com/scholar?q=Alexandre+D%C3%A9fossez%2C+Laurent+Mazar%C3%A9%2C+Manu+Orsini%2C+Am%C3%A9lie+Royer%2C+Patrick+P%C3%A9rez%2C+Herv%C3%A9+J%C3%A9gou%2C+Edouard+Grave%2C+and+Neil+Zeghidour%2E+2024%2E+Moshi%3A+a+speech%2Dtext+foundation+model+for+real%2Dtime+dialogue%2E) [our card](https://md.research.elamur.ai/free/papers/defossez-2024-moshi-speech-text-foundation-model/)
9. Gemini Team, Google DeepMind. 2023. Gemini 1: Unlocking multimodal understanding. [https://storage.googleapis.com/deepmind-media/gemini/gemini_1_report.pdf](https://storage.googleapis.com/deepmind-media/gemini/gemini_1_report.pdf). [↗](https://storage.googleapis.com/deepmind-media/gemini/gemini_1_report.pdf)
10. Hao-Han Guo, Yao Hu, Kun Liu, Fei-Yu Shen, Xu Tang, Yi-Chen Wu, Feng-Long Xie, Kun Xie, and Kai-Tuo Xu. 2025. Fireredtts: A foundation text-to-speech framework for industry-level generative speech applications. [find](https://scholar.google.com/scholar?q=Hao%2DHan+Guo%2C+Yao+Hu%2C+Kun+Liu%2C+Fei%2DYu+Shen%2C+Xu+Tang%2C+Yi%2DChen+Wu%2C+Feng%2DLong+Xie%2C+Kun+Xie%2C+and+Kai%2DTuo+Xu%2E+2025%2E+Fireredtts%3A+A+foundation+text%2Dto%2Dspeech+framework+for+industry%2Dlevel+generative+speech+app)
11. Wei-Ning Hsu, Benjamin Bolte, Yao-Hung Hubert Tsai, Kushal Lakhotia, Ruslan Salakhutdinov, and Abdelrahman Mohamed. 2021. Hubert: Self-supervised speech representation learning by masked prediction of hidden units. IEEE/ACM Trans. Audio, Speech and Lang. Proc., 29:3451–3460. [find](https://scholar.google.com/scholar?q=Wei%2DNing+Hsu%2C+Benjamin+Bolte%2C+Yao%2DHung+Hubert+Tsai%2C+Kushal+Lakhotia%2C+Ruslan+Salakhutdinov%2C+and+Abdelrahman+Mohamed%2E+2021%2E+Hubert%3A+Self%2Dsupervised+speech+representation+learning+by+masked+prediction+of) [our card](https://md.research.elamur.ai/free/papers/hsu-2021-hubert-self-supervised-speech-representation/)
12. Yichong Leng, Zhifang Guo, Kai Shen, Xu Tan, Zeqian Ju, Yanqing Liu, Yufei Liu, Dongchao Yang, Leying Zhang, Kaitao Song, Lei He, Xiang-Yang Li, Sheng Zhao, Tao Qin, and Jiang Bian. 2023. Prompttts 2: Describing and generating voices with text prompt. [find](https://scholar.google.com/scholar?q=Yichong+Leng%2C+Zhifang+Guo%2C+Kai+Shen%2C+Xu+Tan%2C+Zeqian+Ju%2C+Yanqing+Liu%2C+Yufei+Liu%2C+Dongchao+Yang%2C+Leying+Zhang%2C+Kaitao+Song%2C+Lei+He%2C+Xiang%2DYang+Li%2C+Sheng+Zhao%2C+Tao+Qin%2C+and+Jiang+Bian%2E+2023%2E+Prompttts+2%3A)
13. Guan-Ting Lin, Cheng-Han Chiang, and Hung yi Lee. 2024a. Advancing large language models to capture varied speaking styles and respond properly in spoken conversations. [find](https://scholar.google.com/scholar?q=Guan%2DTing+Lin%2C+Cheng%2DHan+Chiang%2C+and+Hung+yi+Lee%2E+2024a%2E+Advancing+large+language+models+to+capture+varied+speaking+styles+and+respond+properly+in+spoken+conversations%2E)
14. Tzu-Quan Lin, Guan-Ting Lin, Hung-Yi Lee, and Hao Tang. 2024b. Property neurons in self-supervised speech transformers. In 2024 IEEE Spoken Language Technology Workshop (SLT), pages 401–408. [find](https://scholar.google.com/scholar?q=Tzu%2DQuan+Lin%2C+Guan%2DTing+Lin%2C+Hung%2DYi+Lee%2C+and+Hao+Tang%2E+2024b%2E+Property+neurons+in+self%2Dsupervised+speech+transformers%2E+In+2024+IEEE+Spoken+Language+Technology+Workshop+%28SLT%29%2C+pages+401%E2%80%93408%2E)
15. Michael McAuliffe, Michael Socolof, Sarah Mihuc, Michael Wagner, and Morgan Sonderegger. 2017. Montreal forced aligner: Trainable text-speech alignment using kaldi. [https://montreal-forced-aligner.readthedocs.io](https://montreal-forced-aligner.readthedocs.io). Version 2.2.5, Accessed: 2025-10-24. [↗](https://montreal-forced-aligner.readthedocs.io)
16. Kentaro Onda, Hayato Futami, Yosuke Kashiwagi, Emiru Tsunoo, and Shinji Watanabe. 2026. Phonological tokenizer: Prosody-aware phonetic token via multi-objective fine-tuning with differentiable k-means. ArXiv, abs/2601.19781. — [arXiv:2601.19781](https://arxiv.org/abs/2601.19781) [find](https://scholar.google.com/scholar?q=Kentaro+Onda%2C+Hayato+Futami%2C+Yosuke+Kashiwagi%2C+Emiru+Tsunoo%2C+and+Shinji+Watanabe%2E+2026%2E+Phonological+tokenizer%3A+Prosody%2Daware+phonetic+token+via+multi%2Dobjective+fine%2Dtuning+with+differentiable+k%2Dmeans)
17. OpenAI. 2024. Hello gpt−4o. [https://openai.com/index/hello-gpt-4o/](https://openai.com/index/hello-gpt-4o/). Accessed: 2025-10-24. [↗](https://openai.com/index/hello-gpt-4o/)
18. Long Ouyang, Jeff Wu, Xu Jiang, Diogo Almeida, Carroll L. Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, John Schulman, Jacob Hilton, Fraser Kelton, Luke Miller, Maddie Simens, Amanda Askell, Peter Welinder, Paul Christiano, Jan Leike, and Ryan Lowe. 2022. Train… [find](https://scholar.google.com/scholar?q=Long+Ouyang%2C+Jeff+Wu%2C+Xu+Jiang%2C+Diogo+Almeida%2C+Carroll+L%2E+Wainwright%2C+Pamela+Mishkin%2C+Chong+Zhang%2C+Sandhini+Agarwal%2C+Katarina+Slama%2C+Alex+Ray%2C+John+Schulman%2C+Jacob+Hilton%2C+Fraser+Kelton%2C+Luke+Miller%2C+)
19. Dario Di Palma, Alessandro De Bellis, Giovanni Servedio, Vito Walter Anelli, Fedelucio Narducci, and T. D. Noia. 2025. Llamas have feelings too: Unveiling sentiment and emotion representations in llama models through probing. ArXiv, abs/2505.16491. [find](https://scholar.google.com/scholar?q=Dario+Di+Palma%2C+Alessandro+De+Bellis%2C+Giovanni+Servedio%2C+Vito+Walter+Anelli%2C+Fedelucio+Narducci%2C+and+T%2E+D%2E+Noia%2E+2025%2E+Llamas+have+feelings+too%3A+Unveiling+sentiment+and+emotion+representations+in+llam)
20. Samir Sadok, Julien Hauret, and Éric Bavu. 2025. Bringing Interpretability to Neural Audio Codecs. In Interspeech 2025, pages 5023–5027. [find](https://scholar.google.com/scholar?q=Samir+Sadok%2C+Julien+Hauret%2C+and+%C3%89ric+Bavu%2E+2025%2E+Bringing+Interpretability+to+Neural+Audio+Codecs%2E+In+Interspeech+2025%2C+pages+5023%E2%80%935027%2E)
21. Christophe Veaux, Junichi Yamagishi, and Kirsten MacDonald. 2019. Cstr vctk corpus: English multi-speaker corpus for cstr voice cloning toolkit (version 0.92). [https://datashare.ed.ac.uk/handle/10283/2950](https://datashare.ed.ac.uk/handle/10283/2950). [↗](https://datashare.ed.ac.uk/handle/10283/2950)
22. Released September 2019. Licensed under Creative Commons Attribution 4.0 International. [find](https://scholar.google.com/scholar?q=Released+September+2019%2E+Licensed+under+Creative+Commons+Attribution+4%2E0+International%2E)
23. Xinsheng Wang, Mingqi Jiang, Ziyang Ma, Ziyu Zhang, Songxiang Liu, Linqin Li, Zheng Liang, Qixi Zheng, Rui Wang, Xiaoqin Feng, Weizhen Bian, Zhen Ye, Sitong Cheng, Ruibin Yuan, Zhixian Zhao, Xinfa Zhu, Jiahao Pan, Liumeng Xue, Pengcheng Zhu, Yunlin Chen, Zhifei Li, Xie Chen, Lei Xie, Yike Guo, and W… [find](https://scholar.google.com/scholar?q=Xinsheng+Wang%2C+Mingqi+Jiang%2C+Ziyang+Ma%2C+Ziyu+Zhang%2C+Songxiang+Liu%2C+Linqin+Li%2C+Zheng+Liang%2C+Qixi+Zheng%2C+Rui+Wang%2C+Xiaoqin+Feng%2C+Weizhen+Bian%2C+Zhen+Ye%2C+Sitong+Cheng%2C+Ruibin+Yuan%2C+Zhixian+Zhao%2C+Xinfa+Zhu)
24. Dan Wells, Hao Tang, and Korin Richmond. 2022. Phonetic Analysis of Self-supervised Representations of English Speech. In Interspeech 2022, pages 3583–3587. [find](https://scholar.google.com/scholar?q=Dan+Wells%2C+Hao+Tang%2C+and+Korin+Richmond%2E+2022%2E+Phonetic+Analysis+of+Self%2Dsupervised+Representations+of+English+Speech%2E+In+Interspeech+2022%2C+pages+3583%E2%80%933587%2E)
25. An Yang, Baosong Yang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Zhou, Chengpeng Li, Chengyuan Li, Dayiheng Liu, Fei Huang, Guanting Dong, Haoran Wei, Huan Lin, Jialong Tang, Jialin Wang, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Ma, Jianxin Yang, Jin Xu, Jingren Zhou, Jinze Bai, Jinzheng He, Juny… [find](https://scholar.google.com/scholar?q=An+Yang%2C+Baosong+Yang%2C+Binyuan+Hui%2C+Bo+Zheng%2C+Bowen+Yu%2C+Chang+Zhou%2C+Chengpeng+Li%2C+Chengyuan+Li%2C+Dayiheng+Liu%2C+Fei+Huang%2C+Guanting+Dong%2C+Haoran+Wei%2C+Huan+Lin%2C+Jialong+Tang%2C+Jialin+Wang%2C+Jian+Yang%2C+Jian)
26. Dongchao Yang, Songxiang Liu, Rongjie Huang, Chao Weng, and Helen Meng. 2023. Instructtts: Modelling expressive tts in discrete latent space with natural language style prompt. [find](https://scholar.google.com/scholar?q=Dongchao+Yang%2C+Songxiang+Liu%2C+Rongjie+Huang%2C+Chao+Weng%2C+and+Helen+Meng%2E+2023%2E+Instructtts%3A+Modelling+expressive+tts+in+discrete+latent+space+with+natural+language+style+prompt%2E)
27. Heiga Zen, Viet Dang, Rob Clark, Yu Zhang, Ron J. Weiss, Ye Jia, Zhifeng Chen, and Yonghui Wu. 2019. Libritts: A corpus derived from librispeech for text-to-speech. [find](https://scholar.google.com/scholar?q=Heiga+Zen%2C+Viet+Dang%2C+Rob+Clark%2C+Yu+Zhang%2C+Ron+J%2E+Weiss%2C+Ye+Jia%2C+Zhifeng+Chen%2C+and+Yonghui+Wu%2E+2019%2E+Libritts%3A+A+corpus+derived+from+librispeech+for+text%2Dto%2Dspeech%2E) [our card](https://md.research.elamur.ai/free/papers/zen-2019-libritts-corpus-derived-librispeech-text/)
28. Xin Zhang, Dong Zhang, Shimin Li, Yaqian Zhou, and Xipeng Qiu. 2024. Speechtokenizer: Unified speech tokenizer for speech language models. In The Twelfth International Conference on Learning Representations. [find](https://scholar.google.com/scholar?q=Xin+Zhang%2C+Dong+Zhang%2C+Shimin+Li%2C+Yaqian+Zhou%2C+and+Xipeng+Qiu%2E+2024%2E+Speechtokenizer%3A+Unified+speech+tokenizer+for+speech+language+models%2E+In+The+Twelfth+International+Conference+on+Learning+Represent)