---
title: "MiMo-Audio: Audio Language Models are Few-Shot Learners"
description: "Together they offer a reproducible blueprint for general-purpose speech models. 61 references, cited by 22 papers here."
authors: ["Xiaomi LLM-Core Team", "Dong Zhang", "Gang Wang", "Jinlong Xue", "Kai Fang", "Liang Zhao", "Rui Ma", "Shuhuai Ren", "Shuo Liu", "Tao Guo", "Weiji Zhuang", "Xin Zhang", "Xingchen Song", "Yihan Yan", "Yongzhe He", "Cici", "Bowen Shen", "Chengxuan Zhu", "Chong Ma", "Chun Chen", "Heyu Chen", "Jiawei Li", "Lei Li", "Menghang Zhu", "Peidian Li", "Qiying Wang", "Sirui Deng", "Weimin Xiong", "Wenshan Huang", "Wenyu Yang", "Yilin Jiang", "Yixin Yang", "Yuanyuan Tian", "Yue Ma", "Yue Yu", "Zihan Zhang", "Zihao Yue", "Bangjun Xiao", "Bingquan Xia", "Bofei Gao", "Bowen Ye", "Can Cai", "Chang Liu", "Chenhong He", "Chunan Li", "Dawei Zhu", "Duo Zhang", "Fengyuan Shi", "Guoan Wang", "Hailin Zhang", "Hanglong Lv", "Hanyu Li", "Hao Tian", "Heng Qu", "Hongshen Xu", "Houbin Zhang", "Huaqiu Liu", "Jiangshan Duo", "Jianguang Zuo", "Jianyu Wei", "Jiebao Xiao", "Jinhao Dong", "Jun Shi", "Junhao Hu", "Kainan Bao", "Kang Zhou", "Linghao Zhang", "Meng Chen", "Nuo Chen", "Peng Zhang", "Qianli Chen", "Qiantong Wang", "Rang Li", "Shaohui Liu", "Shengfan Wang", "Shicheng Li", "Shihua Yu", "Shijie Cao", "Shimao Chen", "Shuhao Gu", "Weikun Wang", "Wenhan Ma", "Xiangwei Deng", "Xing Yong", "Xing Zhang", "Xu Wang", "Yifan Song", "Yihao Zhao", "Yingbo Zhao", "Yizhao Gao", "Yu Cheng", "Yu Tu", "Yudong Wang", "Zhaojun Huang", "Zhengju Tang", "Zhenru Lin", "Zhichao Song", "Zhipeng Xu", "Zhixian Zheng", "Zihan Jiang"]
year: 2025
venue: "arXiv:2512.23808"
source: "https://arxiv.org/abs/2512.23808"
doi: "10.48550/arxiv.2512.23808"
arxiv: "2512.23808"
licence: "CC BY 4.0 (arXiv)"
licence_url: "http://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/team-2025-llm-mimo-audio-audio-language/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/team-2025-llm-mimo-audio-audio-language/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2025](https://md.research.elamur.ai/free/en/papers/2025/page/4/) / MiMo-Audio: Audio Language Models are Few-Shot Learners

# MiMo-Audio: Audio Language Models are Few-Shot Learners

**Authors:** [Xiaomi LLM-Core Team](https://md.research.elamur.ai/free/en/authors/xiaomi-llm-core-team/), [Dong Zhang](https://md.research.elamur.ai/free/en/authors/dong-zhang/), [Gang Wang](https://md.research.elamur.ai/free/en/authors/gang-wang/), Jinlong Xue, Kai Fang, [Liang Zhao](https://md.research.elamur.ai/free/en/authors/liang-zhao/), Rui Ma, [Shuhuai Ren](https://md.research.elamur.ai/free/en/authors/shuhuai-ren/), Shuo Liu, Tao Guo, [Weiji Zhuang](https://md.research.elamur.ai/free/en/authors/weiji-zhuang/), [Xin Zhang](https://md.research.elamur.ai/free/en/authors/xin-zhang/), Xingchen Song, Yihan Yan, Yongzhe He, Cici, Bowen Shen, Chengxuan Zhu, Chong Ma, Chun Chen, Heyu Chen, Jiawei Li, [Lei Li](https://md.research.elamur.ai/free/en/authors/lei-li/), Menghang Zhu, Peidian Li, Qiying Wang, Sirui Deng, [Weimin Xiong](https://md.research.elamur.ai/free/en/authors/weimin-xiong/), Wenshan Huang, Wenyu Yang, Yilin Jiang, Yixin Yang, Yuanyuan Tian, Yue Ma, [Yue Yu](https://md.research.elamur.ai/free/en/authors/yue-yu/), Zihan Zhang, [Zihao Yue](https://md.research.elamur.ai/free/en/authors/zihao-yue/), Bangjun Xiao, Bingquan Xia, [Bofei Gao](https://md.research.elamur.ai/free/en/authors/bofei-gao/), Bowen Ye, Can Cai, [Chang Liu](https://md.research.elamur.ai/free/en/authors/chang-liu/), Chenhong He, Chunan Li, [Dawei Zhu](https://md.research.elamur.ai/free/en/authors/dawei-zhu/), Duo Zhang, Fengyuan Shi, Guoan Wang, Hailin Zhang, Hanglong Lv, Hanyu Li, [Hao Tian](https://md.research.elamur.ai/free/en/authors/hao-tian/), Heng Qu, Hongshen Xu, Houbin Zhang, Huaqiu Liu, Jiangshan Duo, Jianguang Zuo, Jianyu Wei, Jiebao Xiao, Jinhao Dong, Jun Shi, Junhao Hu, Kainan Bao, [Kang Zhou](https://md.research.elamur.ai/free/en/authors/kang-zhou/), Linghao Zhang, Meng Chen, [Nuo Chen](https://md.research.elamur.ai/free/en/authors/nuo-chen/), [Peng Zhang](https://md.research.elamur.ai/free/en/authors/peng-zhang/), Qianli Chen, Qiantong Wang, Rang Li, Shaohui Liu, Shengfan Wang, Shicheng Li, Shihua Yu, Shijie Cao, Shimao Chen, [Shuhao Gu](https://md.research.elamur.ai/free/en/authors/shuhao-gu/), Weikun Wang, Wenhan Ma, Xiangwei Deng, Xing Yong, Xing Zhang, [Xu Wang](https://md.research.elamur.ai/free/en/authors/xu-wang/), [Yifan Song](https://md.research.elamur.ai/free/en/authors/yifan-song/), Yihao Zhao, Yingbo Zhao, Yizhao Gao, [Yu Cheng](https://md.research.elamur.ai/free/en/authors/yu-cheng/), Yu Tu, [Yudong Wang](https://md.research.elamur.ai/free/en/authors/yudong-wang/), Zhaojun Huang, Zhengju Tang, Zhenru Lin, Zhichao Song, [Zhipeng Xu](https://md.research.elamur.ai/free/en/authors/zhipeng-xu/), Zhixian Zheng and Zihan Jiang · **Year:** 2025 · **Venue:** arXiv:2512.23808 · **Licence:** [CC BY 4.0 (arXiv)](http://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://arxiv.org/abs/2512.23808) · [PDF](https://arxiv.org/pdf/2512.23808) · [DOI 10.48550/arxiv.2512.23808](https://doi.org/10.48550/arxiv.2512.23808) · [arXiv 2512.23808](https://arxiv.org/abs/2512.23808) · [OpenAlex](https://openalex.org/W7117988839)

**Citations:** 2 citations (OpenAlex, 2026-10-14) · [22](https://md.research.elamur.ai/free/en/papers/team-2025-llm-mimo-audio-audio-language/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
Most current audio language models need separate fine-tuning for each speech task. This work asks whether the GPT-3 recipe, scaling next-token prediction pretraining, also gives audio models few-shot generalization. The Xiaomi team built MiMo-Audio around a tokenizer designed to keep paralinguistic detail. They used a patch-based architecture that shortens long audio token sequences before they reach the LLM. Starting from MiMo-7B-Base, they pretrained on more than 100 million hours of speech in two stages: understanding first, then joint understanding and generation. Once the data passed a certain scale, the base model began learning from in-context examples. It could do voice conversion, style transfer, denoising, speech translation and speech editing from a handful of demonstrations, and it also produced realistic speech continuations. An instruction-tuned variant with chain-of-thought "thinking" data reached open-source SOTA in audio understanding, spoken dialogue and instruct-TTS. The models are released openly, together with the evaluation suite. Together they offer a reproducible blueprint for general-purpose speech models.

## Key points
- **MiMo-Audio-Tokenizer**: a 1.2B-parameter Transformer with 8-layer RVQ, running at 25Hz and producing 200 tokens/s. It was trained from scratch on 10 million hours with combined semantic and reconstruction objectives.
- **Architecture**: a patch encoder merges 4 RVQ timesteps into one patch, giving the LLM a 6.25Hz input. A patch decoder then autoregressively restores the full 25Hz token stream.
- **Evaluation of the base model**: MiMo-Audio-7B-Base was tested with a new few-shot benchmark that includes SpeechMMLU, a speech-synthesized version of MMLU. On it, speech-input and speech-output scores come close to text MMLU.
- **MiMo-Audio-7B-Instruct results**:
  - It leads open-source models on MMSU, MMAU, MMAR and MMAU-Pro.
  - On Big Bench Audio it scores 72.90 (S2T) and 60.20 (S2S), second only to gpt-4o.
  - On MultiChallenge Audio it scores 15.15 (S2T) and 10.10 (S2S).
  - It beats gpt-4o-mini-tts on InstructTTSEval.
- **Dialogue data and limitations**: spoken dialogue training data were generated with MiMo-TTS-7B, which was trained on more than 7M hours. The authors report that the thinking mode helps speech understanding but hurts sound and music understanding, because the model hallucinates during reasoning.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

📚 Shelves (5): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 16,039 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 15,891 works · [🔊 Signal Processing](https://md.research.elamur.ai/free/en/shelves/computer-science/signal-processing/) · 452 works

🧭 Topics (2): [🧠 Language models & pretraining](https://md.research.elamur.ai/free/en/topics/language-models/) · 3,320 works · [🎙️ Speech & spoken language](https://md.research.elamur.ai/free/en/topics/speech/) · 480 works

## Cite

Cite as: Xiaomi LLM-Core Team, Dong Zhang, Gang Wang, Jinlong Xue, Kai Fang, Liang Zhao, Rui Ma, Shuhuai Ren, Shuo Liu, Tao Guo, Weiji Zhuang, Xin Zhang, Xingchen Song, Yihan Yan, Yongzhe He, Cici, Bowen Shen, Chengxuan Zhu, Chong Ma, Chun Chen, Heyu Chen, Jiawei Li, Lei Li, Menghang Zhu, Peidian Li, Qiying Wang, Sirui Deng, Weimin Xiong, Wenshan Huang, Wenyu Yang, Yilin Jiang, Yixin Yang, Yuanyuan Tian, Yue Ma, Yue Yu, Zihan Zhang, Zihao Yue, Bangjun Xiao, Bingquan Xia, Bofei Gao, Bowen Ye, Can Cai, Chang Liu, Chenhong He, Chunan Li, Dawei Zhu, Duo Zhang, Fengyuan Shi, Guoan Wang, Hailin Zhang, Hanglong Lv, Hanyu Li, Hao Tian, Heng Qu, Hongshen Xu, Houbin Zhang, Huaqiu Liu, Jiangshan Duo, Jianguang Zuo, Jianyu Wei, Jiebao Xiao, Jinhao Dong, Jun Shi, Junhao Hu, Kainan Bao, Kang Zhou, Linghao Zhang, Meng Chen, Nuo Chen, Peng Zhang, Qianli Chen, Qiantong Wang, Rang Li, Shaohui Liu, Shengfan Wang, Shicheng Li, Shihua Yu, Shijie Cao, Shimao Chen, Shuhao Gu, Weikun Wang, Wenhan Ma, Xiangwei Deng, Xing Yong, Xing Zhang, Xu Wang, Yifan Song, Yihao Zhao, Yingbo Zhao, Yizhao Gao, Yu Cheng, Yu Tu, Yudong Wang, Zhaojun Huang, Zhengju Tang, Zhenru Lin, Zhichao Song, Zhipeng Xu, Zhixian Zheng and Zihan Jiang. 2025. MiMo-Audio: Audio Language Models are Few-Shot Learners. arXiv:2512.23808.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@misc{team2025mimo,
  title = {{MiMo-Audio: Audio Language Models are Few-Shot Learners}},
  author = {Xiaomi LLM-Core Team, Dong Zhang, Gang Wang, Jinlong Xue, Kai Fang, Liang Zhao, Rui Ma, Shuhuai Ren, Shuo Liu, Tao Guo, Weiji Zhuang, Xin Zhang, Xingchen Song, Yihan Yan, Yongzhe He, Cici, Bowen Shen, Chengxuan Zhu, Chong Ma, Chun Chen, Heyu Chen, Jiawei Li, Lei Li, Menghang Zhu, Peidian Li, Qiying Wang, Sirui Deng, Weimin Xiong, Wenshan Huang, Wenyu Yang, Yilin Jiang, Yixin Yang, Yuanyuan Tian, Yue Ma, Yue Yu, Zihan Zhang, Zihao Yue, Bangjun Xiao, Bingquan Xia, Bofei Gao, Bowen Ye, Can Cai, Chang Liu, Chenhong He, Chunan Li, Dawei Zhu, Duo Zhang, Fengyuan Shi, Guoan Wang, Hailin Zhang, Hanglong Lv, Hanyu Li, Hao Tian, Heng Qu, Hongshen Xu, Houbin Zhang, Huaqiu Liu, Jiangshan Duo, Jianguang Zuo, Jianyu Wei, Jiebao Xiao, Jinhao Dong, Jun Shi, Junhao Hu, Kainan Bao, Kang Zhou, Linghao Zhang, Meng Chen, Nuo Chen, Peng Zhang, Qianli Chen, Qiantong Wang, Rang Li, Shaohui Liu, Shengfan Wang, Shicheng Li, Shihua Yu, Shijie Cao, Shimao Chen, Shuhao Gu, Weikun Wang, Wenhan Ma, Xiangwei Deng, Xing Yong, Xing Zhang, Xu Wang, Yifan Song, Yihao Zhao, Yingbo Zhao, Yizhao Gao, Yu Cheng, Yu Tu, Yudong Wang, Zhaojun Huang, Zhengju Tang, Zhenru Lin, Zhichao Song, Zhipeng Xu, Zhixian Zheng and Zihan Jiang},
  year = {2025},
  doi = {10.48550/arxiv.2512.23808},
  eprint = {2512.23808},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2512.23808}
}
```

- [← previous M-MAD: Multidimensional Multi-Agent Debate for Advanced Machine Translation Evaluation](https://md.research.elamur.ai/free/en/papers/feng-2025-m-mad-multidimensional-multi-agent/)
- [next → MiniCPM-V 4.5: Cooking Efficient MLLMs via Architecture, Data, and Training Recipe](https://md.research.elamur.ai/free/en/papers/yu-2025-minicpm-v-4-5/)
- [↑ back to papers of 2025](https://md.research.elamur.ai/free/en/papers/2025/page/4/)

## References

61 references, 23 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. P. Anastassiou, J. Chen, J. Chen, Y. Chen, Z. Chen, Z. Chen, J. Cong, L. Deng, C. Ding, L. Gao, et al. [Seed-tts: A family of high-quality versatile speech generation models](https://md.research.elamur.ai/free/en/papers/anastassiou-2024-seed-tts-family-high/). arXiv preprint arXiv:2406.02430, 2024.
2. Z. Borsos, R. Marinier, D. Vincent, E. Kharitonov, O. Pietquin, M. Sharifi, D. Roblek, O. Teboul, D. Grangier, M. Tagliasacchi, and N. Zeghidour. [Audiolm: a language modeling approach to audio generation](https://md.research.elamur.ai/free/en/papers/borsos-2022-audiolm-language-modeling-approach-audio/), 2023. URL https://arxiv.org/abs/2209.03143 . [↗](https://arxiv.org/abs/2209.03143)
3. A. R. Bradlow and T. Bent. Perceptual adaptation to non-native speech. Cognition, 106(2): 707–729, 2008. ISSN 0010-0277. doi: https://doi.org/10.1016/j.cognition.2007.04.005. URL https://www.sciencedirect.com/science/article/pii/S0010027707001126 . — [doi:10.1016/j.cognition.2007.04.005](https://doi.org/10.1016/j.cognition.2007.04.005) [↗](https://doi.org/10.1016/j.cognition.2007.04.005)
4. T. Brown, B. Mann, N. Ryder, M. Subbiah, J. D. Kaplan, P. Dhariwal, A. Neelakantan, P. Shyam, G. Sastry, A. Askell, S. Agarwal, A. Herbert-Voss, G. Krueger, T. Henighan, R. Child, A. Ramesh, D. Ziegler, J. Wu, C. Winter, C. Hesse, M. Chen, E. Sigler, M. Litwin, S. Gray, B. Chess, J. Clark, C. Berner… [find](https://scholar.google.com/scholar?q=T%2E+Brown%2C+B%2E+Mann%2C+N%2E+Ryder%2C+M%2E+Subbiah%2C+J%2E+D%2E+Kaplan%2C+P%2E+Dhariwal%2C+A%2E+Neelakantan%2C+P%2E+Shyam%2C+G%2E+Sastry%2C+A%2E+Askell%2C+S%2E+Agarwal%2C+A%2E+Herbert%2DVoss%2C+G%2E+Krueger%2C+T%2E+Henighan%2C+R%2E+Child%2C+A%2E+Ramesh%2C+D%2E+Ziegle)
5. Curran Associates, Inc., 2020a. URL [https://proceedings.neurips.cc/paper_files/paper/2020/file/1](https://proceedings.neurips.cc/paper_files/paper/2020/file/1) 457c0d6bfcb4967418bfb8ac142f64a-Paper.pdf . [↗](https://proceedings.neurips.cc/paper_files/paper/2020/file/1)
6. T. B. Brown, B. Mann, N. Ryder, M. Subbiah, J. Kaplan, P. Dhariwal, A. Neelakantan, P. Shyam, G. Sastry, A. Askell, S. Agarwal, A. Herbert-Voss, G. Krueger, T. Henighan, R. Child, A. Ramesh, D. M. Ziegler, J. Wu, C. Winter, C. Hesse, M. Chen, E. Sigler, M. Litwin, S. Gray, B. Chess, J. Clark, C. Ber… — [Language Models are Few-Shot Learners](https://md.research.elamur.ai/free/en/papers/brown-2020-language-models-few-shot-learners/)
7. H. Bu, J. Du, X. Na, B. Wu, and H. Zheng. Aishell-1: An open-source mandarin speech corpus and a speech recognition baseline. In 2017 20th Conference of the Oriental Chapter of the International Coordinating Committee on Speech Databases and Speech I/O Systems and Assessment (O-COCOSDA), pages 1–5,… — [doi:10.1109/icsda.2017.8384449](https://doi.org/10.1109/icsda.2017.8384449) [find](https://scholar.google.com/scholar?q=H%2E+Bu%2C+J%2E+Du%2C+X%2E+Na%2C+B%2E+Wu%2C+and+H%2E+Zheng%2E+Aishell%2D1%3A+An+open%2Dsource+mandarin+speech+corpus+and+a+speech+recognition+baseline%2E+In+2017+20th+Conference+of+the+Oriental+Chapter+of+the+International+Coord)
8. Y.-A. Chung, Y. Zhang, W. Han, C.-C. Chiu, J. Qin, R. Pang, and Y. Wu. W2v-bert: Combining contrastive learning and masked language modeling for self-supervised speech pre-training. In 2021 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU), pages 244– 250. [find](https://scholar.google.com/scholar?q=Y%2E%2DA%2E+Chung%2C+Y%2E+Zhang%2C+W%2E+Han%2C+C%2E%2DC%2E+Chiu%2C+J%2E+Qin%2C+R%2E+Pang%2C+and+Y%2E+Wu%2E+W2v%2Dbert%3A+Combining+contrastive+learning+and+masked+language+modeling+for+self%2Dsupervised+speech+pre%2Dtraining%2E+In+2021+IEEE+Autom)
9. J. Copet, F. Kreuk, I. Gat, T. Remez, D. Kant, G. Synnaeve, Y. Adi, and A. Défossez. Simple and controllable music generation. Advances in Neural Information Processing Systems, 36: 47704–47720, 2023. [find](https://scholar.google.com/scholar?q=J%2E+Copet%2C+F%2E+Kreuk%2C+I%2E+Gat%2C+T%2E+Remez%2C+D%2E+Kant%2C+G%2E+Synnaeve%2C+Y%2E+Adi%2C+and+A%2E+D%C3%A9fossez%2E+Simple+and+controllable+music+generation%2E+Advances+in+Neural+Information+Processing+Systems%2C+36%3A+47704%E2%80%9347720%2C+2023%2E)
10. A. Défossez, J. Copet, G. Synnaeve, and Y. Adi. [High fidelity neural audio compression](https://md.research.elamur.ai/free/en/papers/defossez-2022-high-fidelity-neural-audio-compression/). arXiv preprint arXiv:2210.13438, 2022.
11. A. Défossez, L. Mazaré, M. Orsini, A. Royer, P. Pérez, H. Jégou, E. Grave, and N. Zeghidour. [Moshi: a speech-text foundation model for real-time dialogue](https://md.research.elamur.ai/free/en/papers/defossez-2024-moshi-speech-text-foundation-model/). Technical report, 2024. URL https://arxiv.org/abs/2410.00037 . [↗](https://arxiv.org/abs/2410.00037)
12. A. Défossez, J. Copet, G. Synnaeve, and Y. Adi. [High fidelity neural audio compression](https://md.research.elamur.ai/free/en/papers/defossez-2022-high-fidelity-neural-audio-compression/), 2022. URL https://arxiv.org/abs/2210.13438 . [↗](https://arxiv.org/abs/2210.13438)
13. Q. Fang, S. Guo, Y. Zhou, Z. Ma, S. Zhang, and Y. Feng. Llama-omni: Seamless speech interaction with large language models, 2025. URL https://arxiv.org/abs/2409.06666 . [↗](https://arxiv.org/abs/2409.06666)
14. Z. Gao, S. Zhang, I. McLoughlin, and Z. Yan. Paraformer: Fast and accurate parallel transformer for non-autoregressive end-to-end speech recognition, 2023. URL [https://arxiv.org/ab](https://arxiv.org/ab) s/2206.08317 . [↗](https://arxiv.org/ab)
15. A. Goel, S. Ghosh, J. Kim, S. Kumar, Z. Kong, S.-g. Lee, C.-H. H. Yang, R. Duraiswami, D. Manocha, R. Valle, et al. Audio flamingo 3: Advancing audio intelligence with fully open large audio language models. arXiv preprint arXiv:2507.08128, 2025. — [arXiv:2507.08128](https://arxiv.org/abs/2507.08128) [find](https://scholar.google.com/scholar?q=A%2E+Goel%2C+S%2E+Ghosh%2C+J%2E+Kim%2C+S%2E+Kumar%2C+Z%2E+Kong%2C+S%2E%2Dg%2E+Lee%2C+C%2E%2DH%2E+H%2E+Yang%2C+R%2E+Duraiswami%2C+D%2E+Manocha%2C+R%2E+Valle%2C+et+al%2E+Audio+flamingo+3%3A+Advancing+audio+intelligence+with+fully+open+large+audio+language+)
16. Y. Gong, L. Jin, R. Deng, D. Zhang, X. Zhang, Q. Cheng, Z. Fei, S. Li, and X. Qiu. [Xy-tokenizer: Mitigating the semantic-acoustic conflict in low-bitrate speech codecs](https://md.research.elamur.ai/free/en/papers/gong-2026-xy-tokenizer-mitigating-semantic-acoustic/). arXiv preprint arXiv:2506.23325, 2025.
17. H. He, Z. Shang, C. Wang, X. Li, Y. Gu, H. Hua, L. Liu, C. Yang, J. Li, P. Shi, Y. Wang, K. Chen, P. Zhang, and Z. Wu. Emilia: An extensive, multilingual, and diverse speech dataset for large-scale speech generation, 2024. URL https://arxiv.org/abs/2407.05361 . — [arXiv:2407.05361](https://arxiv.org/abs/2407.05361) [↗](https://arxiv.org/abs/2407.05361)
18. D. Hendrycks and K. Gimpel. Gaussian error linear units (gelus). arXiv preprint arXiv:1606.08415, 2016. [find](https://scholar.google.com/scholar?q=D%2E+Hendrycks+and+K%2E+Gimpel%2E+Gaussian+error+linear+units+%28gelus%29%2E+arXiv+preprint+arXiv%3A1606%2E08415%2C+2016%2E)
19. D. Hendrycks, C. Burns, S. Basart, A. Zou, M. Mazeika, D. Song, and J. Steinhardt. [Measuring massive multitask language understanding](https://md.research.elamur.ai/free/en/papers/hendrycks-2020-measuring-massive-multitask-language-understanding/), 2021. URL https://arxiv.org/abs/2009.0 3300 . [↗](https://arxiv.org/abs/2009.0)
20. W.-N. Hsu, B. Bolte, Y.-H. H. Tsai, K. Lakhotia, R. Salakhutdinov, and A. Mohamed. [Hubert: Self-supervised speech representation learning by masked prediction of hidden units](https://md.research.elamur.ai/free/en/papers/hsu-2021-hubert-self-supervised-speech-representation/). IEEE/ACM transactions on audio, speech, and language processing, 29:3451–3460, 2021.
21. K. Huang, Q. Tu, L. Fan, C. Yang, D. Zhang, S. Li, Z. Fei, Q. Cheng, and X. Qiu. Instructttseval: Benchmarking complex natural-language instruction following in text-to-speech systems, 2025. URL https://arxiv.org/abs/2506.16381 . [↗](https://arxiv.org/abs/2506.16381)
22. W. Kang, X. Yang, Z. Yao, F. Kuang, Y. Yang, L. Guo, L. Lin, and D. Povey. Libriheavy: a 50,000 hours asr corpus with punctuation casing and context, 2024. URL [https://arxiv.org/ab](https://arxiv.org/ab) s/2309.08105 . [↗](https://arxiv.org/ab)
23. KimiTeam, D. Ding, Z. Ju, Y. Leng, S. Liu, T. Liu, Z. Shang, K. Shen, W. Song, X. Tan, H. Tang, Z. Wang, C. Wei, Y. Xin, X. Xu, J. Yu, Y. Zhang, X. Zhou, Y. Charles, J. Chen, Y. Chen, Y. Du, W. He, Z. Hu, G. Lai, Q. Li, Y. Liu, W. Sun, J. Wang, Y. Wang, Y. Wu, Y. Wu, D. Yang, H. Yang, Y. Yang, Z. Ya… — [Kimi-Audio Technical Report](https://md.research.elamur.ai/free/en/papers/kimiteam-2025-kimi-audio-technical-report/)
24. J. Kong, J. Kim, and J. Bae. Hifi-gan: Generative adversarial networks for efficient and high fidelity speech synthesis, 2020. URL https://arxiv.org/abs/2010.05646 . [↗](https://arxiv.org/abs/2010.05646)
25. S. Kumar, Šimon Sedláček, V. Lokegaonkar, F. López, W. Yu, N. Anand, H. Ryu, L. Chen, M. Plička, M. Hlaváček, W. F. Ellingwood, S. Udupa, S. Hou, A. Ferner, S. Barahona, C. Bolaños, S. Rahi, L. Herrera-Alarcón, S. Dixit, S. Patil, S. Deshmukh, L. Koroshinadze, Y. Liu, L. P. G. Perera, E. Zanou, T. S… — [arXiv:2508.13992](https://arxiv.org/abs/2508.13992) [find](https://scholar.google.com/scholar?q=S%2E+Kumar%2C+%C5%A0imon+Sedl%C3%A1%C4%8Dek%2C+V%2E+Lokegaonkar%2C+F%2E+L%C3%B3pez%2C+W%2E+Yu%2C+N%2E+Anand%2C+H%2E+Ryu%2C+L%2E+Chen%2C+M%2E+Pli%C4%8Dka%2C+M%2E+Hlav%C3%A1%C4%8Dek%2C+W%2E+F%2E+Ellingwood%2C+S%2E+Udupa%2C+S%2E+Hou%2C+A%2E+Ferner%2C+S%2E+Barahona%2C+C%2E+Bola%C3%B1os%2C+S%2E+Rahi%2C+L%2E+Herrer)
26. M. Lehet and L. L. Holt. Nevertheless, it persists: Dimension-based statistical learning and normalization of speech impact different levels of perceptual processing. Cognition, 202: 104328, 2020. ISSN 0010-0277. doi: https://doi.org/10.1016/j.cognition.2020.104328. URL https://www.sciencedirect.com… — [doi:10.1016/j.cognition.2020.104328](https://doi.org/10.1016/j.cognition.2020.104328) [↗](https://doi.org/10.1016/j.cognition.2020.104328)
27. T. Li, J. Liu, T. Zhang, Y. Fang, D. Pan, M. Wang, Z. Liang, Z. Li, M. Lin, G. Dong, et al. [Baichuan-audio: A unified framework for end-to-end speech interaction](https://md.research.elamur.ai/free/en/papers/li-2025-baichuan-audio-unified-framework/). arXiv preprint arXiv:2502.17239, 2025.
28. J. H. Lim and J. C. Ye. Geometric gan, 2017. URL https://arxiv.org/abs/1705.02894 . — [arXiv:1705.02894](https://arxiv.org/abs/1705.02894) [↗](https://arxiv.org/abs/1705.02894)
29. Z. Liu, H. Mao, C.-Y. Wu, C. Feichtenhofer, T. Darrell, and S. Xie. A convnet for the 2020s, 2022. URL https://arxiv.org/abs/2201.03545 . [↗](https://arxiv.org/abs/2201.03545)
30. Z. Ma, Y. Ma, Y. Zhu, C. Yang, Y.-W. Chao, R. Xu, W. Chen, Y. Chen, Z. Chen, J. Cong, K. Li, K. Li, S. Li, X. Li, X. Li, Z. Lian, Y. Liang, M. Liu, Z. Niu, T. Wang, Y. Wang, Y. Wang, Y. Wu, G. Yang, J. Yu, R. Yuan, Z. Zheng, Z. Zhou, H. Zhu, W. Xue, E. Benetos, K. Yu, E.-S. Chng, and X. Chen. Mmar:… — [arXiv:2505.13032](https://arxiv.org/abs/2505.13032) [find](https://scholar.google.com/scholar?q=Z%2E+Ma%2C+Y%2E+Ma%2C+Y%2E+Zhu%2C+C%2E+Yang%2C+Y%2E%2DW%2E+Chao%2C+R%2E+Xu%2C+W%2E+Chen%2C+Y%2E+Chen%2C+Z%2E+Chen%2C+J%2E+Cong%2C+K%2E+Li%2C+K%2E+Li%2C+S%2E+Li%2C+X%2E+Li%2C+X%2E+Li%2C+Z%2E+Lian%2C+Y%2E+Liang%2C+M%2E+Liu%2C+Z%2E+Niu%2C+T%2E+Wang%2C+Y%2E+Wang%2C+Y%2E+Wang%2C+Y%2E+Wu%2C+G%2E+Yang%2C+J)
31. T. Miyato, T. Kataoka, M. Koyama, and Y. Yoshida. Spectral normalization for generative adversarial networks, 2018. URL https://arxiv.org/abs/1802.05957 . — [arXiv:1802.05957](https://arxiv.org/abs/1802.05957) [↗](https://arxiv.org/abs/1802.05957)
32. V. Panayotov, G. Chen, D. Povey, and S. Khudanpur. Librispeech: An asr corpus based on public domain audio books. In 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 5206–5210, 2015. doi: 10.1109/ICASSP.2015.7178964. — [doi:10.1109/icassp.2015.7178964](https://doi.org/10.1109/icassp.2015.7178964) [find](https://scholar.google.com/scholar?q=V%2E+Panayotov%2C+G%2E+Chen%2C+D%2E+Povey%2C+and+S%2E+Khudanpur%2E+Librispeech%3A+An+asr+corpus+based+on+public+domain+audio+books%2E+In+2015+IEEE+International+Conference+on+Acoustics%2C+Speech+and+Signal+Processing+%28ICAS)
33. A. Radford, J. W. Kim, T. Xu, G. Brockman, C. McLeavey, and I. Sutskever. [Robust speech recognition via large-scale weak supervision](https://md.research.elamur.ai/free/en/papers/radford-2022-robust-speech-recognition-large-scale/). In Proceedings of the 40th International Conference on Machine Learning, ICML’23. JMLR.org, 2023.
34. A. W. Rix, J. G. Beerends, M. P. Hollier, and A. P. Hekstra. Perceptual evaluation of speech quality (pesq)-a new method for speech quality assessment of telephone networks and codecs. In 2001 IEEE international conference on acoustics, speech, and signal processing. Proceedings (Cat. No. 01CH37221)… [find](https://scholar.google.com/scholar?q=A%2E+W%2E+Rix%2C+J%2E+G%2E+Beerends%2C+M%2E+P%2E+Hollier%2C+and+A%2E+P%2E+Hekstra%2E+Perceptual+evaluation+of+speech+quality+%28pesq%29%2Da+new+method+for+speech+quality+assessment+of+telephone+networks+and+codecs%2E+In+2001+IEEE+in)
35. S. Sakshi, U. Tyagi, S. Kumar, A. Seth, R. Selvakumar, O. Nieto, R. Duraiswami, S. Ghosh, and D. Manocha. Mmau: A massive multi-task audio understanding and reasoning benchmark, 2024. URL https://arxiv.org/abs/2410.19168 . [↗](https://arxiv.org/abs/2410.19168)
36. S. Sakshi, U. Tyagi, S. Kumar, A. Seth, R. Selvakumar, O. Nieto, R. Duraiswami, S. Ghosh, and D. Manocha. MMAU: A massive multi-task audio understanding and reasoning benchmark. In The Thirteenth International Conference on Learning Representations, 2025. URL [https://openreview.net/forum?id=TeVAZXr…](https://openreview.net/forum?id=TeVAZXr…) [↗](https://openreview.net/forum?id=TeVAZXr…)
37. V. Sirdeshmukh, K. Deshpande, J. Mols, L. Jin, E.-Y. Cardona, D. Lee, J. Kritz, W. Primack, S. Yue, and C. Xing. Multichallenge: A realistic multi-turn conversation evaluation benchmark challenging to frontier llms, 2025. URL https://arxiv.org/abs/2501.17399 . [↗](https://arxiv.org/abs/2501.17399)
38. H. Siuzdak. Vocos: Closing the gap between time-domain and fourier-based neural vocoders for high-quality audio synthesis, 2024. URL https://arxiv.org/abs/2306.00814 . — [arXiv:2306.00814](https://arxiv.org/abs/2306.00814) [↗](https://arxiv.org/abs/2306.00814)
39. X. Song, M. Xing, C. Ma, S. Li, D. Wu, B. Zhang, F. Pan, D. Zhou, Y. Zhang, S. Lei, Z. Peng, and Z. Wu. Touchtts: An embarrassingly simple tts framework that everyone can touch, 2024. URL https://arxiv.org/abs/2412.08237 . — [arXiv:2412.08237](https://arxiv.org/abs/2412.08237) [↗](https://arxiv.org/abs/2412.08237)
40. A. Srivastava, A. Rastogi, A. Rao, A. A. M. Shoeb, A. Abid, A. Fisch, A. R. Brown, A. Santoro, A. Gupta, A. Garriga-Alonso, et al. [Beyond the imitation game: Quantifying and extrapolating the capabilities of language models](https://md.research.elamur.ai/free/en/papers/srivastava-2022-imitation-game-quantifying-extrapolating-capabilities/). arXiv preprint arXiv:2206.04615, 2022.
41. J. Su, M. Ahmed, Y. Lu, S. Pan, W. Bo, and Y. Liu. [Roformer: Enhanced transformer with rotary position embedding](https://md.research.elamur.ai/free/en/papers/su-2021-roformer-enhanced-transformer-rotary-position/). Neurocomputing, 568:127063, 2024.
42. M. Sumner. The role of variation in the perception of accented speech. Cognition, 119(1):131– 136, 2011. ISSN 0010-0277. doi: https://doi.org/10.1016/j.cognition.2010.10.018. URL https://www.sciencedirect.com/science/article/pii/S0010027710002556 . — [doi:10.1016/j.cognition.2010.10.018](https://doi.org/10.1016/j.cognition.2010.10.018) [↗](https://doi.org/10.1016/j.cognition.2010.10.018)
43. M. Suzgun, N. Scales, N. Schärli, S. Gehrmann, Y. Tay, H. W. Chung, A. Chowdhery, Q. V. Le, E. H. Chi, D. Zhou, and J. Wei. [Challenging big-bench tasks and whether chain-of-thought can solve them](https://md.research.elamur.ai/free/en/papers/suzgun-2023-challenging-big-bench-tasks-whether/). arXiv preprint arXiv:2210.09261, 2022.
44. C. H. Taal, R. C. Hendriks, R. Heusdens, and J. Jensen. A short-time objective intelligibility measure for time-frequency weighted noisy speech. In 2010 IEEE international conference on acoustics, speech and signal processing, pages 4214–4217. [find](https://scholar.google.com/scholar?q=C%2E+H%2E+Taal%2C+R%2E+C%2E+Hendriks%2C+R%2E+Heusdens%2C+and+J%2E+Jensen%2E+A+short%2Dtime+objective+intelligibility+measure+for+time%2Dfrequency+weighted+noisy+speech%2E+In+2010+IEEE+international+conference+on+acoustics%2C+spe)
45. A. van den Oord, O. Vinyals, and K. Kavukcuoglu. Neural discrete representation learning, 2018. URL https://arxiv.org/abs/1711.00937 . [↗](https://arxiv.org/abs/1711.00937)
46. D. Wang, J. Wu, J. Li, D. Yang, X. Chen, T. Zhang, and H. Meng. Mmsu: A massive multi-task spoken language understanding and reasoning benchmark, 2025. URL https://arxiv.org/abs/2506.04779 . — [arXiv:2506.04779](https://arxiv.org/abs/2506.04779) [↗](https://arxiv.org/abs/2506.04779)
47. J. Wei, Y. Tay, R. Bommasani, C. Raffel, B. Zoph, S. Borgeaud, D. Yogatama, M. Bosma, D. Zhou, D. Metzler, E. H. Chi, T. Hashimoto, O. Vinyals, P. Liang, J. Dean, and W. Fedus. [Emergent abilities of large language models](https://md.research.elamur.ai/free/en/papers/wei-2022-emergent-abilities-large-language-models/), 2022a. URL https://arxiv.org/abs/2206.07682 . [↗](https://arxiv.org/abs/2206.07682)
48. J. Wei, X. Wang, D. Schuurmans, M. Bosma, F. Xia, E. Chi, Q. V. Le, D. Zhou, et al. [Chain-of-thought prompting elicits reasoning in large language models](https://md.research.elamur.ai/free/en/papers/wei-2022-chain-thought-prompting-elicits-reasoning/). Advances in neural information processing systems, 35:24824–24837, 2022b.
49. B. Wu, C. Yan, C. Hu, C. Yi, C. Feng, F. Tian, F. Shen, G. Yu, H. Zhang, J. Li, M. Chen, P. Liu, W. You, X. T. Zhang, X. Li, X. Yang, Y. Deng, Y. Huang, Y. Li, Y. Zhang, Z. You, B. Li, C. Wan, H. Hu, J. Zhen, S. Chen, S. Yuan, X. Zhang, Y. Jiang, Y. Zhou, Y. Yang, B. Li, B. Ma, C. Song, D. Pang, G.… — [Step-Audio 2 Technical Report](https://md.research.elamur.ai/free/en/papers/wu-2025-step-audio-2-technical/)
50. Y.-C. Wu, I. D. Gebru, D. Marković, and A. Richard. Audiodec: An open-source streaming high-fidelity neural audio codec. In ICASSP 2023-2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 1–5. [find](https://scholar.google.com/scholar?q=Y%2E%2DC%2E+Wu%2C+I%2E+D%2E+Gebru%2C+D%2E+Markovi%C4%87%2C+and+A%2E+Richard%2E+Audiodec%3A+An+open%2Dsource+streaming+high%2Dfidelity+neural+audio+codec%2E+In+ICASSP+2023%2D2023+IEEE+International+Conference+on+Acoustics%2C+Speech+and+Sign)
51. L.-C.-T. Xiaomi. Mimo: Unlocking the reasoning potential of language model – from pretraining to posttraining, 2025. URL https://arxiv.org/abs/2505.07608 . [↗](https://arxiv.org/abs/2505.07608)
52. D. Xin, X. Tan, S. Takamichi, and H. Saruwatari. Bigcodec: Pushing the limits of low-bitrate neural speech codec. arXiv preprint arXiv:2409.05377, 2024. — [arXiv:2409.05377](https://arxiv.org/abs/2409.05377) [find](https://scholar.google.com/scholar?q=D%2E+Xin%2C+X%2E+Tan%2C+S%2E+Takamichi%2C+and+H%2E+Saruwatari%2E+Bigcodec%3A+Pushing+the+limits+of+low%2Dbitrate+neural+speech+codec%2E+arXiv+preprint+arXiv%3A2409%2E05377%2C+2024%2E+%E2%80%94+%5BarXiv%3A2409%2E05377%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2409)
53. J. Xu, Z. Guo, J. He, H. Hu, T. He, S. Bai, K. Chen, J. Wang, Y. Fan, K. Dang, B. Zhang, X. Wang, Y. Chu, and J. Lin. [Qwen2.5-omni technical report](https://md.research.elamur.ai/free/en/papers/xu-2025-qwen2-5-omni-technical/), 2025. URL https://arxiv.org/abs/ 2503.20215 . [↗](https://arxiv.org/abs/)
54. Z. Ye, P. Sun, J. Lei, H. Lin, X. Tan, Z. Dai, Q. Kong, J. Chen, J. Pan, Q. Liu, et al. Codec does matter: Exploring the semantic shortcoming of codec for audio language model. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 39, pages 25697–25705, 2025a. [find](https://scholar.google.com/scholar?q=Z%2E+Ye%2C+P%2E+Sun%2C+J%2E+Lei%2C+H%2E+Lin%2C+X%2E+Tan%2C+Z%2E+Dai%2C+Q%2E+Kong%2C+J%2E+Chen%2C+J%2E+Pan%2C+Q%2E+Liu%2C+et+al%2E+Codec+does+matter%3A+Exploring+the+semantic+shortcoming+of+codec+for+audio+language+model%2E+In+Proceedings+of+the+A)
55. Z. Ye, X. Zhu, C.-M. Chan, X. Wang, X. Tan, J. Lei, Y. Peng, H. Liu, Y. Jin, Z. Dai, H. Lin, J. Chen, X. Du, L. Xue, Y. Chen, Z. Li, L. Xie, Q. Kong, Y. Guo, and W. Xue. Llasa: Scaling train-time and inference-time compute for llama-based speech synthesis, 2025b. URL https://arxiv.org/abs/2502.0412… — [arXiv:2502.04128](https://arxiv.org/abs/2502.04128) [↗](https://arxiv.org/abs/2502.0412…)
56. J. Yu, H. Chen, Y. Bian, X. Li, Y. Luo, J. Tian, M. Liu, J. Jiang, and S. Wang. Autoprep: An automatic preprocessing framework for in-the-wild speech data, 2023. URL https://arxiv.org/abs/ 2309.13905 . [↗](https://arxiv.org/abs/)
57. N. Zeghidour, A. Luebs, A. Omran, J. Skoglund, and M. Tagliasacchi. [Soundstream: An end-to-end neural audio codec](https://md.research.elamur.ai/free/en/papers/zeghidour-2021-soundstream-end-end-neural-audio/). IEEE/ACM Transactions on Audio, Speech, and Language Processing, 30: 495–507, 2021.
58. A. Zeng, Z. Du, M. Liu, K. Wang, S. Jiang, L. Zhao, Y. Dong, and J. Tang. [Glm-4-voice: Towards intelligent and human-like end-to-end spoken chatbot](https://md.research.elamur.ai/free/en/papers/zeng-2024-glm-4-voice-intelligent/). arXiv preprint arXiv:2412.02612, 2024.
59. D. Zhang, S. Li, X. Zhang, J. Zhan, P. Wang, Y. Zhou, and X. Qiu. [SpeechGPT: Empowering large language models with intrinsic cross-modal conversational abilities](https://md.research.elamur.ai/free/en/papers/zhang-2023-speechgpt-empowering-large-language-models/). In H. Bouamor, J. Pino, and K. Bali, editors, Findings of the Association for Computational Linguistics: EMNLP 2023, pages 15757–15773, S…
60. X. Zhang, D. Zhang, S. Li, Y. Zhou, and X. Qiu. [Speechtokenizer: Unified speech tokenizer for speech large language models](https://md.research.elamur.ai/free/en/papers/zhang-2023-speechtokenizer-unified-speech-tokenizer-speech/). arXiv preprint arXiv:2308.16692, 2023b.
61. Y. Zhang, W. Han, J. Qin, Y. Wang, A. Bapna, Z. Chen, N. Chen, B. Li, V. Axelrod, G. Wang, et al. Google usm: Scaling automatic speech recognition beyond 100 languages. arXiv preprint arXiv:2303.01037, 2023c. — [arXiv:2303.01037](https://arxiv.org/abs/2303.01037) [find](https://scholar.google.com/scholar?q=Y%2E+Zhang%2C+W%2E+Han%2C+J%2E+Qin%2C+Y%2E+Wang%2C+A%2E+Bapna%2C+Z%2E+Chen%2C+N%2E+Chen%2C+B%2E+Li%2C+V%2E+Axelrod%2C+G%2E+Wang%2C+et+al%2E+Google+usm%3A+Scaling+automatic+speech+recognition+beyond+100+languages%2E+arXiv+preprint+arXiv%3A2303%2E01037)

*Updated 1 November 2026*